The purpose of this study was to compare effectiveness and student satisfaction of distance education versus face-to-face interaction in delivering therapeutic crisis management skills content to Associate Degree Nursing (ADN) Students as measured by test scores, overall grade point average (GPA), class grade, and student satisfaction survey results. One group of students was taught via face-to-face interaction in the traditional classroom setting with case studies and group work. The other group of students was taught via distance education with the same instructor presentation followed by the same case studies and distance education group work. This researcher believed that test scores would be higher in the distance education setting because the students would have to take the time to read and respond to discussion board questions. Personal experience led to this hypothesis because students have verbalized in the past that they enjoy a distance education environment where students can set the pace. No difference between the two separate groups or enhanced performance by those who receive instruction through distance education could indicate that distance education is an effective teaching modality for therapeutic crisis management.
Methods:
The study was a quasi-experimental, post-hoc causal comparative, two group post-test only design. There were two conditions: face-to-face classroom delivery of course content and distance education with online course content. The study was designed to compare effectiveness of distance education and face-to-face interaction through reviewing test grades, overall GPA, and class grades. Additionally students were asked to rate their satisfaction of the different modalities.
Institutional Review Board approval was received from two agencies involved in the research study. Participation for the study was voluntary. Students’ agreement to participate in the study was obtained through volunteer signatures. Consent was obtained from all volunteers prior to the study through the Student Consent Form. Students who had completed the third semester psychiatric mental-health nursing course were invited to participate in the survey. Students were not asked to complete the survey until they had completed all work associated with the course.
Test score data was collected from test questions related specifically to the therapeutic crisis management content. Test questions were part of a larger exam. Test questions were validated with data analysis including Point Biserial and content validity. Question analysis was gathered from ParScore©, a test analysis computer software program. Content validity was established through the use of questions on previous tests during previous years with data from previous statistical analyses. Overall GPA and class grades were gathered from student records. Surveys of student satisfaction were developed by the researcher, peer reviewed by two faculty members and were piloted with a cohort of 65 students. No formal validity measure (Cornbach alpha) have been reported for prior uses of these measures.
An independent samples t-test was employed to identify therapeutic crisis management test performance differences between students in face-to-face interaction and distance education sections. A t-test was also performed to assess differences in test grades, course grades and GPAs of students who were in their preferred setting and those who were not. Type I errors were controlled for by using SPSS software.
A quantitative analysis regarding satisfaction was performed with a series of questions on a researcher-developed survey. For categorical responses on the questionnaire, such as age, gender, ethnicity, learning styles, and satisfaction counts and percentages are presented. All tests were conducted at a significance level of 0.05. The primary hypothesis was to investigate differences between test scores, overall GPA, and class grade of students who were taught therapeutic crisis management techniques via face-to-face interaction and those who were taught through a distance education format.
Results:
There were 110 participants who were eligible and agreed to participate in the study. There were 63 participants in the distance education group and 47 participants in the face-to-face interaction group. The majority of participants were 18 to 29 years of age (59.1%, n=65), female (84.5%, n=93), and Caucasian (79.1%, n=87). The age distribution of the remainder of the participants was as follows; 23.6% (n=126) categorized themselves as between the ages of 30 to 39, 17.3% (n=19) categorized themselves as between the ages of 40-59. There were 15.5% (n=17) male participants. The ethnicity of the other participants was 9.1% (n= 10) African American, 5.5% (n=6) Hispanic, 2.7% (n=3) Asian, and 3.6% (n=4) classified themselves as Other Ethnicity. The majority of the participants, 42.7%, (n=47) categorized themselves as visual and auditory learners, 32.7% (n=36) categorized themselves as tactile and visual learners, 24.5% (n=27) categorized themselves as other style learners.
The first research hypothesis was to investigate if there were differences between test scores, overall GPA, and class grades of students who were taught therapeutic crisis management techniques via face-to-face interaction and those who were taught through a distance education format. A series of Independent Samples t-tests, with an alpha .05, were performed to assess the mean difference between the section of the course as the dependent variable and test grade, overall GPA, and class grade as the independent variables. Data were characterized for their distributional characteristics using descriptive and graphical methods where they were tested for equal variance and passed. Where the assumptions for the t-test were not met data was transformed to reduce skewness and number of outliers, and improve the normality and linearity of any residuals. Analysis was performed using SPSS.
The mean test grade was 82.1 out of 100, SD=5.88 for the distance education group and 82.8, SD=5.20 for the face-to-face interaction group. No statistically significant difference was noted between test grades (t58=.704; DF=108, p=.483). The overall mean GPA for the distance education group was 3.1 compared to 3.0 for the face-to face interaction group. This difference also was not statistically significant (t58=.765; DF=108, p=.446). The mean class grade was 82.4, SD=4.23 for the distance education group and 82.8, SD=4.34 for the face-to-face interaction group. This difference in class grades was not statistically significant (t58=.429; DF=108, p=.668).
The second research question was to investigate if there were differences in nursing student satisfaction between distance education and face-to-face interaction when learning therapeutic crisis management techniques. The same methods listed above were used to assess the distributional characteristics for each of the variables of interest. The satisfaction scores were taken from a Likert scale with twenty-eight total points possible. A higher score is indicative of less satisfaction while a lower score is more indicative of higher satisfaction. Total satisfaction scores were grouped into three separate categories; one to nine points indicated satisfaction, ten to nineteen points indicated neutral, and twenty to twenty-eight points indicated not satisfied. Overall satisfaction score means were 9.3, SD=3.63 for the distance education group and 8.6, SD=2.48 for the face-to-face interaction group. This difference was not statistically significant (t58=1.12; DF=108, p=.264).There were no significantly statistical differences (t58=.169; DF=108, p=.87).
Overall, there were 67% of the students who were in the preferred section and 39% who were not in the preferred section. The mean test grade was 82.8, SD=4.38 for the students who were in the preferred section and 82.2, SD=7.16 for the group that was not in the preferred section. No statistically significant difference was noted between test grades (t58=.478; DF=104, p=.634). The overall mean GPA for the group that was in the preferred section was 3.1, SD= .353 compared to 3.0, SD=.414 for the group that was not in the preferred section. This difference was not statistically significant (t58=.646; DF=104, p=.520). The mean class grade was 82.8, SD=3.99 for the group that was in the preferred section and 82.3, SD=4.80 for the group that was not in the preferred section. This difference in class grades was not statistically significant (t58=2.68; DF=104, p=.563).
Conclusion:
Nursing schools have had to develop ways to handle faculty and space shortages. Distance education must be an option for nursing schools facing today’s difficult challenges. No difference between the face-to-face and distance education group could indicate that distance learning is an effective teaching modality when compared to face-to-face interaction. Despite the limitation of the size of the study large differences between groups (large effects size) would have been detected if present.
It is important to note that student preference for a particular teaching modality did not impact the overall test grade. Difference in overall GPA and overall course grade were also not statistically significant. There were also no statistically significant differences of overall satisfaction between students who were and those who were not in their preferred section. This may suggest that providing both face-to-face interaction and distance education sections will not adversely affect student outcomes for the course.
Neither research question was supported by the data from the study. No differences were found between test scores of students who were taught therapeutic crisis management techniques via face-to-face interaction and those who were taught through a distance education format. No differences in the degree of nursing student satisfaction between distance education and face-to-face interaction when learning therapeutic crisis management techniques was found either.