Content Specific Simulation-Supported Learning and High-Stakes Exams: Longitudinal Outcomes

Purpose: The use of human patient simulation in nursing curriculum has intensified over the last several years. As more schools of nursing begin and/or continue to implement the use of human patient simulation objective evaluation tools must be created and utilized to measure learning outcomes. The benefits of human patient simulation have been demonstrated in the nursing students' ability to improve skill acquisition, knowledge attainment, critical thinking skills, and development of clinical judgment. Although the many benefits of human patient simulation have been thoroughly examined, the relationship between specific human patient simulated clinical experiences and retention of content specific knowledge via success on high-stakes standardized nursing exams has not been examined. The results of this study add to the growing body of research related to human patient simulation as an effective teaching modality for utilization in nursing education related to scores on nursing students' standardized tests.

The purpose of this longitudinal quantitative study was to (1) examine the trajectory of cardiovascular content related scores obtained on high-stakes standardized nursing exams after exposure to cardiovascular specific simulation-supported learning, and (2) compare the trajectories scores of the experimental group (cardiac specific simulation) with the control group (course related simulation).

Methods: This longitudinal, experimental pre-test/post-test quantitative study compared two groups of baccalaureate nursing student who experienced simulation-supported learning. The research question was: How do students who experience a human patient simulated clinical experience perform on content specific standardized high-stakes exams; specifically how do the scores on the exam differ by group (those who received the cardiovascular specific simulation [experimental] versus those who did not [control])? This was examined by presenting the usual cardiovascular specific information in class and subsequent completion of a cardiovascular content specific standardized exam (pre-test). After completion of the pre-test, students were randomized into two groups: control or experimental. During the selected semester each student completes a human patient simulation experience. Students completed the simulation in teams of two, with the simulation for the experimental group being cardiovascular specific, while the control received the usual course simulation. After all students had completed participation, a second cardiovascular specific standardized exam (post-intervention) was administered. Scores on the cardiovascular specific content of other high-stakes standardized exams taken at course completion and exit from the nursing program were examined for trajectory and compared by groups. The independent variable was the cardiovascular specific simulation. Dependent variables included scores on the (1) pre-test for the cardiovascular specific standardized exam, (2) the post-test 1 for the cardiovascular specific standardized exam after simulation completion, (3) the post-test 2 cardiovascular specific high-stakes standardized exam at course completion, and (4) the post-test 3 cardiovascular specific content on the first attempt on the final semester high-stakes program completion exit exam (required for graduation). Approval of the institutional committee for the protection of human subjects was obtained.

Results: Ninety four students completed all waves of testing. Prior to knowing about group randomization, students in the control group scored significantly higher on the pre-test (F(1, 93) = 21.54, p < .000). Students in the control group achieved a mean score and standard deviation of 977 and 157, respectively; while the experimental group mean and standard deviation were 823 and 157, respectively. This relationship reversed for the first post-test. The experimental group scored significantly higher when compared to the control group (F(1, 93) = 5.04, p < .027). The mean and standard deviation of the experimental group was 982 and 171, respectively; while the control group mean and standard deviation were 900 and 184, respectively. No significant differences were found between groups on the second (course completion) and third (program exit) high-stakes post-test exams. This study also examined differences in percent of change between groups over the three post-test exams. There was a statistically significant difference between groups in percent of change from pre-test to first post-test (F(1, 92) = 38.19, p < .000). The mean and standard deviation of the experimental group changed from 832 and 157, respectively to 981 and 171, respectively. In the control group the mean and standard deviation changed from 977 and 145, respectively to 899 and 184, respectively. There was also a statistically significant difference between groups in percent of change from pre-test to the high-stakes program exit exam, post-test 3 (F(1, 92) = 19.16, p < .000). The mean and standard deviation of the experimental group changed from 832 and 157, respectively to 948 and 105, respectively. In the control group the mean and standard deviation changed from 977 and 145, respectively to 938 and 90, respectively.

Conclusion:

It is interesting, yet unexplained that the non-intervention group scored significantly higher on the pre-test. This reinforces the positive finding that students who participated in a cardiovascular simulation may have gained more knowledge as demonstrated by the fact that they performed significantly better on the first post-test than those students who do not receive the targeted simulation. Not unexpectedly, this difference in scores does not persist over time, although the experimental group percent of change in mean scores was significantly different from the control group. In fact, the experimental group means score increased from pre-test to post-test 3 (high-stakes program exit exam), while the control group mean score actually decreased from pre-test to high-stakes exit exam. This may indicate a longitudinal effect of the targeted cardiovascular simulation experience, or may be due to unknown and confounding factors.

This longitudinal quantitative study examined the trajectory of cardiovascular content related scores obtained on high-stakes standardized nursing exams after exposure to a simulation learning experience. Initially, students in the experimental group demonstrated increased scores, indicating better internalization of didactic content, but this difference between groups did not last over time. It is important to remember that the control group scored higher at pre-test, yet lower at the high-stakes exit exam (post-test 3); while the experimental group’s trajectory of change was in the opposite direction. More research is needed to discover possible factors that may explain these results.