I developed the pre- and post- knowledge surveys based upon the work of Wirth and Perkins (2005) and Jarzemsky, McCarthy and Ellis (2010) for use with professional, pre-licensure nursing students. In addition I created a demographic survey, a clinical simulation performance evaluation and finally, I created evaluation items for this study, I used an experimental, randomized pre-test, post-test design and, participants received didactic learning materials and participated in a high fidelity simulation specifically focused upon end-of-life care.
Using non-parametric tests using chi square (with Phi or Cramer’s V) as well as the t-test for paired samples, I evaluated whether or not the use of a knowledge survey increased knowledge and clinical skill/ability. The participants reported a perception of improvement in their knowledge and clinical skill/abilities; they overwhelmingly overestimated their knowledge; they over/under estimated their abilities to perform clinical skills; and their perceptions did not reflect actual ability to safely demonstrate clinical skills/abilities. These findings may be consistent with the Dunning-Kruger effect. This effect is found when low performing students over-estimate their ability and high performing students’ under-estimate or have an accurate perception of their ability. I also used non-parametric tests, including chi square test (with the McNemar test and Phi correlational coefficient) to evaluate whether there was a relationship between student perceptions and their cognitive knowledge and clinical skills/abilities. It was difficult to determine whether the student’s perceptions of their knowledge and clinical skill/ability were strongly or directly related to their actual ability and, student’s perceptions of their knowledge and clinical skill/ability were more strongly and positively related to their actual clinical ability during a simulation.
There are several strengths and limitations of this study. The strengths included: there are no published studies in nursing regarding the use of a knowledge survey; the literature does not reveal any discipline that has assessed behavioral components using a knowledge survey and this study introduces a behavioral component; there is the potential to expand a knowledge survey across an entire semester of a nursing course rather than focus upon a single area of content; and, finally, the incorporation of the QSEN competencies illustrates how profession-specific competencies can be incorporated into a knowledge survey. The limitations of this study included the fact that this was a pilot study; the sample size was small; the use of on-line surveys may affect student accuracy; the lack of debriefing items in the surveys; the lack of diversity of participants, the use of non-parametric statistics; and the difficulty of ensuring that participants did not talk with one another and influence the findings of the study.
There are a number of directions for future research and nursing education practice. The directions for research include: planning ahead when creating a knowledge survey in order to ensure the validity/reliability of items; planning to control for bias by using participants not known to the researcher; exposing students to the concept and function of knowledge surveys prior to conducting a research study; providing a thorough description of the knowledge survey with students prior to recruitment; administering the knowledge survey in class rather than on-line; consider offering extra credit or a small incentive for students to participate in knowledge survey research; collect data from a knowledge survey that is administered over several semesters; and lastly, categorize the skill of participants into levels of high, mid and low.
There are several ways in which the use of knowledge surveys can impact nursing education practice. For instance, it may be helpful to explain to students the goal of using knowledge surveys and how the use of such surveys can improve metacognition. In addition, educators should be mindful of the possibility of the Dunning-Kruger effect; closely monitor student activities in the clinical setting; provide timely and accurate feedback to improve metacognition; recognize students may not accurately self and/or peer evaluations for simulations; and finally, encourage the practice of reflective judgement.