Measuring What Matters: A Multi-Site Study of Self-Reported and Objectively Measured Nursing EBP Knowledge

Purpose: Research findings from fields as diverse as psychology and medicine (Blanch-Hartigan, 2011; Zell & Krizan, 2014) suggest that when tasked with evaluating one's own knowledge and abilities, people often over- or under-estimate themselves when their self-ratings are compared to more objective measures. The purpose of this study was to evaluate nurses' self-reported EBP knowledge, attitudes, and skills compared to EBP knowledge measured using a previously developed standards-based, objective test of EBP knowledge. Findings from the current study are compared with past research to provide a wider perspective on the state of nurses' EBP knowledge levels and approaches to measuring EBP knowledge, skills, and abilities.

Methods: Practicing nurses from two Magnet®-designated hospitals in the Midwest United States were recruited to the study. In proctored data collection sessions, subjects completed a 17-item demographic and professional characteristics questionnaire, the self-report Evidence-based Practice Questionnaire (EBPQ; Upton & Upton, 2006), and the Evidence-based Practice Knowledge Assessment in Nursing (EKAN; Spurlock & Wonder, 2015), a 20-item multiple-choice EBP knowledge test developed based on two widely-adopted US EBP competency frameworks. Data from N= 151 subjects indicated subjects were 95.1% White/Caucasian, 92% female, and had an average age of 40.9 years (Range 23-66 years). Nearly all (99.4%) of subjects reported English as their primary language and 57.7% reported having a bachelor's degree in nursing as their highest degree.

Results:  Subjects provided positive ratings overall on the three subscales of the EBPQ. Considering a maximum score of 7, subjects provided the highest average ratings on the EBP attitudes subscale (M = 5.51, SD = .98), followed by EBP knowledge (M = 4.68, SD = .81), and then on practice/use of EBP (M = 4.48, SD 1.37). The mean sum score on the EKAN was 10.58 (SD = 2.87) out of a possible 20 points. One-way ANOVA (with familywise error correction for alpha) showed no statistically significant difference on any subscale of the EBPQ across nurses’ educational levels (e.g., associate’s, bachelor’s, and master’s degree levels). When EKAN scores were compared based on subjects’ highest level of education, a statistically significant difference in EKAN scores was observed (9.0 for associate’s degrees versus 12.7 for master’s degrees; F_3,159 = 11.84, p < 0.001). While scores from each of the subscales of the EBPQ were statistically significantly correlated with each other (r = .350 - .595, p < .01), correlations between EBPQ subscales and the EKAN sum score were small (r = .017 - .123) and statistically nonsignificant. To further evaluate effect size, a two-step hierarchical regression analysis showed that educational level, but not EBPQ subscale scores, predicted EKAN sum scores (F_(1,149) = 30.43, p < .001, R²= .170).

Conclusion:  In this study, nurses’ EBP attitudes and self-reported knowledge and skills showed a near zero correlation with more objectively measured EBP knowledge. Higher levels of education were associated with higher scores on the EKAN knowledge measure but not with EBP attitudes, practices, or self-reported knowledge. These findings are consistent with those from a broad range of fields showing that accurate self-assessment is often difficult, as illustrated by the low concordance between self-assessed and objectively measured knowledge. Nurses' EBP knowledge levels in the current study are similar to those reported elsewhere (e.g., Spurlock & Wonder, 2015). Implications for leaders and educators in academic and clinical environments include re-evaluating strategies for assessing learning outcomes with a focus on using more direct, objective approaches when possible.