Paper
Tuesday, November 6, 2007
691
A Pilot Study of a Metacognitive Skills Diagnostic Test for Nursing Students
Judith E. Garrett, PhD, Office of Educational Development, University of Arkansas for Medical Sciences, Little Rock, AR, USA, Sarah J. Rhoads, DNP, APN, College of Nursing, University of Arkansas for Medical Sciences, Little Rock, AR, USA, Larronda Rainey, MSN, BS, College of Nursing (former), University of Arkansas for Medical Sciences (formerly), Maumelle, AR, USA, and Linda Calhoun, RN, MNSc, College of Nursing, University of Arkansas for Medical Sciences, Little Rock, AR, USA.
| Learning Objective #1: differentiate between cognitive and metacognitive skills |
| Learning Objective #2: interpret outcomes of validation measures used in the pilot study |
Objective: To assess the feasibility of developing a valid and reliable diagnostic pretest of cognitive skills related to metacognition. Methods: Cognitive skills hypothesized to be related to metacognition in two entry-level baccalaureate nursing courses were identified and assessment items were constructed. Although the N was small(54), exploratory factor analysis was used to examine the structure of cognitive tasks involved in pretest items. Regression analysis, including multicollinearity assessment, was used to explore the contribution of each type of skill to achievement. Participants’ scores on both the diagnostic test and the two criterion measures were divided into a high and low group. Crosstabs output was used in specificity and sensitivity analyses to assess the extent to which pretest scores accurately identified high and low scoring students on two criterion measures. Results: Cronbach’s Alpha for the overall pretest was 0.89. Preliminary exploratory factor analysis suggested that some skills were indeed multi-dimensional depending on whether stimulus materials for test items involved a single concept (e.g., visualizing a sarcomere) or principle (e.g., visualizing how a sarcomere contracts and relaxes). The ability to condense information was a significant in both courses and multicollinearity diagnostics indicated that results were not affected by intercorrelations among variables. Of the students who scored below the group mean in Exam 1 in Health Assessment, 59% had a positive ‘pretest’ result; i.e., they scored below the group mean (sensitivity) while 81% who scored below the group mean on Exam 1 also had a negative pretest result; i.e., they scored above the group mean (specificity). The magnitude of similar analyses for Foundations was somewhat less (sensitivity = 52% and specificity = 77%). Conclusion: Preliminary results suggest that courses have different metacognitive skills demands. However large-scale tryouts are needed to validate these pilot results.