Poster Presentation
Thursday, 20 July 2006
10:00 AM - 10:30 AM
Thursday, 20 July 2006
3:00 PM - 3:30 PM
This presentation is part of : Poster Presentations II
Assessment of Cognitive Level and Item-Writing Flaws in Multiple-Choice Questions in High Stakes Nursing Examinations
Marie Tarrant, RN, MPH, PhD1, Sasha Hayes, RN, MN1, Aimee Knierim, RN, BSN2, and James Ware, MA, MB, FRCS, DMSc2. (1) Department of Nursing Studies, University of Hong Kong, Hong Kong, Hong Kong, (2) Faculty of Medicine, Chinese University of Hong Kong, Hong Kong, Hong Kong
Learning Objective #1: Identify common item writing flaws found in multiple-choice questions (MCWs) in nursing examinations
Learning Objective #2: Identify the impact of item writing flaws and cognitive level on the test performance of MCQs in nursing examinations.
Multiple Choice Questions (MCQs) are a frequently used assessment method in nursing examinations.  Few nurse educators, however, have formal preparation in constructing MCQs.  An unfortunate consequence is that many MCQ items on baccalaureate nursing tests and examinations have item writing flaws (IWFs).  All nursing assessments should have validity and a common threat to validity is the presence of IWFs.  An ongoing project shared by three Hong Kong nursing schools seeks to improve the assessments in high stakes nursing examinations. At one of the schools over 2800 MCQs have been collected, and are currently being classified and entered into an item test bank. At present, 824 of these items have been chosen randomly and further classified according to the cognitive level tested and the frequency of 19 accepted IWFs. Two cognitive levels were used for item classification: testing only simple recall of isolated fact (K1) and test of knowledge application, reasoning and problem solving (K2).  Item discrimination properties and difficulty index were also assessed for each question and compared across IWFs and cognitive level using chi-squares and ANOVAs.  Over 80% of the items were K1 and over 40% of questions contained IWFs. The frequency of IWFs among K1 items was 43.7% as compared to 12% for the K2 items (p<0.001). Common IWFs included the use of negatives or except in the question stem, the presence of logical clues, gratuitous information, ambiguous or unclear questions and failure to pass the hand-cover test. Questions containing IWFs were significantly more likely to have lower discrimination ability and more likely to negatively affect the difficulty index (p<0.05). Further analysis is currently underway to classify and examine the IWFs in all 2800 questions and to examine their impact upon question performance..

See more of Poster Presentations II
See more of The 17th International Nursing Research Congress Focusing on Evidence-Based Practice (19-22 July 2006)