The Effect on Inter-Rater Consistency Using a Standardized Assessment Tool/Framework in Musculoskeletal Examinations

This quality improvement (QI) project was conducted to determine if a standardized assessment framework/tool for clinician use when conducting musculoskeletal examinations for disability claims would increase inter-rater consistency. The problem of inconsistent musculoskeletal examination outcomes was identified when clinicians performing disability examinations at a large healthcare organization reported a lack of criteria on which to base the determination of “functional loss.” The finding of “functional loss” was being made independently by each clinician without any specific criteria and the examination outcomes lacked inter-rater consistency. A survey of the clinicians revealed that several examiners concluded that an examinee experienced "functional loss" in a large percentage of the examinations they completed, while other clinicians rarely indicated a "functional loss" existed, despite similar findings in examination metrics, such as range of motion and/or pain. This resulted in unfair compensation for the disability claimants. Standardized assessment tools have been used widely in a variety of settings in various disciplines and have proved to be beneficial in identifying and quantifying body function and structure limitations; this improves inter-rater consistency and clinical practice. A thorough evaluation of available tools and frameworks for use in musculoskeletal examinations was conducted and an appropriate framework was selected for use in this QI project by the clinicians performing the disability examinations. Rhetorical education, lunch-time discussions, slide presentations, and case studies were carried out to convey all educational information necessary regarding the purpose, function, and use of the selected framework. Clinicians were required to achieve an 80% pass rate on a post-education examination to assure appropriate use of the selected framework. A three-month period of post-education shoulder examination data was collected for comparison with the previously collected data. A Pearson’s Chi-Square test performed to analyze the data and revealed a statistically significant association (a=0.05 level) between post vs. pre-intervention for “functional loss” findings by clinicians, along with a seven-fold increase in overall positive findings for “functional loss” by clinicians using the standardized framework to perform the musculoskeletal examinations.