![[ Visit Client Website ]](images/banner.gif)
Learning Objective 1: Examine assignment grading rubrics for validity and reliability;
Learning Objective 2: Develop accurate guidelines for grading threaded discussions in an online program
This presentation will describe research conducted on the TD grading rubric thus far. The following questions were addressed in the initial phase of the research: “Do faculty members apply the TD grading rubric similarly, and does the grading rubric reflect the student knowledge and skills intended by faculty?” Faculty members randomly selected weekly posts of 20 students in an evidence-based practice nursing course, resulting in 196 sets of posts related to TD questions. Two faculty members simultaneously applied the rubric to each student’s set of posts. Cohen’s kappa was used to estimate inter-rater reliability.
The first study, completed in 2011, resulted in a Cohen’s kappa of 0.227 with overall agreement of 56% between the two raters indicating inconsistency in grading. Based on these results, the rubric was revised with specific rubric guidelines rewritten for students and faculty.
The study was repeated in 2012 using two different raters and a different course. There were minimal changes in the outcomes of the study. The Cohen’s kappa was 0.049 with an overall agreement of 52%. Faculty is currently investigating the specific issues in the second study causing the low Cohen’s Kappa.
The research group will then revise the guidelines and directions for the grading rubric to develop accurate guidelines for grading TDs and assignments and to pursue a program to continually improve grading and feedback for students.
See more of: Oral Paper & Poster: Evidence-Based Practice Sessions