While learning analytics have been used to analyze the association between student activity and performance in online higher education (Yu & Il-Hyun,2014; Zhang, 2016), studies on this topic were sparsely found in the nursing literature (Patzer et al., 2017). Analytics provided by a learning management system (LMS) (Blackboard Learn, 2018) may provide information on student engagement and improve the quality of online courses by making changes in learning activities, assignments and the learning environment (Deitz-Uhler & Hurn, 2013a; Gazza & Hunter, 2014). Additionally, analytics can be used to examine student activity data within a course to make predictions about learning outcomes and institute appropriate interventions to improve those outcomes for the future. The purpose of this study was to analyze the relationship between engagement, as measured by course analytics (course access or the number of times students accessed the course, minutes spent in the course, interactions with the instructor or students, and submissions in the course) with course grade.
Methods: The conceptual framework for this study was based on the work of Alexander Astin (1975, 1984, 1985, 1993). According to Astin (1975; 1984; 1985; 1993), students are described as highly involved if: 1) they interacted with faculty more, 2) participated actively with fellow students on a more frequent basis, and 3) devoted more time to studying. Likewise, those students who neglected studying, had less frequent interactions with faculty and were on the opposite end of high levels of involvement (Astin,1985).
A retrospective, correlational design was used to analyze the relationship between admission GPA, course analytics (course access or the number of times students accessed the course, minutes spent in the course, interactions with the instructor or students, and submissions in the course) with course grade. Associations with age, gender, major, and geography were also completed. The sample consisted of (n=360) MSN students enrolled at a large, private, urban university hosting an online MSN program in the northeast part of the United States. Analysis included correlation coefficients, analysis of variance (ANOVA), and multiple linear regression using the backwards elimination method. Specifically, Pearson product moment correlation coefficients and Spearman correlation coefficients were produced to assess relationships between the input variables (GPA, major, access, minutes, interactions, submissions) and the output variable (grade). ANOVA was used to assess the impact of categorical predictor variables on grade, and backwards elimination within a multiple linear regression analysis was used to produce a model that only included variables that significantly predicted grade at the alpha = .05 level. The non-parametric Kruskal-Wallis test was used to confirm the results of the ANOVA. All analyses were performed using the Statistical Analysis System (SAS) software package version 9.4.
Results: The factors that most impacted grade were entry level GPA, age, interactions with instructor or students and submissions in the course. Each additional increase in submissions, resulted in an increase in course grade by 0.33% (p<0.0001). Additionally, each one-point increase in entry level GPA was associated with an increase in course grade by 1.93% (p = 0.0289). Each one-year increase in age, demonstrated a course grade decrease of 0.17% (p<0.0001).
Conclusion: A primary finding from this research was the fact that interactions and submissions had the highest impact on the outcome of course grade. This is consistent with Astin’s model of involvement (engagement) which states that students who participate actively with faculty and students and devote more time to studying, are highly involved. Further study needs to be conducted using larger samples to determine if Astin’s model is verified in that entry GPA does determine student grades. While mildly associated in this study, more work should be done to see if these results are replicated. Moreover, if grades do have a relationship between age, access and minutes in a more intensive way as found in this study, additional work needs to be done for corroboration. One interesting fact from this study was that older students tended to have lower grades. While grades do appear to be related to interactions and submissions, replication of the study could determine whether those classes which vary in numbers of assignments still demonstrate an association with interactions and submissions.