LMS Data and Student Achievement: Which Variables Are Meaningful?
It has been suggested that analysis of LMS tracking data may allow early identification of students at risk of academic failure. But which online tracking variables indicate meaningful activity in relation to learning? We’ll present some of our work showing that only 15 of over 100 possible LMS tracking variables demonstrated any significant simple correlation with a student’s final grade in online courses and that a logistic model using only 3 of these correctly identified 81% of students who eventually failed. Importantly, we’ll discuss how course design must be considered in the predictive efforts of learning analytics. Participants will (1) consider the potential and limitations of LMS tracking data for predicting student achievement, (2) discuss the basic challenges of institutional data gathering and integration, and (3) explore the availability of additional indicators of learning and achievement in order to strengthen future predictive models.
- Slides are available here.
- The 5 challenges of LMS learning analytics and predictive modeling.
- How do educators track student learning progress? Traditional summative assessments typically occur too late in learning and they offer limited insight that is timely.
- Pioneers in LA “Wang & colleagues” (2000, 2002) logins linked to student performance and “Campbell & colleagues” (2006, 2007) focus on retention and attrition and the inclusion of the LMS login data to a strengthened model of predicting student success.
- The key tension of LA – the wish to move beyond data mining to the need to select data variables that are meaningful.
- Challenge #1 – Getting access to the Data
- Most LMSs capture and store user activity but a small portion is available via “tracking tools”.
- Little information is available about how/why displayed variables were selected for tracking.
- Challenge #2 – Finding the meaningful variables
- The key is to find the variables that can correlate data between the LMS and the SIS final grades.
- Challenge #3 – Students are not univariate actors
- Challenge #4 – Predictive models must consider course design
- Tool implementations, tool use, online activities, use of discussion boards and assessments, etc.
- Challenge #5 – How do we measure learning
- Measures of learning engagement such as time spent in peer engagement activities, and data such as patterns of discussion board use.