Skip to content
April 11, 2012 / dlitgroup

LMS Data and Student Achievement: Which Variables Are Meaningful?

It has been suggested that analysis of LMS tracking data may allow early identification of students at risk of academic failure. But which online tracking variables indicate meaningful activity in relation to learning? We’ll present some of our work showing that only 15 of over 100 possible LMS tracking variables demonstrated any significant simple correlation with a student’s final grade in online courses and that a logistic model using only 3 of these correctly identified 81% of students who eventually failed. Importantly, we’ll discuss how course design must be considered in the predictive efforts of learning analytics. Participants will (1) consider the potential and limitations of LMS tracking data for predicting student achievement, (2) discuss the basic challenges of institutional data gathering and integration, and (3) explore the availability of additional indicators of learning and achievement in order to strengthen future predictive models.

Manager, Evaluation and Learning Analytics
The University of British Columbia
  • Slides are available here.
  • The 5 challenges of LMS learning analytics and predictive modeling.
  • How do educators track student learning progress?  Traditional summative assessments typically occur too late in learning and they offer limited insight that is timely.
  • Pioneers in LA “Wang & colleagues” (2000, 2002) logins linked to student performance and “Campbell & colleagues” (2006, 2007) focus on retention and attrition and the inclusion of the LMS login data to a strengthened model of predicting student success.
  • The key tension of LA – the wish to move beyond data mining to the need to select data variables that are meaningful.
  • Challenge #1 – Getting access to the Data
    • Most LMSs capture and store user activity but a small portion is available via “tracking tools”.
    • Little information is available about how/why displayed variables were selected for tracking.
  • Challenge #2 – Finding the meaningful variables
    • The key is to find the variables that can correlate data between the LMS and the SIS final grades.
  • Challenge #3 – Students are not univariate actors
    • Correlation and causality or not the same thing.
    • Students have multiple combinations of multiple activities.
  • Challenge #4 – Predictive models must consider course design
    • Tool implementations, tool use, online activities, use of discussion boards and assessments, etc.
  • Challenge #5 – How do we measure learning
    • Measures of learning engagement such as time spent in peer engagement activities, and data such as patterns of discussion board use.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: