Why should educators measure learning analytics?

As a Solutions Architect for Leo Learning (one of our Moodle Partners), Peter Dobinson uses “big data” to measure results.

In his MoodleMoot UK & Ireland 2017 presentation, Peter shared with us how educators can use analytics to measure the impact of learning.

He also explored how doing so can improve teaching and learning experiences. In this article, we summarise Peter’s presentation, which you can, of course, view from the link below.

So let’s start with….what is learning analytics?

Learning analytics is the measurement, collection, analysis and reporting of student data, which can then be used to optimise learning results.

Why should educators measure learning?

Peter’s presentation began with a quote from Williams Edwards Deming (American engineer, statistician, professor, well known for using data to improve economic outcomes post World War II): “In god we trust, everyone else must bring data”.

Peter explained that educators who don’t measure results are at best guessing, and, at worst using inbuilt bias to force learning on their students.

The real-world value of measuring learning data was summarised by Peter as:

• Proof of ROI and effectiveness.
• Highlighting of areas which will benefit from targeted learning.
• Proof of successful teaching and learning.
• Resulting in effective learning programmes.
• Allowing learning personalisation: meaning educators can see the best way people have learnt to customise the learning to benefit them.

What are the objections to Learning Analytics?

Following his introduction, Peter discussed the general or most common objections he believed educators had towards learning analytics:

• It is not a valid experiment.
  The academia world has commented that variables cannot be controlled, making learning analytics an invalid experiment. However, Peter argued that although all the variables cannot be controlled, a chain of evidence can be identified which suggests a conclusion.
• It’s too hard.
  Peter suggested that the right people and tools make can make it easy.
• It’s too expensive.
  Although this may be the case in some circumstances, there are cheap and free open source learning analytics tools available.

A Learning Analytics model

At Leo Learning, Peter described a model which includes seven steps to assist educators in identifying how to get started with learning analytics:

1. Work out what you want to do.
   Peter suggested that educators start small by running an A/B test through two courses. He discussed the benefit of not just measuring learning, but also identifying KPIs (key performance indicators) to set measurable goals.
2. Get buy in from all levels of the business.
   People can be fearful of tracking data, so it is essential that all data is communicated to all parties and that the information is used to improve the accuracy of learning.
3. Design an experiment.
   The next step is for educators to work out what they want to compare, as data collected is of no use in isolation. Peter suggests that this can be achieved by identifying quantifiable outcomes, for example, split testing.
4. Get the data.
   Tools such as xAPI and Caliper were suggested because they have portable interfaces capable of collecting data more complex than a “pass” or “fail”.
5. Measure your impact and try to disprove your hypothesis.
   Peter reminded #MootIEUK17 attendees that although correlation doesn’t equal causation, it is a good place to start when analysing data results.
6. Communicate your findings.
   Although Peter argued that it’s not ethical to withhold findings, he believed that it was important to pick carefully how findings were communicated. He suggested presenting the one “big finding” and “shouting” about that.
7. Use the findings to improve what you do.
   For those who were unsure how to use findings to implement change, Peter suggested seeking the help of a company which does.

Practical tools for learning analytics

Peter also shared tools which can be used to build up a learning ecosystem for learning analytics:

• Moodle
• Watershed
• Yet
• Learning locker
• gomoLearning
• xAPI apps
• Kaltura
• Lectora16
• SurveyGizmo

Summarising his presentation Peter reminded attendees of three main benefits of using learning analytics:

• Better outcomes for learners.
• Better outcomes for organisations
• Justification for budgets through learning analytics evidence.

Find out more about why should educators measure learning analytics, watch Peter’s full presentation below: