Learning analytics or crystal balls: Which one works best?



This post is a brief attempt at a “so what” from my previous post where I mentioned my growing skepticism about the predictive potential of learning analytics. It is also tying this post with some previous posts that I believe are relevant.

Broadly speaking, if we consider institutional learning and teaching environments as complex systems, or more correctly, complex adaptive systems, we change how these systems are managed. In linear systems cause and effect are evident and therefore, predicting the future states of these systems becomes possible. In complex systems, the agents are interacting with each other and the environment, which means the systems are in a constant state of evolution. It gets even more interesting when the individual human agents interacting within the system are known to have multiple identities and are temporally unpredictable. This is why prediction is almost impossible in complex systems; there are simply too many variables with unpredictable and disproportionate effects.

Much of the rhetoric around learning analytics is talking about its potential to use data about what has happened to predict what will happen. As I said in my previous post, this ignores the interactions and subsequent changes that occur after the prediction. This alone places real-world limits on the predictive potential of learning analytics, something that the commercial entities are unlikely to admit. Human beings feel threatened by uncertainty, which feeds our fears. Hence we strive to eliminate uncertainty by trying to predict the future so as to eliminate uncertainty.

“The study of the psychology of risk perception has found that one of the most powerful influences on fear is uncertainty.”

I have a number of concerns about learning analytics and predictive modeling. Firstly, our ability to use learning analytics for predictive modeling is inherently limited. Secondly, human nature compels us to try and reduce uncertainty by anticipating future states through prediction. Thirdly, commercial entities and consultants know about human nature and are playing to our fears by associating their products with an ability to make predictions. If their product X is so good at predictive modeling, why aren’t they making a killing on the share market?

All of this is a long way of saying that predictive modeling with learning analytics is interesting and potentially valuable but is not the only way that learning analytics can be applied. I live in fear of the one-off hegemonic approaches that organisations love to take. This comes back to my previous post on situation awareness whereby learning analytics has a role to play in better representing the present. A better map if you like, about what is the current state of the system and agents right now. This feeds into sense making where sense making is how we develop an understanding of what we are sensing to that we can take action. To me, this is where learning analytics can really make a difference with an increasingly complex higher education landscape. It’s also a lot easier than having to learn all those complicated statistics.


2 thoughts on “Learning analytics or crystal balls: Which one works best?”

  1. In the end I wonder if you’ve back-pedaled too far when you see “predictive modeling with learning analytics is interesting and potentially valuable but is not the only way that learning analytics…”. I wonder if there’s any value at all in predictive modeling? Especially if you’re predicting something quite a way in the future? Especially given some of the findings from Gasevic et al (2015).

    Wondering where the best value will come from
    Complicate prediction – Increasing the number of predictive models to increase prediction quality?
    Focus on sense-making – Focus work on providing actors with better ideas of what is happening so they can react further and check?

    Or is it the case of either or? Doesn’t sensemaking lead to making plans and taking action on the assumption that it will achieve some goal (i.e. prediction)? I wonder if what is meant by prediction (gee an academic argument about semantics) is at play here.

    Wonder what Gasevic et al (2015) suggested? ahh

    it is imperative for learning analytics research to take into account instructional conditions when developing predictive models.

    They also suggest

    The course specific models of academic success can offer valuable in
    sights for instructors regarding how to improve their instructional practice

    Is there value in course specific models of “what the hell is going on” rather than “success”?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s