There are many publications and presentations espousing the potential of learning analytics to contribute to improved decision-making in education. For example:
“Basing decisions on data and evidence seems stunningly obvious, and indeed, research indicates that data-driven decision- making improves organizational output and productivity.”
(Siemens and Long 2011)
This quote was based on very interesting research that showed improved performance across a bunch of publicly listed companies that better utilized data-driven decision-making (Brynjolfsson, Hitt, and Kim 2011). I like the Siemens quote and the research it was based on for two reasons, one good and one bad. Firstly, I work in a regional Australian university and I know that higher education can do a lot better with regards to making decisions based on evidence. Secondly, the Siemen’s quote appeals to me because (a) I have great respect for his opinion and (b) there is a part of me that believes that better data and information will lead to better decisions.
Note that I used the work ‘believes’ deliberately in the previous sentence. Upon reflection, this is probably more of a faith thing than a scientific reality. The trouble is that better and particularly more detailed data does not necessarily lead to better sense-making and/or decision-making (Aaltonen 2008). Human beings are just not wired that way.
“We are a bricolage of cognition, emotion, intuition, information consumption, doubt and belief”
This highlights the importance of including human beings in the learning analytics cycle. Something that Doug Clow has previously noted:
“Previous work in the literature has emphasised the need for and value of human meaning-making in the process of interpretation of data to transform it in to actionable intelligence.”
So even though we know that humans are essential in the information / learning analytics cycle, we also know that humans are bad at making ‘rational’ decisions based on data. We filter our observations of the world through our cognitive frameworks. Our frameworks are individual to each and every one of us and include things such as experience, intuition and instinct. Throw in that learning analytics is closely coupled with IT where the main considerations are likely to be precision, rigor and reproducibility rather than the human consumers (Norman 1993).
“The logic behind many investments in IT tools and big data initiatives is that giving managers more high-quality information more rapidly will improve their decisions and help them solve problems and gain valuable insights. That is a fallacy.”
(Marchand and Peppard 2013)
A long way of saying that I’m wondering if we need to spend more time on the recipients of the information rather than the data side of things?
Aaltonen, Mika. 2008. “Multi-ontology, sense-making and the emergence of the future.” Futures 41:279-283. doi: 10.1016/j.futures.2008.11.017.
Brynjolfsson, Erik, Lorin M Hitt, and Heekyung Hellen Kim. 2011. “Strength in numbers: How does data-driven decisionmaking affect firm performance?” Available at SSRN 1819486.
Clow, Doug. 2014. “Data wranglers: human interpreters to help close the feedback loop.” Proceedings of the Fourth International Conference on Learning Analytics And Knowledge.
Marchand, Donald A., and Joe Peppard. 2013. “Why IT Fumbles Analytics.” Harvard Business Review 91 (1):104-112.
Norman, Donald A. 1993. Things that make us smart: Defending human attributes in the age of the machine: Basic Books.
Siemens, George. 2006. Knowing knowledge: Lulu. com.
Siemens, George, and Phillip Long. 2011. “Penetrating the Fog: Analytics in Learning and Education.” EDUCAUSE Review 46 (5):9.