In a previous post I talked a little about how universities (well their leaders anyway) are attracted to the predictive potential of learning analytics, and how this approach is fundamentally flawed. In a subsequent post I suggested that situation awareness can provide a useful theoretical platform that may help universities broaden their learning analytics implementation approaches beyond prediction-based endeavours. This new post has been inspired by a couple of fascinating LAK18 papers that talk about learning analytics implementation.
Universities in Australia generally fall into two camps based on their leadership approach to learning analytics implementation; top-down and bottom-up (Dawson et al., 2018).
Top-down or instrumental approaches to learning analytics adoption are often based on preconceived recipes or prescribed methodologies and are all too often doomed to failure (Dawson et al., 2018). The top-down implementation of analytics related technology is the easy part, getting uptake and generating impact based on how and if people use the technology is where these systems fail (Marchand & Peppard, 2013). In other words, these systems are easy to install but are unlikely generate the desired uptake and impact. It is well known that top-down implementation approaches are “less than ideal” when it comes to learning analytics (Beer & Jones, 2014, p. 244).
Bottom-up or emergent approaches to learning analytics takes a much more consultative approach and usually begin on a small scale (Dawson et al., 2018). However, bottom up approaches are difficult to scale up beyond the local context to “a more holistic and complex organisational level” (Dawson et al., 2018, p. 236). So while the bottom-up approach might meet the needs of a small number of learning and teaching contexts, it may fail to scale beyond this due to the diversity of contexts found in a typical university.
As LA research continues to grow there is a very real danger of a widening gulf between identified research needs and outcomes and applied practice. (Dawson et al., 2018, p.242)
While seemingly discrete, I suspect these two approaches are unfortunately linked. Technology adoption in Australian Higher Education is dominated by vanilla implementations and centralised approaches (D. T. Jones & Clark, 2014). So even if the learning analytics system at an institution has been developed with a bottom-up approach and has a track record of uptake and impact, it may still be perceived as “feral” or “risky” due to its decentralised and perhaps unconventional origins (Spierings, Kerr, & Houghton, 2014).
In talking to various folk at the recent ALASI2017 conference, there seems to be a trend whereby universities are wanting to quickly bring learning analytics to the enterprise. In at least one case that I am aware of, a platform developed using a bottom-up approach, is being replaced with a commercial off-the-shelf product that is to be implemented using a top-down, centralised approach, in spite of the evidence against such an approach.
Once an innovation such as [learning analytics] achieves a high public profile, it can create an urgency to ‘join the bandwagon’ that swamps deliberative, mindful behavior (Beer & Jones, 2014, p. 243)
There are two things at play here that I’m thinking about with regards to my PhD studies. The first is how learning analytics is being conceptualised by these universities. No matter the university, learning analytics is an applied research project (Dawson et al., 2018) and not an IT project. I suspect that this mistake/misinterpretation also contributes to the high failure rates experienced by analytics related projects (Marchand & Peppard, 2013). The second is the role of meso-level practitioners and they can potentially contribute to bridging the gap between the two approaches (Hannon, 2013). Meso-level practitioners assuage the tension between the small-scale, local interactions, and the large-scale policy and institutional processes (C. Jones, Dirckinck‐Holmfeld, & Lindström, 2006; Uhl-Bien, Marion, & McKelvey, 2007).
As a personal aside with regards to LAK18, my application to the doctoral consortium got accepted and I was very much looking forward to attending. Unfortunately, I had to withdraw at the last minute due to a family illness. It is a fantastic event that sees the world’s foremost experts in learning analytics gather to talk and share their stories. I dearly hope I can make LAK19 next year.
Beer, C., & Jones, D. T. (2014). Three paths for learning analytics and beyond: Moving from rhetoric to reality. Paper presented at the Australasian Society for Computers in Learning in Tertiary Education: Rhetoric and Reality, Dunedin, New Zealand. Conference Publication retrieved from http://ascilite2014.otago.ac.nz/files/fullpapers/185-Beer.pdf
Dawson, S., Poquet, O., Colvin, C., Rogers, T., Pardo, A., & Gasevic, D. (2018). Rethinking learning analytics adoption through complexity leadership theory. Paper presented at the Proceedings of the 8th International Conference on Learning Analytics and Knowledge.
Hannon, J. (2013). Incommensurate practices: Sociomaterial entanglements of learning technology implementation. Journal of Computer Assisted Learning, 29, 168-178.
Jones, C., Dirckinck‐Holmfeld, L., & Lindström, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer-Supported Collaborative Learning, 1(1), 35-56.
Jones, D. T., & Clark, D. (2014). Breaking Bad to Bridge the Reality / Rhetoric Chasm. Paper presented at the ASCILITE2014 Rhetoric and Reality, Dunedin, New Zealand. Conference publication retrieved from http://ascilite2014.otago.ac.nz/
Marchand, D. A., & Peppard, J. (2013). Why IT Fumbles Analytics. Harvard Business Review, 91(1), 104-112.
Spierings, A., Kerr, D., & Houghton, L. (2014). What drives the end user to build a feral information system? Feral Information Systems Development: Managerial Implications, 161-188.
Uhl-Bien, M., Marion, R., & McKelvey, B. (2007). Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era. The Leadership Quarterly, 18, 298-318. doi:10.1016/j.leaqua.2007.04.002