In my previous post I described situation awareness as it applies to learning analytics in complex adaptive systems. The aircraft analogy I used, compared the black-box flight recorder with the cockpit instrumentation to differentiate the role of real-time sensemaking with retrospective analysis. As David commented, learning and teaching is more complex than flying an aircraft, which means the instrumentation and the black-box need to be more configurable and adaptable than the analogy would suggest. Irrespective of the analogy, my suspicion is that a majority of learning analytics projects are too focused on retrospective data analysis. This analysis has limited value in complex contexts when there is a need to act and adapt in the here-and-now.
That said, whether the learning analytics data is retrospective or real-time, they are both used by humans to make sense of something. Sensemaking is a well-researched phenomenon and I think it has the potential to help us improve learning analytics. Even my favorite learning analytics definition alludes to the important role that sensemaking has:
“Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs”
What is sensemaking?
Sensemaking refers to how we structure the unknown so as to be able to act in it (Weick, 2005). Sensemaking involves coming up with a plausible understanding, a map, of a shifting world; testing this map with others through data collection, action, and conversation; and then refining, or abandoning, the map depending on how credible it is (Ancona, 2010). Action is not a separate or later step in sensemaking, but is an integral part of it (Ancona, 2010). This aligns neatly with how agents act in complex adaptive systems whereby they probe, sense and respond. The unanticipated and unintended consequences of acting within a complex adaptive system make upfront analysis less valuable than typical organizational ways of doing things would suggest. (Weick, 2005) summarises this nicely when he says:
“ To work with the idea of sensemaking is to appreciate that smallness does not equate with insignificance. Small structures and short moments can have large consequences”
I mention organisations (such as universities) deliberately because the risk averse SET mindset drives approaches (such as those involving learning analytics) and are based on upfront analysis and one-off projects (I would say at the expense of sensemaking). The sense-making and decision-making models that I would associate with the SET mindsets are outmoded models based on linear and stable environments (Mika, 2008). In these models, analysis and upfront design make rational sense, but they do not match the unstable and turbulent contexts that we see today.
One of things that I notice with learning analytics is that data from information systems receives most, if not all the focus. According to (Ancona, 2010) when sensemaking, you “seek out and combine many different types of data. This includes system data and narrative of people involved”. The area of narrative is an area that I’m quite interested in, an interest learned from Dave Snowden’s Cynefin podcasts from sometime ago. I just wonder if the learning analytics community is a little too focused on data from information systems when there is an untapped human sensor network available?
The following are some other interesting quotes I found while scanning the sensemaking literature that I have to consider further:
“Failure is part of sensemaking”
People create their own environments and are then constrained by them.
“much of the effort to design information technology to support cognition in organizations has not addressed its distributed quality”
“Sensemaking is about the interplay of action and interpretation rather than the influence of evaluation on choice”
“ignorance and knowledge coexist, which means that adaptive sensemaking both honors and rejects the past”
Ancona, D. Framing and Acting in the Unknown.
Boland Jr, R. J., Tenkasi, R. V., & Te’eni, D. (1994). Designing information technology to support distributed cognition. Organization science, 5(3), 456-475.
Mika, A. Multi-ontology, sense-making and the emergence of the future. Futures, 41, 279-283. doi:10.1016/j.futures.2008.11.017
Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (2005). Organizing and the process of sensemaking. Organization science, 16(4), 409-421.