Situation awareness, complex adaptive systems and learning analytics

According to (Endsley, 1988) situation awareness is:

“the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and projection of their status in the near future”

While situation awareness has received particular attention in situations where spatial awareness is important (such as aircraft piloting), it is also applicable in non-moving systems such as industrial instrumentation systems (Pew, 1994). Situation awareness has become particularly important due to the introduction of advanced technologies and automation in many of today’s systems, which are being blamed for a reduction in situation awareness in many areas (Pew, 1994).

The “situation” in situation awareness has been defined as:

“a set of environmental conditions and system states with which the participant is interacting that can be characterized uniquely by a set of information, knowledge and response options”
(Pew, 1994).

There are a number of elements of awareness in situation awareness including:

  • Current state of the system including all the relevant variables
  • Predicted state in the “near” future
  • Information and knowledge required in support of the person’s activities
  • List of current goals
  • Time
  • Information and knowledge needed to support anticipated “near” future contexts

The “awareness” in situation awareness has been defined as the information resources that are available that can contribute to awareness:

  • Sensory information from the environment
  • Visual and auditory displays
  • Decision aids and decision support systems
  • Extra and intra team communication
  • Team member background knowledge and experience

It is worth knowing that situation awareness is the product that results from situation assessments. It is also important to note that situation awareness is not exclusively knowledge nor exclusively process, but is, an adaptive, externally directed consciousness (Smith & Hancock, 1995). In other words, situation awareness is a component within an adaptable cycle of knowledge, action and information.

The process of situational assessment requires active effort to achieve, effort that competes with other aspects of task performance. The situation awareness process requires someone to be attentive to numerous pieces of information, usually from a variety of sources. Some information will be ignored while some will be determined to be relevant. The structure of the information received from sensory inputs is critical as it determines how quickly and easily the input can be processed (Pew, 1994). This is an important consideration as it is well recognized that:

“human processing capabilities are not well suited to a multiplicity of tasks”
(Pew, 1994).

So what? What has this got to do with learning analytics?

To answer this, requires some explanation about complex adaptive systems. We believe, and have to some extent shown in a previous paper, that learning analytics is data resulting from the interactions of agents within complex adaptive systems (Beer, Jones, & Clark, 2012). Complex adaptive systems are systems containing agents that adapt and change as they interact (Holland, 2006). To cut a long story short, making predictions within complex adaptive systems is futile, as the patterns observed are never likely to be reproduced (Mason, 2008). This limits the value of retrospective (or out of context) analysis of agent behavior within complex systems, as the patterns uncovered are not likely to happen again. Hence, the recommended approach when dealing with complex adaptive systems is manage the situated present rather than targeting idealistic future states (Kurtz & Snowden, 2003).

This is where I see a role for learning analytics data and I think it links to David’s sentiments about why dashboards suck. The retrospective nature of business intelligence style dashboards limits their usefulness in the here and now. I’m not quite as anti-dashboard as David, I think they do have their uses, but I also think that far too much time, effort and money is being spent on retrospective analysis of data. We need more real-time data that can aid decision-making in the here and now. The analogy I’m thinking of is the comparison between an airliners black box flight recorder and the cockpit instrumentation.

IMG_9839

The black-box records what is happening so in the event of an accident, it can help retrospectively determine what transpired. The instrumentation in the flight deck is about providing the pilots with real-time situation awareness.

_MG_9288

A process of analysis is required to analyze black-box recordings whereas cockpit instrumentation is in-context sense-making. Based on my (limited) understanding of how learning analytics is being deployed around the world, I wonder if we could benefit from more real-time situation awareness rather than allocating all of our resources to predicting the future?

Things to do

The following is just a parking spot for some things to improve upon or explore based upon this post:

  • Look at specific examples whereby learning analytics is providing real-time decision support.
  • Explore the link between learning analytics, complex adaptive systems, situation awareness and the sense-making literature.
  • Explore the link between this post and (P)IRAC.
  • Explore distributed situation awareness along with distributed cognition
  • Explore the hypothesis linking SET mindsets/non-complex systems with retrospective data analysis.

 References

Beer, C., Jones, D., & Clark, D. (2012). Analytics and complexity: Learning and leading for the future. Paper presented at the ASCILITE2012 Future challenges, sustainable futures, Wellingtone, New Zealand.

Endsley, M. R. (1988). Situation awareness global assessment technique (SAGAT). Paper presented at the Aerospace and Electronics Conference, 1988. NAECON 1988., Proceedings of the IEEE 1988 National.

Holland, J. (2006). Studying Complex Adaptive Systems. Journal of Systems Science and Complexity, 19(1), 1-8. doi:10.1007/s11424-006-0001-z

Kurtz, C. F., & Snowden, D. J. (2003). The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal, 42(3), 462-483. Retrieved from http://ezproxy.cqu.edu.au/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=tnh&AN=10654356&site=ehost-live

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=5386804

Mason, M. (2008). What Is Complexity Theory and What Are Its Implications for Educational Change? Educational Philosophy and Theory, 40(1), 35-49. Retrieved from http://onlinelibrary.wiley.com/store/10.1111/j.1469-5812.2007.00413.x/asset/j.1469-5812.2007.00413.x.pdf?v=1&t=h672i83c&s=4c8fb933073ab8cf0e246a0ede0d3ad149ac5866

Norman, D. A. (1993). Things that make us smart: Defending human attributes in the age of the machine: Basic Books.

Pew, R. (1994). An introduction to the concept of situation awareness. Situational Awareness in complex systems, 17-26.

Smith, K., & Hancock, P. (1995). Situation awareness is adaptive, externally directed consciousness. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 137-148.

Advertisements

7 thoughts on “Situation awareness, complex adaptive systems and learning analytics”

  1. The cockpit instrumentation and the blackbox metaphor is a good one and has some linkages to stuff I’m playing with at the moment for a talk on Thursday.

    But that same stuff also suggests that the metaphor quickly breaks down and is perhaps the source of my hatred of dashboards. When you are flying (or driving) you have the same instrumentation as inherently you are doing the same work and taking the same actions.

    I’m not sure that this applies to teaching or learning. I’ll focus on teaching and use some simplifications.

    Teaching should occur within a specific learning design. Depending on what you want students to learn, the learning design will be different. The information that will be useful and the interventions/changes you might like to implement will be different depending on the learning design.

    e.g. what I’d like to know and do within a discussion forum that’s been configured for a ice-breaking activity is going to be very different from what I’d like to know and do within a discussion forum that’s been configured to host a debate.

    1. Yes. I agree that the metaphor can only take you so far and it demonstrates an example of bias (my bad). I have a background that included some time flying around the country side in a Cessna 182. Hence when I think of the instrumentation in a light aircraft, I’m thinking about artificial horizon which is great for combating vertigo, turn coordinator for nice even turns, magneto switch position, engine temp, voltage, VOR & DME gauge yada yada yada. So the metaphor works better for me with some understanding of how the light aircraft instrumentation is configured, than for someone else. I wonder if the “glass cockpit” concept is a better, although still flawed, metaphor. Its much more configurable than standard instrumentation. See example at https://commons.wikimedia.org/wiki/File:Cessna_Citation_Mustang_glass_cockpit.jpg

  2. Quick idea re: your last “thing to do”.

    Select a list of institutions or parts thereof.
    Gather a description of all the types of learning analytics projects they’ve done.
    Gather records of all the policies, documents, speeches etc that the people (perhaps the leaders) have written about learning and teaching and how to improve it, perhaps including learning analytics.
    Do a metaphor analysis on what they’ve written/said.
    Are there any patterns?
    Is anyone within Universities using metaphors that don’t reveal a SET mindset?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s