This post provides a very succinct summary of my PhD for those strange folk out there who might be interested 🙂
Universities are struggling to develop the capabilities required to implement meaningful learning analytics (Colvin et al., 2015). This struggle is linked with the approach that universities take with learning analytics implementation whereby detailed planning is designed to achieve a predetermined and idealistic future state (D. T. Jones & Clark, 2014). The dominant managerial and bureaucratic approach that universities apply to technology adoption manifests in top-down centralised approaches that are deemed to be organisationally efficient (D. T. Jones & Clark, 2014). However, learning is a complex social activity that is situated in complex and diverse social environments (Macfadyen & Dawson, 2012). Consequently, learning analytics is a multifaceted construct with many interdependent and contributing variables, and is highly dependent on specific contextual variables (Clow, 2014). The misalignment between the complex nature of learning analytics implementation, universities as complex socio-technical systems, and the strategic operations of universities (Colvin, Dawson, Wade, & Gasevic, 2017) represents a challenge for universities trying to develop the capabilities for meaningful learning analytics implementation.
Other complex socio-technical systems, most notably healthcare, have applied an alternative ontological conceptualisation based on complex adaptive systems theory in an effort to move beyond hierarchical and mechanical models (Boustani et al., 2010; Plsek & Greenhalgh, 2001). This theory describes systems that are non-causal, non-linear and are comprised of many interacting and interdependent agents (Holland, 2006, 2014). The application of an alternative ontological conceptualisation to learning analytics shifts the epistemological approach from planning and strategy, to an approach to implementation based on learning and improvisation (Juarrero, 1999; Kurtz & Snowden, 2003). While an alternative conceptualisation of learning analytics may assist with implementation and uptake, it also needs to fit within current strategic and hierarchical operating norms of universities. The orthodox approach to technology related implementations is to apply deliberate strategy and detailed planning (Kezar, 2001) (Reid, 2009), which raises a question about how can we apply a complex adaptive systems lens in this environment.
The embryonic nature of learning analytics and the complexity of issues that influence its systemic uptake go some way to explaining the paucity in large-scale implementations and highlights a need for further empirical and methodological studies (Colvin et al., 2017). Considering learning analytics as either top-down or bottom-up is unlikely to lead to meaningful learning analytics implementation. Top-down approaches are unlikely to meet the needs of specific learning and teaching contexts while bottom-up approaches are unlikely to scale across multiple learning and teaching contexts. Considering learning analytics as either top-down or bottom-up also risks overlooking a key area of translation in between that is often overlooked (Hannon, 2013). Much of the work of implementation occurs between the top-down and bottom-up, at the meso-level. This is the level within the organisation that sits between the small scale, local interactions and the large-scale policy and institutional process (C. Jones, Dirckinck‐Holmfeld, & Lindström, 2006). The meso-level practitioners assuage the tension between the upper and lower levels (Uhl-Bien, Marion, & McKelvey, 2007) and can negotiate a balance between the contextual complexity of learning analytics and the conventional centralised approach to data services common to universities.
This project aims to produce design principles derived from complex adaptive systems theory to guide the contribution of meso-level practitioners in order to help universities address the challenges of institutional learning analytics implementation. The principles aim to improve and enhance the integration between tools, actionable data and educator practices in real-world settings. The principles will be iteratively tested through a cycle of design-based research at a regional Australian university with the broad goal of improving student outcomes. The study aims to answer the following research question:
How can a complex adaptive systems conceptualisation help meso-level practitioners enhance and transform university learning analytics implementation capability?
Boustani, M. A., Munger, S., Gulati, R., Vogel, M., Beck, R. A., & Callahan, C. M. (2010). Selecting a change and evaluating its impact on the performance of a complex adaptive health care delivery system. Clinical Interventions In Aging, 5, 141-148.
Clow, D. (2014). Data wranglers: Human interpreters to help close the feedback loop. Paper presented at the Proceedings of the fourth international conference on learning analytics and knowledge, Indianapolis, IN, USA.
Colvin, C., Dawson, S., Wade, A., & Gasevic, D. (2017). Addressing the Challenges of Institutional Adoption. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (Vol. 1, pp. 281 – 289). Australia: Society for Learning Analytics Research.
Colvin, C., Rogers, T., Wade, A., Dawson, S., Gašević, D., Buckingham Shum, S., . . . Corrin, L. (2015). Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement. Canberra, ACT: Australian Government Office for Learning and Teaching.
Hannon, J. (2013). Incommensurate practices: Sociomaterial entanglements of learning technology implementation. Journal of Computer Assisted Learning, 29, 168-178.
Holland, J. H. (2006). Studying complex adaptive systems. Journal of Systems Science and Complexity, 19, 1-8. doi:10.1007/s11424-006-0001-z
Holland, J. H. (2014). Complexity: A very short introduction. Oxford, England: Oxford University Press.
Jones, C., Dirckinck‐Holmfeld, L., & Lindström, B. (2006). A relational, indirect, meso-level approach to CSCL design in the next decade. International Journal of Computer-Supported Collaborative Learning, 1(1), 35-56.
Jones, D. T., & Clark, D. (2014). Breaking Bad to Bridge the Reality / Rhetoric Chasm. Paper presented at the ASCILITE2014 Rhetoric and Reality, Dunedin, New Zealand. Conference publication retrieved from http://ascilite2014.otago.ac.nz/
Juarrero, A. (1999). Dynamics in action: Intentional behavior as a complex system. Cambridge, Massachusetts: MIT press.
Kezar, A. (2001). Understanding and facilitating organizational change in the 21st century. ASHE-ERIC higher education report, 28(4), 147.
Kurtz, C. F., & Snowden, D. J. (2003). The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal, 42, 462-483. doi:10.1147/sj.423.0462
Macfadyen, L. P., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan. Journal of Educational Technology & Society, 15(3), 149-163.
Plsek, P. E., & Greenhalgh, T. (2001). Complexity science: The challenge of complexity in health care. BMJ: British Medical Journal, 323, 625-628.
Reid, I. C. (2009). The contradictory managerialism of university quality assurance. Journal of Education Policy, 24(5), 575-593. doi:10.1080/02680930903131242
Uhl-Bien, M., Marion, R., & McKelvey, B. (2007). Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era. The Leadership Quarterly, 18, 298-318. doi:10.1016/j.leaqua.2007.04.002