David’s recent post suggested that, while learning analytics has potential to contribute to data driven decision making in higher education, the ways in which it will most likely be implemented, will limit its effectiveness to say the least. As it happens, I read David’s post shortly after reading a paper by Jeffery Wayman entitled “Involving Teachers in Data-Driven Decision Making: Using Computer Data Systems to Support Teacher Inquiry and Reflection”. A number of things within this paper resonated with me and also with David’s post. While this paper was talking about K12 schools, there are some similarities with higher education.
Wayman (2005) describes schools as being ‘data rich’ and ‘information poor’. This is something that higher education is almost famous for. We collect an enormous amount of data from our various information systems but only rarely make any use of the data, much less make effective use of the data. Learning analytics proposes to change this but I think this is extraordinarily optimistic, given the way that universities are currently operating. Our ASCILITE paper from last year touches on some of the things that are causing my skepticism.
“The mere presence of data does not automatically imply that usable information is available; educators need support to use these data to the fullest extent” (Wayman, 2005). Most higher education institutions have business intelligence areas that maintain complex data warehouses that are capable of producing many interesting reports. Like David says, they do not yet appear to be directly helping teachers and students within the learning context and as such, are only retrospective and abstract representations of what has happened. To take a “glass half full” perspective, I hope that the current learning analytics fad may help move these systems and areas to more directly supporting teachers and students. The systems should help teachers become more effective practitioners (Wayman, 2005). Perhaps they should also help students be more effective learners?
One particular facet of the Wayman paper that I am interested in relates to teacher professional development. Wayman (2005) suggests that teacher-to-teacher interaction had a strong positive impact on teacher use of technology, whereas training provided by the organisation did not. I think there is a lesson here for folk like us trying to develop systems around learning analytics. The ASCILITE paper I am currently working on makes the point that universities need to spend less time and effort on centralized interpretation and analysis of data, and much more time on getting the data to the folk operating within the data’s context, and more importantly, providing support on sensemaking, reflection and action.
Wayman, J. C. (2005). Involving Teachers in Data-Driven Decision Making: Using Computer Data Systems to Support Teacher Inquiry and Reflection. Journal of Education for Students Placed at Risk, 10(3), 295-308.