There appears to be some debate around the efficacy of Purdue’s Signals Project and the effect it is having on student retention. You can see examples of this debate here, here and here. I guess I would classify myself as a believer in Purdue’s course signals work, but my belief does not stem from their prediction algorithm. I think the novelty in their work is in the way that they represent the data and provide, albeit basic, affordances for action. They calculate the chance of a student’s success and provide this information to the professors and students. I think this is a key and perhaps, overlooked, point.
Last year we argued that data represented by learning analytics arises from interactions between a whole bunch of complex systems. We suggested that correlations within learning data tend to be quite distinct at the macro levels while being far less distinct at the micro levels. For example, student LMS activity correlates with student grades across large numbers of students but becomes far less distinct when you drill down to individual units or students. Human behavior along with the plethora of factors that influence it, are incredibly diverse and complex. I know that my behavior in online courses can be extraordinarily unpredictable, depending on what is happening elsewhere in my life on any particular day.
I’m no statistician (actually if I died, the net statistical knowledge in the world would probably increase) but I think we have a tendency as a sector, to spend too much time over-analyzing data. I wonder if the value of Purdue Signals lies within the fact that it catalyzes some form of action whether by the student or professor? And, at least to some extent, the value of this action cannot be measured especially with something as enormously complex as student retention. It may just be me, but I fail to see how we can adequately measure the impact of a single factor on something as complex as retention with a zillion contributing factors. I’m sure my statistical friends will yell at me now but I think George Siemens summed it up nicely:
‘Who knows why people do what they do? The point is they do it, and we can track it and measure it with unprecedented fidelity’
(Siemens & Long, 2011)
1. Siemens G, Long P. Penetrating the Fog: Analytics in Learning and Education. Educause Review [Internet]. 2011 13/8/2012; 46(5):[9 p.]. Available from: http://www.educause.edu/ero/article/penetrating-fog-analytics-learning-and-education.