This post came about from conversations with folk at ASCILITE who are also trying to use captured LMS data to inform and improve teaching and learning.
Academic analytics is the process of analysing institutional data captured university enterprise systems for decision making and reporting purposes (Campbell & Oblinger, 2007). A simple example of this is the Indicators project that is using data from an enterprise LMS and an enterprise student records system to create information that can assist and benefit university teaching and learning. Note the term information! Not data! Data is the raw material for data processing and relates to events and transactions. Information is data that has been processed in such a way as to be meaningful to the person who receives it and universities are under increasing regulatory pressure to make better use of their data.
Higher education is entering into a new era of heightened scrutiny as governments, accrediting agencies, staff, students and parents call for new ways of monitoring and improving student success (Campbell & Oblinger, 2007). The Bradley review into higher education in Australia specifically mentions benchmarking activities in areas like student engagement as indicators of institutional quality (Commonwealth Government of Australia, 2008). At the same time assessment, accreditation, regulation and increased competition are all driving the need for more information and analysis.
The trouble is that while universities collect a great deal of data that can potentially assist in addressing problems such as retention and attrition, the data is not being adequately converted into meaningful information (Goldstein, 2005. Norris, Leonard, & Strategic Initiatives Inc, 2008). As a result, university and faculty leadership are in the unenviable position of having to make important decisions based on less than comprehensive information that could be provided by better utilization of the available data. This, I’d describe as a strategic problem but the same issues arises at the tactical level.
Teaching staff are often unaware of how the data collected by enterprise LMS can assist with their decisions at the course level. A simple example is the influence of course discussion forums and instructor presence on student engagement as provided by the Indicators project. So what is the problem and why isn’t the available data being converted to information that is usable by administration and teaching staff?
Unfortunately, and like David, I believe that the issue is organizational and relates to the ownership of the data. Teachers, faculty and IT all have some (arguably) legitimate claims to ownership of parts or all of the data. Note that we are talking about ownership of the data! Until the time that data becomes information, it is a waste of disk space and this is the key point.
From conversations with colleagues who work at other institutions, IT are seen to be the gatekeepers of data sources such as those required for academic analytics. There are arguably sound reasons for IT restricting and limiting access to institutional data and the main one is security. No one can deny that institutional data is valuable intellectual property and it should be well insulated from the likes of viruses and hacking but of what use is the data if it does not become information that can be used to the institution’s advantage? Additionally there is another problem arising from IT’s ownership of institutional data, and that is the skills required for data analysis.
Teaching and learning in a university environment is complex and diverse. Every student, teacher and course is different and the enterprise LMS has to cater for all. At the core of academic analytics is the LMS. It records every staff and student click within the system along with results and other useful information that, when combined with other enterprise information systems, can provide a university with a wealth of information that can be harnessed for competitive advantage (Dawson & McWilliam, 2008; Goldstein, 2005; Heathcoate & Dawson, 2005).
In order to extract meaning (information) from institutional data sources such as the LMS, it has to be interpreted against a backdrop of educational effectiveness and this requires analysis by folk with knowledge of teaching and learning. To do this requires a collaborative approach by both the folk who guard the data and the folk who can interpret the data or else the data remains a waste of disk space. Its a bit like hiring a security guard to guard your house against intruders but he won’t let anyone in,including you.
Campbell, J. P., & Oblinger, D. G. (2007). Academic Analytics. Educause Article.
Commonwealth Government of Australia. (2008). Review of Australian Higher Education o. Document Number)
Dawson, S., & McWilliam, E. (2008). Investigating the application of IT generated data as an indicator of learning and teaching performance: Queensland University of Technology and the University of British Columbia. (A. L. a. T. Council o. Document Number)
Goldstein, P. (2005). Academic analytics: The uses of management information and technology in Higher Education o. Document Number)
Heathcoate, L., & Dawson, S. (2005). Data Mining for Evaluation, Benchmarking and Reflective Practice in a LMS. E-Learn 2005: World conference on E-Learning in corporate, government, healthcare and higher education.
Norris, D. M., Leonard, J., & Strategic Initiatives Inc. (2008). What Every Campus Leader Needs to Know About Analytics o. Document Number)