What does LMS feature adoption tell us?

A learning management system (LMS) enables teaching staff to add resources or activities to an online course via a web interface with little or no knowledge of the underlying web (html) technology. Often the design and development of this course is the responsibility of the teaching staff and this is an important consideration when looking at the adoption of features within an LMS over time. If it is the teaching staff who are adding resources and activities to a course, then their experience with the teaching medium and their conception of teaching influences the features that will be used in a particular course. Additionally, Siemens (2004) says its only recently, and in limited ways, that LMS vendors have begun extending tools and offerings beyond simple content sequencing and discussion forums.

I realize that this is an over-generalization but let’s say that LMS tools and features tend to favour content dissemination type tools. The staff teaching via an LMS were themselves most often taught via a “lecture” or “sage on the stage” modality. Does this mean that pattern entrainment comes into play and they would tend to teach in a similar way that they were taught?

The following is a figure demonstrating feature adoption over time in one particular LMS. Whilst potentially another over-generalization, I’ve grouped features that support content dissemination and features that support communication to see if there is a trend.

Content and Communication Feature Adoption

LMS discussion forums

It has been said that for deeper learning to occur, learning must be social, active, contextual, engaging and student owned (Carmean & Haefner, 2002). Others have suggested that the learning process is transitioning from cognitive theories based on the individual to theories that stress the importance of the social nature of learning (Dawson, 2006) . Dawson et al (2006) states, numerous authors have espoused the importance for developing social learning opportunities where students actively debate, exchange and clarify ideas with other peers. Irrespective of learning theory, there is little doubt that social interaction plays an important part in learning and for the fully online student in a university course, the LMS discussion forum is the main enabler for social discourse and interaction during their course.

The following figures demonstrate the difference between on-campus and distance students in how they interact with the LMS.

What isn’t shown here is that when forum discussions are embedded into the course to meet specific course objectives as opposed to just providing a space for spontaneous student interaction, the proportion of the course that is forum activity increases.

Carmean, C., & Haefner, J. (2002). Transforming Course Management Systems into Effective Learning Environments [Electronic Version]. Educause, November/December 2002, from http://net.educause.edu/ir/library/pdf/ERM0261.pdf

Dawson, S. (2006). Online forum discussion interactions as an indicator of student community. Australian Journal of Educational Technology, 22(4), 495-510.

LMS content and communication

An quick extract from something I’m currently working on about LMS features that support content and LMS features that support communication. The following chart shows the evolution of these features over time in one particular LMS and indicates the dominance of content related features.

The following chart demonstrates the break down of distance student hits on these features.

This dominance of content features would appear to be a problem as it is contrary to research showing that effective learning occurs when students learn with computers rather than from computers (Oliver, 2005). Others have suggested that the learning process is transitioning from cognitive theories based on the individual to theories that stress the importance of the social nature of learning (Dawson, 2006) .

References.

Dawson, S. (2006). Online forum discussion interactions as an indicator of student community. Australian Journal of Educational Technology, 22(4), 495-510.

Oliver, R. (2005). Using blended learning approaches to enhance teaching and learning outcomes in higher education. Paper presented at the International Association of University Presidents’ Teaching Showcase, Joondalup, WA.

Long overdue blog update

Its been a while since I’ve updated this blog so I thought I’d jot down some thoughts while I had the time. Aside from work, life has been pretty hectic of late with a distinct lack of fishing occurring due to good weather coinciding with other engagements and various other non-fishing things happening in my world. My wife used the fact that I purchased another V8 Ford as an excuse to buy a kitten. My argument that we can always sell the car but not the cat didn’t seem to work.

Anyway onto the happenings at work.

The university where I work has recently appointed a new VC who quickly established an externally hosted blog where he is sharing his strategic vision and direction for CQUniversity. To me, this is a refreshing and welcome development in two respects:

• Firstly, by using an externally hosted blog provider such as Blogspot, tends to break a tradition of only using technologies and services that are provided by in house systems. In blogging terms, using an in house system restricts the potential audience and conversation participants to only authenticated users. Thereby, other stakeholders such as industry, government and even future students are excluded.
• Secondly, and more importantly, the blog is being used to air ideas and planning on strategic direction for CQUniversity in a way that is transparent and invites comment and conversation from all stakeholders.

In the latest post on his blog the VC writes about blended teaching and mobile learning which is something of interest to us as it has some links to our recent work on the Indicators project which is looking at data captured by learning management systems (LMS) and how organizations make sense of this data.

The following is a quick mind dump from my perspective that summarises the Indicators project.

Whilst LMS have some underlying flaws (Beer & Jones, 2008), most institutions are using an LMS of one form or another (Deakin University, 2002) and therefore further studies into how they are being used is warranted. When our unit was supporting the Blackboard LMS we realized that there was an enormous amount of captured data automatically stored by the system and we began ‘tinkering’ with the data to see what information could be extracted with a view to informing and improving teaching and learning at CQUniverisity.

We began to see some interesting patterns such as the one below, which indicates a correlation between hitcounts (indicative of engagement) and grades.

This distinct correlation got our attention and we set about looking at previous research that others have done in this area. There are many examples of researchers examining data captured by an LMS (Dawson, Winslett, & Burr, 2004; Deakin University, 2002),(Romero, Ventura, & ´, 2007),(Black, Dawson, & Priem, 2008),(Berg, Maijer, & Benneker, 2004) but in particular, Heathcoate (Heathcoate & Dawson, 2005) suggest that quantitative methods have tended to focus on usability and specific technical criteria while qualitative methods have been predominantly implemented to evaluate the achievement of stated learning outcomes.

One particular exception to this (Malikowski, Thompton, & Theis, 2007) presented an LMS independent model that looked at the features staff adopted over time and suggested that most LMS courses use an information dissemination mode with fewer moving on to use features that support higher order learning goals. We’ve done some comparisons with two LMS at CQUniversity and have submitted a paper on this, to a conference to be held later this year.

A gap in the research into LMS usage appears to be in cross institution and cross LMS comparisons. This is where the main body of our research is heading and it’s especially fortunate that CQUniversity currently has three LMS in operation as Moodle has been chosen to replace the other two. Potentially this enables the researchers to do some in house comparisons between the three LMS in order to paint a clearer picture of how the systems are being used and their effects on student outcomes. This information can not only be used by CQUniversity to inform T&L but can be used to assist other institutions in paralleling our research in their own institution and therefore gaining a broader perspective on how LMS are being used.

For more information on the Indicators project be sure to check out David’s and Ken’s respective blogs. I’ve also noticed that Rolley and Nona have had posted some blog updates recently with Rolley’s most recent post causing me to sympathize for his neighbors.

References.

Beer, C., & Jones, D. (2008). Learning Networks: Harnessing the power of online communities for discipline and lifelong learning. Paper presented at the 2008 Lifelong Learning Conference. from http://hdl.cqu.edu.au/10018/13162

Berg, A., Maijer, V., & Benneker, F. (2004). Blackboard 6 usage patterns and implications for the University of Amsterdam. Unversiteit of Amsterdam, Amsterdam.

Black, E. W., Dawson, K., & Priem, J. (2008, 17 March 2008). Data for free: Using LMS activity logs to measure community in online courses. Internet and Higher Education, 11, 65-70.

Dawson, S., Winslett, G., & Burr, L. (2004). Toward A Quantitative Analysis Of Online Communities. Paper presented at the Online Teaching and Learning Conference: Exploring integrated learning environments. Retrieved 25/1/2009, from http://eprints.qut.edu.au/2084/

Deakin University. (2002). Learning Management Systems at other Australian Universities. from http://www.deakin.edu.au/itl/research-eval/lms-evaluation/otherInstitutions.htm

Heathcoate, L., & Dawson, S. (2005). Data Mining for Evaluation, Benchmarking and Reflective Practice in a LMS. E-Learn 2005: World conference on E-Learning in corporate, government, healthcare and higher education.

Malikowski, S., Thompton, M., & Theis, J. (2007). A model for research into course management systems: bridging technology and learning theory. Journal of educational computing research, 36(2)(2007), 24.

Romero, C. b., Ventura, S. n., & ´, E. G. (2007). Data mining in course management systems: Moodle case study and tutorial. ScienceDirect Computers and Education(51 (2008)), 368-384.