Identifying ‘at risk’ students is the easy bit

We are currently trialling a simple system designed to help teaching academics identify students who may be at risk of failing. We are hoping to identify struggling students earlier than we previously could so as to more efficiently target support and academic interventions. Basically we are amalgamating data from the student information system (SIS) and the learning management system (LMS) which in our case, is Moodle. At my university we have a large array of academic disciplines and students can study via a number of modes other than online or distance learning. This makes predictions based on previous student results, student demographics and student patterns of online behavior from the LMS very difficult at best. Our thinking, based in complexity science, is the information needs to be directed to the point and time where it can best be used to influence the outcome. In our case, and for this particular trial, the point of need sits with the academic teacher.

The following is a ‘dummy’ screen grab from the system which is explained further down.

• Mail merge, simply allows the teaching academic to select multiple students in order to send a personalized email.
• Prior fail indicates whether or not this student has failed this particular course previously.
• Pass rate is the number of courses this student has passed out of the number they have attempted.
• Load is the number of courses that this student is attempting this term.
• GPA is a self explanatory.
• Clicking on the student name takes the teacher to another page with more details on the students academic history.
• Week. This is arranged by weeks of the term. (Moodle courses at my university are made available to students two weeks prior to the official term start). The numbers in these columns are simply the number of clicks that these students have made within the course site during that week.

As the title of this post suggests, this is the easy part. Once a student has been identified by the teaching academic as potentially being at risk of failing, then what? The reasons that students fall into the ‘at risk’ category are as extraordinarily diverse as our students. Some may be struggling academically, some financially or personally, and some (like me) are just struggling for time. Every student’s situation is different and a one-size-fits-all approach to intervention is not going to work. This is why I continue to be fascinated by the extraordinary effort universities are putting into refining their ‘at risk’ student identification algorithms. Accurate statistical models are all well and good, but  all rather pointless without an intervention strategy that is capable of dealing with diversity and complexity.

The decline of the compliant learner

I’m reading an interesting paper by Peter Goodyear titled “Environments for lifelong learning: ergonomics, architecture and the practice of educational technology“. It talks about the changing face of instructional design in the Web 2.0 era. I’ve cherry picked some quotes that I found relevant.

• “it is becoming increasingly difficult to imagine that we can design learning technology systems around an assumption that they will be used by learners in the ways we prescribe.”
• ” a shift from the creation of ‘tasks-in-objects’ to ‘environments-for-activities’.”
• “Evaluative data seems to be indicating that, however good the learning software, students will rarely use it unless they are obliged to do so.”
• “It is now part of the folklore of information systems design and software engineering that technology needs to be built around a well founded understanding of what people actually do. Just listening to the managers’ account of how work should be done is a recipe for disaster and leads to unused and unusable software systems. Why should higher education be any different?”

Another point he makes and one that I’ve thought about with regard to PLEs relates to learning communities and has been proven by the present day LMSs. A learning community grows of it’s own accord and any attempt to force the growth will lead to a situation that we see with our own LMS which is poor student participation.

Personal Learning Environments.

== What are Personal Learning Environments? ==

Introduction.
Metaphorically, it can be said that each individual human being is a self contained virtual world. The way I perceive life and the happenings in it are as specific to me as DNA and finger prints. This could be interpreted to say that the optimal method for me to learn is also somewhat unique and probably won’t match the next person. With developments in technologies and an understanding that traditional teaching doesn’t transpose to teaching online effectively with current technologies, teachers are seeking better mechanisms for disseminating knowledge to the student in a more focused and efficient manner. A pedagogically efficient way to teach individuals online matched with individual learning styles is the Holy Grail of online learning. One of the stepping stones to this goal could be the Personal Learning Environment.

The concept of E-Learning has been around for more than ten years now and moved from a radical idea shunned by the traditional higher education institutions to becoming an integral part of their teaching and learning with close to universal participation. A revolution of similar proportions is happening now within this online realm as the large scale adoption and acceptance of social networking and consumer created content, known as Web 2.0, takes hold (Chambers 2007[12]). If Web 1.0 can be described as the ‘go get it’ web then Web 2.0 can be called the ‘come to me’ web with content being created and manipulated by the average internet user to suit their own individual requirements and delivered to them rather than them having to seek it out.

So far the effect of Web 2.0 on universities and the systems universities use to deliver content has been small but the rate of adoption and acceptance of Web 2.0 concepts by the wider community is unprecedented and can no longer be ignored. Universities, and in particular XXX, now have an opportunity to capitalize on Web 2.0 popularity as well as engaging with a much larger audience than was traditionally possible by using a Web 2.0 concept called Personal Learning Environments or PLEs. This paper will give the reader a brief introduction to Personal Learning Environments and compare some of the philosophical differences between PLEs and Learning Management Systems as well as listing some of the problems that each system possesses. The technical and administrative aspects of supporting Personal Learning Environments as well as the cultural changes required for their inception in the university environment are only lightly touched on in this document but are important enough to warrant serious consideration in their own right.

Before elaborating on Person Learning Environments we need to understand how self directed learning works and what advantages does it offer. According to Heimstra (Heimstra. R. 1994 [10]) self directed learning utilizes the learner’s willingness and ability to take control of their learning. Heimstra stresses the following points about self directed learning.
* Empowers learners to take responsibility of decisions associated with the learning Endeavor.
* Self-direction doesn’t mean isolation.
* Knowledge and skills are transportable to other learning situations.
* Teachers can take an effective role in a self directed learning environment.
Removing some of the limiting factors in education like fixed terms and course boundaries, empowers the learner to learn what they want to learn and when they want to learn it, but only recently with the potentials of Web 2.0 that the concept can be taken to the world of online learning.

A recent survey by the Collaboratory for Advanced Research and Academic Technologies (CARAT 2007 [11]) from the University of Michigan indicates that self directed learning , using exploratory and experimental techniques might be the preferred way for students to learn. They surveyed 1700 students and 391 faculty members and produced the following graph about their preference for learning technologies.

[[Image:CARAT.jpg | 800px]]

(2007 The Millennial Instructor.[11])

What are Personal Learning Environments?
A Personal Learning Environment can be a difficult thing to define as it is more a concept than a quantifiable system like a Learning Management System or LMS. The definition I like is by Ron Lubensky, who is a deliberative process designer and online learning consultant working at the University of Sydney. He says ‘a Personal Learning Environment is a facility for an individual to access, aggregate, configure and manipulate digital artifacts of their ongoing learning experiences” (Lubensky 2006 [1]). A PLE could empower the learner to manage and control their learning throughout their learning lifetime whether the learning is formal or informal. It could be a desktop application or a collection of web services that interconnect to serve the needs of the learner, but the key point is its learner centric.

In theory, Personal Learning Environments last the lifetime of the learner and allow the learner to access a single location that aggregates artifacts of their learning past, present and future. An individual’s PLE is fluid and will not be the same in the future as what it is now as their life circumstances dictate. Technologies will change but more importantly the learner’s requirements and expectations of the PLE will change over time as the learner’s life situations dictate.

Following is a quote from Sebastian Fiedler [9] that describes the underlying philosophy behind Personal Learning Environments.
“I treat personal learning environments more as a psychological perspective. What forms my ‘personal learning environment’ at a given point in time, and for a particular purpose or goal (that drives a learning project), is largely determined by the range of resources that I am able to perceive, locate, link to, access, manage, and so forth.” (Fiedler. S. 2006 [9]).
The following diagram from Jeremy Heibert is an excellent graphical representation of the time and process flow behind a PLE.
[[Image:PLE_Process.jpg | 800px]]

(Heibert. J. 2006 [8])
In a University environment, it’s the decoupling from the teaching institutions that, on a conceptual level, set PLEs apart from other E-earning technologies such as Learning Management Systems or LMS. A PLE is centered on the learner and is tailored to suit the requirements of the learner as opposed to Learning Management Systems that are typically designed to meet the administrative requirements of the institutions.
Learning Management Systems are part of the organizational infrastructure that facilitates learning and teaching and as a consequence, pedagogical practices within the organization are driven by the course based LMS which can be to the detriment of the learner.

Developments in educational technology and in particular Web 2.0 have lead to the point where a PLE could be viable and provide a more effective learning environment for the both the student and the institution in the realm of online learning than the traditional stand-alone LMS. Prior to Web 2.0 a few content authors provided content for an audience of passive listeners. Contrastingly with Web 2.0 the users can create, repurpose and consume shared content to suit there own requirements and the rate of adoption of Web 2.0 applications is astounding. According to the Blog search engine Technorati at the time of writing, they are tracking more than 100 Million Blogs and over 250 Million pieces of tagged social content (Technorati 2007 [3]). It’s this shift in online technology where the consumer has the ability to create and repurpose content to suit their individual needs, that has lead to the concept of the Personal Learning Environment and I believe there is an opportunity for higher education institutions to engage with a much larger audience than was previously possible under institution specific Learning Management Systems.

Show me a PLE!
So far I have been talking at the conceptual level of Personal Learning Environments. How do they work in practice? A very simplistic example is one I’m using at the moment to learn the guitar. I’m not using a single resource but a whole range of different online resources to aid my learning. Rather than visit all of these different sites and check for updates I’ve centralized all of the web resources I commonly use into a central location which is on a Wiki.
http://cddu.cqu.edu.au/index.php/PLE_simple_example

As you can see the four YouTube videos at the top are songs I’m currently learning or wanting to learn. Underneath that are 3 columns which are RSS feeds taken from web sites I regularly visit and often contain content I’m interested in. The bottom item is a feed from my guitar Blog which I use as a scratch pad to jog my memory about whatever it is I’m trying to learn at the time.

This example over simplifies the concept of a PLE and the layout and content won’t appeal to everyone but the point is it works for me and only contains the content I want it to contain. It also doesn’t contain much in the way of interaction with other people as music tastes and solo guitar playing are highly individual and don’t lend themselves to constructive discussion easily. A better example which I intend to create when time permits is to have separate pages for every song or method I’m learning so I have an archive of content I’ve learned over the years which in time builds to a useful database of learning material specific to my requirements. Expand this further and I can add other categories to the Wiki depending on what I’m learning at the time. For example a category on the Wiki devoted to EDED21007 Flexible Learning, where the contents from the Blackboard course automatically feed in along with all of the associated resources collected and tagged around the internet.

A far more utopian example would be where universities have modularized learning outcomes, learning objectives and assessment for every subject area and the PLE learner can mix and match their learning objects based on what they want to learn. The courses as we know them now might be divided up into smaller units each with a learning outcome based on learning objectives and assessment. For example a Math 101 course now consists of Addition, Subtraction and quadratic equations with a single end of term assessment piece. This could be divided into the three sections with each section individually assessable so the adult learner who needs to demonstrate knowledge of quadratic equations for some career objective can cherry pick the learning object to suit their individual requirements. However the addition, subtraction and quadratic course content doesn’t necessarily have to come from a single educational institution. Instead of the university delivering a specific product, in this case Math 101 it could follow the strategy of Amazon™ who have moved from being a retailer to a service provider. You can still visit Amazon to buy a book but the books are supplied by other retailers with Amazon providing the portal into the book retail world. This would mean in our example you may choose to do addition and subtraction at the University of Queensland and the quadratics at CQU. Add to this the learning objectives and learning outcomes are open to the public and can be studied by anyone who chooses to do so. One of the obvious problems with this is how a university generates income with such a structure and to some degree the smaller regional universities will be disadvantaged by their inherently smaller budgets.

How does online learning work with Personal Learning Environments?
Don’t think of a Personal Learning Environment as an object or software package that has hard boundaries. What we are really talking about when discussing PLE are Web 2.0 concepts and how online learning is considered with Web 2.0 in mind. According to Stephen Downes there are 3 principles associated with online learning with particular reference to Web 2.0 and they are.
* Interaction.
* Usability.
* Relevance.
(Downes. S. 2006 [5])
Interaction is communicating with other people via available online tools. These people may be other students studying the same or similar topics or even experts in the field. Most of current online learning is based on the broadcast model often referred to as ‘Sage on the Stage’ method of teaching which is asymmetrical. This means the conversation is predominately one way from the teacher to the learner cohort with little or no symmetrical interaction between the involved parties. True interaction is the ability to communicate with other people interested in the same topic or using the same online resources or information so knowledge can be shared among the group. To do this we must place ourselves, not the content in the centre of our learning, by utilizing the Web 2.0 tools that are available like email, mailing lists, online news, Blogs, instant messaging and Skype etc in order to build a community of people interested in a common learning goal.

Usability is consistency and simplicity in the way we interact with the technologies available to support our learning. Prime examples are Google and Yahoo whose web interfaces are often consistent across their various applications while being simple to use. With a PLE you need to match the methods and technologies that best assist your learning. For example, you learn best via audio you can seek out Podcasts and MP3s to support your learning of the particular topic and feed them into you the applicable are of your PLE. Summarize the information you find in a consistent manner into a content management system like a Wiki using your own language for your own understanding remembering that usability is personal to the learner.

Relevance is getting what you want when you want it and where you want it. By utilizing tools like RSS you can feed the information you need into the mechanism that best suits you requirements. For example, if you are comfortable with Wikis you can channel your RSS feeds into the Wiki in an area that suits your preference. The best time for learning is highly personal and unique to the individual so you choose the time and place of you learning by having the relevant information available for when you need it. Information discrimination is a very important part of online learning to ensure the information being received is relevant and timely.

PLEs address these three principles by utilizing available technologies to create an area that the learner can configure, develop and interact with according to their requirements. A PLE is specific to the learner which means the learner can set their own goals and manage the content and delivery processes of their learning as well engage in social environments that link them to other learners in similar situations. Rather than providing a set of tools in a specific context a PLE seeks to maintain connections between the users and a wide range of services as required by the learner in their context as indicated in the diagram following.

[[Image:Map.jpg | 800px]]

([2]. Wilson, Scott. Liber, Oleg. Beauvoir, Phillip. MIlligan, Colin. Johnson, Mark. Sharples, Paul. (2006[2])

But what’s wrong with Learning Management Systems? Why change?
Many of the problems with LMS systems are typical of Web 1.0 and traditional IT dominant design methodologies in general. To quote Walter S. Mossberg[7] from the Wall Street Journal,
‘Information technology departments are the most regressive and poisonous force in technology today. They make decisions based on keeping technology centralized. Although lesser known software may be better, technology departments are likely to use big name products for their convenience. That may keep costs down for an organization but it puts consistency above customization, preventing individuals from exploring what technology products are best suited to their own needs’
To change this institutional culture to the point where technology departments accept even the possibility of a PLE is going to take time and will probably be driven by the end users rather than the institution. The concept of a PLE directly contrasts with the current IT governance protocols of control and centralize.

LMS are course based and institution centric and therefore they have a limited ability to deliver practice-based learning and usually don’t support the learner beyond their institutional enrollments. Some implementations of LMS, in particular some of the open source LMS like Moodle and Sakai, have tended to be more dynamic in that they allow customization and open APIs but the same underlying design flaws affect not only the pedagogical effectiveness of these systems but tend to stifle teaching innovations due to the restrictive environment provided by the LMS.

LMS are typical of the dominant design concept as defined by Abernathy and Utterback 1979 which states that innovative activity is directed to improving the process by which the dominant design is delivered rather than exploring alternatives. In a paper titled ‘Person Learning Environments: Challenging the dominant design of education systems” Wilson, Liber, Beauvoir, Sharples and Milligan 2006 [2] list some of the characteristics of an LMS that relate to the Dominant Design Theory. They include:
* The design of an LMS follows a particular design model into which tools and content are organized into courses or modules. Typically data can’t be shared outside this course/module enclosure.
* These course based tools for organization and creation are often richer for the teacher than the learner which means the users are relegated to a passive role in which they are often told to be creative and innovative. This relationship can be described as Asymmetric.
* In this course/module based system all learners see the same content organized in the same way using the same tools which ensures a common experience of learners within this context whereas under lifelong learning the experiences is required to be individualized and tailored to personal needs and priorities.
* Standards and specifications have developed to assist in integrating LMS and other systems like Peoplesoft whereas other specifications have achieved widespread usage without impacting the LMS. RSS is an example of this.
* Typically the LMS restricts access to the content and conversation to the enrolled learners often to secure copyrighted content from publishers. This content is not available to the student after the course has finished which counters the concepts of lifelong learning.
* Organizational scope. The LMS models are typically installed and configured to meet the requirements of the installing institution which makes it difficult to engage external organizations and unregistered learners.

Personal Learning Environments have associated problems to consider.
So far we have looked at the inherit problems with Learning Management Systems and how Personal Learning Environments can overcome some of these problems by empowering the learner to control their learning environment but what potential issues to PLEs bring to the table?
* LMS interoperability. Initially for a PLE to be successful the various LMS the learner is coordinating with must be able to interoperate with the PLE. This requires secure gateways between the LMS and the PLE which will need to be constructed or worked around specific to the LMS involved.
* Intellectual property. Some of the content within the LMS is copyrighted material for a specific student cohort and this must be password protected under the agreements with the copyright owners. Do these agreements stretch to the mix and mash world of the PLE?
* Identity. How is the individual’s online identity established across the many different systems they encounter in a lifetime of learning? The expected solution will be an open ID of some sort which is similar to your Windows login name and password except instead of establishing your identity to the local network it can establish your identity across the internet’s many participating applications. This too will be a problem as the key word in the previous sentence is ‘participating’.
* Plagiarism. With one of the key benefits of a PLE being the ability to mix, match and rehash other sources of information how is plagiarism going to be addressed? How do we assess the learner in such an environment?
* Maintainability. Overtime the quantity and value of the information held in the PLE will increase and mechanisms need to exist for the learner to transport this data for safekeeping.
* Migration issues. A university isn’t going to migrate from LMS to PLE overnight and there is a plethora of issues surrounding the changes required both in terms of technical and user behaviors.
* Mindsets. Traditional IT governance structures are very rigid in their thinking which in turn stifles innovations and experimentation with the systems implemented by these structures. A PLE will be seen as the enemy by these communities as their ability to control is eroded and the old adage of ‘academic privilege should apply’ probably won’t. The teacher will use their own individualized variant of the PLE to teach from which will also require some radicalization of thinking on their behalf as they move from the tools they traditionally use.
* Offline Learning. Typically the PLE will be an online application hence won’t be available while the student is offline or mobile.
* Restrictions on free internet applications. What becomes of the PLE if changes are made to major component applications by the companies that run them? For example if the learner is using a Wiki and the Wiki owners decide to make the service a pay for service based operation.
* For a PLE to be easy to use by the average learner, Web 2.0 developers will have to agree some sort of common API to allow other applications to mix and mash the data they are managing.
* The problem of cost and quality of internet services is common across all online learning.

As you can see Personal Learning Environments, from an institutional standpoint are much less attractive than the LMS because of the nature and the seriousness of the issues they raise should their adoption be considered. However it’s possible that the drive to implement a PLE will come from the end users voting with their feet by choosing the university that best matches their individual requirements by offering methods and practices that are most relevant to their requirements.

Summary.
The Internet and more recently Web 2.0 have changed the notion of knowledge. Knowledge, and in particular the technology of knowledge dissemination, is changing faster than traditional institutions can adapt to meet the affordances required. The personal learning environment is an evolutionary step from the Learning Management System, which has typically tried to mimic the classroom environment to a model where the learner is the hub of their learning ambitions and controls every aspect of their learning.

The changes required in universities to adapt to a concept like a PLE will be enormously difficult if not impossible considering the inertia to change that exists in higher education governance structures today. Fundamentally, adopting the PLE philosophy will require universities to transition from product delivery organizations to organizations that provide a service and it’s difficult to see the changes extending to the unbundling of educational offerings. However, this will be a change that will be driven by the consumer culture and not the universities themselves. It’s even arguable that given the rate of adoption of Web 2.0 in the online community that universities have been quite slow to react and capitalize on a potentially large market in the general online community. How often have we seen the 4 Cs in technology adoption? That’s where a technology evolves from cute to convenient to compelling to compulsory. Do you remember when spreadsheets followed this evolution? Whatever the online technology used to deliver education will be in the future, there is one certainty. Universities will experience a Darwinian process of adapt or die. Will PLEs be part of the future? We can only find out if we experiment, reflect on the results and experiment some more.

References

1. Lubensky, Ron. (2006), eLearning & Deliberative Moments, http://members.optusnet.com.au/rlubensky/2006/12/present-and-future-of-personal-learning.html
2. Wilson, Scott. Liber, Oleg. Beauvoir, Phillip. MIlligan, Colin. Johnson, Mark. Sharples, Paul. (2006), Personal Learning Environments: Challenging the dominant design of educational systems, http://hdl.handle.net/1820/727
3. Technorati (2007), http://technorati.com/about
4. Scott Wilson © 2004-2007, http://www.cetis.ac.uk/members/scott/blogview?entry=20050125170206
5. Downes, Stephen. http://www.downes.ca
6. Jones, David (2007). http://cq-pan.cqu.edu.au/david-jones/Publications/Presentations/TeleologicalReason/
7. Walter Mossberg, “The Most Poisonous Force in Technology « CIOh-no,” The most poisonous force in technology., http://pmasson.wordpress.com/2007/07/05/the-most-poisonous-force-in-technology/ (accessed September 16, 2007).
8. http://headspacej.blogspot.com/search?q=ple. Jeremy Heibert (2006).
9. Sebastian Fiedler, “Seblogging :: :: by Sebastian Fiedler,” http://seblogging.cognitivearchitects.com/2006/03/01#a1724 (accessed September 17, 2007).
10. Hiemstra, R. (1994). Self-directed learning. In T. Husen & T. N. Postlethwaite (Eds.), The International Encyclopedia of Education (second edition), Oxford: Pergamon Press
11. CATS 2007 Millennial Instructor. 2007 CARAT @ University of Michigan.
12. John Chambers. Cisco CEO (2007) http://networks.silicon.com/webwatch/0,39024667,39167942,00.htm

[[Category:PLE]]