Self and Peer Assessment Tool

CQUniversity recently undertook a mapping project that evaluated the levels at which the university’s eight graduate attributes were represented in all undergraduate courses. This process was part of a systematic approach to embed its nominated generic skills and attributes into the curriculum, teaching and assessment practices of the institution (Fleming, Donovan, Beer, & Clark, 2010). With the mapping process almost completed, it has become apparent that the teamwork graduate attribute is the most under-represented graduate attribute across the curriculum.  As part of a strategy to address this deficit, the educational development team looked at ways that teamwork could be introduced into the university’s online learning spaces. Self and peer assessment was discussed as one approach that may contribute to the promotion of teamwork in the online curricula.

Extant literature espouses the virtues of self and peer assessment in higher education and in particular online education. It has been said that assessment should play a vital part in the learning process itself and the act of self assessment can be a force pushing students to engage more actively in their own learning (Roberts, 2006). Self and peer assessment can help students develop lifelong learning skills and can aid in students’ critical reflections of their own, and other’s work in parallel (RMIT, 2008). Given that CQUniversity has recently adopted Moodle as its single LMS, there existed limited technological means by which teachers could implement self and peer assessment for its online students.

One of the comments often made with CQUniversity’s LMS implementation project was the need to keep Moodle as ‘vanilla’ as possible to maximise maintainability and security. While the basis for this decision is arguable, it limited the opportunity for teaching staff to introduce functionality into their online learning that did not exist in the default Moodle system. Recognising this deficiency and the gap with the teamwork graduate attribute, the educational development team set about developing a small, simple web based system to assist the teaching staff in facilitating self and peer assessment.

CQUniversity’s Moodle LMS has a simple process whereby teaching staff can assign students to groups. These groups can then access resources and activities based on their group membership. For example, students posting to a forum that has been configured for group-work will only be able to see the posts and replies made by their team members. The process of assigning students to groups is quite simple and there is even a process whereby group allocation can be automated. The issue faced by the educational development team was that there was no opportunity within Moodle to leverage these groups to create a self and peer assessment process.

It was decided to develop version 1 of our self and peer assessment tool outside of Moodle and, by using access to a read only copy of the Moodle database, we could extract a range of information useful for the self and peer assessment web system (known as SPA). Information such as course and group membership, email addresses and names could be extracted from the Moodle database via an update script created for this purpose. AJAX drag and drop functionality was utilised where possible to maximise the usability of the system and a simple, clean user interface was the goal. Authentication is handled via standard LDAP libraries that interface with the university’s authentication systems. While we recognise that this self and peer assessment tool lacks the features of more mature systems designed specifically for self and peer assessment, it does integrate with our existing systems and makes it very easy for the teaching staff to implement self and peer assessment in their courses within the CQUniversity context. Our thinking is that by making it easier for teaching staff to assess group-work, they will be more likely to experiment with assessment strategies that fall outside their ‘comfort zone’. The following are some screenshots taken from the system.

The standard login screen where users are authenticated.

SPA Login Screen

The following screen is the main SPA page where the users are presented with a list of their courses taken from the Moodle system. They can add a self and peer assessment to their course by selecting the Add SPA link or review the status of existing self and peer assessments in the SPA status tab.

 

SPA Main Page

The next screen shows the SPA status screen where the teacher can monitor the number of student responses for existing self and peer assessments.

SPA status

On the SPA configuration screen there are three tabs; questions, groups and emailing. Questions are added to the self and peer assessment by dragging them to the right hand column.

Question selection

Groups from Moodle are selected in the same way. Note that on this screen it will show students who are not included in the selected groups and will therefore require the teacher to go to the Moodle site and fix up their group allocations.

Moodle group selection

Once the configuration has been completed and the students have finished their group task within Moodle. All the teacher has to do is to click send in the email tab and a link will be sent out to each student. The link is unique to each student and they will asked the selected questions about themselves and the other members of their groups. The following is an example of what the students will see with the names blanked out to protect the innocent. Note that there is no submit button as their selections are saved automatically. The students can revisit their selections at any time up until the due date expires.

Student Screen

This system is being piloted this term with a small group of students (31). We will expand the pilot next term to cater for all the interest we are getting in the system. Most of the credit for this system is attributable to Rolley for his awesome visual design skills and Damo for both his patience with teaching this old dog new programming tricks, and his ability to make complex code look simple.

Our todo list remains long and includes leveraging some of the better charting libraries to develop the reports that staff require. This is more complex than first thought as there are least two different interpretations of self and peer assessment among the two teaching staff assisting the trial. The first person wishes to use the system to identify discrepancies between  what the student self evaluated and how their peers evaluated them. The second is to give the students the raw feedback from their peers along with a grade based on the average responses. Both are applicable in their own context and the reporting functionality needs to reflect these requirements. Any thoughts or comments would be most welcome.

Fleming, J., Donovan, R., Beer, C., & Clark, D. (2010). A whole of university approach to embedding graduate attributes: A reflection. Paper presented at the DEHUB Education 2011 Summit. Retrieved from https://beerc.files.wordpress.com/2010/11/graduateattributespaper.pdf

RMIT. (2008). Self and Peer assessment. Retrieved from http://mams.rmit.edu.au/71ra0k9io8yzz.pdf

Roberts, T. S. (2006). Self, Peer and Group Assessment in E-Learning. Bundaberg, QLD, Australia: Information Science Publishing.

Advertisements

3 thoughts on “Self and Peer Assessment Tool”

  1. Col, good to see you blogging about the work you guys are doing.

    While the applications looks professional and almost certainly works well, I know the quality of the work you guys do.

    However, is it wrong of me to wonder if developing yet another self and peer assessment tool is the best use of the team’s skills?

    There are some good reasons for providing this sort of service (you include some of them in the post), but I wonder if you having to develop another such app is an indictment of the institutional context in which you have to operate.

    Of course, I thought much the same thing about the curriculum mapping project.

    BTW, doesn’t the workshop module in Moodle support peer assessment in someway?

    1. I think some of the phrasing in that comment could be taken the wrong way. Especially the second paragraph/sentence.

      To clarify.

      I know the guys who worked on this do good work. So, the app itself is almost certainly quite good.

      But I do wonder whether those talents are being wasted, or at best used inefficiently.

  2. I don’t necessarily disagree with your comments and some of these came up in our internal discussions. The trouble as we look at it is that teaching academics are typically time poor and this inhibits their ability to learn and try new things. I would suggest that this little system is an example of a ‘probe’ in terms of a complex system and an attempt to stimulate a little evolution in a small corner of learning and teaching.

    On the practical side a expenditure of effort for three of us can be made up in the time saved by the teaching staff at the other end.

    Col.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s