eAssessment Scotland 2014: Final Answer? The question of summative eAssessment

The eAssessment Scotland day conference has rapidly become a fixed event in the year for many practitioners from across the UK. Each year a rich mixture of stimulating keynotes and case studies, hosted by the University of Dundee, gives food for thought and ideas for approaches to the rapidly-changing world of eAssessment for Further and Higher education.

We were fortunate that several members of the team were able to attend the 2014 conference, and so this post comprises the key “take-aways” selected by Wilma, Mark and Stuart.

The opening keynote from Peter Reed of the University of Liverpool offered a refreshingly practical approach to summative assessment and what it means for today’s Universities. Peter emphasised the need to reflect on our assumptions and aims, when designing assessments, and made use of Brookfield’s four lenses when thinking about innovative practice, support for staff, and overcoming barriers to adoption. He insisted that all four lenses are of equal importance and relevance in this context, and all the stakeholders in the assessment process should be considered in any workflow redesign: students, tutors and administrators.

When considering drivers for change, perhaps we are guilty of over-weighting the student experience aspect (in reaction to NSS scores perhaps?) and possibly going for a “quick fix” – patching up an already dysfunctional system instead of seizing the initiative by redesigning courses and programmes to meet changed requirements and expectations, of both staff and students.

He also made an interesting point about our existing tools being outdated – the digital tools that we still struggle to get people to engage with may already be outdated and inappropriate for new approaches to assessment, e.g. authentic assessment approaches which often place students as producers. We may also end up driven by the wrong parts pf the system – e.g. the timing of assessments may be decided by administrators and systems rather than pedagogic requirements.

Mark Glynn from Dublin City University presented the afternoon keynote on using data from eAssessment. Like Peter Reed, he combined some theoretical explorations with demonstrations of three pilot projects. These are using learning analytics to investigate student activity and grades at a course (module) level and consider what the effects of class size may have on the robustness of findings. He also referred to the growing body of work on the ethical and legal issues around analytics applied to student data and especially the development of some guiding principles for Learning Analytics based on the work of, amongst others, Sharon Slade of the Open University.

The parallel sessions offered a wide variety of demonstrations and case studies, some perhaps more successful than others.

Martin Hawksey from ALT talked about Open Badges. The most interesting thing here was his answer to the question “do badges count?” … his answer was a very simple “yes” … if an individual chooses to display the badge then it counts. Is that a measure of success for a badge system? How many users take the bother to ‘surface’ the badge against how many receive it and forget about it?

Mark Glynn, standing in at the last minute for Jasmin Hodge’s workshop, did a useful ‘hands on’ intro to a variety of open and cloud tools that might be useful in a classroom context:

Open Badges for tracking achievements – which can even be time limited & require re-validation.
Kahoot for simple, online, real-time quizzes … “game-based blended learning” and “classroom response system”
Wambiz providing a private social network, built for education, free for classroom use.
Blikbook free with no statistics, providing an LTI connection, manages discussion and classroom interaction.

Steve Draper, Sarah Honeychurch and Niall Barr talked about work they are doing in the University of Glasgow to further develop a “two-way classroom”, using a Twitter feed during lectures to provide the student “back channel”. There was some useful reflection on the variety of social software available for activity of this sort, with an awareness that some are more “attention-grabbing” than others, e.g. Facebook is notorious for sucking people into “lost time”.

At the end of the day, many of the themes were picked up and skilfully woven back into the issues flagged up in the keynotes. My notes for this point are a series of questions rather than answers – How do we manage the new types of information available to us through digital media and learning analytics? How do we ensure the students benefit from this analysis? What are the respective responsibilities of the institution, staff and students? What are the models of teaching and learning underpinning predictive use of analytics and how do we apply this in an ethical fashion? Should we be doing this at all?

The day conference was supplemented by an online conference which ran sessions from 8-19^th September, in partnership with Transforming Assessment. The programme and recordings from this event are available from TA, along with many other worldwide webinars from the project.

Thanks are due to the event sponsors who also provided an interesting suppliers’ exhibition, and to the ever-resourceful eAssessment Association committee for the usual excellent organisation.