Promoting innovative practices in assessment: Use of Media Hopper for video assignments

As the new EDE advisor for assessment and feedback, I would like share my experience using Media Hopper (Kaltura) to enhance assessment and feedback at the University of Dundee.

We are living in the Digital Age, yet the methods that we use for assessing our students and providing feedback are, essentially, the same ones used a century ago. The Australian educator David Boud said once that, if students can escape, with difficulty, from the effects of poor teaching, they cannot escape the effects of poor assessment. Assessment design should be aimed at fostering learners’ creativity and gaining a deeper understanding of the academic subject. If we want to promote deep learning, we need to embed innovation in our methods of assessment, which are currently more focused on awarding a grade than on fostering creativity. Technology can help us to foster this innovation by transforming traditional written essays into multimedia assignments, which provide students with the digital and soft skills that the Information Society demands.

At the University of Edinburgh, innovation in assessment and feedback is led by Susan Rhind, Assistant Principal for Assessment and feedback, whose aim is to introduce the use of Media Hopper as a vehicle for establishing a feedback dialogue between tutors and students. Susan has done some research using audio feedback in a veterinary curriculum to improve students’ satisfaction.

Kaltura, the software behind Media Hopper, is a video management platform that can be used to develop video assignments as well as to provide video and audio feedback. At the Medical School of the University of Dundee we pioneered the use of Kaltura as part of the assessment strategy of our postgraduate programmes. The video assignments were used in clinical skills modules to assess the interview and communication skills of our students in workplace scenarios. Students recorded themselves interviewing real patients and prescribing treatment and submitted their video using the Kaltura assignment link in the School’s VLE. Students used their smartphones to record their interviews.

Tutors were able to provide video and audio feedback using the same Kaltura assignment link and via the Kaltura Media Gallery. Although we used Moodle as the School’s VLE, the process can also be easily implemented in Blackboard Learn.

Benefits of using video assessment

Video assessment can be used in a variety of academic disciplines, not only Medicine: Law, Languages, Multimedia, Engineering, etc. It can be used to assess generic skills, such as communications, leadership, problem-solving, team work, etc., as well as subject-based specific knowledge. There is enough empirical evidence to support the benefits of using video (and audio) in assessment and feedback, but I will not quote references here, because I want to mention the benefits that I saw from my own experience:

Video assessment improves student engagement by reaching those students who may feel alienated or disenfranchised when using traditional assignment methods. In education, emphasis has been put on producing diverse learning materials that can suit different learning styles, yet the majority of the assessment methods only support a verbal-linguistic learning style.

Formative assignments based on video submissions can increase student participation. Some authors consider that formative feedback is better internalised by students than summative feedback because it does not include the judgement and “punishment” of summative assignment (Norcini and Burch 2007) (Epstein (2007). Yet some students miss this opportunity of receiving this type of feedback because they see formative assignments as time-consuming and non-compulsory. Formative video assignments introduce an element of novelty and creativity that spur students’ curiosity contributing, therefore, to increase the submission rates. As an evidence of this, I can mention one of the modules of our MSc in Diabetes Care, Education and Management in which we replaced the traditional formative draft (students had to submit the first draft of their work-based project) by a video assignment in which students were asked to submit a 5 minutes video relating to their work-based projects.

For distance learning and blended learning courses we discovered that replacing traditional ice-breaking activities with video submissions improved students’ online engagement. For example, most of our modules included a “Saying Hello” forum hosted during the induction week. Replacing that forum with an “Introductory selfie” activity hosted in Kaltura increased students’ online participation for the rest of the activities during the induction week.

Those tutors who used video feedback (using Kaltura or Voicethread, another web platform for supporting video assessment and feedback) were able to establish a more personalized relationship with their students. Research suggests that students see video and audio feedback as more “individualised” and “easier to comprehend and act upon” (West and Turner 2015).

Video assessment and feedback helped engage students in collaborative working and peer assessment. In postgraduate part time programmes, an obstacle to overcome is how to engage students in team work and peer assessment when both activities do not contribute to their final marks. Likewise, many peer assessment activities do not have the expected student turnouts as students find the peer-marking software confusing or obsolete-looking. The Kaltura Media Gallery, integrated into the online activities supported by Blackboard Learn (e.g. forum) can be used as a peer assessment tool that students find easy to use and in which they can provide video feedback to their peers.

What are the cons of using video assessment?

In order to develop the video assignment activities at the University of Dundee, we also had to overcome some challenges, most of them caused by changing the assessment paradigm rather than the technological implementation:

Time allocation: Video assignment requires more time in comparison with a traditional written assignment. Tutors and students need to familiarise themselves with the video management software and the way in which the software is integrated with the VLE. Students need to know how to record and submit their videos and tutors need to know how to review the videos and provide feedback.

It is important that tutors consider the amount of time that students will spend preparing the video assignment to allocate enough time during their teaching week. Likewise, it is important that the workload model for academics also recognises the time spent taking part in innovative teaching practices, such as video assignment.

Logistics and technical support: Some tutors panic that creating a video assignment activity may end in sorting the logistics of providing video cameras to students or liaising with complex multimedia queries. Other tutors assume that every student has a smartphone and know how to record a video from the smartphone and submit the video using the Kaltura Native Mobile app.

Both assumptions are dangerous and the reality is somewhere half way: supporting a video assignment activity is not as time consuming as it looks like, and some students may need to book recording equipment and may need additional help submitting their videos to Kaltura. It is important to count with video or FLIP cameras that students can borrow as well as to have technical support for the activities. At the University of Edinburgh, this support is provided locally in some Schools or via the IS.

How to grade a video? Some tutors may find difficult to mark and provide feedback for a video, in particular for summative assignment.

When using video assignments, tutors should be upfront with the students about the marking process, since video assignment is also a new medium for most of the students. I always advise to attach a marking grid/rubric to the video assignment to make clear to students what is being assessed and how is being assessed. It is a good practice that tutor and students discuss the marking grid before the submission to make sure that students understand what is expected from them.

If you are interested in using Media Hopper for video assignment and feedback, get in touch with me, Mari-Cruz García, at maricruz.garcia@ed.ac.uk

_________________________________________________________________________________________________________________

References:

Epstein, Ronald M. (2007). Assessment in Medical Education. Medical Education. The New England Journal of Medicine. Massachusetts Medical Society.

Norcini, J., Burch, V. (2007). Work-based assessment as a medical tool: AMEE Guide No.31. Medical Teacher.

West, J. & Turner, W. (2015). Enhancing the assessment experience: Improving student perceptions, engagement and understanding using online video feedback. Innovations in Education and Teaching International, 38(2), 240-252.