We’ve been thinking about how to invigilate digital exams. I have found this hard, I’m not sure I agree with it. Somehow it feels far more intrusive digitally than in a room with an invigilator. However, I do understand that sometimes it may be necessary and not all assessment can be redesigned (particularly not with the timescales we have) to minimise cheating, although I do agree with Tim Fawns in this tweet about assessment. And of course, Anne-Marie Scott (previous Head of DLAM, and I’m desperately trying to work out how she did this job so well and stay sensible….) wrote this excellent, but a-bit-scary blog post about assessment, proctoring, ethics and trust. Oh. And skelfs.
I’m not going to write about proctoring in general, my colleague Myles Blaney is running this project and he has many reflections too, and I don’t want to steal his next blog post from him whilst he is on holiday (that would be a bit mean, although I’m secretly jealous he’s on holiday so maybe not so worried about being mean? 😛 ). And I’m also secretly hoping that the M&M Podcast will discuss proctoring at some point (hint, hint). However, I do have a few reflections to make from a personal perspective.
The personal perspective is that my partner recently had to do a proctored exam. He is an IT Contractor and very much into keeping up with the latest in technology and recently completed a week long training course with an exam at the end. In the past he’s headed to London (or Brussels!) to sit exams to get qualifications from big-IT-companies-who-shall-remain-nameless but obviously, because of Covid, this is not currently possible, and the exams need to be done online. So he booked his exam at 8pm on a Friday night, at the end of a long week of training, scoffed his dinner down so he had time to clear out our spare room (where I’m currently working) to remove my computer equipment from the room (so that the invigilator wouldn’t think he had multiple computers) and set himself up on the desk I work on by 7.45pm. I took our dog out around that time to give him some peace and quiet to get his exam started. It was supposed to last 1.5 hours, so should have been done by 9.30 but at 8.45 I started to return home with Charlie our schnauzer. I got a message from him saying his exam still hadn’t started. They were having ‘technical problems’. So I sneaked into the house when we got home around 9 to find him pacing around downstairs on a call. He still hadn’t started his exam. He was on hold, they were trying to give him some help. He said he thought it was to do with the virtual machines he has running on his computer (which most techies probably do…) but they weren’t really sure. He’d made it through the on-boarding part of the process where his computer had been checked to make sure that nothing was running that shouldn’t have been but when he tried to start the exam, the software just died and wouldn’t run.
He was still pacing around by 9.45pm At this point, his exam should have been finished. He was still on the phone. They were trying to find someone to help him but the supervisors were all busy or on a break. Eventually at about 10pm, they agreed he could reschedule the exam.
This sort of exam probably works when a company has thousands of questions/question combinations and where users don’t sit an exam simultaneously. But what would have happened if he’d been sitting his exam in 3rd year Software Engineering or 4th year Medicine? We don’t have thousands of questions/question combinations. He’d have to sit the exam later, after the rest of the class had already completed it. Which would mean preparing at least another equivalent exam paper, possibly more. With a class of 500, how often would these technical issues occur? And how would this have made an anxious student feel? It wasn’t pleasant for my partner who is very experienced at these types of exams. It was at best a complete waste of an evening, at worse very stressful.
I’ve been interested that for at least one of the proctoring services we’ve looked at, they could only take 75-100 concurrent users in an hour. So for that class of 500… we’d need to rethink how this assessment worked. And if we are rethinking how the assessment worked anyway… maybe we can rethink it in a way which means you don’t really need a proctoring service? And hey, we are back to that tweet of Tim’s!! How did that happen?
