Myself and Karen Howie have been working on a paper regarding online proctoring and whether the technical solutions currently in the market are fit for purpose. When I say fit I mean do they meet the huge amount of variation in exam practise used at the institution, can they cater for varying numbers of students, what is the process and are they any good.
Proctored exams – a personal reflection – https://t.co/AKYl6P1NQZ— Karen Howie (@karenhowie2008) July 13, 2020
Maybe we should rewind a bit to the age of PP (no E) or what I like to call is pre-pandemic. Exams were usually in physical buildings at a set time with an invigilator/proctor present in the room to ensure the prescribed exam conditions were met. People would sit less that 2 metres apart and even sneeze without being called a leper and subsequently judged and hung on the spot. Campuses were busy and the only use for yellow and black hazard tape was on Halloween to transform your cosy abode into a zombie den.
Now PPEOW (post pandemic end of the world) with the age of social distancing and the potential for local lockdowns and campus restrictions we need to review how we could deliver exams when a physical location is not available. Now we all know how universities love building as much as they love Pokemon, and how much they love creating innovative spaces (then buying desks and chairs from boringofficekit.com) however imagine a world where this space wasn’t available …… oh hang on! and welcome to the new normal (covid-19 lingo bingo full house!).
Online proctoring isn’t new and it’s not revolutionary however its a hot topic in edtech. So we are back to the review myself and Karen are currently conducting of the TECHNICAL resources (Myles has said TECHNICAL a few times in caps and bold he might be about to make an important point). Open a new tab and search ‘Online proctoring’ on the web and browse the title and summary of pages returned. Go on then…….I ain’t going anywhere.
You may have a lot of ads on online proctoring providers, some articles by private and HE institutions on how great its been however you may notice some article about anxiety or how to cheat online exams. This highlights the complexity of what these platforms are delivering, It’s not just a link to a reading, or a dropbox for an assessment a student has 2 weeks to complete (although its shares a lot with this) online proctoring is usually framed around high stakes exams in education. Physical (in a hall or room) exams can be and from my own experience are very stressful so replicating this into a digital environment carries over those stresses and adds a whole new layer of complexity regarding digital, distance and desire (by institutions to replicate exam conditions and catch those pesky cheaters).
Off the top of my head TECHNICAL concerns (and this is excluding the actual exam platform….)
- What about accessibility and disability? (can it be used by students who require assistive tech, is it W3C WCAG compliant)
- What about my local device? (what OS, what browser, mobile, tablets, i don’t have a device etc)
- What about my internet connection? (satelite, dial up support, internet cafes)
- What do I need to click when? (profile creation, onboarding, on the day)
- What about being watched? (desktop, web cam, audio, your local device)
- What about exam conditions (it states i cant leave the room but I need the loo)
- What about being flagged for a potential breach (and having to wait for days)?
- What about my data? (where, when, how long and why)
- What happens when a student has an issue? (live chat, email, human interaction)
- What about the reliance and quality of AI
- What about inclusivity
Technically there’s alot to consider however we see vendors ensure us they can accommodate large cohorts (but maybe not all starting the exam at the same time) on stable (cloud based data centres located all over the globe), easy to use platforms (it has buttons and text and images what can go wrong………) that anyone can use (although some don’t have accessibility statements or can provide accessibility testing information) whilst increasing exam integrity at speed and scale (similarity detection vendors like to talk about integrity however I get a bit confused on how a 23% text match maps to an integrity model and do they exist?).
There is alot to investigate however with all these things to consider I do see a use case where online proctoring could be useful. For example resits by students not on campus (which is a cost saving for the institution and student), smaller online courses, low stakes assessment that require invigilation etc. The technology is getting better (expanding functionality and configuration) more accessible (providing accessibility statement and working to W3C compliance), more secure (EU/UK based data centres, end to end encryption) however maybe the question we need to ask isn’t whether the online proctoring is technically ok, Maybe the question is why do we need proctored exams?
Let's not just try to do the same thing online as we did on campus. Assessment design requires thoughtfully integrating purpose, theory, affordances and care.— Tim Fawns (@timbocop) June 29, 2020
In particular, proctored exams are the worst of all worlds: exams plus surveillance. https://t.co/V9GjuZ0259
I don’t intend to discuss that topic as others have recently asked the same question and are far more knowledgeable however the technology has been designed to meet a physical proctored requirements:
- replicate exam conditions
- we need to see the user
- we need to see the users desk
- we need to see the room they are in
- we need to do an ID check
- we need to check they are not conferring with anyone or anything else
- we need to ensure they don’t act dishonestly or unfairly in order to gain an advantage (aka cheat)
- we need a space which can accomodate everyone
- we need cohorts to sit the same exam a the same time (to avoid anyone gaining an unfair advantage
Which reminds me of this…..
So what came first the exam or the requirement to replicate the exam online? Plus is online proctoring a victim of providing a service that in its crudest form (and with no consideration for ethics, user experience, accessibility and inclusion) meets the above?