Exams, human paperclips and 600 buildings
I spoke to staff in the University of Edinburgh’s 24 schools & Deaneries in summer/autumn 2023 about how they examine students and what some of the challenges are. It got me thinking about exams… what are they, what’s changed recently and what’s coming next?
The emperor receives a candidate during the Palace Examination. Song dynasty. Ming Dynasty Painting, Public Domain
Exams are nothing new
Examinations to assess knowledge, skill or aptitude pre-date Universities. The earliest are thought to have been used in China to recruit to the state bureaucracy during the Sui Dynasty (581-618). Over the centuries, women, butchers, artisans, priests, physicians, merchants and certain castes were excluded from these ‘Imperial Exams’, so they could never achieve high ranking state roles. In Europe, exams came later, possibly in the early 13th Century in Bologna. Candidates would give a public lecture on civil or ecclesiastical law and then answer questions and debate their position. Perhaps this is the origin of the viva voce.
The earliest evidence of written exams is from Trinity College, Cambridge in 1702 although they probably began much earlier in the early 16th Century. Although there was much scepticism in Europe about exams because of their Chinese origin (the German philosopher Christian Wolff was forced to leave his position at the University of Halle after being accused of atheism when he praised Confucianism in a 1721 lecture), they have become ubiquitous.
The first Master of Arts degree at Edinburgh in the late 16th Century had an entrance exam to test potential students’ command of Latin, since all classes were to be taught in Latin. The University has therefore had over 400 years of exams although it’s difficult to find written records. If you have time to browse the Centre for Research Collections, you can take a look at a Manuale Scolarium from the late 15th Century where two students from the University of Heidelberg (Germany’s oldest surviving University, established in 1386) discuss subjects such as examinations, fresher initiations and the dangers of loving women.
Exams today
Although the majority of courses at the University of Edinburgh now have no exams at all (Figure 1), almost 800 exams were scheduled in May 2023. Of course, Covid changed how exams were delivered, with all exams in 2020 and 2021 (except some written and clinical exams within the College of Medicine and Veterinary Medicine) being conducted online via systems such as Blackboard Learn, Turnitin and Gradescope. The process was familiar to students who regularly submitted in-course assessments to Learn. Additional time was added at the end of the exam to upload documents and resolve any technical issues and on the whole this seemed to work ok.
Figure 1 Data from DRPS Oct 2023
Professional Services staff told me these online exams were easier to administer and troubleshoot; no juggling room bookings, no enormous print runs and no humphing exam papers to exam halls. Marking was done online, and many Academics have said that online marking can be done more quickly and with more fulsome feedback.
Show me what’s under your desk
However, some Schools described an increase in academic misconduct during this time. There are concerns about opportunities to communicate with peers and access reference materials undetected during an exam. Whole books have been written about attempts to cheat the old Chinese Imperial Exam. Some candidates were found to have written copies of texts on their underwear, so I don’t doubt some people will try to cheat in modern times.
How do you prevent cheating when you don’t have an Invigilator observing the exam? Say hello to online proctoring. I’ve had the misfortune to have sat a proctored exam at home. Carrying my laptop around my dining room with the camera on to show an anonymous invigilator I wasn’t hiding an extra laptop under the desk or person in the corner wasn’t a great start. The issues around online proctoring were described eloquently in Myles Blaney’s Aug 2020 blog so I’m not going to replay them here. Suffice to say, no one wants to do it.
AI and human paperclips
Schools also told me that Academics are worried about the potential for AI to be misused in remote exams, where it’s more difficult to lock down or prevent access to other technology. Of course, we don’t know the full potential of AI yet but if you are interested in reading about how AI might turn humans into paperclips, try Dr Gina Helfrich’s Nov 2023 blog. The issue of how AI might influence assessment practices is exercising many at the moment.
It is contradictory for teaching practices to be undergoing digitalization, yet assessment still paper-based” (Chan 2023)
Given these concerns, perhaps I shouldn’t be surprised that by May 2023, 82% of timetabled exams at the University had reverted to in-person. Medicine and Veterinary Medicine use risr/Assess (formerly Practique) for most in-person exams. For MBChB students, all exams are completed and marked electronically, as was the case pre-Covid.
However, the vast majority have reverted to hand-written exams. Most Science and Engineering exams are completed on paper, scanned into Gradescope and marked online (Academics were keen to continue online marking). Most Arts, Humanities and Social Science exams are completed and marked on paper, having reverted almost completely to pre-Covid practices. For old yins like me, handwriting is second nature but the world has changed. Exams must be one of the only places today’s students are required to handwrite and I wonder what they think of this.
All that space
We have a real challenge with competition for suitable PC lab space for exams; fitting a large cohort of several hundred students into a single exam hall with locked-down laptops and sufficient power sockets is currently impossible. This means even if Schools want to have online in-person exams, there are some barriers. Schools have tried to split exams over several labs (needing additional invigilators and IT staff to configure laptops) or sittings (keeping cohorts separate until both sittings are complete). One School must hire commercial space so all students can be accommodated. You’d think with over 600 buildings we would have sufficient space to run online exams, but this is not the case.
What next?
Perhaps we should be looking at alternatives to exams. IAD’s approach to assessment design includes a useful list of pros, cons and resource requirements of different assessment types that may be considered as an alternative to exams, and for the student perspective take a look at Kent-Waters et al’s 2018 compendium of assessment techniques in higher education: From students’ perspectives.
It will be interesting to see how Curriculum Transformation tackles this. The Assessment and Feedback Group (Chaired by Professor Tina Harrison) is looking at assessment volume, pace, role and types and I look forward to reading more about their findings.
A paper from the PC-enabled Exam, Teaching and Study Spaces group will be going to Senate about the issue of lab space for exams in early 2024, so watch this space.
Karen McPhail (Business Analyst, DLAM, Information Services)
Share
