As part of our series of usability testing showcases in collaboration with the Learn Foundations project, we worked with the School of Law to uncover usability issues witnessed when staff are using Learn.

Throughout 2018/2019, the User Experience Service worked with a number of different Schools to find out what’s working well, and not so well, for users of different Learn course environments. In June, for the first time, we conducted usability testing with staff members in Learn.

At a showcase session held at Old College, colleagues from across the University were invited to come and see for themselves how different staff members in one School tackle common, representative tasks in a typical Learn course. As a room, we then identified and prioritised all the usability issues we witnessed by using the tried-and-trusted red route prioritisation technique.

Learn more about the red route method

Format of the usability tests

We ran usability tests with four members of staff from the Law School. Two of them were academics involved in updating Learn. The other two were professional services staff with responsibilities around Learn.

We set each of them a scenario and the same four common tasks, each of which staff members would be expected to be able to complete:

1. Uploading a PowerPoint file for a seminar
2. Finding information about a reading
3. Promoting an extra-curricular event
4. Adding contact details of a course rep

What we saw

Attendees of the showcase event identified 20 different usability issues seen in the four videos. This is a higher number than any of the five rounds of usability testing we have conducted with students.

These issues were collaboratively prioritised as either low, medium, serious or critical using the red route prioritisation flowchart. We found five issues in each category.

But perhaps most notably, we witnessed staff following a range of different strategies and approaches to completing these tasks. These are four common tasks that are pivotal to our students’ experience when using Learn. And yet four staff members from the same School ended up putting content in a variety of places, and presenting them in differing ways.

This underlines one of the main issues that the Learn Foundations project is trying to tackle — students’ frustration with inconsistencies in Learn. By watching staff members using Learn and seeing the usability issues staff encounter, we have developed a stronger understanding of how we can best support staff to use Learn and thereby improve students’ experience in Learn.

Through the user research we’ve been conducting, we have listened to students talk about the problems inconsistencies cause, and watched them get frustrated when trying to complete common tasks in Learn.

By undertaking usability testing with staff members, we saw that staff members even within the same School can have radically different approaches to organising their content in Learn. Watching just 45 minutes’ worth of footage of staff using Learn gave us a very stark demonstration of how these inconsistencies are arising.

Evidence-based enhancements

The Learn Foundations project team have used the findings from this usability testing to identify how we can better support staff using Learn. In particular, this session highlighted some opportunities for the training programme that is being developed. We will also highlight relevant user interface findings to Blackboard, the vendor that supplies Learn.

We were really keen to explore how our academic and professional services staff in Law interact with Learn every day, to check if our School template was working as expected from their point of view, but also to expose any other barriers they might be facing. Working with the UX team was a great experience as we got really valuable feedback which we can now take forward through staff guidance and training.
— Neil Davidson, Learning Technologist, School of Law

Further information

• Blog post introducing the usability testing cycle
• Learn Foundations project
• User Experience Service