On 1 March, the IS User Experience (UX) Service, in partnership with the School of Informatics, ran a Learn usability testing showcase event. Participants from across the University watched screencasts of students using an Informatics Learn course, before prioritising the usability issues identified.
Five students in total took part in the testing – four from Informatics, including those enrolled on single programmes, joint programmes with other Schools, and those from our Undergraduate Apprenticeship Scheme – and one from the School of Economics. Each was presented with the following scenario and four ‘typical’ tasks to perform.
A copy of the Learn course for Computer Security was used for testing purposes. This was chosen as it aligned closely with the Learn template developed for the School.
Scenario
You’re a third year student on the joint programme BSc Artificial Intelligence and Computer Science. This semester you are studying a course called Computer Security. It’s week 3 of the course, and you’re preparing for your first piece of coursework.
Tasks
Task 1: You want to check the deadline for the first piece of coursework and see if it clashes with any other coursework deadlines. Using the Learn course site, find out the deadline for the first piece of coursework, and then see if it clashes with coursework deadlines for any other courses on which you are enrolled.
Task 2: You missed the third lecture of week 1 because of sickness. You’d like to watch the recording so you can catch-up. Using the Learn course site, find and play the third lecture of week 1.
Task 3: You’re going away for the weekend and you’d like to do some reading while you’re away. You’re not sure you’ll have access to the internet, so you’d like offline access to your reading. Using the Learn course site, find the required textbook for the course and see if you can download or print a section of the textbook.
Task 4: You’d like to familiarise yourself with the content of the last lecture you attended, called Cryptography – asymmetric encryption. Can you open the lecture notes from this lecture?
Results
Task 1: Most users found the coursework deadline relatively quickly and with ease. One student checked both the Course Information and Course Content pages prior to selecting the Coursework and feedback page.
However, no users were able to easily find the link to the personalised coursework planner. This was expected, and one of the reasons why I included it in the task.
Task 2: Most users found the link to the lecture recording overview page with relative ease. Some users were expecting to find a direct link in the table on the Course content page. This was not surprising as the Semester 1 course Informatics 1: Introduction to Computation includes this.
There was, however, a significant usability issue identified for all users when it came to identifying a particular recording from the Media Hopper Replay course overview page. This was caused by the unhelpful automatic naming convention of recordings (see below). Users performed a lot of cross-checking between different pages on Learn, various online calendars and and the Media Hopper Replay course overview page to identify the recording from the “third lecture of week 1”.
Task 3: The course organiser had used Leganto, the centrally supported Resource List tool, for the course. Users could access the text on Leganto via both an in-text link, or an icon associated with the service link. Most Informatics users found the link to the required textbook with relative ease. There was one instance of users navigating to the table on the Course content page where references to specific chapters are included.
This particular textbook was behind an EASE login. As the students were using a dummy account, they were prompted to enter their EASE credentials which would not be the case when logged in as themselves.
Interestingly, the student from Economics searched for the textbook by navigating to the Handbook. This highlights the different approach to content curation and the various roles course and programme handbooks perform across the University.
Task 4: Four users found the link to the lecture notes with ease. One (visiting) student initially checked the Coursework and feedback page. It was noted that the terms lecture notes and lecture slides are sometimes used interchangeably.
Action Points
- Feature request for Media Hopper Replay team: can we automate naming of recordings by date? Venue information would also be helpful here.
- Request for Media Hopper Replay team: can we facilitate production of individual URLs for each recording which will work for enrolled users – *even when they haven’t selected initial LTI link*. Only when this can be a achieved, should we encourage course instructors to include links to Media Hopper Replay recordings in the table on the Course Content page.
- Promote coursework planner across the school. Posters / monitor displays etc.
- Include a thumbnail of a ‘typical’ coursework planner in the next iteration of the template.
- Can the coursework planner display full course name rather than acronym?
- Enquire into possibility of responsive design for coursework planner.
- Can we have the coursework planner work for tutors (eg marking loads)? Do we need this?
- Work with the web and communication team to research how Informatics students use the yearly handbook.
Reflection
I thoroughly enjoyed working with Duncan Stephen on this mini project. The feedback was informative, encouraging, and a call to action. I’m looking forward to embedding similar practice across the School for alternative platforms for content delivery.
The results of the ‘prioritisation of issues’ aspect of the workshop can be found below. If you would like to know more about this particular round of testing, or would like to use your course for further testing, please don’t hesitate to get in touch.
Further links
- User Experience Service: http://www.ed.ac.uk/is/ux
- Join the UX community: http://bit.ly/UX-meetup-blogs
- UX mailing list: http://bit.ly/uoe-ux-mail
- Steve Krug’s Rocket Surgery resources: http://bit.ly/1I1muXo
- David Travis’s prioritisation flowchart: http://bit.ly/1I1mCWW
