A service design approach to Learn Foundations
Throughout 2018/2019, the User Experience Service has been collaborating with the Learn Foundations project team to undertake a comprehensive programme of user research with students and staff. Through this we have discovered how students’ experience in Learn is closely intertwined with how staff work with it. This post summarises all our work, and outlines how we have ended up taking a service design approach.
Improving services for staff so that everyone can improve services for students
Learn Foundations is all about improving the experience of staff members so that we can all improve students’ experience when using Learn.
The approach we have taken to the user research in this project has reminded me of service design principles.
Service design is the activity of planning and organising a business’s resources (people, props, and processes) in order to (1) directly improve the employee’s experience, and (2) indirectly, the customer’s experience.
— Definition of service design from Sarah Gibbons, Nielsen Norman Group
Front stage and backstage
In service design, the metaphor of the stage is often used. When an audience member (or user) attends a play at a theatre, they see what happens on the front stage. This is as far as the user’s “line of visibility” goes.
Yet, a host of backstage actions are also needed for the show to go smoothly — people and processes involved with lighting, sound, sets, and so on.
Beyond even this, there are what we can call behind-the-scenes processes. These include things like legislation, policies, budgets and a whole host of support processes that are required to make the play happen even beyond the front and back stage.
While this theatre metaphor is common in service design, we may as well be talking about a lecture theatre.
Our students have a direct line of visibility with their lecturers on the “front stage” of the lecture theatre. Backstage, we can think about local (School-based) learning technologists and support staff who help ensure teaching activities go to plan. Behind the scenes, we have central services based in departments like Information Services, and projects like Learn Foundations, as well as wider policies and legislation.
Understanding how to improve staff actions and student experience
Our user research began with open interviews with both students and staff. Findings from these have been summarised in two previous blog posts.
For most of our findings about students, we have found something about staff that is driving that experience.
After our interviews with students, we presented the findings as “a semester in the life of students using Learn”. This shows us how students’ experience of Learn changes as they encounter various tasks throughout the academic year.
Because we found strong associations in what staff were experiencing, we presented the findings from our staff interviews in a similar format, so that the two boards can be read together.
Learning how to use Learn
Both students and staff experience issues around familiarising themselves with using Learn. Students expressed feeling overwhelmed when they first access Learn. Staff face a similar issue around understanding how to use Learn and get the most out of it to support their teaching.
Assignments and feedback
Assignments were found to be a major pain point for both students and staff. Many people were remarked on inconsistent processes from course to course and School to School.
Similarly, students’ concerns around the consistency of feedback were mirrored in staff members’ comments. We identified a complex set of emotions and experiences around the issue of feedback. There is a challenge around providing feedback in a consistent format for students. Staff members often put a lot of effort to provide good feedback, and feel frustrated if students don’t always read it.
Learn as a communications tool
We found that many students feel overwhelmed by the volume and timing of notifications and announcements in Learn. Meanwhile, staff members told us that they love using Learn as a comms tool.
This is perhaps the most challenging finding. If there’s one thing staff members like about Learn above all else, it’s using it as a comms tool. But this makes students feel like they being bombarded with irrelevant messages.
We saw students demonstrate that they lacked trust in existing Learn templates implemented by Schools.
We discovered that staff using Learn are often not told why templates exist, or why they’re set up the way they are. So staff lacked a full understanding of where to place content within templates. This explains the inconsistencies that students have been encountering, even when a template has been set up by the School.
Usability testing with students and staff
On top of these interviews, we conducted a regular cycle of usability testing with students and staff, in collaboration with six different Schools from across the University.
Overall we have conducted 27 usability tests. These were played back to staff members during eight different showcase sessions. We identified and prioritised 94 usability issues:
- 21 critical priority
- 26 serious
- 28 medium
- 19 low
All of these findings have been influencing the development of the Learn Foundations solutions throughout.
Not understanding the full picture can lead to you building the wrong thing
In the past, people might have approached a project like Learn Foundations like the following.
School-based learning technologists may ask Information Services to develop new announcements functionality because lecturers want it to be easier to post announcements.
A team in a central service like Information Services might then go ahead and build that functionality, based on a list of requirements given to them by local staff.
This pleases the lecturers, who go ahead and start posting announcements.
But for students, more announcements is the last thing they wanted.
This might happen because students weren’t engaged properly, or perhaps they weren’t even engaged at all. So the feature may be designed badly from the perspective of the end user. It might even be the wrong feature entirely.
The service design approach is helping us truly meet users’ needs
We have undertaken a service design approach, incorporating multiple user experience methods, to avoid this problem. Our comprehensive programme of user research has enabled us to develop a better understanding of this entire complex ecosystem of interactions.
We have conducted qualitative face-to-face research with 34 different students from 15 different Schools. We’ve also received 4,321 responses from students to our quantitative studies. This has given us detailed insight into what our audience needs.
With staff members, we’ve conducted 15 qualitative research sessions with different types of staff from nine different Schools. We’ve also received 264 staff responses to our top tasks survey. Through this we’ve built a picture of what’s currently happening, and what’s needed on the front and backstage.
This has enabled us to build a very rich and detailed picture of what students and staff need to do in Learn, and why — and how staff members can improve students’ experience in Learn. In other words, we now have a better idea of how to design behind-the-scenes processes, to better support what’s happening backstage and front stage, in order to ultimately meet the audience’s needs.
All blog posts about Learn Foundations user experience
We have carried out a large amount of user research. That means there are a lot of blog posts about it. Here is a categorised list of all the blog posts:
Usability testing cycle
- Improving student experiences in Learn: usability testing showcase and workshop — introducing the regular cycle of usability testing
- Report on round 1 with the Royal (Dick) School of Veterinary Medicine
- Report on round 2 with the School of History, Classics and Archaeology
- Report on round 3 with the School of Informatics
- Report on round 4 with the School of Philosophy, Psychology and Language Sciences
- Report on round 5 with the School of GeoSciences
- Report on round 6 with the School of Law — usability testing conducted with staff members
Contextual enquiry and interviews
- Interviews with students to understand users’ needs and contexts around Learn
- Contextual enquiry with members of staff working with course materials digitally
- The elements of a better user experience in Learn — introduction to our programme of quantitative research
- Top tasks surveys have identified what really matters to students using Learn
- Card sorting has informed a new information architecture for Learn courses
- Using tree tests to refine an information architecture
- First click tests — building up the elements of user experience for Learn Foundations
- UX Service Showcase December 2018 – slides, videos & feedback — contains a presentation about the early stages of this work
- User research into the needs of students and staff using Learn — contains a presentation from the middle of our research programme
- UX Service showcase July 2019 – videos and slides — contains a presentation from near the end of this phase of work