Phase Three of Learn Foundations has carried out important research to investigate how Learn is supporting the needs of staff and students as they continue to teach and learn in the hybrid context, brought about by the Covid-19 pandemic.

The Learn Foundations project seeks to make Learn better. It strives to ensure students and staff have the best possible user experience of Learn, and that this experience is consistent across the University.

Service design in Learn Foundations

Phase One of the project began in 2018 and from the start, it has been led by service design. Service design is a discipline which recognises that to achieve a good service, the needs of those using the service as well as those operating and providing the service must be fully understood and applied. Only with a clear view of all perspectives can organisations develop services which work.

Service design research activities from Phase One of Learn Foundations included usability testing, contextual enquiries and interviews with staff and students, and key pieces of quantitative research such as a card sort study and a top tasks survey. This research provided a sound understanding of  staff and students’ needs from Learn, and a clear picture of the way they wanted and expected to use it.

You can read more about the service design approach to the project and the work from Phase One in Duncan Stephen’s blog about the project:

A service design approach to Learn Foundations

Learn Foundations template: Stability in a time of rapid change

The Phase One research and Phase Two engagement stages of Learn Foundations led to the development of a template with the following nine ‘core’ menu items to support task-based navigation of Learn:

• Welcome
• Course Information
• Announcements
• Course Materials
• Library Resources
• Assessment
• Course Contacts
• Have Your Say
• Help and Support

The template also included an optional ‘plus two’ menu items, which Schools could use to customise the interface to meet needs specific to their context.  As well as the template, a glossary of terminology was developed to help keep the language and terms used within Learn consistent across Schools and Deaneries.

In Phase Two, 10 Schools and Deaneries had signed up to work with Learn Foundations, and to adopt the template. This was increased to 20 as a result of the Covid-19 pandemic, and Schools that had adopted the Learn Foundations template adapted more easily to delivering their teaching online.

Consistent templates paved the way for a seamless move into hybridity in a time of crisis.

– Jon Jack, Learning Technology Team Manager

The importance of Learn in helping to ensure continuity in teaching and learning across the University was clear. Staff were using the Learn template for their teaching and students were relying on Learn as the primary interface to support their learning.

Research to understand the new hybrid context

Increased uptake of the Learn template in response to the pandemic showed despite the changed circumstances, the template was robust and still fit for purpose. In other words, Learn seemed to be fulfilling one of the fundamental principles of good service design:

A good service should respond to change quickly .

– Principle 13 of 15 principles of good service design by Lou Downe

To validate these assumptions about Learn’s performance in the new hybrid context, we needed to carry out research. We wanted to understand:

• Are student and staff needs from Learn the same in the hybrid context?
• Are the templates still valid in the new context?
• Is Learn continuing to support a good student and staff experience?

Answering these questions would help us assess whether Learn was still fulfilling its purpose (or solving the right problem).

Upskilling the team – making research a team sport

Learn Foundations strives for continual improvement of Learn for staff and students, and the service design process supports this, driving sustainable solutions and optimal experiences based on a common understanding of the needs of service users and providers.

Research is particularly crucial in the service design method. Through research we can obtain a clear view and shared understanding of the needs of service users and providers. Those doing the research activities have the chance to gain a holistic understanding of how people interact with the service, and are well-placed to feed this knowledge into shaping future iterations of the service.

To this aim, the research in Phase Three of Learn Foundations was not, as in Phase One predominately carried out by the UX and Digital Consultancy team. Instead, research became a ‘team sport’. The UX and Digital Consultancy team worked closely with staff from both the Learn Foundations project and from Schools to equip them with the right guidance, skills and toolkits  – to empower them to carry out and take ownership for research.

There were several benefits to opening up (or ‘democratising’) research in this way:

• Increased visibility and understanding of UX and service design research practices and recognition of their value in providing evidence to make design decisions
• Greater efficiency in the process – with research findings experienced first-hand rather than having to be separately communicated
• Greater empathy and understanding with student and staff needs emerging from the research – as more members of the team get to see these findings for themselves
• Greater capacity for creative solutions and multi-disciplinary decision-making – as involving more people brings new perspectives to issues presented by the research findings
• Establishment of user research as a core skill embedded in the Learn Foundations team to help sustain the service design approach for the longer term.

As in Phase One, we adopted a ‘triangulation’ of research methods for the research in Phase Three, combining quantitative and qualitative methods to unravel the ‘why’ as well as the ‘what’ aspects to the research questions.

Quantitative research – top tasks and card sort

In Phase One of the project, two key pieces of quantitative research were instrumental in helping us understand what staff and students wanted to use Learn for, and how students expected to navigate it.

These were: A top tasks survey and a card sort. A top tasks survey helps identify what really matters to the people using a service by asking them to vote for the tasks they deem most important. A card sort is a study designed to identify how users expect information to be arranged within a service, so they can easily navigate to the right place to complete their tasks.

You can read more about these techniques, how they were applied and what we found in Phase One of Learn Foundations in Duncan Stephen’s blog posts:

Card sorting has informed a new information architecture for Learn courses

Top tasks surveys have identified what really matters to students using Learn

Repeating these two studies in the changed, post-Covid-19 context would be an efficient effective way to gauge what had changed for staff and students with the move to hybrid teaching and learning, and therefore a way to address the research questions we wanted to answer.

Tweaking the set-up to reflect the hybrid context, and making changes based on feedback from the findings from Phase One, we repeated the top tasks survey and the card sort.

You can read more about these studies and the results in these blog posts:

A card sort study has revealed how students expect to navigate Learn for hybrid learning

A top tasks survey has shown what staff and students prioritise in hybrid teaching and learning

Qualitative research – usability tests and semi-structured interviews

To complement the quantitative research, a series of usability tests and semi-structured interviews were carried out with staff and students in selected Schools.

Usability testing centred around common tasks staff and students would want to complete in Learn to support hybrid teaching and learning. For example:

• Posting details of a seminar in a course area (staff)
• Finding and viewing a recorded lecture (students)

Details of the usability tests and their findings are included in a separate blog post:

Usability tests have revealed how staff and students use Learn for hybrid teaching and learning

Semi-structured interviews sought to probe into staff and students’ experiences of using Learn to discover how they used it to achieve their teaching and learning tasks, and to identify pain points within their experiences. You can read more about the findings from the interviews in this blog post:

Using Learn for hybrid teaching and learning – perspectives from staff and students

What we found about staff and students needs in the hybrid context

Students’ needs from Learn were the same as they were pre-Covid-19

Analysis of both sets of quantitative research data – from the top tasks survey and the card sort – indicated that student needs from Learn had not significantly changed in the shift to hybrid. Seven out of the top eight things students said they wanted to use Learn for in the hybrid learning context were the same as in the pre-Covid-19 context.

Items like lecture slides, lecture notes and reading lists were still as important to students when they used Learn as they had been before the pandemic.

Furthermore, the card sort research data indicated students also expected to navigate Learn in the same way as had been the case pre-Covid-19. Comparing the Phase Three data to the Phase One data, students had grouped approximately 80% of the items they were presented with into the same four main categories – ‘Course materials and content’, ‘Course information’, ‘Help and support’ and ‘Assessment’, as those in the Phase One study had done.

Therefore, despite learning and teaching being fundamentally different, students were expecting to find things in the same place in Learn, they weren’t expecting to see a separation of physical and online items.

Staff needs had shifted slightly

Looking at the top tasks survey responses from staff, it was clear that staff were prioritising things slightly differently than they had done pre-Covid-19. In the top eight tasks voted for by staff, just over 60% were the same as the tasks identified by staff from the Phase One survey.

Staff still ranked things like lecture notes, assignment submission details and course curricula highly. Some items (like ‘Resources for preparing for seminars/lab/practicals’) had dropped in importance while others had become more important (for example ‘Announcements’). The change in staff priorities when using Learn was to be expected, as it indicated that staff had had to change their thinking in order to deliver teaching differently.

What the findings meant for the Learn template

The card sort data told us students were expecting to navigate Learn in the same way they had before the pandemic. This was a strong indicator that the template of nine fixed menu items (plus two which could be adapted at a School level) did not need to be changed to account for the hybrid context.

Students’ needs from Learn, and the things they regarded as important when they were using Learn had not changed which further supported this conclusion.

Staff needs from Learn had altered slightly, however, not to the extent that these needs could not be accommodated within the existing templates. Applying service design thinking, it was important to consider both staff and student perspectives when deciding whether a change was necessary. Altering the template to accommodate staff needs could result in a change that students could find confusing, since the existing template was fulfilling their needs and expectations.

Data from the card sort showed most students created four large categories to group 80% of the total items. These categories were well-aligned with the menu items of the Learn templates. Students typically placed 9-13 items in each category, some of which were quite varied. For example, in the category ‘Course content and materials’ students placed items as diverse as ‘Lecture slides’, ‘Virtual classrooms (live lectures)’ and ‘Resources for practicals’. Despite this variation, the research did not point to the need to add further structure to organise items within these larger categories.

What the findings meant for the Learn terminology

Since the card sort required students to create terms to label groups of items, this study was an important source of information about the terminology students expect to see and are familiar with.

Comparing the terms emerging from the Phase One and Phase Three card sort studies, there were no significant differences, suggesting there hadn’t been a shift in the way students were referring to things, and therefore no adjustment to the glossary was required.

Filtering the Phase Three data to compare terms used by first years (who had only experienced teaching and learning in the hybrid context) to those used by non-first years showed subtle differences. For example use of the term ‘Announcements’ was more common in non-first years and use of ‘online’ was more common in first years. This variation was so slight it did not suggest changes to the terminology were necessary.

More blogs about Learn Foundations user research

Phase Three Learn Foundations has included both qualitative and quantitative research. You can read about the different pieces of work in separate blog posts:

A card sort study has revealed how students expect to navigate Learn for hybrid learning

A top tasks survey has shown what staff and students prioritise in hybrid teaching and learning

Using Learn for hybrid teaching and learning – interview perspectives from staff and students 

Usability tests have revealed how students and staff use Learn for hybrid teaching and learning 

Information about the research and findings from the Phase One Learn Foundations user experience programme are included in Duncan Stephen’s post:

A service design approach to Learn Foundations