In preparation for moving to Learn Ultra it was important to find out staff and students’ experiences of actually using Learn Ultra courses. This insight is guiding the continuing development and iteration of Learn Ultra has helped the project team identify areas of improvement. 

The project to upgrade to Learn Ultra is part of the Virtual Learning Environment (VLE) Excellence programme which aims to enhance the overall staff and student experience. Therefore, feedback from staff and students has been central to the project at all stages. The latest programme of user experience research was completed between May and October 2022 and looked at the staff and student experience of Learn Ultra, the upgraded version of Learn. A particular focus was the experience of Learn Ultra courses, in preparation for this part of the Learn upgrade at the start of the 2023 to 2024 academic year.

Continuing a service design approach

Understanding Learn Ultra as a service – and planning user research accordingly – helps ensure decisions about its ongoing implementation remain as user-centred as possible. The Learn service ecosystem comprises a complex set of interactive processes carried out by staff and students. How successfully Learn Ultra delivers service value – in terms of teaching and learning outcomes – depends on combined actions and interactions of its ‘actors’ (staff and students) and their integrated use of the resources available.

 Value is co-created by multiple actors, always  including the beneficiary –  one of the axioms of Service-Dominant logic, Robert Lusch and Stephen Vargo, 2008

It follows that, when researching the user experience Learn Ultra provides, it is important to take into account perspectives, expectations and perceptions of both staff and students, to gain a full perspective of the service ecosystem.

Read more about the application of service design to Learn in these blog posts:

Understanding what staff and students need from Learn for hybrid teaching and learning

A service design approach to Learn Foundations

Previous Learn Ultra user research

A programme of research activities has supported each stage of the Learn Ultra roll-out. Ahead of the upgrade to the Learn Ultra interface, navigation and landing page (termed Ultra Base Navigation) at the start of the 2022 to 2023 academic year, discovery research was completed with students to learn how they sought to interact with Ultra Base Navigation and their associated needs and expectations, and feedback from staff was gained through an Early Adopter programme.

Read more about these areas of research in the related blogs:

Sketching workshops have provided insight into students’ needs and expectations for Learn Ultra

Understanding how students interact with Learn Ultra and Ultra Base Navigation

Objectives of the latest programme of user experience research

This programme of research with staff and students aimed to achieve the following:

• Test assumptions that Learn Ultra, an upgrade from Learn, offers a better user experience for staff and students
• Investigate how effectively Learn Ultra supports staff and students to complete priority tasks
• Assess the usability of Learn Ultra by students using different devices and viewports
• Validate design choices for aspects of the interface and development of the Blackboard Learn App
• Engage with staff involved in the Early Adopter programme to understand how they have been using and experimenting with Learn Ultra and feed this knowledge into the development of staff-centred training and guidance
• Identify areas for improvement ahead of the roll-out of Learn Ultra courses.

Methods used in the user experience research

Usability testing – where staff and students used Learn Ultra to complete high-priority tasks (identified in top task surveys from previous Learn Foundations research) – was used to understand how effectively the platform supported staff and students in their core uses of Learn. Where possible, usability tests were carried out using real course content, however, since Learn Ultra courses had not been rolled out at the time of the research, some usability tests were carried out with staff and students in a test environment. Semi-structured interviews were also used. These interviews were an informal, conversational way to understand staff and students’ perceptions and expectations of and responses to specific aspects of Learn Ultra. Through these interviews, it was possible to identify potential routes for improvement and gauge associated impact. Structured questions (asked of students) enabled specific design decisions to be probed, to seek evidence for validation.

Findings – student experiences of Learn Ultra

The research with students revealed an overall positive response to Learn Ultra, students were easily able to complete the top tasks and most students said they would use the Blackboard Learn App. Some students experienced difficulties differentiating between aspects of the user interface and there were comments that the route to obtain assessment feedback was not as smooth as they had expected.

Read more detail about the research with students in the related blog post:

User research with students has helped guide the Learn Ultra implementation

Findings -staff experiences of Learn Ultra

As for students, staff perceptions of Learn Ultra were positive on the whole – staff were able to complete most high-priority tasks with ease, there were positive comments about the less hierarchical flatter structure of the course content and some staff preferred the Learn Ultra discussion boards to those in the original version of Learn. Several areas were identified where improved training and guidance would support staff to make best use of the Learn Ultra features and functionality – these related to structuring and styling content, setting up groups and using updated Learn Ultra communication tools like the discussion boards and activity stream.

Read more detail about the research with staff in the related blog post:

Staff user research has helped shape and steer the Learn Ultra upgrade

Addressing the findings of the staff and student research

Following the research, all findings were collectively analysed, with members of the UX team and the Learn project team working together to identify priorities and decide on courses of action. Resulting actions fell into several categories:

• Confirming the course template for the 2023 to 2024 academic year through consultation with Learning Technologists
• Developing guidance and resources (such as an Image Bank) to help staff create visually-pleasing yet accessible course content
• Focused workstreams on assessment, including a holistic review of assessment guidance, and engagement with governance groups to address matters around Own Work Declaration
• Improving guidance (such as The Good Practice Guidance) and training courses (such as ’10 things to get started in Learn Ultra’) in response to insights and questions raised by staff
• Creation of specialised guidance and training to support staff in particular aspects of course design (such as use of folders and learning modules) and migration of existing courses to Learn Ultra
• Collaboration with Blackboard to influence improvement of areas of the user interface in response to staff and student feedback.

Further detail about each of these actions, and the findings that prompted them, can be accessed in the blog posts relating to staff and students research.

Further UX research planned for the 2023 to 2024 academic year

As iterative development on Learn Ultra continues and the platform continues to evolve, there is a need for continuous UX research to maintain a good understanding of what staff and students need and expect from Learn Ultra. Research has been scheduled for the start of the 2023 to 2024 academic year, when Learn Ultra courses will have been applied across all courses and both staff and students will have experienced using it in an authentic teaching and learning context.

Areas to be investigated will continue to be added as more insight comes from the Learn project team working with staff and from the Early Adopter programme, however, initial areas identified for further UX research include the following:

• Staff requirements and experience of using Learn Ultra discussion boards and groups
• Students’ experience of Learn Ultra discussion boards and groups
• How students interpret and use course template items to navigate to content
• Student responses to different banner image options
• Students’ understanding and interpretation of iconography in the Learn Ultra interface
• How students find content in the context of different course structures (for example, navigating between learning modules and course folders).