Undergraduate Open Day pop-up research
Our team went on campus during the recent Undergraduate Open Day to do some pop-up research and check some design concepts and live services with prospective students and their parents.
Events like these are great opportunities for us to bring the team together and get everyone closer to our target users.
One of the more onerous aspects of my role is participant recruitment to take part in user research. So when we have 4000+ of our target audience visiting campus for the day, it’s too good an opportunity to miss.
Pop-up testing is the perfect approach.
Preparation
We spend the week before getting everyone up to speed on what we would be doing on the day. We have different levels of experience doing this kind of research across the team, so this was a chance to offer others to get more experience and confidence to go out and do this kind of research themselves.
As well as agreeing the focus and goals of the tests with various stakeholders, and then designing the test scenarios with Neil, I needed to organise a schedule so everyone knew what they were doing when, and run a demo session with the team so everyone could get some confidence with the scripts.
In addition to our team, we had a colleague from Student Recruitment and Admissions join us, plus colleagues in user experience and design roles from other teams observe to help them prepare to run testing of their own.
What we tested
For our pop-up research activity on campus, there were multiple areas that we wanted to investigate. The new degree finder requires a number of changes to its design, so this was an opportunity to put as many ideas as we could in front of prospective students and see what they thought of them. Over the course of our time on campus we were able to test:
- In-page navigation concepts – how would students interact with a potential new feature on desktop and mobile that supports navigation on long pages of content?
- Content format concepts – would new styling and spacing of subheaders and paragraphs make scrolling and scanning behaviour more effective and efficient?
- Entry requirements – wording – would students be able to better differentiate between different types of entry requirements and identify what applied to them?
- Virtual Visit – we wanted to explore the perceived value of the University’s Virtual Visit platform for open day visitors, and identify any big usability problems that might be hindering engagement
How long it took and participant numbers
Pop-up research is typically a quick way to test and get insight into a specific area you want.
At the Open Day we operated in 4 pairs and over the course of two hours on campus we managed to hit or exceed our recruitment targets:
- In-page navigation – tested with 5 participants
- Content format concepts – tested with 11 participants
- Entry requirements – tested with 7 participants
- Virtual Visit – tested with 11 participants
We worked in pairs to approach students and ask them to interact with our online services using phones and iPads
This was a fantastic result, exceeding my expectations. I had planned for an hour or two extra in case progress was slower, but we managed to get back to the office and begin the collaborative review process because we’d worked so fast.
Undoubtedly, the lovely sunny weather helped us. Lots of people were sitting about between open day sessions, just enjoying the weather which made it very easy for us to approach them and ask for a little bit of their time.
But I’d say that weather aside, people at events like this are typically very receptive, so long as you’re not interfering with their schedule. When the weather isn’t so great (which is a lot of the time in Edinburgh) the abundance of cafes and indoor meeting spaces across the campus are also very productive locations for pop–up user research.
Getting access to real prospective students
At many points in the year, it can be difficult to get access to prospective students. We are fortunate that we have lots of on-programme students available to test with for much of the time, but there is added value in being able to test our ideas with actual prospective students.
This is because they are at the stage in their journey where they are interacting with our services. They can give an opinion on something that they are currently using, and draw on experiences happening right now, whereas our current students will often have to recall what they were doing or thinking up 18 months previously.
This is especially relevant for something like Virtual Visit, which is a service aimed at prospective students to help them get an understanding of what the campus and facilities are like and the city itself.
Collaborating with other teams
I’ve noted how we were testing Virtual Visits as part of our research. This is a service managed by our colleagues in Student Recruitment and Admissions.
Not everyone has the experience or confidence to put together a pop-up research activity, so when we do, we try to involve others outside our team. This Open Day, we were able to work with and support our colleagues in Student Recruitment and Admissions.
We worked with them to put together a usability testing guide where we could identify the things they wanted to learn about their service. We were also able to work with a member of their team, so they could not only see how users were interacting with their service, but also get experience to hopefully feel more confident in the future to put an activity like this together for their own team.
We’ll be coming together with various stakeholders in this service in the coming month to run a collaborative playback session, so even more colleagues can see first-hand how prospective students interact with the Virtual Visits website.
Over 4000 students and parents were on campus to learn more about studying at Edinburgh. It was a perfect opportunity to trial a variety of services and features we’re working on.
We learn quickly how we should update our designs
In the afternoon, we run a playback session on the area we were testing. We watch the videos of what we tested and together we identify and prioritise the top usability issues that we see. As a team, we discuss these issues and put each one through a flow chart that helps us understand if the issues should be classed as a low or high priority.
Once we have our issues prioritised we are in a position to start thinking about what decisions we need to make to the design we tested. This process is a core part of how we work because we work in an iterative process where we test something, learn something and make the required changes to the design so we can test again. This ensures that whatever design we hand over to be developed is done so with the confidence that it meets the needs of the user.
We tested in a number of different areas this time, so we did a playback for what we learned around how students interacted with our new entry requirements design. I’m setting up further playbacks for the days and weeks after the Open Day.
The results from our wording changes for entry requirements were incredibly positive as students all made the right choices based on their circumstances. So this was a really nice way to round off the day together.
The Open Day pop-up research was a great success as we gained insight into a number of different areas but were able to support another team in the University to get insight into something they wanted too.
Interested in pop up testing?
If you’re a colleague in the University and are interested in trying this approach out for yourself, feel free to drop me a line, and I’d be happy to chat through your approach.
1 reply to “Undergraduate Open Day pop-up research”