I tagged along with user researcher Nicola Dobiecka on her quest to understand the postgraduate research (PGR) application experience to discover the points where user research and analytics overlap.

Background

As part of the overhaul of the University’s undergraduate (UG) and postgraduate (PG) degree finders we are undertaking user research to better understand what prospective PGR students are doing and how to enhance their experience.

Blog posts about the degree finders project

One of the main questions of this research is whether the postgraduate degree finder is the best location for information on PGR programmes.

Some might think that this could be answered by just looking at the user behaviour on our website. But analytics on its own cannot answer this question.

Why? Because we can’t collect data on the reasons behind users’ actions.

A common thing I say to people about analytics is that it can tell us what is happening, not why it is happening. The why is exactly what user research tries to answer by having conversations with the humans behind the processes we aim to understand.

So, how can analytics help user research?

Analytics and user research can, and should, be used in parallel to gain insights into a process and inform the goals of any human-centred design project.

Before user research begins

At the beginning of the user research process, we need to define the goals and objectives of the project and decide on:

• the scope and dimension of the project
• who we want to speak to and why
• how many interviews or discovery sessions to conduct

At this stage, analytics can be used to:

• get a general idea of what the process looks like
• understand average user behaviour
• scope out the dimensions and distribution of our audiences
• identify bottlenecks in the process

This information can then be validated or questioned by user research.

Analytics questions in the PGR research discovery phase

During the discovery phase of the PGR research project we had some general questions about PGR provision. We then came up with a subset of questions that could help inform our decisions using data available to the team.

What are the dimensions of PGR provision?

• how many PGR degrees are available at the University?
• how many PGR applicants and applications does the University get every year?

What do we know about general user behaviour?

• how many visits related to PGR do we get on the PG degree finder?
• how long are users spending reading PGR pages?
• what do users do after looking at a PGR degree finder entry?

Can we find any bottlenecks in the process?

• are there pages that lead to many enquiries?
• are there pages where users are doing U-turns?
• are there pages that should be easy to read but have a long time on page or vice versa?
• how many enquiries about PGR programmes do we get each year?

Who is our audience?

• how many staff deal with PGR applicants/applications?
• how many students are currently matriculated in PGR programmes?
• how many PGR students are there across colleges and schools?

What is the process life cycle?

• how often are programmes updated?
• is there a standard procedure for programme updates?
• how much has the information on programme pages changed over the past three years?
• what is the application process? Has it changed in the last three years?

During and after user research

Once we begin user interviews we start to get an understanding of the reasoning behind user behaviour and the logic behind the current application process. We also begin to confirm the information gathered during discovery sessions.

Ideally, we want to interview enough individuals to get a reliable account of what the PGR application process entails but there are always limits in time and resources.

With this in mind, we need to take measures to avoid introducing bias into our research by making sure that the information we are getting from user research is accurate. Here, we can use analytics to confirm or verify information we gather from our research.

As an example, if someone says that they think most PGR students and applicants are international students we can check our applications and admissions records to verify the accuracy of this statement.

I’ve heard from a couple of staff that the vast majority of PhD students are international. I wonder if there’s any way to see if this is the case from the data we have?

The information we gather from both analytics and user research activities can be then used in the decision-making process at later stages of a project. For example, it can help build a foundation for a content model or help create personas in the development phase of a project.

My takeaways

My short foray into user research has taught me that validating user research and analytics information is a two-way street and we should communicate more with each other to check that we’re not introducing bias into our research and analyses.

I also learnt that there is a lot more workload involved in user research than I expected. And, though I shouldn’t have been surprised by this, there is a lot of talking in user research!

I am in awe of colleagues who can do four user research interviews in a day. I was exhausted after one! Though I’ll happily look at a spreadsheet for 8 hours of my day.

Another takeaway from this collaboration was the realisation that results and outputs from user research don’t show the enormous amount of effort it takes to produce them. A striking similarity with analytics work!

This is because making complex data and processes understandable to a variety of audiences requires a lot of time and effort. So, I’ll leave you with my favourite approach to making sure anyone can understand my work, and you’re welcome to use it.

Anytime you’re trying to gauge if your work is readable and digestible, ask yourself: would my 96-year-old gran get this? If not, then keep working on it until the answer is yes.

For more about this project, read Nicola’s blog post:

Exploring the prospective Postgraduate Research application experience

If you have any questions, please get in touch with me

Carla’s contact details

When analytics met user research… / Future student online experiences by blogadmin is licensed under a Creative Commons Attribution CC BY 3.0