UX techniques tend to be more successful the earlier they are applied. Striving for a more proactive UX Service strategy, I have been considering the practicalities of incorporating user-centred approaches at the point of software purchase.

Managing the University’s UX Service, my remit is to guide the best possible experiences of students and staff when using our digital products, systems and services. Being part of the Information Services Group, we are often contacted by colleagues involved in the rollout of new and updated software, to help ensure technical changes have a positive impact on student and staff experiences, rather than the adverse effect.

How soon is now? Is procurement too early for UX?

Our consultancy approach means we can offer help at any stage of a project or product lifecycle, and I am often asked ‘when is the right time to involve UX? My response often drifts to a mantra from one of my favourite laptop stickers: ‘You should have invited me earlier’ (courtesy of Content Design London). Reflecting on past projects we have worked on, the earlier UX are brought in, the more time we have to research and understand the needs and expectations of the users concerned, and make appropriate recommendations for the best ways to develop and configure software in response.

So far in my time managing the UX Service, however, we have not been directly involved at the procurement stage, and I have wondered about both the value and the practicalities of involving UX at the point of software purchase. On the one hand, procurement seems like a tantalising stage to be involved, offering the chance to make the right decisions from the start, therefore ensuring the procured technology supported and exceeded user needs and expectations proactively, rather than trying to retro-fit further down the line. On the other hand, to do UX well requires a clear sense of the priority tasks that a product, system or service will enable, and a unified commitment to these priorities, taking into account both the needs of the team responsible for managing the software, and the needs of the people using it. Knowledge of and alignment on these priorities might be in place in the case of replacing long-running software, but not necessarily in the case of upgraded or new software offering the promise of extra or different functionality than before.

I recognised that learning more about this area could help me advise prospective users of the UX Service on the best UX approach for their individual projects and circumstances. I therefore resolved to try and learn more about procurement to help me understand how UX might fit with it.

Difficulties drafting a best practice guide with the UCISA Procurement Group

In 2023, Rob Moore, chair of the UCISA Procurement Group reached out to the UX Group I co-chair to collaborate on creating a best practice guide for User Experience in Procurement. We wanted to address tendencies for suppliers to treat UX as a ‘pass/fail’ criterion, often with little or no actual testing with those intended to use the product being procured. The goal was for the guide to be available for procurement professionals, to ensure user experience was considered and accounted for appropriately. We found it straightforward to make the case for incorporating UX in procurement (increased efficiencies, reduced costs and efforts spent re-engineering and finding workarounds post-procurement), however, our guide fell short when it came to defining practical advice for procurement professionals to follow. Without knowing details of the particular software being procured, how could we provide abstracted tangible pointers that were useful to help procurement colleagues establish user needs, understand user journeys and define meaningful metrics – so that all of these factors could be included in a scoring process that would apply in a universal sense?

Accessible procurement: Jisc event to explore challenges and identify collaborative solutions

In June 2024, the University of Edinburgh hosted a workshop facilitated by colleagues at Jisc, aimed at examining opportunities and barriers for innovation and growth of accessible digital and AI technologies in Higher Education as part of the Accessible Digital Futures project. The event was attended by colleagues from the University, including staff from procurement, Schools, and other parts of ISG, as well as people from other Scottish institutions. I was interested to take part in this event, as I recognised that learning about aligning accessibility with procurement could help me ideate about making UX considerations at this stage, too.

Learning how accessibility is built into procurement at the University of Edinburgh

Viki Galt, Head of Disability Information gave a helpful overview of how accessibility testing is carried out as part of procurement at the University. The process includes stages such as:

• Working with project teams to either select a standard set of questions to ask potential suppliers at procurement or develop a customised question set
• Gaining access to a test system from potential suppliers that Viki’s team can use to carry out both automated and manual accessibility tests (and score them based on test results)
• Asking potential suppliers about their support for specific pieces of accessibility software used by the University (including, among others, JAWS and Dragon)
• Liaising with potential suppliers about improvements, building accessibility commitments into contractual agreements, and implementing continuous improvement cycles.

Some of her reflections on the existing process included:

• It was difficult to score supplier products quantitatively (as was required by procurement scoring spreadsheets) given much of the data gained from accessibility testing was qualitative
• This process was resource-intensive, and often difficult to execute within the required procurement timescales
• Having good working relationships with procurement colleagues helped the process.

UX-related themes arising from break-out discussions

Viki’s talk was followed by several lightning talks giving a sense of the evolving software procurement challenges in Higher Education – notably the rapid expansion of AI technologies, changing procurement regulations and innovation fatigue (typically associated with large digital business change programmes). We joined break-out groups to think about collaborative solutions to the challenges identified applying the lens of accessible procurement as a potential solution approach. Several ideas and themes emerged from the discussions which helped me understand the potential for UX within procurement:

Putting user needs front-and-centre

Having identified user input as crucial to an accessible procurement process, there were several ideas about how to involve users. Having evaluation panels of willing participants was a suggestion, however, this would require management, and possibly recompense for the participants. Questions of the right time to involve participants were also raised – it was felt beneficial for them to be involved at the start to define requirements, but also for them to have input at later stages, to practically test out playground/sandbox products from suppliers. Making data gathered from user research into student and staff needs available in a central repository or insights bank was also suggested, so that this data could be referred to when procuring new software in the future.

Collaborative sharing and frameworks

Recognising the shared challenges of achieving accessible procurement within Scottish Higher Education, ideas around knowledge-sharing (for example, results of software accessibility tests), and adoption of common frameworks were an attractive prospect to increase efficiencies and enable closer collaboration. Data sharing between institutions using the same types of software could encourage working as a collective, to influence suppliers as a consortium. Caution was exercised over a ‘one-size-fits-all’ framework, however, recognising the nuanced needs of individual institutions. Furthermore, it was noted that the effort required to maintain up-to-date repositories of testing data sufficient to support multi-institutional needs and timescales could present its own set of challenges.

Learning from work of colleagues at other universities

In November 2024 the UCISA UX Group ran a webinar on the topic of applying UX expertise to procurement. We included a talk from Chris Sherwood, Quality Assurance and UX Lead in Digital Services at the University of Swansea, with the title: ‘Get in early: UX and Evidence-based procuring’.

Chris began by noting the multi-year impact software procurement had – therefore making the case to involve UX considerations in the decision-making. He had taken part in several procurement exercises and shared the different UX approaches he had used in each, reflecting on which were most successful, and showing there was always a way to incorporate UX into procurement at some level, even if this was later in the process than was desirable.

From Chris’s talk and the discussion that followed, I identified several factors that help ensure user-centred procurement:

• Support and buy-in for UX from the procuring team
• Involvement of UX practitioners in the procurement meetings and discussions
• Access to a sandbox/trial software environment that can be tested by users (beyond the sales demo)
• Knowledge of software configurability and adaptability (system constraints)
• A clear shared sense of the user tasks the software is to support
• Availability of heuristic principles and system scores (to support not substitute user testing).

Pulling together what I’ve learned so far

Revisiting my original aim to learn more about procurement and the potential to embed UX within it, I am convinced of the value of ‘getting in early’ and feel positive about the opportunities to incorporate user-centred considerations at the software purchasing stage. There are several learnings that I will take forward.

Aligning UX and procurement is a similar challenge to aligning UX and Agile

Thinking about how to fix UX into the process of procurement reminded me of my previous work developing and iterating UX approaches to align with existing software development processes, documented in my blog post from 2023:

Making Agile and UX work together – reviewing the UXD process for the Web Publishing Platform project

Procurement processes can be rigid, so UX approaches need to flex

UX as a discipline is very experimental, meaning that there are a range of design approaches and techniques to choose from for any given scenario. Conversely, procurement processes are often fixed and well-established. Therefore, for UX to have an impact on procurement, it is likely to be necessary to select and tailor an appropriate UX approach for the circumstances. To keep things moving forward, regular feedback loops need to be established between UX work and procurement decisions, to ensure user-centred knowledge is informing the choices being made in the process.

Understanding what students and staff need from software is good preparation for UX

Procurement may occur at any time, and there may not be the opportunity to carry out user research of associated needs once the process has begun. To avoid UX being reactively considered, and potentially becoming an ‘after-thought’, a baseline understanding of the current state of staff and student needs when it comes to key pieces of software could help to guide the start of the process. At the very least, it would be useful to understand ‘who uses what and why’, so that this information can be drawn upon in the event of software being replaced and updated.

Knowing the priority tasks supported by software can help guide a shared outcome

Many decisions need to be made during procurement, and there are lots of pieces of supporting data and scores collected to guide informed choices. Having a clear sense of the core user tasks the software is intended to support presents a powerful prioritisation mechanism, to ensure the attention of the procurement is not pulled away from these core tasks towards new software features included in new releases or demos. The Top Tasks survey method (pioneered by Gerry McGovern) is a useful way to obtain a user-informed set of priority tasks for a given product or system.

Read how the UX Service successfully used the top tasks method to understand students’ needs from Learn during Covid-19 in this blog post from the Learn Foundations project:

A top tasks survey has shown what staff and students prioritise in hybrid teaching and learning

Having meaningful UX metrics to assign scores could help user-centred procurement

The process of procurement is about making reasoned choices based on the data available, with final decisions coming down to weighing up scores. Putting numbers on aspects of UX is always difficult – as user experiences are typically gauged qualitatively, by examining human behaviour, sentiment and patterns of interaction. In order for UX to be included as a criterion for consideration in procurement, however, I could understand the logical need for UX measures to carry weight alongside other factors, and to add in to the final decision-driving scores.

What’s next – what I would still like to learn

From what I have learned so far, I feel I have enough knowledge to assist with bringing UX considerations into a procurement process. To help me to do this in the best way, however, there are certain areas I would like to continue to understand a bit better.

Obtaining a deeper understanding of procurement processes – through examples

Meeting procurement colleagues through Jisc and UCISA I have been able to establish that in procurement processes, details count, and the complexities of multiple, interconnecting stages exist for good reason, having been previously established over long periods. It was difficult to appreciate the intricacies of these processes in the abstract (for example when attempting to write good practice guidance with the UCISA Procurement Group), however, when considering real-life examples (for example, provided by Chris Sherwood in his talk), the reasoning became clearer. Becoming more familiar with procurement processes applied in the University context would enable me to familiarise with suitable opportunities to embed UX approaches more effectively, to facilitate the application of an evidence-led approach to selecting appropriate UX techniques for given scenarios and circumstances.

Improve my knowledge of potential UX metrics, data and indicators

I have a good working knowledge of universal usability measures like the System Usability Score, however, I have always been skeptical about their use to assess user experience. Bearing in mind the importance of numbers and scores in the procurement process, however, I would like to apply revised thinking to areas like heuristic principles and score-sets to examine the potential to apply weighting or scoring to these to use them for the purposes of UX assessment. I would also like to consider the forms of data collected about University software use through a UX lens, to assess what, if anything, this may indicate about user experiences of the software.

Further my grasp of prioritisation mechanisms and frameworks

With so many factors to consider and scoring options to take into account when purchasing software, prioritisation is a critical aspect to keep any procurement process moving forwards. The same applies to practising UX, when deciding on the most important aspects of UX research data to act upon and respond to. The top tasks method is a powerful mechanism, enabling prioritisation on the basis of the key actions people want to perform with software, but other frameworks for prioritisation could also be useful to know about, both in order to flex UX to align with procurement processes and also to enable the most important factors to be surfaced to guide objective user-centred decision-making with minimal bias.