This post is all about identifying key evaluation stakeholders and setting up Evaluability Assessment workshops. And it also tries to answer the overarching question: Why is all this important for Knowledge Mobilisation?
What are Evaluability Assessment Workshops?
In the last blog post I introduced how we scope and plan for evaluations at the very start of an evaluation project (the “Evaluability Assessment” phase).
I focused on the “3 Questions” which we ask the local government to help us understand more about the aims and purpose of the evaluations and the intended audience.
In the next step of the Evaluability Assessment, we run workshops with key stakeholders of the particular intervention we aim to evaluate. The workshops have several purposes. They help us:
- further refine the logic model and evaluation questions
- critically appraise evaluation approaches
- identify, as well as prioritise, knowledge mobilisation opportunities.
In a nutshell, the workshops are an opportunity to co-create a starting point for the evaluation protocol, taking into consideration local knowledge, expertise and (potentially) competing priorities. In doing so, we aim to produce an evaluation that key stakeholders have had an opportunity to shape, and are interested and invested. We want them to feel it is useful, so that the evaluation findings will ultimately be used. This chapter from Patton (2003) underlines the importance of “utilization-focused evaluations”.
However, rather than focusing on the content and purpose of these workshops, I would like to focus on the attendees – the “key stakeholders”.
Who are the “key stakeholders”?
I have always found “stakeholder” to be a vague term, and it can mean different things to different people. What we, as an evaluation team, mean with the term could be very different for the local government and vice versa. To get everyone on the same page, I have developed a prompt/checklist for the local government to use when identifying key stakeholders to invite to the workshops.
The importance of flexibility
Of course, the nature of each intervention, and the culture, context and existing relationships in each local government is different. So we remain flexible, especially regarding the approach to the workshops. We adapt our approach by asking the following questions:
- Should the workshops be online or in-person? Or should we do one of each?
- In person workshops are much higher quality, but sometimes the sacrifice for this can be reduced attendance.
- Who should send out the workshop invites: Are people more likely to attend if the invites come from someone they know (e.g. the key contact at the local government) or from an independent research institution (i.e. us!).
- We have had a mix of preferences here. A middle way could be for the key contact at the local government to send out a stakeholder briefing written by the research team.
- Has the local government had events or workshops with a mixture of service users/residents, and manger staff before (i.e. everyone in one room)? What is their experience with this? What is their preference moving forward? How could this inform the Evaluability Assessment workshops?
- Sometimes the local governments prefer keeping things separate. For instance, local governments have mentioned challenges in navigating power dynamics, or challenges in getting everyone on the same page for nitty-gritty/“technical” discussions. In these cases, we can run different workshops with different groups.
And why is this relevant for Knowledge Mobilisation?
There’s lots of reasons!
- Organising and running the workshops is a great way to refine information on key audiences Whilst the 3 Questions help identify key audiences, the EA workshop process helps connect with them, build relationships, and further understand their evaluation aims and contextual factors.
- The workshops have the potential to generate buy in and interest. This helps increase the likelihood of key stakeholders engaging with the evaluation, and ultimately the outputs and recommendations.
- The workshops are an opportunity to directly ask about Knowledge Mobilisation strategies. I tend to explore things like: preferred knowledge mobilisation outputs (e.g. report, infographic, slide deck), available communication channels within and beyond the local government, key contacts for engagement and dissemination, any key events or timepoints they would like (preliminary) results to feed into (e.g. a community event, a recomissioning deadline, conferences)…
