In today’s blog, Laura Cockram from the Research Support Office’s Knowledge Exchange team takes us through her experience of the Engage Academy, run by the National Coordinating Centre for Public Engagement (NCCPE) in Bristol.
Last month I attended the NCCPE’s Public Engagement event with the Research Academy in Bristol. I’m part of a cohort of 30 engagement professionals doing a 9-month programme on the culture, strategy and practice of engaging the public in research.
We had a really useful session with Sophie Duncan, NCCPE’s Deputy Director, on that thorniest of questions – how do you (meaningfully) evaluate public engagement with research? The stand-out points for me were:
- Evaluation is a really helpful tool to help you and your project team test your theory of change throughout your project.
- Evaluation is not just a reporting chore. It helps you learn from what you do and make it even better next time.
- Evaluation does not have to be via questionnaire! In fact, it might even be a fun activity.
Evaluation is useful throughout your project
How many times have we quickly knocked up a questionnaire to ‘evaluate’ an event we are doing with a group of the public at the end of a project? Sophie invited us to look at evaluation in a different way – throughout the project, and as a useful feedback mechanism to test your theory of change. Your theory of change is the crucial bit. What assumptions are you making when you anticipate your outcomes e.g. About what people will do next or about what they will learn? How will you test whether these were in fact correct, and where you may have got it wrong? How will. This inform decisions adjusting your research questions or activities?
Evaluation is valuable for learning, not just reporting
The ‘e’ word doesn’t usually make my heart sing; but seeing it as a tool for learning makes me feel way more interested. We distinguished between formative evaluation, that is the questions you ask at points during the project, and summative evaluation, that is the questions you ask at the end. Formative evaluation gives you the information you need to improve the project as you go along. So, when you evaluate your event, it’s with a view to making it even more effective next time. Summative evaluation gives you the information you need at the end of a project to say whether, or the extent to which, your project met its objectives.
We were also (gently!) reminded of the difference between monitoring and evaluation. Often, evaluation forms focus on counting how many people attend your event and how many are women/under 25/wearing purple… that is monitoring!
Evaluation methods beyond the questionnaire
We learnt about a huge range of creative ways methods appropriate for all sorts of audiences. This bit was fun! Here are a few of my favourites:
- Photograph diary. Participants take photos of any moment they think is important during the activity. Use these to have a group discussion
- Graffiti wall. Pin a large piece of paper to the wall and invite participants to write or draw their comments, observations or thoughts. You can leave it free-form or specify some questions or areas where you want people to focus.
- Opinion postcards. Provide postcards with specific prompts, such as “I feel…” or “What did you think of the event?” Ask people to fill them in and stick them on wall / post in box.
- Thought boxes. At the start of an activity, ask everyone to write down their thoughts about a specific question and post them in a box. Take out a few and use them to start an ice-breaker discussion with the group (comments are anonymous). At the end, ask the same question on your questionnaire to get a before and after view.
- Mystery shopper. Brief some of your participants beforehand to test out your engagement activities. Did they work as you hoped?
Looking for good resources, advice or training?
The College or Arts Humanities and Social Sciences Knowledge Exchange and Impact website is a treasure trove of useful information and resources. In addition the NCCPE have a really useful section on evaluation on their website. You might also be interested in the public engagement skills training offered by the Beltane Public Engagement Network.
If you are planning public engagement with research activities as part of your project, and would like to discuss your ideas, there are a number of resources and expertise at your disposal!
Best wishes,
Laura
Laura Cockram is the Knowledge Exchange and Communications Advisor for the College of Arts, Humanities and Social Sciences. She is currently on secondment at the Scottish Parliament.
- https://www.publicengagement.ac.uk/work-with-us/cpd-training/public-engagement-academy-2016-2017
- https://www.publicengagement.ac.uk/plan-it/evaluating-public-engagement
- http://www.beltanenetwork.org/opportunities/beltane-training/
More from this author
