Evaluating Workshop Outputs

Evaluating the qualitative outputs from co-design workshops is a nuanced process that hinges on understanding the context and the participants involved. I hope my workshops will be characterised by their collaborative and participatory nature, yield rich, descriptive data that can provide deep insights into the pupil’s experiences, needs, and desires. To effectively assess these outputs, one must adopt a multi-faceted approach.

In my reading I have discovered a few important aspects to this:

Firstly, thematic analysis is key; it involves identifying patterns and themes within the qualitative data. This requires a careful and iterative process of reading through the data, coding it, and looking for recurring ideas or concepts. Secondly, the validity of the findings can be strengthened through triangulation, using multiple data sources or methods to cross-verify the results.

Moreover, participant feedback is invaluable. Engaging with the pupils to validate and refine the findings ensures that the interpretations are aligned with their perspectives. One question for is time – will I have time to engage in this process fully?

In conclusion, a robust evaluation of co-design workshop outputs demands a thorough analytical approach, corroborated by participant engagement, to derive meaningful insights that can drive for my project.

Selecting Participants

As I begin to put my plans into place I want to ensure that I am reaching the right participants. Selecting the right participants is crucial for the success of a co-design workshop. Best practices suggest a diverse group that can bring different perspectives and expertise to the table. It’s important to include users, staff, volunteers, and other stakeholders who are actively involved in the co-design process. The Interaction Design Foundation recommends that designers act as facilitators, guiding participants through the design process to harness collective wisdom and insights, especially from end-users. This is crucial for my project.

The importance of including people with lived experiences and considering various types of diversity when assembling co-design teams is also key. The role of the research facilitator in maintaining momentum and ensuring that participants view themselves as co-owners of the intervention, which fosters a sense of engagement and ownership is important and one that will need thought. Effective participant selection not only contributes to the creation of more inclusive and useful outcomes. Therefore, carefully planning the participant mix by considering these best practices can significantly impact the effectiveness and efficiency of the co-design workshop.

The question for me is how do I ensure the above happens?

Beginning to narrow in on my project

a group of people crowded around an indoor market stall.
A stall at a Futures Bazaar (via)

Having had helpful conversations over the last couple of weeks, I’m beginning to develop a more concrete plan, particularly with regards to the focus of my work.

The future of Scottish qualifications is going to provide the context, specifically the recommendations set out in the Hayward Review. What I want to do is use critical design methods to elicit young people’s views about the recommendations, but more generally what they think they should leave school with. In doing so, I want to be able to critically evaluate three of these methods and how they might help young people articulate educational futures.

I had initially thought I would concentrate on a single method. Recently, provotyping looked promising. However, on reflection, I think a range of methods might be best. To this end, I want to look at:

  1. Provotyping
  2. Design Fiction/Narrative work
  3. Co-creating speculative objects with the young people (perhaps using a model similar to Stuart Candy’s Futures Bazaar)

The first two would be created by me. I imagine the workshop might revolve around examining the artefacts and using them as stimuli for semi-guided conversations. For the third workshop, I would use the same semi-guided conversation approach but would do so while creating the speculative objects. Practically, I imagine I would involve a different group of young people for each method. It’s worth noting that this potential approach will require careful thought about the selection of participants.

Participatory Provotyping

a woman works at a desk surrounded by craft materials

In what ways might participatory provotyping help young people articulate preferred educational futures with particular reference to the future of qualifications in Scotland?

While it’s a clunky question it is a first step on the way to my research question. Looking back over my blog posts it’s starting to get at the issues I want to explore namely:

  • Using participatory speculative methods with children and young people in order to help them articulate education futures
  • Creating tangible diegetic works alongside children and young people
  • Using outputs from these methods to inform policy
  • A grounding in a live educational policy issue within Scotland

What remains in terms of fleshing out my project is the specific area of focus. Is this a project about critically examining participatory provotyping as a methodology or am I more interested in exploring the outputs as they relate to the future of Scottish qualifications?

 

Children & Young People as Protagonist

Drawing together a few strands of my thinking; when it comes to using design to help children and young people articulate their educational futures, a key element should, I think, understand them as protagonists (Ref: 1, 2, 3).

I’m drawn to this approach (see my previous post) as it connects to two important policies relevant to my context of Scottish education.

The first is the Scottish Approach to Service Design (SAtSD). My project would sit in the first of the “double diamonds” (4) and the SAtSD framework provides a good ethical approach to my work (and crucially, one that is endorsed by central government). The SAtSD sets out 7 principles for using the approach. Principle 4 states:

4. We use inclusive and accessible research and design methods so citizens can participate fully and meaningfully.

In order to do this I am drawn to Education Scotland’s policy document Realising The Ambition:  Being Me . This document sets out guiding principles for working with young children, placing emphasis on their autonomy within their environment:

We often talk about the environment in terms of physical spaces, but the key part of the environment for children is the human, social environment of positive nurturing interactions. Experiences are also part of the environment. Children need to learn things for themselves, but this does not mean they should always do so by themselves. We can, by following and building on children’s motivations and interests, support young children to make the most of the environment for learning and development.

Connecting these two ideas; the research and design methods I use in my project must allow children and young people to participate fully (i.e be the protagonist). In order to this I must allow them to learn for themselves, building on their interests and motivations, rooted in their own environment. The question then becomes how? Which design methods might I use and how might I use them?

1 Iversen, O.S., Smith, R.C. and Dindler, C., 2017, June. Child as protagonist: Expanding the role of children in participatory design. In Proceedings of the 2017 conference on interaction design and children (pp. 27-37).

2 Clark, Alison. “Young Children as Protagonists and the Role of Participatory, Visual Methods in Engaging Multiple Perspectives.” American journal of community psychology 46, no. 1–2 (2010): 115–123.

3 Schepers, Selina, Katrien Dreessen, and Bieke Zaman. “Rethinking Children’s Roles in Participatory Design: The Child as a Process Designer.” International Journal of Child-Computer Interaction 16 (2018): 47–54.

4 Double Diamond graphic showing two phases. One - Designing the RIGHT thing (Discover and Define). Two - Designing the THING right (Develop and Deliver).

Ethnographic Experiential Futures

I was struck by Candy and Kornet’s use of the term Ethnographic Experiential Futures to describe a methodology that aims to:

invite engagement with diverse participants [1]

In their article they go on to describe a framework [2] for the methodology. What was particularly interesting, from the point of view of my project, was the attempt to use this framework to

prod at the traditional overreliance in the field on words and corresponding underutilisation of other media. [3]

As the aim of the project is to use design research methodologies to help children and young people articulate their preferred education futures this provides another way to engage participants – avoiding the potential problem of using methodologies inappropriate to their age and stage.

 

 


Turning Foresight Inside Out: An Introduction to Ethnographic Experiential Futures Stuart Candy and Kelly Kornet Journal of Futures Studies, March 2019, 23(3): 3-22 DOI:10.6531/JFS.201903_23(3).0002

2 Reference as above:

3 Reference as above.

Dream Project

Inspired by the Dream Project post idea here’s an attempt to flesh out just that.

I would like to work with several diverse groups of similarly aged children or young people. We would have two or three school days together to work  use a variety of design research methods to consider their preferred educational futures, We would do three things:

  1. Collaboratively create a speculative scenario focussed on an educational future. I would like to ground the stages of the work that follow in this co-created scenario (rather than simply presenting the children or young people with a scenario I have created).
  2. Respond to this scenario in a way that creatively articulates how they want education to be. The question remains to focus on one design research method or try and create a set of shorter activities utilising a variety of methods. The children/young people would work in smaller groups and have the opportunity to explore each others work.
  3. Create a public group display, statement or manifesto synthesising the children/young peoples work. Thereby giving them an authentic audience/outlet.
  4. A final fourth step might be to connect the different groups work to the current policy landscape. This could be done by utilising some type of trend map or final report.

An important part of this work is the diverse nature of the groups I would work with. Scotland – like any nation is diverse in all sorts of ways (geographic, cultural, socio-economic) there is an obligation to present the type of future forecasting I’m considering here in a way that reflects that diversity, rather than gloss over it.

This approach is inspired by a variety of sources discovered during my EFI courses and wider reading. These include:

  • Schooling, Education and Learning 2030 and Beyond (Published by Scottish Parliament & Scottish Futures Forum 2020)
  • Little Book of Speculative Design for Policy Makers (Imagination Lancaster 2020)
  • NESTA Playbook for People Power
  • THE FUTURES BAZAAR A PUBLIC IMAGINATION TOOLKIT (BBC; Filippo Cuttica and Stuart Candy)
  • The Future Everything Manual (Drew Hemment)

Making it concrete

grayscale photography of concrete blocksAs I get my project off the ground, one of the aspects I’m struggling with is context. I know generally what I want to work on, but I’m a little at a loss for how to ground it. More specifically I am trying to decide if it is best to ground using design research methods with children and young people to help them think about preferred educational futures in the general sense, or to tie it to a specific context. On the one hand having a very specific focus (for example influencing policy around a new pupil portfolio system) perhaps makes the work (for the children and young people involved) seem less abstract. On the other hand there is something appealing about keeping it general – it allows the methods to really be stress tested as it were…

Blog Task 2 – Methods & Research Design

a blank diagram of the double diamond service design model

Blank Double Diamond

What approach do you want to take to your research?

I know I want to look at how we might use design research methodology to help children and young people (C&YP) articulate their preferred educational futures. There are a few key questions I am currently working through. Namely:

  1. Do I concentrate on one method (for example using design fiction) or use a range?
  2. Do I concentrate on thinking about educational futures generally or take a specific practical element of educational policy currently being worked on? Here in Scotland we’re currently thinking how we might design and implement new forms of national assessment – specifically an e-portfolio for all learners aged 3-18. Using this approach my first question would again come into play do I use one method, or a range?
  3. More generally I have been struck by this article¹ on involving children in the design process. The authors argue for an understanding of children as protagonist and

encourage children to be the main agents in driving the design process and thereby to develop skills to design and reflect on technology and its role in their life. (Iversen et al, 2017)

In this was I think I can begin to make connections between design research and my ultimate goal of helping C&YP articulate their preferred futures.

What methods interest you?

Two two main strands are speculative and participatory methods. I think the Scottish Approach to Service Design is helpful here as it gives a formal, ethically appropriate approach to this work. Looking specifically at methods; design fiction and diegetic porotypes as well as storytelling around utopias seem promising as do more participatory methods like some of the activities from the Liberating Structures work as well as Lego Serious Play. I found NESTA’s Report Playbook For People Power a helpful overview on the practicalities of using some of these types of methods

What training or development are you pursuing to support your project and why? 

I’m very much coming at this as a teacher who is new to design research. To this end I am am grateful to have had the opportunity for some informal coaching from a colleague in the Scottish Government’s service design team who has experience of using exactly these methods in the educational futures space. I have also reached out to academics here at EFI who have been generous with time and advice.

I updated this title on 1-11-23 to make it clear the post was engaging with the research design task


¹Ole Sejer Iversen, Rachel Charlotte Smith, and Christian Dindler. 2017. Child as Protagonist: Expanding the Role of Children in Participatory Design. In Proceedings of the 2017 Conference on Interaction Design and Children (IDC ’17). Association for Computing Machinery, New York, NY, USA, 27–37.

Blog Task 1

a brutalist building with a large sign reading "the future"
Photo by Haley Truong on Unsplash

 

During the last academic year I was involved in creating and delivering the  pupil engagement sessions for the Scottish Government’s National Discussion on Education. While we had huge numbers of learners (26,000) take part and many pieces of evidence (20,000) submitted, sifting through the evidence, it was clear that children and young people struggled to articulate their preferable futures when it came to education. They had definite ideas about the liked and didn’t like about the present, but they didn’t have the tools or language to express what the future might look like.

Taking EFI courses last year showed me how design research methodologies might be one set of tools to help young people articulate educational futures. I want my project, therefore, to do exactly that. As I wrote in my last post:

Broadly speaking my interest is in using design research methods (particularly speculative design) to involve children and young in the creation of education futures. Following on from that I’m keen to explore how both how these methodologies might be usefully “packaged” so that teachers and other educators might use them, and how the outputs from might be used to inform policy making.

As I flesh this out further I have more questions than answers:

  • There is clearly too much in my initial ideas for one project – how best to narrow the scope?
  • I want to explore a range of participatory design methods, how best to do that?
  • There is part of me that is unconvinced in tying the use of these methods exclusively to “plan” the future. In other words, is a better approach to use these methods not to simply to help adults create a future policy direction, but instead to help children and young people make meaning of their present situations?