Category: impact

SAT Visited North-West Highlands and Islands

matjazvidmar/ / impact, Innovation, Outreach, Post, science, Space, teaching, Travel

Between 25th June and 3rd July, our Space and Astronomy Tour visited North-West Scotland, in particular, the islands of Skye and Lewis and Harris as well as Sutherland and Inverness. (Here are some of the past tours to Islay, Orkney and Sutherland).

This was to continue my engagement with the Highlands and Islands communities in the context of potentially expanding space industry, which is one of my research interests at the University of Edinburgh, however, this year, in particular, the visits are also to celebrate the 200th Anniversary of the birth of astronomer Charles Piazzi Smyth (CPS200) and the 50th anniversary of the Moon landings.

As such, the programme was organised as part of the CPS200 celebrations, as well as working with the local astronomical societies Highlands Astronomical Society, Stornoway Astronomical Society and Tarbert Astronomy Group, and other groups and organisations, in particular, the local libraries and community centres and the Travelling Scholars initiative.

Over the 9 days, we travelled 875 miles in total and spent 24 hours talking to our audiences. We delivered rocket and constellation making workshops at seven primary schools (Staffin, Kilmuir, Sir E Scott, Lionel, Lochinver, Achiltibuie, Bonnar Bridge) with a total of 270 pupils and 20 teachers, as well as six evening talks (Portree, Tarbert, Stornoway, Lochinver, Lairg, Inverness) with a total audience of 102. We also held drop-in family activities at Stornoway Library, called “Rocks, Stars and Rockstar Rockets”.

We spoke of early “astronomers” keeping time using stone circles and calendar stones, of 19th century industrial and maritime revolutions and all the way to the modern satellites, providing us with GPS, telecommunication and Earth Observation services. With several potential spaceports being planned both on the islands as well as the mainland in North-West Highlands, there was particular interest in the developments of the Scottish Space Industry, too, questions I attempted to answer as per my recent academic paper.

In all, over 100 (paper) rockets were made and successfully launched as well as over 150 (personalised) constellations. Deep questions were discussed, including what does it feel like to be inside a black hole and what might happen if we ever meet some aliens. Hopefully, fun was had by all and several promises were exchanged to visit again in the near future.

 

The trip was funded by IoP in Scotland Public Engagement Grant and Royal Society of Edinburgh’s Charles Piazzi Smyth 200 Anniversary Project, with generous contributions from Highlands Astronomical Society, Stornoway and Tarbert Libraries, An Lanntair Centre, High Life Highland Libraries and the Ferrycroft Visitor Centre, Davar B&B and Lochinver Community Hall, and many individuals who gave up their time to help organise and promote these events, as well as warmly embraced us with warm Highland hospitality. Thank you all!

 

Developing a Framework for Innovation Intermediation

matjazvidmar/ / benchmarking, evaluation, impact, Innovation, methods, outcomes, policy, Post, process, research, science

My exciting journey with the Innovation Caucus started one rainy morning in Spring 2017, when by chance I spotted an advertisement for internship applicants doing the rounds over email. This was followed by an email from my supervisor, asking all of his PhD students if we have seen the call and whether we were interested. Not being someone who declines any opportunity, my reply was immediate – yes!

Having found out about the Innovation Caucus and its work some months previously, when putting together a notice for the departmental newsletter about our engagement with policy, I was really excited by the opportunity to further translate my research interest into useful knowledge for policy-making. Having applied and made it through to the interview, I was ecstatic! Speaking to Tim and his team was interesting and inspiring, and once I was offered the internship, it took even less time than before to say “yes” and accept it.

As I am really passionate about my PhD research topic (social aspects of technology development and innovation) and my subject matter (Space Industry – yes, the stuff “up there”) I took quite some convincing to take on new challenges within the Innovation Caucus brief. In part, this was because I really wanted to create a new space of shared knowledge and sense-making, i.e. to challenge the theoretical concepts with empirical findings and policy realities – and I could only envisage doing so within the topics about which I was already somewhat knowledgeable.

However, in discussion with Innovate UK and the Economic and Social Research Council (ESRC), I did eventually reshape my interest into developing a broader framework and typology of innovation intermediation in any geographically bound sectoral system of innovation. This was of value to Innovate UK, since it supported the ongoing development of their portfolio of Catapults and Knowledge Transform Networks, as well as other projects and policies.

This experience was a great lesson for me, not only in working with and delivering for a policy-making system, but also in expanding my own research interests into domains I did initially find uncomfortable. Presenting the headline findings of this work at one of the most prestigious innovation conferences in the world, DRUID (2018), helped me appreciate the power of broader generalisation of academic knowledge, in order to achieve more substantial societal impact.

The lessons learned and experiences from this project also enabled me to engage better with new concepts, unfamiliar settings and unknown stakeholders in my subsequent work. For instance, these skills have proved critical in working on a consultancy project for the OECD and as a Research Assistant in academia.

I have to express my big thanks to Tim and his team for their support and mentorship and to all involved with the Innovation Caucus, particularly Innovate UK and the ESRC teams involved with my internship. It was your determination and generosity that turned this project from a 3-month desk-job into a transformational professional journey.

lxvchcxo_400x400

This post has been published in October 2018 at Innovation Caucus blog: Developing a framework for innovation intermediation.

Find out more about Innovation Caucus.

Co-creation, research work and collaboration: Developing graduate attributes through research practice

matjazvidmar/ / impact, Post, research, science, STEM, teaching

I was delighted to contribute to the Edinburgh University’s Teaching Matters blog again, writing about using research as a tool for teaching and learning co-creation.

You can find the blog here.

tmblog

From Orkney to Outer Space (Again!)

matjazvidmar/ / arts, impact, Innovation, outcomes, Outreach, Post, research, science, Space, STEM, Travel

I have been invited to deliver a series of knowledge exchange and outreach events for Orkney International Science Festival (OISF) between 6th-12th September 2018. This is my third appearance at OISF, having visited the 2015 and 2017 editions of the festival.

I am particularly excited that though some of the events I will deliver will be held on Orkney mainland I will yet again also visit some of the more remote islands in the archipelago.

Okrney-Fix

Hence, on 8th and 9th September I will be visiting the islands of North Ronalsday and Sanday, where I will speak to the local community about going From Standing Stones to Blasting Rockets: Scotland is Off to Space!

The talk is based on my extensive ESRC-funded doctoral research of the development of (“New”) Space Industry in Scotland, in particular examining emerging networks, innovation intermediaries and changes to new product development processes. With the recent announcement of UK Government investment into the creation of vertical launch capability in the North of Scotland, this is a particularly topical contribution to the discussion in many communities.

Furthermore, I have also been invited to present some thoughts on the deep connections between scientific research, arts and society as explored through my participation in the recent Social Dimensions of Outer Space network’s Edinburgh Futures Institute project. I will outline the main points from my essay on the topic in a talk entitled Universe: The Love Story, as part of the skyranMoon exhibition in Stromness on 11th September (7pm).

skyran-moon-2

Finally, in partnership with a young team of rocket engineers from Spacelink Institute, we will have a plenary discussion about the future of small-scale spaceflight in an event called Make it, Fire it Into Orbit! in King Street Halls in Kirkwall (Wednesday, 12th September, 11.30am). Details and tickets here.

As is my practice with such visits, I will also speak to pupils of local schools, both on Sanday as well as at Kirkwall Grammar School, to discuss a variety of opportunities to engage in science, technology and innovation, including studying it form a social-scientific perspective.

As such, I am delighted to be yet again complementing a variety of colleagues form the College of Science and Engineering, including Prof Peter Higgs, who will support the festival with a host of other events and activities.

I am very grateful for the generous support of my OISF 2018 visit provided by the festival, as well as Science, Technology and Innovation Studies Group at the University of Edinburgh.

 

 

When will Space Mining be a “Thing”?

matjazvidmar/ / asteroid mining, impact, Innovation, research, Satellites, science, Space, Space Exploration, space resources, Video

Earlier this year I have been asked by the BBC to do a short comment piece about space mining opportunities and development timelines.

This was to accompany an introduction to the UK’s first space mining company, the Asteroid Mining Corporation.

Here is the resulting video (via BBC Science and Environment):

The 22-year-old running UK’s only asteroid mining business

BBCnewsStartClip

 

I-SAT: Heavy on Flavours, Heavy on Space!

matjazvidmar/ / impact, LHC, Post, Space, University of Edinburgh, value

This is a slightly late update on another outreach project I was involved with this Summer. Between 10th-14th July 2016, “Heavy Flavour – Quo Vadis?” workshop was organised by the Particle Physics Group, School of Physics and Astronomy, The University of Edinburgh, at the Ardbeg Distillery on Islay.

The aim of the workshop was to the future direction of research for heavy flavour physics, so-called as it involves the study of the heavy beauty and charm quarks that are produced in large numbers at the CERN Large Hadron Collider.

In parallel with the workshop, a series of public outreach events were held at Bowmore High School between 12th-14th July 2016, organised on behalf of the Particle Physics Group by Alan Walker, Director of Particle Physics for Scottish Schools (PP4SS), and the local Science teacher, Russell Pollock. This comprised of three joint exhibitions: Particle Physics for Scottish Schools Exhibition, From Maxwell to Higgs Exhibition (Royal Society Edinburgh) and I-SAT: Islay Space and Astronomy Tour.

Poster 2

 

The latter, Islay Space and Astronomy Tour (I-SAT), was a new invited project I designed specifically for this event on the basis of an IoP in Scotland Public Engagement Grant. In particular, I was asked to develop an interactive display related to my research, which concerns the applications of basic research and innovation partnerships between scientists and local entrepreneurs in the Space Industry in Scotland and to give a talk about science, technology and innovation in Astronomy and Space Science at the evening public session.

Interactive Display and Exhibition Talks

The display included a rolling loop explaining key features and current finding of my research and a selection of relevant info-sheets produced by Science and Technology facilities Council (STFC) and Institute of Physics (IoP).

I-SAT tweet

I was near the display for most of the time exhibition was open and have regularly given short (15min) informal talks small groups of people to explain my work further.

It IS Rocket Science! – Building Scotland’s Space Sector

Building on detailed micro-level studies of innovation process my research suggests that the further development of Scottish Space Sector rests on matching the right people and institutions, who can then trade resources, knowledge and skills

Hence, I am using Social Network Analysis to map out the knowledge network(s) of the Space Industry in Scotland in order to identify its key systemic characteristics and outline the contours of the interactions (knowledge flows) between new product development (NPD) processes and its environment are the key for understanding which external factors most significantly influence the success of commercialisation of emerging Space technologies.

However, illuminating this network and characterising its effects is not trivial – after all, who said it was not rocket science?

Evening Talk

The series of events also included a public session with two presentations, one by the organiser of the workshop, Prof Franz Muheim, on “Higgs Bosons, Antimatter and all that” and the other my own on “Astrotechnology – and how it changed the World”. The event was well attended and well received by the local community (see below).

Evening tweet.JPG

 

Astrotechnology – and How it Changed the World?

Astronomy research is both driving the technological advances as well as being shaped by them. From ancient telescopes to modern infra-red detectors simulations, the practice of Astronomy is very much rooted in physical objects – most of them located down here on Earth!

Using an example of such an object, an ordinary DSLR camera, this talk will afford a fly-by tour of the key astronomical technologies, how did they come about and what did they do for the science as well as for the society. We will examine the way Space innovation is allowing us to see the past and the future, the distant and the invisible, to experience reality well beyond our imagination and then emerge in the world to change it for the better.

What have the Space ever done for us, again?

So, how did it go?

Though our initial estimate was to attract about 300 visitors to the exhibition and the evening event combined, this turned out to be relatively ambitious, in particular as the events happen to coincide with unusually good weather for Islay, which led to many (in particular younger) audiences to try and spend more time outdoors. In the end, we have had approximately 100 unique visitors to the events in total, the evening talk being attended by around 30 people. Given the small population on Islay (around 3000 total), this is still a significant turnout.

More importantly, the visitors were very interested in the events and thought them to be of high quality. Some commented:

“Great interactive exhibition.”

“Great initiative – never had anything like that on islands when I grew up.”

“When are you coming back?”

Importantly, we engaged with several groups of high school children, who are thinking about studying science, and have said that this positive experience motivated them even more. Furthermore, noticing the event activities on Twitter (announcements alone reached over 6,700 impressions and 115 engagements), some were already advocating future locations to visit, which we would be more than happy to follow up!

More information (including more photos) about the activities on Islay can be found

Finally, none of this would be possible without financial support from the IoP in Scotland Public Engagement Grant and the Particle Physics Group at the School of Physics and Astronomy, and the generous hospitality of the Islay High School, Bowmore.

Networking 101

matjazvidmar/ / impact, Innovation, methods, outcomes, policy, Post, research, ROE, Royal Observatory Edinburgh, science, Space, University of Edinburgh

This is a belatedly late post about some of my (early) research findings  – and my efforts to disseminate them.

This is build around Phase 1 of my research into the facilitation of technology transfer from basic research into high-tech industry, which is (hopefully) leading to overall economic growth and prosperity.

I am focusing on the Space Sector, the historic development of which is in its 3rd phase, sometimes referred to as the “New Space”, – after the initial state monopoly (1st phase) and the technology being commercialised by large multinational corporations (2nd phase), it is now being democratised through innovation and entrepreneurship as the (previously complex and expensive hardware becomes smaller, more standardised and cheaper (Space IGS, 2010). In the context of this transition and growth of the sector, there is a government target of increasing the UK share of global Space market from 7% to 10% by 2030 (Space IGS, 2010).

For small and medium size enterprises (SMEs) – which are the subject of my research – this is also denoting a transition in SMEs from “technology-push” to “market-pull” new product development (NPD) orientation, based on a shift from “supplying” to larger companies to “selling” to end customers/users, which is also moving from “mass-manufacturing” to developing Complex Products and Services (CoPS) (Hobday, 1998) and vertical value chain integration.

In order to help facilitate technology transfer at this transition point – one of the objective of my research – a key link between the systemic understanding of the sector and the innovation processes and practices in companies must be established. Given the prevalence of network research – which is at the core of Innovation studies both at macro as well as micro level (Freeman, 1991), due the fact that it covers the crucial aspect of knowledge commercialisation, namely interaction – this is an obvious choice to bridge the two.

In particular I am using ego-centric Social Network Analysis (SNA) (Scott, 1988; Giuliani, 2007) to plot the business network for each of the three studied companies (see table above). This is based on a survey-style questionnaire, with multiple-choice answers, but options for other (more expanded) answers as well. This enabled me a comprehensive analysis of the knowledge network, in particular the flows of knowledge and a qualitative analysis of the NPD process, by examining the specific “innovation moments” (i.e.  points at which decisions about developing an innovation into a product take place) (Edwards et. al., 2000), the role external partners play in those instances and how the differences in companies’ network make-up affect the creation of new products (and vice-versa).

Presently, the study is in its pilot incarnation, focusing on three case studies, carefully selected to represent key elements of the population of the field (upstream /downstream; hardware/software; start-up/spin-off; “Classical”/”New Space”; products/services), i.e. a set of typical cases (Yin, 2009), as summarised here:

ResTable
Analysed SMEs and their properties (Vidmar, 2015)

In short, these are the key findings so far:

  1. The networks are larger for the “New Space”, rich with open innovation, and significant knowledge flows across the firm’s boundary are detected (see the network plots below). This dynamics is gradually diminishes as we examine the other two cases, with the “classical” Space company having a very one-directional flow (and retain all of their IP within the company) and a relatively small core network.
  2. The more “New Space” a company is, the more it relies on knowledge flow from public institutions for R&D, BD and commercial transactions, and the more such partners it has. In contrast a more “classical” Space company has more private sector partners, mainly engaged in purely commercial activities, such as distributors and suppliers, through which it is sourcing components and maximising the reach of its supply.
  3. Interestingly, when analysed through the framework of Open Innovation (Chesbrough, 2006), we again find a very clear divide between hardware and software; upstream and downstream; and “classical” and “New Space”, with hardware upstream (“classical Space”) companies tend to exhibit more “closed” innovation models than software and downstream (“New Space”) ones.
  4. Analysis of the companies participating in the pilot suggests that the more the innovation process is “open” the less hierarchical it is, but also the more structured/standardised and formalised. This is in line with anecdotal experience from most successful high-tech areas, where more formalised, yet less restrictive, NPD protocols are being to be established in order to capitalise on as much innovation as possible (Neapole, -2015).
ResProfile
The network plot for Classical (left); Transitional (centre) and New Space (right) SMEs. (Vidmar, 2015)

Of course, these are only preliminary observations and substantial further research is being planned to further this work, including expanding the study to analyse all Scottish Space Sector companies.

If you are interested in a more full account of my research click through for the full paper presented at Reinventing Space 2015 conference  (Oxford, 9th-13th November 2015)

Here is also a photo of my poster (from SUPA Cormack Meeting 2015):

WP_20151123_007
At SUPA Cormack Meeting, Royal Society of Edinburgh, 23th November 2015.

Space is Up!

matjazvidmar/ / impact, Innovation, policy, Post, research, science, Space, Travel

The Global Space Innovation Conference (GLIC) 2015, which was held in Munich between 23rd and 25th June 2015, was preceded by an “unconference” based on the Space Up template – Space UP GLIC.

This template means, that the “participants decide the topics, schedule, and structure of the event”, which in this instance was themed around Innovation and Entrepreneurship in the Space Sector. We aimed quite high; for instance, one of the (preparatory) questions from Twitter reads:

The day was mainly filled with short presentations from a variety of speakers presenting on topics ranging from protection against Near Earth objects (NEOs):

to dancing in zero gravity:

I also added my ramblings to the mix; I ended up wrapping up the presentations section with my talk “It IS Rocket Science!” about studying innovation, New Product Development (NPD) and networks in the Space Industry:

This was followed by some debate time drawing on two key questions: What is innovation? and Does investment stimulate innovation or vice versa?

The outcomes of this discussion and a summary of our contributions was also presented at the close of the “main” conference, including a video vlog of the day:

For my part, on top of being introduced to a variety of interesting ideas and projects, I was also introduced to so many exciting and very talented people, all passionate about space and ready to actively participate in the future of Space exploration. That in itself made this event the place to be for any budding Space enthusiast and I am looking forward to meet my new friends again, as soon as might be.

Embedded image permalink
“On top of Munich!”

Perhaps, we might even get SpaceUp to Scotland some time soon…

Think Global, Act Global!

matjazvidmar/ / impact, Innovation, outcomes, policy, Post, research, science, Space, Travel, value

Few and far in-between are times when an interdisciplinary researcher working across different fields, such as myself, find themselves at a conference comfortably covering nearly the entire theme of their work. Most often, I myself sit with two crowds – innovation studies researchers’ meetings cater for participating in academic discussions and I attend the gatherings of the space community to keep up to speed in the trends in my research area, the Space sector.

However, the importance of bringing together of these two crowds has recently been noted within the international Space community and from 23rd to 25th June 2015, I have found myself in Munich attending the “conference of my dreams” – the Global Space Innovation Conference.

GLIC conference banner, (c) IAF

For me, the event was two part, as the “main” conference was preceded by the SpaceUp GLIC “unconference”, which provides an alternative platform for facilitating networking and discussions amongst (younger/young at heart!) Space enthusiasts. Little snippets from this event are published separately in the post “Space is Up!”.

The theme of the conference was “From Government Programmes to Entrepreneurial Actions” and was based on two main premises:

1. There is significant appetite, both from the government as well as industry point of view for more private-public partnerships or direct private enterprise to get involved in the (national and international) Space exploration efforts, including but not limited to benefiting from the (civil/non-space) applications of technology originally developed for Space exploration. 

2. There are important (negotiated?) roles for all stakeholders (public, private, academic, non-profit, etc.) in the entrepreneurial system the industry is embedded in. This system was defined along the lines of (Gnyawali and Fogel, 1994):

  • Socioeconomic Environment,
  • Entrepreneurial and Business Skills,
  • Non-financial Support,
  • Financial Support, and
  • Government Policies and Procedures

These five elements were also the themes of the GLIC’s five key discursive panels (2-6):

  • Panel 1. The View from Entrepreneurs
  • Panel 2. Socioeconomic Environment for Entrepreneurs
  • Panel 3. Entrepreneurial Education and Training
  • Panel 4. Non-Financial Assistance for Venture Creation
  • Panel 5. Financial Support for Venture Creation
  • Panel 6: Policies and Laws for Entrepreneurship

The key findings, in my view, are that stakeholders have to work to reduce the burdens on the entrepreneurs by, on one hand, (public and private) funders increasing investment in early-stage technology development and on the other, providing clever institutional support (mainly in business development) to the entrepreneurs. In theory, this should also lead to a (more) competitive and dynamic Space economy, including breaking corporate monopolies currently supported by targeted government procurement.

However, the appreciation of the complexity of the products in the Space sector is necessary and my own research here is in many ways an embodiment of the above programme and an expression of the anxiety associated with it in particular with respect to the need for a more detailed understanding of the product development processes (NPD) and the role the different stakeholders play in it. In particular, the entrepreneurial networks through which expertise and action are channelled play a crucial role here as, the project complexity requires an increased knowledge flow into the NPD or innovation process.

In fact, this was the main part of my contribution to the event, made in the context of the SpaceUp GLIC, where I outlined these important concerns and my plans to address them in the context of my research in Scotland/UK.

WP_20150623_004
Presenting the talk: “It IS Rocket Science!” at SpaceUp GLIC, Munich, 23rd June 2015

More of those plans to follow in a white paper draft to be published later this year…

Quantified Correlated Impacts (QCI) – ER4

matjazvidmar/ / cost benefit analysis, evaluation, evidence, impact, logic model, methods, outcomes, Post, research, value

To start building a (more) coherent picture of impact evaluation in science and technology programmes, we need to look for a constellation of many different methods to provide a meaningful insight into the need for, and success of, an intervention. Consequently, evaluation research is/should be organisational modus operandi, rather than a set of separate top-level exercises.

I propose a new paradigm in impact evaluation of investment and development in science, technology and innovation, namely Quantified Correlated Impacts (QCI). This approach is based on both quantitative as well as qualitative data collection, as bibliometric and econometric figures are correlated with ethnographic methods – interviews, focus groups and surveys – to determine the perceived causal contribution of the different factors, with particular focus on those pertaining from the intervention.

At it’s core, QCI are underpinned by a logic model; which is connecting the intervention with the evidence justifying the planned outputs; and leads form the inputs through action towards short-, mid- and long- term outcomes.

LogicModelBlog4EvalRes
Logic model for the proposed networking strategy in UK/Scottish Space Sector

As part my research project, for example, I am involved in an intervention driving economic growth in the UK/Scotland through stimulating the collaboration across the UK/Scottish Space industry by increasing sectoral networking. This is a particularly important part of my research in business incubation in the Scottish space sector and the related (sectoral) systemic properties, such as institutional framework, networks of actors, and knowledge creation and dissemination (following Malerba’s Sectoral Systems of Innovation approach (Malerba, 2005)). Also, there is a wealth of evidence about the importance of networking for the success (and growth) of small businesses (Brüderl and Preisendörfer, 1998; Ostgaard and Birley, 1996).

The suggested action to generate these positive effects is to support the growth of small to medium sized businesses by integrating them in a wider network across the sector and wider. This will be facilitated by the creation of, and enrolment of actors into, an on-line database/forum/platform to provide easy access to contacts. Having established that, there are also provisions to host networking events (thematic or generalist), to solidify the ties and introduce more actors into the network, particularly from the non-core businesses.

In terms of evaluation, key facilities need to be established prior to the beginning of the evaluation of outputs (database and its uptake, and the networking events). The database growth can be analysed quantitatively (i.e. number of enrolled individuals, organisations, etc), while the networking event qualitatively (i.e. interviews, feedback, ethnography).

The key next step is to tie the intervention with the outcomes/impacts through an advanced cost benefit analysis. In the example given, this can be done by analysing the investment made with respect to the growth and revenue of the companies most interconnected within the newly established network, comparing to the more peripheral ones, or ones outside the network.

The last part is the crucial correlation, which provides tangible benchmarking for the overall success of a programme (within the cost benefit analysis). This is done by comparing the noticed trends in key parameters (in our case job creation, revenue growth, etc.) with corresponding regional, sectoral, national or global trends. The key objective is to trace any significant difference which can then be (in part! – see below) attributed to the intervention.

Crucial information, however, comes from the collected qualitative data which maps the action to its value for the participants, i.e. what was the contribution of a specific intervention to the overall change. For instance, in the example above we investigate the effect/importance of the networking on business success. This data can only be obtained by interviewing the participants in networking events, and running surveys and focus groups with representatives of the companies/individuals on the database. The key questions to ask will be: What made the difference?; How?; and How significant was it? We can then comment on the part the intervention played in the difference found between the participants performance and correlated trends.

Overall, this approach enables the evaluator to marry the desirable clarity of cost benefit analysis, where standards of success/failure can be contested, with a more balanced set of criteria and tangible links. The key features are quantified data (engagement figures, costs, returns, growth, etc) about the intervention, which is qualitatively (interviews, focus groups, etc.) examined as a contribution towards the difference in participants’ performance with respect to correlated background trends (sector growth, national job creation, GDP, etc.) – revealing the impact of the programme.

As said, this new, Quantified Correlated Impacts (QCI), framework is currently under development and I am sincerely opening its tenets to comments and suggestions. (And, please, do have a look at the other posts in the series, too: ER1, ER2, ER3.)

Many thanks in advance!

Cost Benefit Analysis: “What Have the Romans Ever Done for Us?” – ER3

matjazvidmar/ / benchmarking, cost benefit analysis, evaluation, impact, outcomes, Post, research

Cost benefit analysis is an attractive evaluation method, as it can provide concrete, often quantified, data about interventions, usually in a form which is easily communicated to the clients, policy makers, funders and the general (lay) public. In its core and at its best, cost benefit analysis is a very direct and straightforward evaluation process, whereby inputs and outcomes are weighted against each other and logical conclusions about the efficacy of a programme can be reached.

However, all three of these elements – inputs or costs, outcomes or benefits, and efficacy or the relationship between the two – are highly contestable. To begin with, defining your parameter space and acknowledging constrains and assumptions is the key element of this approach to evaluation. These decisions, even if very well argued for, are ultimately just decisions; a global cost benefit analysis, if such a thing was ever possible, would need to encompass much of the factors and effects left on the other side of the dividing line for the evaluation to be a true representation of the net impact of the programme.

Secondly, even though the aim is to have a quantified data as possible – best if every input and impact are turned in some sort of monetary measure – both costs as well as benefits are often indirect or intangible. In Cellini and Klee’s most stark example (2010, p. 500): what is “the value of wilderness or an increased sense of community”? Furthermore, even if a measure can be put to notions such as wellbeing, another – perhaps most challenging of all – decision has to be made, namely what ratio between costs and benefits defines effectiveness of even efficiency?

However, in my limited experience, cost to benefit analysis is effective if the intervention being evaluated is narrow and well defined in terms of the available resources, the scope and the intended outcomes, or better still, when all of the above have an intrinsic monetary value attached. The intended outcomes I look for in my research are related to innovation and consequently increased economic activity, contributions to GDP, business growth, job creation, etc., hence quantification of these parameters is not very difficult as they often come as monetary values to begin with.

The most challenging for me is to benchmark the efficacy of this cost to benefit ratio and, to be honest, even though it would be to a degree possible to put a judgment on how significant the benefits have to be to deem a programme a success, I prefer to correlate these ratios to background trends such as global economic activity, comparisons to global GDP growth, global business and job creation, and add qualitative data where possible, as I believe the later provides a broader judgment on how the intervention is impacting those in and close to it.

This advanced cost benefit analysis can then feature prominently in a new paradigm of impact evaluation – the Correlated Quantified Impacts (QCI) – the topic of the next post.

“Means, Motive, Opportunity” – ER2

matjazvidmar/ / evaluation, impact, outcomes, policy, Post, process, research, value

In order to frame this enquiry, let’s begin with a small the exploration of the motivations behind commissioning and performing the evaluations in the first place. Though examples here are from social research, these are easily compared with parallels in any intervention, including investment in the development of the science, technology (and business support facilities and services (for example STFC, 2014:5-7).

Firstly, an important part of the evaluation research is process evaluation (Rossi, 1972:34), used in order to improve on the delivery of the intervention, or – as beautifully listed in an interview with Waverley Care (a Edinburgh charity) CEO – “what we need to stop doing, what we want to keep doing and what we are not doing that we should be doing”. When working along this strand of evaluation, it is crucial that the researcher provides recommendations that can be acted upon. The best way to carry out such evaluation is often to focus on a specific small area of the intervention, for example how does an organisation collect feedback and implement changes reflecting the concerns raised by internal and external customers. Having said that, conclusions and recommendations can often be very general.

In the process evaluation, there is further check on the identifying emerging needs and (geographical, social, economic) individualisation of the delivery of outputs. This is particularly important for social projects (such as the Waverley Care), where there is significant variation across the different locales in which they work. However, this is also important in terms of social and geographical inclusiveness of science and technology investment. Hence, evaluation research in this context can provide important checks on the “fairness” of the intervention whilst it is underway.

Then there is the often missed – but in my opinion very important objective in evaluation – the inward facing component, i.e. the improvement of morale of the people engaged in the programme/intervention/organisation by celebrating their success. It is very important for the staff to appreciate the whole picture, “take a step back” to frame their work within a wider context. This is both a good motivation for future work as well as a huge morale boost as one can see how they personally and as a team are making a significant difference to people’s lives.

Finally, the primary motivation for impact evaluation is (always?), to understand the impact/difference an intervention/organisation is making. Evaluation is often considered important for funding applications, i.e. both assessing the need for the intervention as well as monitoring the delivery of outcomes (to evaluate the VALUE generated).

My research is similarly linked to the need for accountability when spending public money (Nutley, Walter and Davies, 2007:254) and in particular the effectiveness of the investment in natural sciences research (mainly cost benefit analysis), which is currently epitomised in cost benefit analysis, but that is already the topic of the next post…

“The Case for Space” – ER1

matjazvidmar/ / big science, evaluation, impact, Innovation, methods, policy, Post, research, Space

To start at the beginning, as you might know my main research is in innovation form (basic) natural sciences and its commercialisation in the form of spin-outs and entrepreneurship. My specific field is Space Technologies here in the UK and in Scotland, so I look at emerging technologies ranging from satellite hardware to the use of the expertise developed in large telescopes for designing medical devices, such as Retinal Densitometer.

My research is tied in with the development of a new Space-related business incubator in Edinburgh, the Higgs Centre for Innovation. The expected growth of this sector is part of a wider UK government’s initiative to grow the UK’s share of the global Space Sector to 10% by 2030.

qjCWLWS
The design for the Higgs Centre for Innovation building (bottom right) at the Royal Observatory Edinburgh. (C) STFC

As such, a major part of my work will be the evaluation of past and present incubation programmes, to learn about their effectiveness and suggest examples of good practice. This work is done in collaboration with my research partners, the Science and Technology Facilities Council (STFC), who are launching the said new incubator and who run frequent impact evaluation exercises to justify the investment of public funds and bid for further funding allocation.

I have worked on some impact evaluation previously, for example I have recently written a summative report about the impact of CERN (the European Organisation for Nuclear Research) on science in general and on the UK economy and society in particular, spanning the first 60 years of its operation (up to 2014), which is partially included in a chapter of the last STFC Impact Evaluation Report (2014).

However, while impact evaluation frameworks in the context of science and technology policy are well developed (see STFC example), I have come to realise that their methodology is less so and that there is little available literature to easily form a new coherent approach to this topic (Autio, 2014; Zuijdam et.al., 2011; Markman, Siegel and Wright, 2008). Crucially, most methodological discussions included in the impact evaluation exercises that I have been drawing upon, often focus solely on econometric parameters and their calculations, rather than discussing any holistic framework of evaluation or any of the qualitative or comparative methodology.

Hence, I am looking at programme evaluation elsewhere to cross-reference the methods I encountered in my past research with the well developed theories of policy evaluation in social sciences, in particular concerning social policies and to come up with a rounded impact evaluation logic. Even though my past and present impact evaluation is about research in natural sciences and its impact on the socio-economic situation in the UK, many themes emerging from social policy evaluation match directly the ones I encounter(ed) in my research.

Impact Evaluation Series – ER0

matjazvidmar/ / evaluation, impact, methods, outcomes, policy, Post, research, science, University of Edinburgh, value

Hello, finally a “proper” post after a while!

In fact, this post may not be so “proper” after all, as it is only marking a start of a short series about impact evaluation, an important part of my research in science, technology and innovation.

The plan is to have four posts (ER1-4): (1) an introduction to my research in (impact) evaluation; (2) an exploration of key themes in evaluation research; (3) an analysis of the cost benefit analysis model, dominant in the policy sphere; and (4) an outline of a new methodology -Quantified Correlated Impacts (QCI).

This is very much work in progress so, perhaps, more posts will appear later on and I would very much like to hear your comments on any of it!

Importantly, this effort is part of 2015 incarnation of  Evaluation Research Methods course, a postgraduate course in the School of Social and Political Science at the University of Edinburgh.

Please, do spare a minute or two and have a look at a host of other contributions at our collective blog and follow our Twitter discussion marked with: #evalres15.