Start where you want to end up: Planning for outcomes

In the last few weeks I had two conversations with colleagues working in different sectors about outcomes.  Both gave cause for concern because the assessment of success and demonstrating achievement of outcomes are essential to continuation of funding.

The Big Lottery Fund explains that outcomes ‘are the result of what you do, rather than the activities or services you provide’.  This might be better lifestyle choices, improved wellbeing, better skills or increased employment.

The first conversation was about assessment of workplace employee physical activity projects for a health and wellbeing award.  There were many entries and very few could clearly demonstrate the difference their project had made to the people they were designed for.  They could say how many staff had been through the programme but not whether these were the same as they always get or new people.  They could not go on to say with any confidence what difference the project had made to their lives, either collectively or individually. There was little thought of how their project contributed to supporting Public Health or organisational priorities, reducing sickness absence for instance.

The second conversation was about discovering the impacts a museum exhibition had made on its visitors.  Long term outcomes were set for visitors, eg ‘to give visitors more confidence to talk about the subject of the exhibition.  However baseline indicator data had not been collected prior to visitors entering the exhibition and so assessing impact afterwards was only based on what visitors said some time after their visit.  As there was no baseline data to compare these findings with, the museum could not report robust findings to funders on whether or not the exhibition specifically gave visitors more confidence to talk about its subject.

There is lots of information around about the importance of measuring outcomes.  Public Health England’s 2016 published report ‘Arts for health and wellbeing: An evaluation framework’ by Norma Daykin with Tim Joss is a great guide to evaluation of the arts (including music, dance, theatre, visual arts and writing).  It covers public health evaluation frameworks, project evaluation tools and gives practical advice on measuring outcomes.  For example

• they ‘need to be sensitive to the characteristics and needs of participants and not detrimental to project delivery.’
• ‘it is not necessary or feasible to capture every project outcome.’

However, there is one alarming statement:

‘outcomes evaluation, which takes place at the end of a project to establish whether it has met its aims and objectives and to assess its effects or impacts on participants’.

Outcomes evaluation clearly does take place at the end of a project.  However, with all the exciting work going into planning, consideration of outcomes is often put into the ‘too hard to think about now’ box and ignored until the project is up and running.  And why wouldn’t you leave it until later as it ‘takes place at the end of a project’?

As a researcher/evaluator, the more successful projects that I have worked on included me as a key member of the planning team to think not only about aims, objectives and key messages but also about outcomes.  On an award winning project aimed at tackling heart disease in men, I developed the evaluation framework that included desk research into prevalence, mortality and risk factor indicators.  We needed to assess these before the project began in order to establish a baseline against which success was measured – the outcomes.  If we had not established the baseline, outcomes left to the end to be measured would have been meaningless.

In a more recent social marketing project around domestic abuse, we knew that a six week campaign was not going to make a measurable impact on perpetrator numbers but we could raise awareness in the local population and this would be a key outcome.  We conducted pre and post campaign surveys and found that awareness of the scale of domestic abuse was raised on a population level.  This campaign was one of three finalists out of over 60 entries for the NICE Shared Learning Awards 2016, not least because we could demonstrate achievement of well thought through outcomes.

The assessment of outcomes need to be planned for and researchers/evaluators have a wealth of tools at their fingertips to help with this.  These include

• desk and/or primary research to establish baseline data such as
  • numbers of people in the target audience
  • a measure of attitudes, knowledge, skills and behaviour amongst the target audience.
• stakeholder analysis to understand who you need to influence to collect your outcome data
• customer journey mapping to understand what actions will make the biggest difference to the planned outcomes
• logic modelling to work back from policy requirements and assess what your medium and short term indicators need to look like
• development of a comprehensive research and evaluation strategy that includes the resource requirement for assessment of outcomes.

So here’s the plea.  Include your researcher/evaluator as early as possible in the planning process to think about meaningful outcomes and how they will be measured throughout the life of the project.  It will take that worry off the mind of everyone else in the planning team and make all the difference when it comes to demonstrating success.