Research and analysis

Uttlesford: UKSPF summary evaluation plan

Published 16 October 2024

Applies to England

Summary of the local place

Uttlesford in north-west Essex, is a geographically large district with low population density. It is an affluent area^{[footnote 1]}, reflected by high property prices, a highly skilled population, and very high levels of car ownership^{[footnote 2]}. However, there are pockets of deprivation - some 12% of houses in Uttlesford are fuel poor^{[footnote 3]}. About 80% of the population lives in rural areas^{[footnote 4]} and parts of Uttlesford are geographically isolated.

Map of Uttlesford

Uttlesford’s UKSPF interventions look to tackle social and economic disparities within the district by increasing awareness and provision of public services. The focus is to make UKSPF genuinely transformational to those that need it the most, particularly outside Saffron Walden, the largest settlement. Uttlesford’s UKSPF interventions have included mobile services to improve access to food and parenting support, outreach work to improve resident knowledge of public services, and SME grants.

Unit of analysis

Geographically speaking, the evaluation will study impact at a district-wide level, given that UKSPF interventions are people, not area focussed, and the comparatively low population density across wards. T

hematically, the evaluation’s main focus will the Communities and Place priority theme as it accounts for over 75% of Uttlesford’s UKSPF investments to date. We will focus on three sub-themes within this: food and nutrition, parenting support, and improving knowledge of available services. The evaluation will also undertake a lighter touch examination of the business support priority area with the Business Start-Up grants programme receiving one of the largest financial investments for a single project.

Methodological approach

We will adopt a mixed-methods approach that combines primary research, secondary data analysis, and quasi-experimental approaches, underpinned by a contribution analysis framework.

Process evaluation

The process evaluation considers the four main elements of UKSPF delivery and management in Uttlesford: fund design, fund implementation, activity delivery, and data collection and monitoring. We will collect evidence on these through reviews of local data and documentation, interviews with local stakeholders (e.g. Uttlesford District Council delivery team, delivery providers, and Local Partnership Panel members), and secondary data (e.g. Census and IMD data). We will analyse evidence using a bespoke process evaluation framework that has 22 process evaluation questions relevant to Uttlesford.

Impact evaluation

Uttlesford’s UKSPF interventions have focused more on social, non-quantifiable, and intangible benefits. Consequently, for most outcome and impact areas, quasi-experimental design is not feasible as there is no consistent way of being able to measure progress. Additionally, with a notable proportion of UKSPF interventions in Uttlesford available to all residents, there is not always a feasible non-beneficiary group to use as a control. Nevertheless, there are two outcome and impact areas where it will be feasible and practical to undertake quasi-experimental approaches: analysis of ward-level Food Bank usage to determine changes in societal marginalisation and life chances, and use of business performance data (e.g. employment, turnover) to assess changes in economic performance.

For all other outcome and impact areas, the evaluation will draw primarily on a theory-based evaluation (TBE) approach. Specifically, it will use a contribution analysis framework to systematically test all the evidence collected to determine whether the Theory of Change is valid explanation of change, and how far external factors may have influenced outcomes more.

Economic evaluation

Following the National Audit Office’s 4Es, the evaluation will consider programme economy (by assessing ability to leverage additional in-kind or financial support), efficiency (by calculating costs per output and costs per outcomes, and benchmarking these against comparator programmes), effectiveness (using contribution analysis and QED to determine whether UKSPF is plausible driver for outcomes), and equity (determining whether UKSPF has gone to the communities most in need of it).

Data to support the evaluation

Primary data collection

Beneficiary surveys will be an important data source. It will not be feasible to survey beneficiaries of all interventions. We will therefore undertake surveys with those participating in the projects as well as the SME grants, our evaluation priority projects. These will help obtain information on the quality of interventions, and as appropriate, details on economic, health and social circumstances.

We will also conduct semi-structured interviews, focused on both impact and process elements. Longer 45-60 minutes with Uttlesford District Council staff and key strategic stakeholders (e.g. Local Partnership Panel members) will cover project-level and programme level perspectives. Interviews with all delivery provider leads will last 30-45 minutes, with lighter touch programme-level discussions. Beneficiary interviews will be 15-20 minutes long, focused on specific key issues.

Through delivery observation of the projects we will assess impact (e.g. by viewing beneficiary reaction), and process elements (e.g. quality of project delivery).

Secondary data sources

Census and Index of Multiple Deprivation data will help us, alongside monitoring data, determine for the process evaluation, how far Uttlesford’s interventions have reached those in greatest need.

Specific to Uttlesford, we will also analysis ward-level data over time for Food Bank and Citizens Advice services. Falls in the usage of these services may help indicate improved access to services through other means (including via UKSPF) and could be a proxy indicator for improved life chances.

Where possible, we will use Crunchbase and the Interdepartmental Business Register to monitor business performance over time for SME grantees, again to help see if UKSPF has had an effect here.

1. According to the Annual Survey of Hours and Earnings in 2023, the median gross weekly pay for those living in Uttlesford was £792, compared to £683 for Great Britain as a whole. ↩

2. According to the 2021 Census, 91% of households have access to at least one car, compared to 77% for the whole of England and Wales. See the Essex design guide. ↩

3. See the Uttlesford Health and Wellbeing Strategy 2023-2028. ↩

4. Uttlesford is categorised by Defra as a ‘Rural 80’ district whereby at least 80% of the population live in ‘larger market towns,’ ‘rural town and fringes’, ‘villages’ and ‘dispersed settlements.’ Further details are available in Defra Classification of Local Authority Districts and Unitary Authorities in England: A Technical Guide. ↩