Research and analysis

South Yorkshire Mayoral Combined Authority: UKSPF summary evaluation plan

Published 16 October 2024

Applies to England

Summary of the local place

South Yorkshire comprises 4 local authority areas (Sheffield, Rotherham, Doncaster, and Barnsley) and collectively forms the South Yorkshire Mayoral Combined Authority. It consists of urban areas with minor conurbations and has a population of 1.4 million people.

Map of South Yorkshire Mayoral Combined Authority

South Yorkshire Mayoral Combined Authority received £38.9 million of ‘core’ UKSPF funding, which is being used across the three investment priorities: Communities and Place (£14.5 million), Supporting Local Business (£14.9 million), and People and Skills (£9.5 million). The aims of South Yorkshire Mayoral Combined Authority’s UKSPF programme are to address the key challenges faced in the region: economic inactivity, poverty, and low productivity.

The programme also builds on some of the region’s strengths: its vibrant arts and culture sector and education and training institutions. Interventions are therefore aimed at increasing tourism and boosting the visitor economy, supporting small businesses to grow and become more productive, and upskilling residents who are economically inactive and face barriers to entering work.

Unit of analysis

The place level evaluation of South Yorkshire will focus on the whole combined authority area due to the delivery of interventions across the authority and the unique governance structure of the combined authority and four local authorities. South Yorkshire Mayoral Combined Authority have adopted a democratic approach to delivery, so all authorities are in receipt of all interventions and delivery is consistent across the region. Therefore, it is not possible to isolate one of the local authorities nor select interventions targeted at specific local authorities.

Due to the size and scale of the funding in South Yorkshire, the evaluation will look across all three investment priorities. Currently, there is no locally commissioned evaluation activity planned in South Yorkshire which is a further case for this evaluation covering the entire combined authority.

Methodological approach

Process evaluation

The purpose of the process evaluation is to answer questions on the relevance, efficiency and effectiveness of fund design and interventions and the efficiency and effectiveness of processes used to manage and deliver the Fund. The approach to the process evaluation will include:

• A review of the evolution of the local context at the interim and final stages.

• Analysis of MI to gather evidence on the progress of delivery of the programme.

• Stakeholder and beneficiary qualitative research to explore the set-up and implementation of the UKSPF investment in South Yorkshire.

Impact evaluation

There is scope to deliver Quasi Experimental Design (QED) approaches in the South Yorkshire case study. These primarily relate to the business support and some small elements of the people and skills programme, where for some interventions individuals were required to apply to receive support. South Yorkshire Mayoral Combined Authority has kept the records of those unsuccessful applicants, so it may be possible to contact these individuals and create a counterfactual case for not receiving support. This could involve stakeholder interviews or surveys, as well as reviewing the Business Structure Database for business support applicants.

For People and Skills, the counterfactual is likely to be much smaller in scale. This is due to the recency of the interventions and hence the limited opportunity for individuals to see any effects or impacts from their unsuccessful applications. For Communities and Place, a counterfactual will not be possible as the events and cultural programmes are taking place across the 4 core town centres, so it would be difficult to find a suitable comparison within the combined authority.

Additionally, contribution analysis will be used to assess what changes have taken place in South Yorkshire, how UKSPF has contributed to outcome achievement and why. The research team will develop a contribution framework for the expected outcomes, linked to the Theory of Change. This will draw insights from:

• Management information will provide data on the progress towards and achievement of outcomes as outlined in South Yorkshire Mayoral Combined Authority’s Investment Plan.

• Primary research with stakeholders to capture data on outcomes from the perspective of stakeholders and critically allow exploration of why outcomes have or have not been achieved.

• Primary research with beneficiaries will allow more in-depth exploration of the outcomes achieved.

• Secondary data sources will provide useful contextual information to support the evaluation, although there may be limitations, such as attribution.

Economic evaluation

The evaluation will include a value for money assessment following the National Audit Office 4E’s approach, assessing the economy, efficiency, effectiveness and equity of the UKSPF interventions.

Data to support the evaluation

Primary data collection

Encouraging a culture and visitor economy

• Feedback from UKSPF funded events
• Stakeholder and beneficiary perceptions

Businesses supported, including stimulating enterprise growth and introducing new products, technology, or processes

• Stakeholder, beneficiary, and applicant perceptions
• Business survey

Building pathways to employment participation and promoting a skills-driven green economy

• Stakeholder beneficiary, and applicant perceptions

Secondary data sources

Encouraging a culture and visitor economy

• Management information
• Local data on footfall collected by local authorities
• Visitor numbers at events and festivals (if available)
• Community Life survey/ Your Community, Your Say
• Social media data

Businesses supported, including stimulating enterprise growth and introducing new products, technology, or processes

• Management information
• Business Structure Database
• Local data on business creation

Building pathways to employment participation and promoting a skills-driven green economy

• Management information
• Claimant count data or Annual Population Survey - small scale of P&S activity in South Yorkshire Mayoral Combined Authority may not be detectable in these datasets