Overview 

The European Social Fund (ESF) was set up to improve employment opportunities in the European Union (EU) and thereby raise standards of living. The Department for Work and Pensions is the Managing Authority (MA) of ESF funds in England. 

The ESF 2014-20 Operational Programme – part of the European Structural and Investment Funds (ESIF) Growth Programme for England – aimed to deliver against priorities to increase labour market participation, promote social inclusion and develop the skills of the potential and existing workforce. As set out in the UK’s Withdrawal Agreement with the EU, the ESF programme continued to invest in projects after the transition period for leaving the EU ended on 31 December 2020, but all funding needed to finish by the end of 2023. By the end of this period, £3 billion worth of funding will have been provided from the EU, with an additional £2 billion in ‘matched’ funding from domestic bodies, and 2.4 million participants will have been taken part in ESF projects. 

The programme is structured around 5 Priority Axes (PAs) based on the EU’s Thematic Objectives. This evaluation focuses on two of these: 

• PA1: Inclusive Labour Markets 
• PA2: Skills for Growth 

Provision for the 2014-20 Operational Programme was delivered through four Co-Financing Organisations: the Education and Skills Funding Agency (ESFA); DWP; National Lottery Community Fund (NLCF); His Majesty’s Prison and Probation Services (HMPPS); Greater London Authority acting as an intermediary body with other organisations such as Greater Manchester Combined Authority having similar status, as well as Direct Providers (i.e. projects which bid directly to the Managing Authority). 

ESF support / activity covers a wide range, including: 

• tackling deep rooted labour market barriers through flexible support 
• social inclusion 
• vocational skills and learning 
• employability skills 
• traineeships, apprenticeships, volunteering work experience 
• in-work progression 

Research context 

This is a report of the evaluation of the labour market impact of the European Social Fund (ESF) 2014-2020 programme in England, covering all ESF participants from 2015 to 2020, allowing for a three-year outcome tracking period. 

The study centres on participant outcomes in terms of employment and benefit receipt. It does not attempt to estimate impact on educational outcomes, earnings or other outcomes that may result from ESF as DWP are still in the process of developing a robust way to estimate these with the data DWP currently holds. It does however include a cost benefit analysis of the labour market component. 

This report forms part of a wider evaluation of the programme in England which aims to assess the programme’s effectiveness, impact and efficiency. It delivers a counterfactual impact analysis of the programme to compare the outcomes of ESF with the outcomes that would have been achieved had ESF not been in place. It contributes to the existing ESF evidence base of reports, including a leavers survey covering the 2016-2019 and 2021-2023 periods, as well as a qualitative evaluation published in 2022 and an impact analysis of the Youth Employment Initiative (YEI). Several other evaluations have also been published by external organisations. 

The evaluation provides a legacy evidence base for informing the design of future employment and skills programmes for those furthest from the labour market and are being used to inform development of future domestic employment support. 

Main findings 

• This analysis provides evidence that participating in ESF statistically increases time in employment and reduces time spent on inactive benefits e.g. ESA (Employment Support Allowance) and increases time spent on benefits which require you to try and actively engage with the labour market e.g. UC (Universal Credit). 

• We estimate that, on average, ESF participants spend around 39.7 more days in employment in the three years post-start than they would have done had they not participated in ESF. 

• Those who start the programme already employed spend an additional 29.7 days in employment, after three years, more than they would have done had they not participated in ESF. 

• Those who start ESF when unemployed spend an additional 34.2 days in employment, after three years, more than they would have done had they not participated in ESF. 

• Those who start ESF when inactive spend an additional 76.0 days in employment, after three years, more than they would have done had they not participated in ESF. 

• These impacts accrue, increasing year-on-year, to reach the quoted figures after three years and remain above zero even at the three-year point. 

• For the exchequer ESF makes a return of £0.69 for every pound spent at three years (i.e. a net loss). This is largely due to the relatively narrow scope of exchequer benefits in this context that do not include non-DWP outcomes such as education and criminal justice. This is when the benefits are compared to the ESF costs of delivering the programme. 

• ESF makes a return of £1.50 for every pound of spend when looking at the societal perspective, which includes increases in economic output. 

Methodology 

The treatment group is restricted to participants who participated in an ESF programme in England, who are aged 19-64, who started on or before September 2020 and who were claiming benefits when they started on ESF. These ages allow us to track for months pre-start for even the youngest and cut off at retirement age for many. This aligns with other impact analyses. Participants are grouped depending on the type of benefit they were claiming at the start – employed, unemployed and inactive. The resulting treatment sample is 201,700. 

For the analysis, non-participants drawn from DWP data are assigned pseudo-start dates that mirror the distribution of participant start dates. Individuals are then tracked for three years post-(pseudo) start, with monthly flags indicating whether they had a live spell on employed, unemployed or inactive benefits or in PAYE employment. These flags are needed for the propensity score matching process, reducing processing time while maintaining match quality. The three-year tracking periods allows sufficient time for some outcomes to accrue, while maintaining a large enough sample size. Though it is also limited to three years due to the data available at the time. 

We estimated the average effect of the ESF employment programme on participants using propensity score matching (PSM). PSM creates a comparison group of non-participants who share key characteristics with participants. This enables us to estimate what a participant’s labour market outcomes would have been without the programme (the counterfactual). By matching participants and non-participants based on their propensity scores, PSM helps reduce the impact of confounding variables and selection bias, mimicking a randomised control trial (RCT). RCT approach was not feasible due to the way the ESF programme was managed and operated, whereby the European Commission want to maximise uptake of support. The effectiveness of PSM relies on the Conditional Independence Assumption (CIA) being met. The CIA states that, after controlling for a set of variables, assignment to ESF is independent of the outcomes. Whilst this is challenging to achieve in voluntary programmes where unobservable traits like motivation play a role, the method used should allow these characteristics to be controlled for by use of a rich dataset. 

In this analysis, we also use difference-in-differences (DiD) approach, comparing labour market status of ESF programme participants and non-participants before and after the programme. This method assumes that, without intervention, both groups would have followed the same outcome trends both pre- and post-treatment, known as the common trend assumption and that only the treatment causes a difference in outcomes. While the voluntary nature of the programme could potentially violate this assumption – since participants might be more motivated – the nature of the analysis will mean that we control for this as much as possible. 

Findings explained 

Findings from Impact Analysis 

A key result across all three benefit groups is that ESF participants are more likely to be in or remain in employment in the three years post programme start than had they not participated on the programme. These results are all statistically significant.  

The ESF programme had a positive impact on employment outcomes for the employed benefit group (i.e. those employed and claiming benefits). Overall, these participants are likely to spend 30 additional days in employment over the three years following participation compared to non-participants.  

Similarly, the programme also had a positive impact on employment outcomes for the unemployed benefit group, with these participants likely to spend 35 additional days in employment over the three years following participation compared to non-participants.  

The analysis showed ESF had a positive impact on employment outcomes for the inactive benefit group, suggesting participation substantially increased an individual’s chances of being in employment over the three-year tracking period. These participants were likely to be in employment for an additional 76 days compared to non-participants. 

The ESF programme increased time on employed and unemployed benefits over the three-year tracking period, meaning participation increased an individual’s chances of being on these types of benefits over this period. This means participants move into the labour market, or are closer to the labour market, than they would have been in the absence of ESF.    

Universal Credit differs from previous benefits, in that it is entirely possible for someone to remain on UC whilst entering employment. UC is an income-related benefit therefore employment and benefit receipt can coexist, subject to the individual’s income level, unlike under for example, JSA (Jobseekers Allowance) which stopped once someone entered employment. Movement into employment, whilst still being on benefit, is still a positive impact both for the individual and the exchequer compared to just receiving benefit alone. 

Overall, employed benefit participants are likely to spend 27 additional days on benefits and unemployed benefit participants are likely to spend 140 additional days on benefits across the three years compared to non-participants. However, participants in these benefit groups are also more likely to be in employment following participation compared to those who did not participate. It is likely they are moving into employment with the support of Universal Credit. Inactive benefit participants spent on average 20 fewer days in receipt of benefits across the three years compared to non-participants.  

Understanding from Differing Impacts 

The ESF programme’s impact varies across benefit groups, with the most pronounced effect observed among inactive benefit recipients, who are typically further from the labour market. However, other groups also obtained positive outcomes, mostly in the form of additional time in employment. 

The size of the impacts was different across each of these groups. Causal factors are difficult to isolate in this type of analysis, although some suggested factors include the participants’ relative labour market starting point as well as the type of provision they received on ESF could influence the relative differences. 

The sharp initial rise in benefit receipt following the start of ESF provision across all groups may be explained by the programme ‘lock-in’ effect. This can delay participants’ transition away from benefits as they continue to engage with the programme, seeking better job outcomes. Participants also report increases in softer outcomes such as enhanced skills, confidence and motivation evidenced by survey data, which may lead to prolonged job searches as participants set higher job expectations. Although, we cannot compare these findings with a counterfactual. 

Social Cost Benefit Analysis findings 

At the three-year tracking point, the CBR (Cost Benefit Ratio) is below £1 for the exchequer but above £1 for the social for the ESF programme. From the exchequer’s perspective, which only considers taxes and benefits related to employment, the benefits do not outweigh the costs, resulting in a net loss of £720 and a CBR of £0.69 for the programme. However, from a social perspective, the benefits outweigh the costs, generating a £1,160 surplus to society and a CBR of £1.50 for the programme. 

The programme’s benefits increase over time as more outcomes are realised and more people complete provision, obtain outcomes and more data becomes available. The CBR is low at the 12-month point because many outcomes take time to accrue. It significantly improves after 24 and 36 months as more people see the impact of ESF. The analysis suggests that the positive impacts of ESF extend beyond the three-year tracking period. The difference between the treatment and non-treatment groups at the 36-month point is not zero, indicating that impacts are still present and therefore additional outcomes could be achieved if data were available for 48+ months. Whilst some early cohorts have reached that point, the sample size is too small to make a meaningful estimate in this report. 

Our analysis also suggests that to “break even” on the exchequer savings, around 52 days of additional employment would need to be detected in the impact. At the current rates, if the three-year impact was sustained, we’d expect this to be achieved if we could track for four years. 

Social Cost Benefit Analysis of Differing Groups 

The analysis has focused on participants who are on benefits upon starting ESF. However, unlike other labour market programmes the characteristics of our participants varied. ESF includes individuals who may have been on UC as well as legacy benefits upon starting, unlike other programmes where all participants are on a single benefit. The analysis examines the impact between conditionality types, specifically comparing those who are unemployed, inactive and employed when they started ESF. This approach helps show a range of potential CBR estimated without creating unlikely scenarios. 

The cost benefit analysis findings indicate that the impact is greatest for the inactive group, which aligns with ESF’s core purpose of helping those furthest from the labour market. The unemployed group shows the next highest impact, while the employed group has the lowest. Caution is advised in interpretating these results as definitive evidence of ESF’s effectiveness for one group over another, given the different characteristics and outcomes across groups.