Research and analysis

Summary: Evaluation of the Health-led Employment Trials

Published 20 April 2023

Becci Newton and Rosie Gloster

Overview

This summary provides insights into the outcomes from the Health-led Employment Trials (‘trials’) 12 months after recruitment. These tested the difference made by Individual Placement and Support (IPS) to the employment, health and wellbeing outcomes of people with mild-to-moderate mental and physical health conditions in primary and community health settings. The trials were implemented in West Midlands Combined Authority (WMCA) and Sheffield City Region (SCR). Both recruited people who were not working (the out-of-work (OOW) trial group). Additionally, SCR recruited people who were working but off sick due to ill-health (the in-work (IW) trial group).

IPS is a well-evidenced voluntary employment support programme associated with positive outcomes for people living with severe/enduring mental illness. It is not well-evidenced for other health conditions. ‘IPS-LITE’ was implemented in the trials, meaning support was time-limited (in standard IPS it is not) with up to 9 months to search for and secure a job and up to 4 months of in-work support after starting a job. IPS-LITE had previously been tested with small samples. The trials therefore contribute to the evidence base for new IPS services on several dimensions.

The sites randomised 9,785 people between May 2018 and October 2019 with 50% of recruits entering IPS support (the intervention) and 50% allocated to the control group.

Research context

The trials’ origins date back to 2015, when the Work and Health Unit (WHU) – a joint unit between the Department for Health and Social Care (DHSC) and the Department for Work and Pensions (DWP) working with NHS England – was awarded a health and work innovation fund to develop, deliver and test new ways of working to improve individual economic, social and clinical outcomes.

The aim was to improve prevailing trends as disabled people had substantially lower rates of employment than average over many years. In 2015, the disability employment rate gap was 32.2 percentage points (ppts). In July 2022 it stood at 23.2 ppts.

This summary covers the trials’ employment, health and wellbeing outcomes and lessons for implementation. It draws on baseline data collected pre-randomisation; surveys of recruits conducted 4 and 12-months after randomisation; qualitative interviews with staff, stakeholders, and recruits; service provider management information (MI), national administrative data sets, and assessment of the causal pathways to outcomes.

Main findings at 12 months

• In WMCA 3,675 individuals were recruited; 2,519 were recruited to the SCR IW group; and 3,591 to the SCR OOW group. Randomisation was effective in producing balanced treatment and control groups.
• Many recruits had not worked for 2 years, and some had never worked. Multi-morbidity of 6+ health conditions was common.
• Impacts on employment, health and wellbeing varied by site and trial group.
• In WMCA, there was a substantial and strongly significant impact on the probability of being employed for 13+ weeks over the year following randomisation. The SCR IW group saw less substantial and weaker impact on employment using the same measure.
• The employment outcome measure is aligned with public employment programmes and more ambitious than used in most IPS studies. This in part explains the lower employment impact compared with other studies.
• Across SCR, strongly significant impacts were seen for health and wellbeing outcomes. These did not emerge in WMCA.
• The economic analysis found that health outcomes produced a stronger return to society and the exchequer than employment outcomes. This led to a return-on-investment for every £1 invested in the IPS services of £0.01 in WMCA, and in SCR, of £2.02 (SCR OOW) and £2.32 (SCR IW).
• Trial designers anticipated impacts would be preceded by improvements to: jobsearch capability, use of health services, and self-confidence. The evaluation showed progress among the treatment group on all dimensions.
• High levels of satisfaction with IPS support were found.
• Different strategic-level configurations and different delivery models were most likely to explain the differences in impact between the sites.
• For delivery, specialised (IW or OOW but not both) and smaller caseloads enabled more employer engagement activity. A mix of short and longer meetings was likely to provide momentum for employment outcomes, whereas less frequent, longer meetings were likely to enable focus on health and wellbeing.
• Building integration between health and employment systems, and increasing understanding of the value of work to health and wellbeing outcomes are important factors for future delivery.

Methodology

The trials and evaluation were underpinned by 3 ‘Theories of Change’ developed in collaboration with stakeholders in each site during an 18-month design phase. These covered the expected pathway to outcomes for the treatment group, health system, and employers. As the trials would operate in health settings, ethical approval was gained through the Health Research Association (HRA).

The evaluation involved process, impact and economic components drawing on quantitative and qualitative data. Baseline data were collected from recruits prior to randomisation, and captured information on demographics, employment and mental health starting points, and identifiers to enable linking to administrative data-sets. Added to this were data from 2 surveys of recruits at 4 and 12-months post-randomisation, which gained a response rate of 55% and 46% respectively. In both a lower response rate was achieved for the control group. Quantitative information on the IPS experience drew on MI collated by service providers. In the final analysis administrative data drawn from DWP and HMRC were added to the evaluation data-set, meaning that employment outcomes could be tracked for a large majority of recruits. The quantitative data were complemented by a range of qualitative interviews with recruits, provider staff including employment specialists, health professionals, employers, other key partners and stakeholders.

Findings explained

Profile of recruits

Generally, individuals were motivated to work on recruitment although some referred from employment services were less willing than others to find work quickly. Many OOW recruits had not worked for 2 years on joining, and some had never worked.

While the trials intended to recruit people with mild-to-moderate health conditions, multi-morbidity was common: half of the recruits had 6+ interacting mental and physical conditions.

Impacts 12 months following randomisation

Final impacts varied by site and trial group (see Table 1).

Table 1: Summary of final impacts at 12 months

	Employment	Earnings	Health	Wellbeing
WMCA	4ppt ***	£150	0.05 sd	0.9sd
SCR IW	3ppt *	£442	0.10 sd *	0.18 sd ***
SCR OOW	-2ppt	-£233	0.10 sd *	0.12 sd *
All SCR	1ppt	£102	0.10 sd ***	0.14 sd ***
All OOW	1 ppt	-£51	0.08 sd **	0.10 sd **

Asterisks indicate impact; number of asterisks indicates level of confidence/significance associated with impacts:
* 90%; ** 95%; *** 99%; ppt = percentage points; sd – standard deviations.^{[footnote 1]}

Source: Final evaluation data set

In WMCA, there was substantial and strong impact on the probability of being employed for 13+ weeks over the year following randomisation (4 ppts at the 99% significance level). The treatment group in WMCA was 20% more likely to have worked for 13+ weeks in the year following randomisation than the control group.

The SCR IW group saw a smaller positive impact on the same measure of employment at a weaker level (3 ppt at 90% significance); a small positive impact on health (90% confidence) and substantial and strong impact on wellbeing (99% confidence). Small, positive impacts for health and wellbeing were seen at the 90% confidence level for the SCR OOW group.

Overall SCR saw strong impacts on health and wellbeing (95% confidence). When the OOW groups were combined impact was found on health and wellbeing at the 95% confidence level.

The employment outcome measure for the trials, which covered the probability of being employed for 13+ weeks over the year following randomisation, was aligned to public employment programmes in contrast to many IPS studies which assess impact on a single additional day in work over the control group. This measure was selected in order that results could be compared with other public employment programmes for similar target groups.

Economic findings at 12 months

Costs of delivering the IPS services varied between sites and trial groups as did the return-on-investment (see Table 2).

Table 2: Summary of the costs and returns for the trials

	Costs per recruit to treatment group	Financial return for every £1 spent	Economic return for every £1 spent
WMCA	£3,893	£0.01	£0
SCR IW	£2,116	£0.02	£2.32
SCR OOW	£2,416	£0	£2.02
All SCR	£2,292	n/c	n/c
All OOW	£3,162	£0	£1.22

n/c – not calculated as impact not observed; financial return based on employment impacts; economic return based on health impacts

Source: Final evaluation data set

Spend per recruit receiving IPS was £3,893.00 in WMCA, £2,292.00 (SCR IW) and £2,116.00 (SCR OOW). This resulted from the different number of recruits in each site and group (there were fixed overheads for service delivery).

Economic benefits arising from improvements in health (using the Health-related Quality of Life (HRQOL) measure) were worth more to society and the exchequer than financial benefits from improvements in employment, particularly as the trials did not generate an earnings impact. For every £1 invested, the IPS services generated:

• an economic return of £2.02 (SCR OOW), £2.32 (SCR IW) and £1.22 (All OOW)
• a financial return of £0.01 (WMCA), and £0.02 (SCR IW)

Journey to employment

The trials’ design expected that final impacts for the treatment group would be preceded by improvements to jobsearch capability, self-confidence and use of health services. Evaluation evidence demonstrated that developing an appropriate job goal, increasing confidence and motivation through IPS support, and improved understanding of skills, were important facilitators of employment outcomes.

The 4-months post-randomisation survey showed significant impact on jobsearch self-efficacy including confidence in performing general and specific jobsearch tasks (95% confidence level - All OOW; 99% confidence - SCR IW). Qualitative evidence showed improved adherence to health advice which supported improved condition management.

In the 12-month survey the treatment group said that the IPS service had been helpful to their confidence in their skills (69%) and ability to decide on the job they wanted (71%). They had a positive perception of support saying their employment specialist had the right skills and expertise (69%) and understood their needs ‘a lot’ (68%).

Explaining differences in outcomes

Qualitative evidence showed a greater connection between the health system and trial in SCR. This may have stemmed from local Clinical Commissioning Groups and local authorities forming a steering group to lead on design, whereas in WMCA this process was contracted out. The survey showed that recruits in SCR made greater use of health support organisations than those in WMCA who showed greater likelihood of accessing employment support. MI indicated that SCR offered longer duration but less frequent meetings than in WMCA which may have been conducive to discussing health and wellbeing.

The MI indicated that enrolment into the IPS service occurred more rapidly in WMCA than in SCR; therefore jobsearch could also commence more quickly. The MI showed a mix of face-to-face and telephone check-in interactions in WMCA which may have increased momentum on jobsearch.

Economic data suggested that caseloads in WMCA were smaller than in SCR. The qualitative data showed that the mixed IW and OOW caseloads in SCR were challenging for employment specialists to manage. The qualitative and economic evidence suggested more focus on employer engagement in WMCA than in SCR, potentially due to a smaller and less complex caseload.

Lessons for delivery and implementation

• Staff training is crucial and must be maintained as new employment specialists join IPS services to maintain fidelity in delivery. Areas to focus on are ensuring capability in both employment and health support and leading employer engagement.
• Securing buy-in from health partners and GPs was perhaps the biggest challenge for the trials. Sustained work to build relationships and a shared agenda, to support links between health and employment systems, is important.
• Co-location of health and employment specialists was effective, but not widespread in the trials. Regular presence with local teams mitigated this; this could be extended to support integration between employment and health systems.
• Rapid enrolment to the IPS service helped ensure the treatment group’s swift engagement in jobsearch. In combination with other factors in the IPS Fidelity Protocol including caseload and employer engagement this should correlate to outcomes.
• Caseload size and mix is a crucial factor: larger or more complex caseloads may reduce time for activities such as attaching referrals or employer engagement.

1. The extent to which an outcome varies. An impact of one sd would move the average person from the 50th percentile of the distribution for that outcome to the 84th percentile. An impact of 0.1 or 0.2 sds would move the median individual from the 50th percentile to the 54th or 58th percentile. ↩