Research and analysis

Theory based evaluation: Health-led Employment Trial Evaluation

Updated 8 April 2024

Applies to England, Scotland and Wales

August 2022

DWP research report no.1037

A report of research carried out by the Institute for Employment Studies (IES) on behalf of the Department for Work and Pensions (DWP).

You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence.

To view this licence, visit nationalarchives

or write to

The Information Policy Team
The National Archives
Kew
London
TW9 4DU

or email psi@nationalarchives.gov.uk

This document/publication is also available on our website at: www.gov.uk

If you would like to know more about DWP research, email socialresearch@dwp.gov.uk

First published July 2023

ISBN 978-1-78659-549-2

Views expressed in this report are not necessarily those of the Department for Work and Pensions or any other government department.

Executive summary

This report is part of a series covering the implementation and outcomes of the Health-led Employment Trials (HLT). It explores how far the planned outcomes for the treatment group, health system and employers were achieved and the mechanisms and contexts which enabled or prevented these. The Theories of Change (ToCs) (Appendices 6.2 to 6.7) which were developed in the design phase of the trials to underpin delivery and evaluation form the basis for this exploration.

The analysis is based on interviews with recruits, trial staff, employers, partners and stakeholders.^{[footnote 1]} It includes sections on the outcomes of the treatment group, health system, and employers achieved by Working Win (WW) in Sheffield City Region (SCR) ^{[footnote 2]} and Thrive into Work (TIW) in West Midlands Combined Authority (WMCA). Each section identifies which major and intermediate outcomes were achieved suggests the contexts and mechanisms that enabled or prevented outcomes, and summarises the causal pathways based on qualitative research evidence. An analysis of management information (MI) covering the profile of recruits to the treatment group, the support, and outcomes is included in appendices 6.8 to 6.9.

The qualitative research showed some outcomes for the treatment group were achieved but there was little or no evidence of major outcomes for employers or health systems.

Evidence of major outcomes achieved for employment and health

The findings suggest that the assumptions outlined in the ToC for the treatment group broadly held true for the following outcomes:

• returning to work or getting a new job
• experiencing improved health and wellbeing

There was evidence that the treatment group, and the employment specialists leading their support, achieved major outcomes such as new jobs or improved wellbeing and attributed these to the Individual Placement and Support (IPS) services. Where major outcomes were not evidenced for individuals, the qualitative data suggested this could be due to them not being ready to work, having serious or multiple health conditions affecting their ability to work, or external factors such as lack of suitable jobs or difficult family circumstances. The form of IPS delivered in the trials could not overcome these barriers for all in the treatment group.

Limited evidence of major outcomes on work supporting health, and earnings

In contrast, findings suggest that the assumptions outlined in the ToC for the treatment group did not hold true for the following outcomes:

• experiencing work as a therapeutic outcome
• improved income

In order for the treatment group to experience work as a therapeutic outcome, three contexts needed to be in place. They needed to have the right kind of work, a health condition that was not exacerbated by working, and not be motivated to work purely by financial reasons. These contexts were not consistently observed in the qualitative research. Qualitative evidence on why the treatment group did not experience improved income was limited, but might be explained by some only being able to access lower quality work, or conversely, by some preferring to take part-time or lower paid work as this was better for their wellbeing.

Employer and health systems: Limited evidence of major outcomes

The findings suggest that the assumptions outlined in the ToCs for employers and health systems did not hold true for the following major outcomes:

• Employers more willing and confident to take on and support disabled people and those with long-term health conditions, and to make use of external health-related support for employees.

• Health partners would view conversations about employment with patients as a key part of their role, have a better understanding of the relationship between work and health, and see it as part of their role to make wider referrals including to employment services.

Health system and employer intermediate outcomes such as greater awareness of the trials were not consistently in place which in turn meant that major outcomes for these groups could not be achieved. There was evidence of individual trial staff engaging effectively with individual employers and health partners, but this was not systematically in place. If there had been more strategic engagement with employers and health partners it seems likely this would have led to greater change. These outcomes may also have needed more time than the trial allowed to be realised.

The limited success in employer and health systems engagement may also have affected the outcomes of some in the treatment group with more substantial needs, as well as the achievement of improved income and experiencing work as a therapeutic outcome.

The final report series for the trials covers:

• Synthesis report: a high-level, strategic assessment of the achievements of the trial, drawing together the range of analyses from the evaluation.

• 4 month outcomes report covering: an analysis of implementation, a descriptive analysis of the survey findings 4 months post-randomisation, and an assessment of impact at 4 months following randomisation.

• 12 month survey report providing a descriptive analysis of the final survey, based on the theory of change for those in the treatment group.

• Context mechanism outcome (CMO) report, reporting evidence on outcomes from the trials and relating these to its theories of change.

• 12 month impact report covering the net effect on employment, health and wellbeing resulting from the trials 12 months after randomisation drawing on administrative and survey data.

• Economic evaluation report exploring the costs and benefits arising from trial delivery, drawing on the administrative and survey data.

• The pandemic and the trial: an analysis of how the trial outcomes may have been affected by the onset of COVID-19.

Acknowledgements

Please see the synthesis report

Authors’ credits

Jess Elmore is a Research Manager at Learning and Work Institute (L&W). She managed part of the process evaluation. She is a qualitative researcher who has worked across a range of projects with a focus on disadvantaged or hard to reach groups and their access to education and employment.

Rosie Gloster is a Principal Research Fellow at IES. She supported the management of the evaluation consortium and contributed to the process evaluation. She is a mixed-methods researcher specialising in employment, and careers. She has authored several reports for the DWP, including the evaluation of Fit for Work.

Naomi Clayton is Deputy Director of Research and Development at L&W. She provided quality assurance for L&W’s contributions to the evaluation. She has specialisms in employment, skills and labour market disadvantage issues, and has worked with partners across the UK on multiple evaluations.

Becci Newton is Director of Public Policy and Research at IES and specialises in research on unemployment, inactivity, health, skills and labour market transitions. Becci has managed the evaluation since its design and contributed to the process evaluation. She has led multiple evaluations for DWP including of the 2015 ESA Reform Trials and the Work Programme.

Rebecca Duffy and Zoe Gallagher are Project Support Officers at IES and proofread and formatted the report.

Glossary of terms

Term	Definition
Area	A defined, geographic area within a trial site
Baseline data collection	Data from the baseline assessment completed by provider staff who recruited people to the trial
Casual link	The connection between a cause and an effect
Casual pathway	A way to explore and understand the journey to outcomes, including the role of intermediates outcomes in the achievement of final outcomes
Clinical commissioning groups	Clinically-led statutory NHS bodies responsible for the planning and commissioning of health care services for their local area
Deep dive	Thematic case studies used in the process and theory of change evaluation with methods varying depending on the selected themes for investigation
Descriptive analysis	Producing statistics that summarise and describe features of a dataset such as the mean, range and distribution of values for variables
Employment specialists	Staff employed by the trials to undertake randomisation appointments, provide IPS support to the treatment group, and undertake employer engagement
Final survey	The survey completed by recruits 12 months after randomisation
4 month survey	The survey completed by trial recruits 4 months after starting the trial
Health-led employment trials	Two trials, funded by the Work and Health Unit, to test a new model of employment support for people with long term health conditions
Individual placement and support (IPS)	IPS is a voluntary employment programme that is well evidenced for supporting people with severe and enduring mental health needs in secondary care settings to find paid employment
IPS fidelity scale	A scale developed to measure the degree to which IPS interventions follow IPS principles and implement evidence-based practice
Provider staff	Those working in provider organisations including employment specialists delivering IPS support, as well as managers and administrators
Randomised controlled trial	A study to test the efficacy of a new intervention, in which recruits are randomly assigned to two groups: the intervention group receives the treatment, while the control group receives either nothing or the standard current treatment
Recruits	People who agreed to take part in the trials and who were randomised to either the treatment or control group
Refer / referral	A recommendation that an individual should be considered for the trial, facilitated by a means to directly connect them to a trial provider
Self refer / self referral	Individual applies for more information about the trial via the trial website or helpline and uses information there (phone number, web form, email) to make contact with the trial provider and request support
Signpost	Recommendation to an individual from a support organisation that they consider joining the trial, by providing them with information (leaflets, reference to website or helpline) leading potentially to the individual self-referring into the trial
Site	The trials were delivered in two combined authorities, which are termed sites
Survey	A research instrument used to collect data by asking scripted questions or using lists or other items to prompt responses. Can be conducted in person face-to-face, by telephone, or by postal or web-based questionnaire
Sustainability and transformation partnerships	A partnership of local NHS organisations and Councils which develop proposals for improved healthcare
Theory of change (ToC)	A description and illustration of how and why a desired change is expected to happen in a particular context. It sets out the planned major and intermediate outcomes and how these relate to one another causally
Thrive into work	The name given to the trial in WMCA
Trial Group(s)	Three trial groups are referred to in the report: two out-of-work (OOW) groups (one in each combined authority), an in-work (IW) group in Sheffield City Region (SCR). These groups are pooled as All OOW and All SCR in different elements of the analysis
Working win	The name given to the trial in SCR

1. Introduction

The report presents the theory-based analysis of the outcomes and causal pathways from the evaluation of the HLTs in SCR and WMCA based on interviews with recruits, staff, employers, stakeholders, and partners. It considers how far intended outcomes were achieved for recruits, health systems, and employers, and interrogates the contexts and mechanisms that enabled these outcomes.

The analysis in this report is based on data collected as part of qualitative interviews with different groups, shown in Table 1.1 and deep dives shown in Table 1.2. The interviews were conducted at 3 points between September 2018 and March 2020, with additional employer, partner and system level interviews held later in 2020 and into 2021. The deep dives were conducted between December 2018 and March 2020, and each explored an aspect of the trials in detail.

Table 1.1 Completed interviews by site

Respondent type	SCR interviews	WMCA interviews	Total
Individuals: treatment group	48	48	96
Individuals: control group	17	18	35
Individuals: longitudinal	34	32	66
Staff	34	36	70
Stakeholders and partners	17	15	32
Employers	8	6	14
Total interviews	157	156	313

Source: Evaluation records

Table 1.2 Deep dives

Theme of deep dive	SCR	WMCA
Employer engagement	Staff focus group and document review	Staff focus group and document review
Delivery of employment support	Treatment group, focus group and document review	6 interviews with treatment group members and document review
Delivery of job development activities	5 interviews with treatment group members and MI analysis	5 interviews with treatment group members and MI analysis
Engagement of primary and community care	3 interviews with partners and document review	Focus group with partners and document review

Source: Evaluation records

The report uses the ToCs (Appendices 6.2 to 6.7) that underpinned the evaluation of the HLTs to explore how far the trial outcomes were achieved. It includes sections on the treatment group, health system, and employer outcomes as achieved by WW in the SCR and TIW in the WMCA. An analysis of management information, including the profile of the treatment group attached to each trial, the support delivered, and the outcomes achieved, is included in appendices 6.8 and 6.9. Each section identifies which major and intermediate outcomes were achieved from the evidence collected in the qualitative research, the contexts and mechanisms that enabled or prevented these outcomes and summarises the causal pathways.^{[footnote 3]} The report indicates where there are apparent differences between the sites in terms of how effectively mechanisms worked to achieve outcomes. These comparisons are necessarily tentative due to the significant differences in context, implementation and reporting between the sites.

Further details about the implementation and delivery of the trials based on the process evaluation data are available in the implementation and 4 month outcomes report. It contains information about the providers, the trial sites, and the recruits to treatment and control groups. It also highlights lessons learned from delivery. Quantitative data covering outcomes is available in the 12 month impact report which covers the net effect of the trials on employment, health and wellbeing 12 months after randomisation. The experience and outcomes of recruits to the treatment and control groups is covered by the 12 month survey report which provides a descriptive analysis of the final survey.

2. Outcomes in the treatment group

2.1 Introduction

This section explores the major outcomes experienced by the treatment group. These were reported during the interviews with individuals and trial staff, and are mapped to those predicted in the ToCs (Appendices 6.2 and 6.3).

The section considers the following causal pathways^{[footnote 4]} through which the HLTs were designed influence outcomes as predicted by the ToC. Causal pathways 1,2 and 3 are relevant to all recruits to the treatment group, namely: WMCA OOW group, SCR OOW group and SCR IW group, while causal pathway 2a is relevant only to the SCR IW group.

• Causal pathway 1: The trial will increase or sustain the income of the treatment group, through them entering and sustaining employment. This will be supported through the employment specialist achieving the best possible job match for the individual based on their skills and interests, which in turn will result in satisfaction in work and thus support work sustainment. Achieving and sustaining employment will also lead to reduced benefit receipt.

• Causal pathway 2: Through being in work, and as a result of the receipt of the IPS services members of the treatment group will improve their health (physical and mental) and make more effective use of health services, thereby reducing the costs on health services in the long run.

• Causal pathway 2a (SCR in-work): Those in work will also be more productive in work, leading to less time off due to ill health (than the control group), with less reliance on statutory and employer sickness support.

• Causal pathway 3: As a consequence of improved health, income and good quality employment, the treatment group will also see their wellbeing improved. The analysis focuses on the contextual factors and mechanisms that contribute to outcomes in relation to: returning to work or getting a new job, experiencing improved health and wellbeing, and experiencing work as a therapeutic outcome.

2.2 New job and return to work

Contexts

At the time of the final survey (12 months after recruits were randomised into the trial), 35% of the treatment group were either in full or part-time work. This section first identifies the contexts that enabled or prevented this major outcome and then considers how these interact with mechanisms and intermediate outcomes.

Contextual factors enabled and prevented the treatment group from starting or returning to work.

For those interviewees in the treatment group who successfully found or returned to work the enabling factors included their previous experience of employment (in some cases within specific sectors of interest), improvements in health since randomisation, and increased motivation to find work for financial and personal reasons. Contextual factors which created barriers to entry into work included ongoing health and wellbeing factors, employer attitudes, individual employment objectives, the availability of suitable opportunities, levels of confidence and motivation, and external factors such as travel and childcare issues.

Previous recent experience of employment was a key contextual factor influencing the treatment group’s entry into new jobs.

OOW interviewees who entered new jobs were more likely to have been in stable employment prior to the onset of their health conditions, although others had been out of work for a number of years. Trial staff also noted that it could be easier to place those who had a recent employment history. Where the treatment group had experience in the particular sector they wanted to find work in, they had a better understanding of their skills and preferences, and this helped to focus their job search activities on areas of interest to them. By contrast, some with high level skills or specialist training reported that they had been unable to find work in their specialist area, or no longer felt able to work in these areas due to their health. These interviewees from the treatment group were more likely to be dissatisfied with the job advice and support offered by their employment specialists, which they said was too generic and overly focused on basic skills to be useful to them.

Across both sites, improvement in health for the treatment group was seen as a key contextual factor in supporting entry to work.

This could include improved management and understanding of health conditions as well as improvements in overall health. In the qualitative research, recruits in the treatment group and employment specialists believed improvements in health meant the treatment group were: better able to focus on applying for work, more confident they would be successful, and more able to work in their new role without exacerbating their health conditions. These improvements were sometimes attributed to support received within the trials, to support from the NHS and other organisations or changes in medication. However, improvement in health is a major trial outcome as well as a context, and this is discussed in section 1.2.2.

Improvement in health was particularly discussed as a key factor in supporting entry into a new job or a return to work for the SCR IW group. For example, a member of the treatment group who entered work reported they had improved the management of their condition as a result of support from their employment specialist, including health support to get on effective medications. The change in medication meant they were more comfortable going back to work. This suggests that health was a greater focus of support received for the IW group and may in part explain the findings from the impact report that the IW group experienced the most significant improvements in their health and wellbeing.

Severe health conditions or difficult personal circumstances were contexts that prevented the treatment group entering employment.

As detailed in the implementation and 4 month outcomes report (section 2.2,3 and 3.2.1), some interviewees in the treatment group had been told by other health professionals that they were not ready to work, and this context meant employment outcomes could not be achieved. In this way, employment specialists identified cases where they were not able to follow IPS principles and progress the treatment group rapidly towards employment. The longitudinal interviews with the treatment group gave an insight into health over time. Where their health was already improving as they entered the trials they could move towards employment, but those with degenerative or relapsing conditions were less able to benefit from support to find work. There could also be setbacks. For example, one of the treatment group interviewees described in the first interview how support from their employment specialist had led to them increasing their working hours. However, at the time of the second interview their worsening health had led them to reduce their working hours again.

A few taking part in the treatment group interviews had been unable to find work due to their personal circumstances, for example, needing a job with specific hours due to family and caring responsibilities, or needing a job which was close to their home to enable an accessible and affordable commute. Others had wider issues to address that they felt were more immediately important than employment, such as finding housing after experiences of eviction and homelessness. When employment specialists were engaged in resolving issues such as homelessness, they were not able to rapidly progress the treatment group to employment. If job brokerage and carving^{[footnote 5]} had been more in evidence, then individuals with personal circumstances such as requiring reduced hours may have been more likely to progress to employment.

It was important that the treatment group was motivated to enter work, a key principle of IPS.

This was often connected with a feeling that it was the right time as their health condition was no longer preventing them working. Employment specialists also discussed the importance of motivation, and felt that the treatment group needed to be motivated to look for work to achieve positive employment outcomes. Motivation is a context as well as an intermediate outcome of the intervention. It is important to note that motivation fluctuated over time although the support delivered nonetheless was expected to improve the treatment group’s motivation overall. Employment specialists generally felt that engagement with the IPS services meant that motivation to work increased and this is reflected in the survey findings (section 7.4 of the 12 month survey report).

Where members of the treatment group taking part in the qualitative research did not know what job role they wanted to do, they appeared less likely to enter employment.

This could include individuals who had experienced lengthy periods of unemployment, had previously held a number of low-skilled jobs or a combination of both. Employment specialists also reported that some in the treatment group had unrealistic ideas about the work they could do, and this made it less likely for them to find work. However, this was a less significant context. In the qualitative research, interviewees in the treatment group who had not entered work as a result of the trials were relatively clear about their future aspirations, and were still in the process of making applications and identifying opportunities.

Employment specialists found that ongoing low levels of confidence in the treatment group prevented entry to work.

This context was not observed frequently within the qualitative research. It generally occurred where a treatment group member had been out of work for a long time, had a severe health condition, or had a lack of awareness of their skills and was also dissatisfied with the support they received. For these, support from their employment specialist did not improve their confidence. Potential factors included the rapid progression to job search which increased anxiety levels for some, communication problems with their employment specialist, or the time-limited nature of the support which meant there was not time to address their barriers.

A lack of skills, qualifications or experience were contexts that prevented the treatment group from entering work.

Employment specialists, particularly in WMCA, framed this in terms of recruits’ starting point on their employment journey at the outset of the trial. They indicated that those in the treatment group with longer durations of unemployment were more challenging to place.

For some in the treatment group there were specific issues, for example a criminal record, not holding a driving licence, or needing a particular vocational qualification for a sector such as childcare that prevented them finding work. In other instances, these barriers were removed. For example, an employment specialist supported an individual in the treatment group to reapply for their driving licence. However, this depended on the eligibility of the individual for funding for additional qualifications.

Skills development was not included as a mechanism or outcome within the ToCs, and there was little evidence of ‘place then train’ in the qualitative research.

The availability of suitable work was an important context that had several dimensions.

Firstly, whether jobs were available in the sectors that the treatment group were interested in, secondly, whether the jobs had good pay and conditions, and finally whether jobs were available to the treatment group given their health conditions and circumstances. Employer engagement activities should have helped to ensure that suitable jobs were available but, as discussed in section 1.4, their effectiveness appeared limited. It is, however, important to recognise that the availability of suitable work is an outcome as well as a context.

Employment specialists discussed the importance of opportunity in local areas to support people into the jobs they were interested in. This was particularly important as they wanted to work in accordance with the IPS Fidelity Scale.

But if they’re genuinely just interested in one thing and I’m trying to widen it because I can’t find anything that’s specific to what they actually want, I feel a bit barred and like I’m not doing IPS.

SCR employment specialist

Employment specialists noted a lack of suitable opportunities available, due to fluctuations in the labour market. In SCR particularly employment specialists felt that many roles were low quality, seasonal, zero-hour, or temporary contracts which might exacerbate health issues and prevent treatment group members from seeking other opportunities. Similarly, WMCA employment specialists discussed the need for employment opportunities to be of good quality and affordable for people wanting to leave the benefits system.

Interviewees from the treatment group also noted that competitiveness in the job market and a lack of suitable opportunities was a barrier to entering work, either due to their sector of interest, the preferred location of work, or contract types. In addition, when discussing the limited availability of jobs which offered preferable contractual arrangements, some indicated it was not sufficiently financially beneficial to move into work if this was on a part-time or temporary basis. There were also some who felt that they were unable to find an employer who would accommodate their health conditions. Those with more severe or degenerative health conditions were more likely to see this as a barrier to them finding employment.

Other interviewees in the treatment group felt their age limited their employment opportunities. They believed that they were getting to an age where employers would not consider them, or that their age might prevent them from accessing opportunities they would like such as apprenticeships, or opportunities to set up their own business. This may indicate that receiving treatment did not overcome them seeing their age as a barrier.

Those treatment group members who described in interviews that age or their health conditions prevented them finding work often experienced other barriers. One with a visual impairment needed more support to make applications as well as feeling that employers did not want to accommodate their needs. Another who felt their age and weight prevented them finding employment also did not have a clear idea about what job they wanted to do. This may indicate that those in the treatment group with multiple barriers were less able to benefit from support mechanisms.

Intermediate outcomes

The major outcome of a new job or a return to work was supported by a series of intermediate outcomes.

These in turn were facilitated by the mechanisms of different features of IPS. The IPS service was intended to lead to a series of intermediate outcomes such as increased confidence and motivation to work, and better understanding of skills and career goals which in some cases then led to work.^{[footnote 6]} The relationship between features of support and outcomes was not always clearly defined. Some in the treatment group were not able to distinguish different aspects of support and attributed change more generally to having a good relationship with their employment specialist. Where employment outcomes were not achieved, a number of intermediate outcomes were still observed for those within the qualitative research, most notably improved confidence and motivation, or a change in their views about work and their condition as a result of the support they had received from their employment specialist. Section 4.8 of the implementation and 4 month outcome report includes a discussion of intermediate outcomes as reported in the survey findings.

Interviewees in the treatment group who had not yet found work were mostly still motivated to keep looking, although some still had concerns about the attitudes of employers to people with health issues as well as a lack of appropriate jobs in their local areas. Some in WMCA were demotivated as they did not think the support from the employment specialist had met their needs, which accords with the survey findings (section 7.1 in the 12 month survey report) where overall the treatment group in WMCA was slightly less satisfied than that in SCR with the support received by their employment specialist, and slightly less likely to feel that their employment specialist understood their needs.

Mechanisms

Mechanisms identified as leading to intermediate outcomes were vocational profiling and action planning, job seeking, a positive one-to-one relationship with their employment specialist, and in-work support. However, not all interviewees in the treatment group who achieved intermediate outcomes also achieved the major outcome of finding work.

Vocational profiling and action planning (planned mechanisms) contributed to intermediate outcomes of more defined career goals, an improved awareness of how to achieve them, and an improved awareness of transferable skills.

This was due to the treatment group having the opportunity to discuss possible options, their concerns, and preferences with their employment specialist. These discussions helped to identify what work was available and realistic, as well options that were of interest to them and would not exacerbate any health conditions. Employment specialists identified that observing the IPS principle that client preferences for employment should be understood and honoured meant they were more likely to be able to support individuals into work due to using these mechanisms.

You’re starting off with a much better understanding of what the individual wants and you’re being driven by what you’ve learnt from them, about them.

WMCA employment specialist

One interviewee in the treatment group reported increased awareness of their transferable skills as a result of updating their CV with their employment specialist, who had asked pertinent questions to help draw out the wider skills they possessed. This process led to a discussion of a wider range of job options and to the interviewee choosing work based on this new understanding of their skills. They noted that support from their employment specialist enabled them to identify and apply for different roles they had not thought of previously which were suited to them and their health condition. These mechanisms effected job outcomes for interviewees with a range of different health conditions, different employment histories and different ages. However, those with more severe health conditions or challenging personal circumstances, such as homelessness or problems with alcohol, were less likely to experience both these intermediate outcomes and major outcomes. As discussed in section 2.3.2 of the implementation and 4 month outcomes report the rapid move to action planning for employment was challenging for these latter groups and may have meant support was less effective for them.

The importance of specific job seeking activities (planned mechanism) varied for interviewees in the treatment group.

Activities such as CV writing, job searches, interview skills, and employment specialists contacting employers on behalf of the treatment group were pathways to employment in some cases. Others in the treatment group found work through their own contacts, after being prompted by employment specialists to use these. As detailed in sections 2.3.2 and 3.3.5 in the implementation and 4 month outcomes report job seeking activities tended to be more focused on basic activities such as internet searches rather than the employment specialist accessing hidden jobs or job carving. Interviewees in the treatment group with a higher level of skills (for example a degree) or a more professional work history (for example they had previously worked in a job which had specific vocational entry criteria) were less likely to report this mechanism leading to employment. Support with job seeking activities did not provide any additionality for these as they already had these skills.

One-to-one support from an employment specialist was a key planned mechanism for achieving employment outcomes as it helped to build resilience and confidence in the treatment group, increasing their motivation to work, and changing their views about work.

This was an effective mechanism for treatment group members with varying work histories and skill levels. However, those with more severe health conditions were less likely to report changes in their views about work while those with low level mental health needs were particularly likely to report changes in confidence. Interviewees in the treatment group reported a change in their views about work brought about by their interactions with their employment specialists. Those in the SCR IW noted that their employment specialist had helped them change their perceptions about what was possible for them in terms of work. They felt without these intermediate outcomes they would not have been as motivated or able to identify, apply for, and be successful with their new roles. One WMCA treatment group member who had entered a new job found that the support from their employment specialist had changed their attitudes to work by encouraging them to think about what jobs were suitable for their needs. This support was particularly helpful for those who had previously thought they would not be able to work again or felt employers may prefer to hire people who did not have health conditions.

That’s something that I really work on because I think that is the difference, a massive difference between people that go out there and really try and that can be resilient… and there’s other people that would say, “Well, there’s no point” if they didn’t get it, or didn’t get the last two “so I’m not going to try again, it’s too painful.

SCR employment specialist

Trial staff identified a causal pathway where mentoring led to improved confidence and motivation followed by further steps towards employment. For example, making more applications, and the treatment group presenting themselves effectively at job interviews. It should be noted though that the extent to which interviewees in the treatment group felt increased confidence had directly led to their entry into work was mixed. Some were unclear about the exact impact it had, and others believed improvements in their health had been more influential than increased confidence.

In-work support was an important planned mechanism for sustaining employment, where it was observed.

There was limited evidence of this mechanism within the qualitative research, and it was mentioned predominantly by interviewees in the SCR IW treatment group. Employment specialists reported that they encouraged disclosure of health conditions, and advised the treatment group about their employment rights and the financial support that they could access as well helping with form-filling. This included an employment specialist directly engaging with an employer to help on disclosure and negotiate improved working arrangements. Few interviewees however reported this. One had experienced a relapse in their physical health issues while employed, and their employment specialist set up a meeting with their line manager to negotiate adjustments which were then taken forward. This support was possible as the interviewee was comfortable discussing their health condition with their employer. Another interviewee found having the employment specialist contact and support them in their workplace helped them to stay in work and consider other options and support available depending on their goals.

MI data from SCR suggests that this mechanism was not in place for the majority of the treatment group who were in or entered work. Employment specialists recorded discussing reasonable adjustments and job brokerage with 19% and 15% of the treatment group, respectively.^{[footnote 7]} However, while the MI data is suggestive of limited in work support, it is important to note that this data does not capture all instances of support. In this way it can be assumed that informal in work support was included in regular catch ups with treatment group members but not formally recorded.

The mechanisms discussed above did not work in all contexts.

Limitations in support (discussed in more detail in sections 2.3 and 3.3 of the implementation and 4 month outcome report) interacted with context. These limitations included:

• Communication barriers between employment specialists and the treatment group:

  In SCR this meant, in a few instances, interviewees had a lack of clarity about the nature of the IPS service and what its purpose was. In WMCA a few had not heard from their employment specialist as frequently as they felt they needed or did not think they understood their needs sufficiently. These were typically interviewees with more severe health conditions or health conditions which affected memory or cognition, as well as an individuals with English as an additional language.

• Limitations in the support that was received:

  In SCR, a small number of interviewees noted that they had difficulties completing forms and applications, and may have benefited from more frequent support, rather than weekly or fortnightly appointments. In WMCA a small number felt the employment specialist had not engaged sufficiently with prospective employers on their behalf, and others would have liked additional help with their CVs. More highly qualified interviewees could find activities such as CV writing of limited use, and some were disappointed that employment specialists were not giving them access to ‘hidden jobs’. This may suggest that the IPS service could have been more tailored to individual needs.

For interviewees in the treatment group with more severe health conditions, time limited support was a factor in preventing intermediate outcomes leading to work. Anxiety about the support ending emerged amongst those who had not secured an employment outcome. There were also instances where interviewees who had found employment at the time of their first research interview had not sustained this in subsequent research interviews and had used up their entitlement to IPS. This meant that they had experienced difficulties either after their entitlement to 4 months in work support had ended (OOW group) or after their 12 months of support had ended (IW group). Some of these interviewees felt their lack of access to further support meant they were less likely to secure new employment.

The limitations to employer engagement discussed in sections 2.3.5 and 3.4.5 of the implementation and 4 month outcome report also prevented progression towards employment. In this way the context of a lack of suitable vacancies can be attributed at least in part to employment specialists not being successful in finding hidden jobs and job carving.

2.3 Improved health and wellbeing

Interviewees from the treatment group discussed improvements in their health and wellbeing as a result of the support they had received. Improved wellbeing was variously described in terms of being more active, having a more positive attitude both generally and towards work, engaging better with health services, and growing more resilient emotionally.

Contexts

The qualitative research showed that interviewees in the treatment group who experienced improved wellbeing had typically either spent a significant amount of time out of work, had been in a variety of short-term roles, had left their previous role due to their health condition, or for SCR IW group, had been unhappy in their current job. Most had depression, stress, anxiety, or other mental health issues.

Interviewees in the treatment group attributed improved health and wellbeing to the trials, although support received elsewhere was also a significant factor. This included counselling, therapy, online mindfulness videos, going to the doctor to receive medication, and better management of their health conditions. Section 7.1 of the final survey report suggests 67% of the treatment group found the support they received beneficial for their health, with SCR IW group most likely to find the support helpful and the WMCA group least likely.

Intermediate outcomes

Improved management of health conditions and improved confidence and motivation were the most common intermediate outcomes for improved wellbeing.

One interviewee felt their employment specialist had given them the skills, confidence, and ability to ‘turn my mental wellbeing around and head it off in a much more positive direction’. In turn, the job they found had enabled them to develop their skills and confidence further. Where interviewees reported improved management of their condition, but that this had not led to improved wellbeing, they were typically still adjusting to changes in confidence as a result of their improved condition management and any changes in work. Where interviewees reported improved confidence and motivation, but it had not led to improved wellbeing or other outcomes, this was typically because they: reported a less positive experience with their employment specialist, had not been able to find suitable work, or their health condition was having an impact on their quality of life.

Mechanisms

The key mechanism to the treatment group’s improved wellbeing was the one-to-one support from their employment specialist, leading to increased confidence and motivation (planned mechanism).

Some interviewees said this was because their employment specialist was a trustworthy person they could talk to about their problems. Those in SCR also described wider factors which contributed to their improved wellbeing, including support to improve their financial management and debt situations, and learning how to balance work-related activity with their health condition. When employment specialists discussed the treatment group achieving health outcomes, they reported building confidence and self esteem as crucial to improving wellbeing.

Better engagement with health services was another mechanism that the treatment group felt had improved their wellbeing (planned mechanism).

The employment specialists’ encouragement to engage with health professionals and follow medical advice meant the treatment group were better able to manage their health conditions. For example, one employment specialist had referred an individual to Improving Access to Psychological Therapies (IAPT) services to support their in-work wellbeing as well as worked with them to disclose information to their employer.

That has been really successful in just getting them some basic support with the anxiety the depression the panic attack and that’s actually meant that they are stayed in work they’re feeling a lot happier at work and it’s also resulted me going in and speaking to the employers as well with the customer’s sort of consent and explaining about the services that we can offer and so the employer realises that the customer is getting the support and actually its very beneficial and that they’re actually doing a better job and they’re more content to stay there and that’s obviously giving sort of a service a really good reputation as well

SCR employment specialist

The importance of making new links with GPs and other NHS services to understand the treatment group’s needs and provide support for them was also identified as an important mechanism for improving wellbeing by employment specialists. However, the limited evidence of case conferencing^{[footnote 8]} as a mechanism meant that not all in the treatment group benefited from these links. Interviewees in the treatment group with more serious health conditions or multi-morbidities talked about the support they received from different agencies although their employment specialists did not have a role in coordinating this support.

Finally, some interviewees in the treatment group identified that entering work or starting volunteering had improved their wellbeing (planned mechanism).

This is discussed in the following section.

2.4 Work as therapeutic outcome

The exploration of work as a therapeutic outcome was limited to those interviewees in the treatment group who reported finding new jobs, returning to work, or increasing participation in work, as well as discussions of what enabled work as a therapeutic outcome with employment specialists. There were three dimensions to work being seen as a therapeutic outcome. Firstly, the treatment group experienced improvements in their health and wellbeing as a direct result of being in work. This typically meant improved confidence and mental wellbeing. Second, the financial benefits of work led to improved health and reductions in stress and anxiety. Finally, the right kind of work meant the treatment group experienced improved wellbeing because they had new opportunities to use their skills and progress in their careers.

Contexts

In combination, the qualitative findings indicated that for the treatment group to experience work as a therapeutic outcome, they needed to have a job that was right for them, a health condition that was not exacerbated by working, and not be motivated to work only for financial reasons.

Work could not be a therapeutic outcome without these contexts. Some in the treatment group, particularly those with more serious or degenerative health conditions, found that work had negative impacts on their health. It was particularly common for the SCR IW group interviewees to report that their health difficulties had been exacerbated by their previous work roles, or that they were unhappy in their work. This was often linked to feeling they had to work for financial reasons even if the work was not suitable for their health. Some interviewees discussed feeling pressured to return to work as a result of reduction in their sick pay, which meant they could not cover their living costs.

Where the treatment group could not find the right kind of work, they were less likely to experience improved wellbeing. This could be because they had entered work that might not have been in their preferred sector or factors such as a long commute or unsuitable hours also made working more difficult. Conversely, when the right kind of work was available, and their health allowed them to experience the benefits of work, there were several key mechanisms and intermediate outcomes that meant the treatment group experienced the therapeutic benefits of work.

Mechanisms

The planned mechanism of one-to-one support leading to increased confidence was important if the treatment group were to find working a positive experience.

Employment specialists emphasised the importance of improved confidence and increased motivation and aspiration to work as this helped change attitudes and meant that the treatment group viewed work as something they could both do and benefit from.

Interviewees discussed their entry into work or improved working conditions as helping to maintain their improved confidence following their engagement with the trial.

Support to find the right kind of work which could then benefit their health and wellbeing (planned mechanism).

Interviewees in the treatment group described how taking part in the trials had enabled them to find jobs they were interested in and passionate about, and that would also support their health needs. One described how their employment specialist had facilitated them getting a job in an organisation which they knew was supportive of wellbeing and where they could take professional qualifications to advance in their career. Another was supported to retrain as a teaching assistant. This came with a loss of income but with a rise in job satisfaction and an increase in wellbeing.

Job carving and ongoing in-work support were important planned mechanisms when observed, although there was limited evidence of them within the qualitative research.

Employment specialists discussed the importance of ongoing in-work support for mental and physical health to enable the treatment group to maintain the benefits of being in work. However, they noted their ability to offer this support was limited where individuals were not willing for their health conditions to be disclosed and as such had worked with them to help them understand and feel comfortable to do this. A few employment specialists also discussed the importance of job carving to help make the experience of work more manageable in relation to individuals’ health. An SCR IW treatment group interviewee described how their employment specialist had helped them negotiate their return to work with reduced management responsibilities leading to improved health and reduced stress. In contrast, another reported their increased confidence had resulted from discussions with their employment specialist that meant they had taken on additional management responsibilities at work.

Employment specialists were still able to provide beneficial in-work support in cases where the treatment group did not want to disclose their health conditions or needs. This was primarily because the treatment group were better equipped to understand the working conditions that would best suit them and manage their conditions at work through more general conversations with their employment specialist about work. This meant that some in the treatment group who found entry into work came with risks for their health (for example, additional stress) had found ways to manage and balance it with the benefits of being in work. In one example, an interviewee found the long commute to their workplace did not support their wellbeing, but felt better to be working in general. Another discussed the importance of entering an agency role which the found less stressful than a permanent contract because they were better able to control when they were working and were able to protect their health this way.

Engagement with employers and healthcare organisations were not consistently observed as mechanisms leading to work as a therapeutic outcome (planned mechanisms).

Some SCR employment specialists felt that more work could have been done in terms of employer engagement and marketing the service to employers so that they may better understand the benefits of improved health to their workforce, and the impact this could have on their business. Nonetheless, one SCR employment specialist described helping reduce attrition in some large companies by working with them on their approach to the health of their employees. In contrast, employment specialists in WMCA felt that GP engagement was more important, to tackle traditional ideas of the role of work in patient recovery, which limited their engagement and referrals to the trial. The limitations of engagement with employers and healthcare organisations are discussed in sections 1.3 and 1.4.

2.5 Summary of causal pathways

Overall, the main outcomes achieved by the treatment group as captured within the qualitative research were related to employment, either through entering or sustaining work, with one-to-one support as the key mechanism. This meant that the treatment group gained the confidence and motivation to find work which emerged through vocational profiling and needs assessment, and activities to find and prepare for entering work. Entry to work was, in some cases, further supported through improvements in individuals’ health.

Where SCR treatment group interviewees reported no intermediate outcomes, the qualitative research indicated an interrelationship with combinations of the severity of health issues and some limitations in the support. Where WMCA interviewees saw limited or no intermediate outcomes there was an interrelationship with significant breakdowns in communication with their employment specialist. However, some of the interviewees who reported limitations in the IPS support also had significant barriers, either communication or health difficulties that had an impact on their ability to engage with support. Findings from the survey (section 7.4 of the 12 month outcomes survey report) show that those with better health and higher mental wellbeing were more likely to have more positive views of the support, and to perceive it had helped them find employment.

In the qualitative research, health and wellbeing benefits emerged for the treatment group a result of encouragement from their employment specialists and others to improve management of their condition, increase their motivation and confidence, and support the sustainment of work through the discussion of reasonable adjustments. These were more evident within the SCR IW group. There was very limited evidence within the qualitative research of the treatment group experiencing increased income as a result of a new job or a return to work. This is consistent with the findings of the 12 month impact report where the trials were not found to have any impact on earnings.

In some instances, initial challenges in engaging with employers and health organisations (as observed in the implementation and 4 month outcomes report, sections 2.2.1, 2.2.5, 3.2.1 and 3.2.5) may have limited improvements in health and wellbeing in the treatment group, and had implications in respect of the limited evidence for therapeutic benefits of sustained work.

Health systems outcomes

3.1 Introduction

This section considers the health systems causal pathway which includes the following major outcomes:

• that health professionals view conversations about employment with patients as a key part of their role

• that health professionals have a better understanding of the relationship between work and health and recognise the therapeutic value of work

• that health professionals see it as part of their role to make wider referrals outside of the NHS, including to employment services.

The trials were designed to effect change within health systems and on health professionals’ behaviour to better integrate work and health services and support for patients (Appendices 6.4 and 6.5). Health system change was anticipated to support outcomes, and to be a valuable outcome for the local areas. The health systems ToC predicted that the trials would achieve change by first creating awareness amongst healthcare services and generating referrals to the IPS services, and then promoting effective partnership working between health teams. This would then lead to major outcomes: that healthcare professionals would value IPS and discuss employment options with patients, conversations about employment between health professional and patients would be normalised, and appropriate referrals to support would be made.

Beyond the lifetime of the trials, it was anticipated that this change would lead to longer term outcomes: more effective use of Fit Notes among GPs, and health professionals referring to non-NHS services alongside any health referrals, with the continued integration of work and health prioritised through the Sustainability and Transformation Partnerships.

During the trials’ design and delivery, some contextual factors affected ways of working, delivery mechanisms and ultimately their ability to realise the ambition to contribute to widespread health system change. Health systems are complex, with several layers. The trials’ development coincided with wholesale changes to health system structures to support integration and public sector reform. Primary Care Networks (PCNs) were being established and new legal entities created. The trial in SCR was commissioned via the Clinical Commissioning Groups (CCGs), and representatives from the CCGs and Integrated Care System (ICS) sat on the steering group. This supported South Yorkshire Housing Associations (SYHA) (the IPS provider) to build links and access healthcare settings, and the strategic level engagement and integration was complemented by direct communications to healthcare staff. In WMCA, a CCG also led the commissioning process. While awareness raising activity to encourage health referrals was led by the primary care engagement lead, supported by ongoing engagement undertaken by the providers, there was less health system involvement in management and governance structures in WMCA (as detailed in section 1.3.1 of the synthesis) report. In WMCA, , the health sector was not as systematically or routinely engaged in the trial design and set-up. This was led by a contracted organisation (Social Finance) rather than local partners as in SCR. Structural issues also affected ease of health sector engagement in WMCA as health services were not part of the remit of the Combined Authority (see the synthesis report).This may help to explain the evidence from the impact report that the SCR trial had a positive impact on health and well-being while the WMCA trial did not.

The following section is based on interviews with staff, stakeholders, and partner organisations. Each planned mechanism of the health system ToC is tested against the evidence gathered in the evaluation to understand how far it operated within the trials and the context which supported or inhibited the planned change to occur.

Awareness of the trials and referrals made

The engagement of health practitioners in referring patients to trials in SCR and WMCA underpinned health system change (planned mechanism).

It was anticipated this mechanism would build health practitioners’ understanding of the influence of employment on health. To enable this change, primary care and community healthcare services were intended to be the main referral sources to the trials. However, in practice, this was not fully or consistently realised. Self-referrals and referrals from other sources were most common, accounting for more than half the referrals in both trial sites. Nevertheless, SCR recorded more direct referrals from health settings (42%, with 18% from a GP, and 24% from specialist care service), than WMCA where 20% of referrals came directly from health service providers (16% from a GP, and 4% from a specialist care service). Local factors help to explain these differences and shed light on the contexts and factors that enabled this change mechanism in healthcare settings.

In both sites it was challenging to get agreement and commitment to refer patients across layers and parts of the healthcare system within the trial design timescales.

In SCR, engagement with health partners during the design phase was more wide-ranging and comprehensive than in WMCA. It was enabled by more established partnership working between employment and health services, good links with social prescribing and IAPT services, and involvement from senior managers in the health system. These structures meant trial staff were better able to undertake successful initial engagement with referral agencies, and to negotiate with gatekeeping organisations. For example, in SCR, an employment specialist noted they first met senior managers within an Orthopaedics Outpatient department, before then being able to engage with all healthcare staff in that setting. Other employment specialists in SCR discussed regularly attending IAPT team meetings.

The design stage underestimated the amount of time required to systematically engage GPs and other parts of the health system and to consider their varying capacity and motivation to engage.

Staff reported there was insufficient activity with health providers and GPs to build initial awareness and engagement and believed more intensive relationship building and contact over a longer lead-in time was needed to gain the number of referrals originally anticipated from GPs (see implementation and four-month outcome report section 2.2.1 and 3.2.1). Delay to the anticipated trial start date, pending approval by the Health Research Authority (HRA), was also reported to have made engaging with health services more challenging.

Advocates within existing health structures were important to encourage referrals (unplanned mechanism).

Leaders such as public health champions or GP leads were noted as being influential advocates for the trials and supported engagement from other health professionals. The existing effectiveness and maturity of health partnerships, and support they could offer the trials, enabled engagement. However, the range of other programmes targeting work and health in both sites sometimes meant some professionals were less keen to get involved.

I think whether or not the local implementation boards, whether they were functioning well, whether there’s a public health lead champion as well I think makes a difference to get that engagement across systems and system partners. Then I suppose it’s the history of what else has happened before in those areas around employment and what the organisational memory is for frontline clinicians.

SCR health systems stakeholder

While there was central co-ordination, trial providers also needed to establish health sector relationships locally.

The organisational context supported the ease of this engagement. It is notable that the NHS Trust provider in WMCA received 28% of their referrals from GPs, 15 percentage points higher than any other WMCA provider. This provider had a different context as an NHS organisation. As such, the employment specialists who were also seen as NHS staff had greater traction and were better able to engage health sector colleagues with the narrative that the trials might reduce demand for wider health services. Being employed by an NHS organisation had other advantages. These employment specialists could send communications to health sector staff which were perceived as internal to the NHS rather than arriving from an external contractor. This built trust, quicker engagement and attracted more referrals. This provider was also able to draw on support from organisational experts about the benefits of good quality employment for individuals’ mental health and wellbeing. In contrast, employment specialists in the WMCA providers specialising in employment services spoke of difficulties in becoming familiar with NHS systems and processes, and differences in the language used. In SCR, the context of collaboration during design with health systems was a key factor that enabled easier engagement.

Neither engagement leads within the programme team, nor employment specialists had capacity to undertake detailed engagement with the full number of potential referral agencies, including successfully negotiating with gatekeeping organisations (for example, CCGs, local medical committees) (a planned mechanism).

When the WMCA programme team tried to incentivise the engagement of GPs (by setting a target number of referrals based on patient populations) (see section 4.2.1 of the implementation and 4 month outcomes report), the proportion of referrals from the healthcare sector increased. This strategy focused on specific settings rather than the whole trial geography. Overall, the number of referrals from GPs was similar between the two sites. The presence of strong and active GP leads who fully engaged with the trials was a useful mechanism to promote referrals. As discussed in the implementation and 4 month outcomes report (sections 2.2.1 and 3.2.1), the operation as a randomised controlled trial (RCT) created reluctance among some healthcare staff to refer patients as some would be allocated to the control group where they would not receive employment support unless they sought it from other services.

Different resources and contexts led to inconsistent engagement with the trial both within and between healthcare settings.

While healthcare settings were able to make referrals, time constraints, and the existing purpose of their patient interaction meant that they did not all have the same motivation or natural opportunity to refer. The capacity of health professionals to discuss employment and refer varied depending on the nature of the appointment and the length of time during which the patient was engaged with their service. Furthermore, the data captured on systems did not always enable health staff to review records or send communications based on a patient’s work situation. These barriers were especially acute for GPs.

I think they’d [GPs] all see the value, but if you’ve got a queue of patients outside your door you can’t maybe give them long enough individually to make this kind of referral.

SCR health systems administrator

As noted in the implementation and 4 month outcomes report sections 2.2.1 and 3.2.1, the large number of self-referrals recorded will include some recruits who heard about the trials from within healthcare settings.

Longer appointment times and ongoing engagement enabled referrals (in smaller GP practices, or as standard for some types of health appointment).

For example, staff and wider stakeholders viewed IAPT services as effective referrers because they regularly engaged with the same patient over several weeks or months. Additionally, the nature of the support meant that these health staff felt it was reasonable to discuss work with patients. It was also mentioned that IAPT staff were more likely to be aware of and supportive of IPS services, which also led to positive engagement. Other health partnerships were felt to be more reactive and less consistent referral sources. For example, they might mention the service if employment came up in a conversation with a patient a GP knew well.

We found IAPT were good at getting people in and then knew who needed to go in, they were particularly good, but GPs were pretty good at it, it varied between practice to practice. I think some of our smaller practices were probably more effective than the really big ones, probably because you see the same GP on a regular basis.

SCR health systems administrator

Referrals from MSK services were not particularly prevalent in either site.

Referrals were only felt to be appropriate from these settings if a patient was engaging because of an issue that affected their work or ability to work. The nature of the interaction and likelihood of fewer consecutive appointments did not support referrals either. This underlines the importance of the relationship between the healthcare professional and patient, and the context and purpose of their appointment. Employment was easier to raise in some situations than others.

Trial stakeholders felt there was a degree of over optimism about uniform primary and secondary healthcare engagement in referrals.

While all settings had access to referral systems, the varied time spent with patients, and the focus of patient interaction, meant that some health settings were more willing to refer patients than others. Locally, different structure and commitment from senior leaders also created a mechanism for engagement:

In our area if you were to highlight the areas where we have got it embedded: a really good GP leader, working in a strong primary care network, that has the ethos behind it, and the leadership to make it happen, all of those four boxes would absolutely be ticked. But it’s not the same everywhere and I think that’s the bit we would probably need to honest about.

WMCA employment and health systems stakeholder

The planned mechanisms to underpin health system change held true in certain contexts.

Different organisational cultures, practices, and barriers the sites experienced show that differentiated and targeted approaches were required to engage the many organisations that form the healthcare system. Health professionals interact with patients for different reasons and over varied timeframes. An understanding of these different contexts is crucial to ensuring the feasibility of the planned mechanisms.

3.3 Partnership working between health teams and trial staff

The establishment of co-working between employment specialists and health teams intended to assist the coordination of support for the treatment group and change in health systems (planned mechanism).

Co-working was intended to support the assumption outlined in the intervention-level ToCs that the integration of employment and health support enables complex patient needs to be addressed in a coordinated way, a key principle of the IPS approach. It was recognised that integration could be implemented in several ways, not least because operational constraints meant that full co-location would not be possible in all cases. Envisaged integration options therefore included full-time co-location, part-time co-location and modified co-location (that is, where IPS specialists were based alongside health teams but provided support in the community).

The pace with which capability and capacity (C&C) checks were granted to enable employment services to deliver the trial in health settings varied between and within the trial sites.

Employment specialists needed to be authorised locally by health systems to assess and confirm whether the organisation had the resources, policies and service users required to successfully deliver the trial to time and target. In WMCA, C&C checks were granted from August 2018, but were not all in place until early 2019, 8 months into trial delivery. In SCR, by contrast, all trusts had completed and been granted C&C checks by May 2018 as trial delivery commenced. In WMCA, these delays prevented the aim of co-location.

Co-working was the exception rather than the rule, but where it occurred it supported integration.

Trial providers were not supported to integrate systematically with health services and employment specialists felt that more planning and momentum was needed to set up effective partnerships. There were other barriers such as a lack of suitable physical space and infrastructure. In WMCA, it was notable that the health sector provider found it easier to make such arrangements, seemingly because they were part of the NHS, and had clearance to operate in health settings. Co-working was also enabled where employment specialists had been recruited from NHS roles as they brought health sector networks and relationships. In SCR, co-location was supported by health partners being part of the trial design team, and from ongoing work and resource to develop social prescribing services and non-clinical support services across the trial site. Health settings that made referrals to employment and community services prior to the trial were most receptive to full engagement with trial providers. The result was some examples of health team co-location in settings keen to embrace the employment specialist, including some GP practices and within IAPT.

Where co-location (planned mechanism) occurred, it had several benefits, as envisaged in the ToC.

Physical integration was intended to support co-working practices and encourage referrals. Providers and health partners reported an increased likelihood of referrals from sites where there was co-location, increased trust (both from potential recruits and referring practitioners) and higher attendance at appointments. It also led to increased understanding of the benefits of IPS support amongst health practitioners, and a better level of support through joined up care, cross-referral, and signposting. Where there were closer relationships with healthcare teams, employment specialists were able to speak to them about the trials and individuals they referred (see the implementation and 4 month outcomes report sections 2.2.1, 2.3.4, 3.2.1, 3.3.4). The context made it challenging to implement this mechanism but where it was possible, it supported health system outcomes.

However, full co-location was not always necessary to gain some of its benefits. For example, there were cases of employment specialists attending ‘GP huddles’ where they discussed the trials and provided a named point of contact for health professionals to answer questions and build trust in the service. While these approaches facilitated awareness, and referrals, they did not support other aspects, such as case conferencing.

3.4 Major outcomes

It was anticipated that the intermediate outcomes would lead to the following major outcomes:

• that health professionals view conversations about employment with patients as a key part of their role

• that health professionals have a better understanding of the relationship between work and health and recognise the therapeutic value of work

• that health professionals see it as part of their role to make wider referrals outside of the NHS, including to employment services

However, because the enabling intermediate outcomes relating to referrals and partnership working were achieved only in some locations, and not habitually across the trials, there are some examples where these major outcomes occurred, but they are not consistent. A greater focus on these in implementation would allow this causal pathway to be better interrogated.

The planned mechanism of positive feedback from patients led to greater understanding among some health professionals of the relationship between work and health

This included positive stories from patients about the support they had received and the difference they felt the trial had made. The planned mechanism of having employment specialists embedded in teams enabled them to have conversations about work and health, and for employment specialists to be a source of support and expertise on employment and related issues.

However, most examples of these outcomes were found among health services that reported they recognised and understood the relationship between health and work prior to the trials. Greater consideration at the outset could have been given to how to begin to change behaviour among staff and create a strong presence and co-working in settings where this was not already the case.

There were isolated examples of the difference made by the trials to health settings. For example, one MSK practitioner in SCR who discussed work and health with patients prior to the trial’s implementation reported that WW gave them somewhere to refer suitable patients rather than signposting them to support from their GP or manager. They felt that for those allocated to the treatment group, the degree of support IPS offered was good and its ethos of working with people’s preferences contributed to a positive referral. This, combined with positive feedback from patients they had referred to the trial, ensured that they continued to refer into the trial throughout its delivery.

3.5 Summary of causal pathways

The complexity of the health system and the time it would take for the trials to engage effectively and methodically was underestimated. The anticipated causal pathways for health systems outcomes were realised in some contexts within primary and secondary care more readily than others. This depended on aspects such as prior knowledge and support for work as a health outcome, and local leadership and engagement structures regarding health and work. The length of time during appointments, the content of appointments and whether they offered the opportunity to discuss work also affected engagement. There were examples of the main mechanisms having the positive impact envisaged on referrals and co-working, but because providers were not able to consistently establish these links in the trial timeframe, the planned health system intermediate outcomes were not consistently evidenced, and consequently neither were the ensuing major outcomes. It might be beneficial in future designs of IPS services to give greater consideration to identifying the parts of the health system most able to engage with the trials, rather than treating the health system as a single entity.

4. Employer outcomes

4.1 Introduction

The analysis in this section of the report is based on the employer causal pathway with the following anticipated major outcomes:

• employers being more willing and confident to employ disabled people and people with health conditions

• employers being more knowledgeable and confident, and therefore more receptive to, providing support to disabled people and those with long-term health conditions in the workplace, through modifications to roles or the provision of support services

• employers being more aware of and having more confidence in the value of external health-related support for their workers and therefore being more likely to make use of it

The ToC predicted that the IPS services in the trials would achieve these major outcomes through the intermediate outcomes of employers developing awareness and trust in the trial, being receptive to job carving and workplace adjustments and employing trial recruits.

The evidence for this section is drawn from interviews with trial staff, recruits in the treatment group, and employers engaged by the trials. The interviews provide a snapshot of findings exploring the intermediate outcomes of the treatment group in relation to employers and the contexts and mechanisms that supported these. It was not possible to recruit the full intended sample of employers and so these findings may not fully reflect the outcomes achieved during the trial. The findings are also potentially limited due to the time it may take for outcomes to be observed at an employer level.

It was acknowledged in design stage interviews that employer outcomes were ambitious. There was also limited evidence that consideration on how to achieve these outcomes had been built into the trial design and delivery. For example, the trials did not have strategic employer engagement plans and employers were not involved in the design of trials, or represented on programme boards. This meant that there was no system level mapping of how and why different employer groups would engage with the IPS services. The achievement of employer outcomes was therefore wholly dependent on individual employment specialists’ success in engaging with individual employers, with limited consideration of how mechanisms such as finding hidden jobs and job brokerage would work.

As noted in sections 2.35 and 3.36 of the implementation and 4 month outcomes report, there was a recognition from trial staff at all levels that employer engagement was not delivered frequently enough, and they were not always able to follow IPS principles. This was particularly an issue in the early stages of the delivery where the key focus was on recruitment and gaining referrals into the trials. This further constrained the time available to observe systems-level employer outcomes.

Additional training was provided to trial staff and dedicated employer engagement resources were allocated. Nonetheless, the limited time employment specialists had to focus on this issue due to the size of caseloads and varying levels of confidence in delivering employer engagement activities remained a challenge through delivery. Trial staff also felt that the IPS fidelity model did not necessarily reflect working culture in England and that often employers were too busy to engage in the way envisaged by IPS.

The scale of the trials was potentially another issue, with several thousand recruits in the treatment group in each site. This may have affected the ability to personalise work search and particularly, the aim to secure hidden jobs. The trials were also only one of a number of employment support interventions in both sites and it may have been difficult for employment specialists to develop and maintain contacts in a crowded field.

4.2 Delivering awareness of, and trust in the IPS service

The planned mechanism of employment specialists building relationships with individual employers underpinned the employer ToCs.

Employer contacts were made through workshops, contact for recruitment purposes, existing relationships, and contact through the treatment group to negotiate adjustments. In SCR, the employer interviewees had mostly engaged for in-work support and described having heard of the trial through stakeholders such as local councils, the South Yorkshire Regional Authority, and South Yorkshire Chartered Institute of Personnel and Development (CIPD). One had heard about the trial through LinkedIn advertising. In WMCA employers reported their relationships with the trial’s IPS service were developed in response to specific job adverts which employment specialists followed-up, particularly in the health and social care sector.

Employment specialists needed to have sufficient time to develop these relationships.

As discussed in the implementation and four month outcomes report sections 2.3.5 and 3.3.5, employment specialists felt unable to prioritise employer engagement while the trials were recruiting, and this was the most significant context that prevented relationships developing. Where the first step towards employer engagement had been made, particular mechanisms and contexts meant a relationship was more likely to develop. The employers interviewed for the evaluation were typically larger organisations with Human Resources (HR) departments and policies in place to support employee health and wellbeing. However, this does not necessarily reflect the kinds of employers who engaged with the trial due to the small number of interviewees.

Where an employer had an existing commitment to supporting their employees’ health and wellbeing, they were more likely to be willing to engage.

Employers interviewed in SCR were interested in supporting their existing staff. This included choosing to engage due to a particular interest in accessing mental health training and awareness raising for their staff. There was limited evidence of employment specialists being successful in engaging with employers where support systems were not already in place.

Employers with recruitment needs were more likely to see the value of the trials.

Employers interviewed in WMCA were seeking to recruit through the trial. Employers who had experienced previous recruitment challenges particularly welcomed contact from employments specialists for this reason. For example, they had received applications that were either of poor quality or from people they did not consider to be serious about a job in their sector. They liked the idea that candidates were ‘pre-screened’ by the trial, were genuinely interested in the role, and felt WMCA employment specialists understood their needs well. Some employers also had recruitment targets under social responsibility schemes which meant they were more likely to see the value of the trial.

Workshops were an effective way of building trusting relationships (planned mechanism).

Attendance at initial workshops and training offered by SCR helped to increase employers’ awareness of what the service could offer and, in some cases, led to specific employees being referred to or self-referring to the trial. WMCA employment specialists also highlighted the importance of workshops that allowed them to create relationships with employers rooted in an initial positive experience and seen as reciprocal rather than transactional. It was noted by one employment specialist in SCR that the events could have been more focused on the entry to work element to encourage interest in recruitment from the trial, which not all employers seemed to be aware of. The time commitment involved in attending workshops meant that these again attracted employers who were already interested and wanted to build their knowledge.

Cold calling was seen as challenging by some employment specialists, although it did lead to some positive outcomes (planned mechanism).

Some of the employment specialists discussed engaging with employers through a ‘cold calling’ approach, and where possible described how personal relationships with employers could be developed and maintained through face-to-face visits to check on recruitment progress and have discussions about specific applicants. Employment specialists who had previously worked in employer engagement roles and already had established relationships with employers were more likely to see this as an effective strategy. It meant they knew who to contact, understood the needs of each organisation, and knew whether they would accommodate disabled people and people with health conditions. The ToCs did not reflect the time it would take employment specialists to build and maintain relationships with employers.

A planned mechanism was that employers would be engaged through employer engagement services, but there was limited evidence of this within the evaluation.

Employment specialists successfully engaged with JCP and employment support charities, but this was generally focused on the needs of the treatment group rather than employers. However, there was evidence that SCR’s approach of raising awareness among employers through local stakeholders such as the South Yorkshire Regional Authority and South Yorkshire CIPD was successful. Further strategic use of other agencies and services may have increased the trials’ reach with employers.

The barriers to developing employers’ confidence and trust in the IPS services meant the possibility of achieving other employer outcomes was limited.

However, the discussion below shows how the causal pathway could work once this initial blockage was cleared. This is illustrated by a case of a highly engaged employer in SCR who connected to the trial through LinkedIn. They had regular meetings with WW to review the needs of their employees and discuss how their workplace could effectively respond and make adjustments. This was a feature of the SCR IW treatment group giving employment specialists access to their employers. The relationship is indicative of the potential of the trials to build effective relationships with the possibility of leading to major outcomes. However, it should also be noted that this was a large employer with an occupational health department and established focus on employees’ mental health due to their industry and ways of working.^{[footnote 9]}

4.3 Receptive to job carving and work place adjustments

Some employment specialists and some individuals in the treatment group reported successful cases of job carving and workplace adjustments, but these tended to be isolated occasions where consent to disclose had been obtained from the individual. Consent to disclose from the treatment group was therefore an important context in enabling this outcome.

In the case of a highly engaged employer, regular contact with trial staff was an important mechanism for ensuring current employees had the adjustments they needed (planned mechanism).

This included fortnightly meetings and email contact to make sure that appropriate support was in place for employees with health conditions. However, as mentioned, this employer was already receptive to making adjustments and contact with WW enabled them to support employees better rather than significantly change their processes.

So, the people that are at risk of losing jobs because [of] sickness and absence. So, we look at those cases individually primarily because we want to make sure people are supported and they’re receiving everything that they should be receiving, which there are certain things that [we] would do as standard for people that are struggling with any kind of physical or mental conditions.

SCR employer

There were examples of employment specialists negotiating for workplace adjustments on behalf of their caseload (planned mechanism).

Two employers discussed having made adjustments for employees as a result of the support from WW. This involved the employment specialist directly negotiating the adjustments on the employee’s behalf. These employers acknowledged that while they may have been happy to make these adjustments anyway, they appreciated the involvement of the employment specialist. They felt this may have made the employee (a recruit to the treatment group) more comfortable to raise their health issue and made the employer more confident in putting new adjustments in place. These were both large employers with an HR department but did not have their own occupational health services. WMCA employer interviewees did not report having made any reasonable adjustments. However, they noted already having experience of hiring people with health conditions and adjusting shift patterns and working arrangements in order to accommodate employee requirements and return to work.

The interview data contained examples of employment specialists negotiating for adjustments, although evidence of the impact on employers is limited. The implementation and 4 month outcomes report (sections w.3.2 and 3.3.2) identified a range of adjustments that had been put in place for members of the treatment group who entered work, including support such as a new chair, reductions in working hours, or changes to job roles. In some of these cases, the employment specialist had intervened directly. However, in others the employment specialist had successfully supported the treatment group to request adjustments for themselves.

Yes, it’s been focused totally on keeping me in that job, because it was identifying the problems, the barriers to that job, what could we do to remove those barriers, what were my rights, you know, and obviously she’s been instrumental in sorting out WRAP plans and stuff like that, that I didn’t even know about.

SCR IW treatment group

Interviews with members of the treatment group suggested that the mechanism of employment specialists negotiating with employers was not consistently in place. Not all had been offered support from their employment specialist to directly discuss workplace adjustments with their employers.

Employers also needed to be supportive of health and wellbeing for this outcome to be achieved.

In the context of an employer who was less sympathetic to employees’ wellbeing the mechanism did not work as well. One interviewee in the treatment group provided an example of how their employment specialist had not been able to successfully negotiate adjustments as their employer was not interested in supporting them.

It is notable that these outcomes were more often seen in the SCR IW group, suggesting it was easier for employment specialists to make links with employers with the trial group that already had relationships with employers. This shows the importance of getting access, but may also reflect that employers are more willing to make adjustments for existing staff rather than new employees. This was not anticipated within the ToC and suggests further consideration of how employment specialists could get access to employers may be valuable in the future design of IPS delivery.

4.4 Confidence supporting ill or disabled staff

The planned mechanisms for increasing employers’ confidence in supporting ill or disabled employees were primarily the brokerage activities discussed above.

Change was limited here for a number of reasons. As noted previously, employers who engaged with the trials typically already had experience of hiring people with health conditions. They were already able to offer training and flexible hours, had experience supporting people back into work, or had targets for supporting disabled people.

The workshops offered by trial staff in WMCA, and SCR were identified by employers as valuable (planned mechanism).

SCR employers valued the mental health workshops and informal sessions they were offered, and felt this had made them personally more confident in supporting ill or disabled staff, although some did not feel able to comment for the company as a whole. A WMCA employer noted that a key barrier when recruiting people with health conditions would be understanding of the cost of reasonable adjustments and how the Government could support these costs. Discussion of this at a workshop was therefore useful for them. General information sharing was therefore a potentially important way to overcome employers’ concerns about supporting people with health conditions.

An SCR employer found that the process of making adjustments had improved both their own and their organisation’s awareness of how to support employees with health issues (planned mechanism).

This had made them more willing and confident to employ people with health conditions in the future. However, as noted previously, they still valued ongoing support from the trial.

There were limited examples from interviews where job brokerage activities (planned mechanism) led to perceived changes in employer attitudes and behaviours.

These included a greater focus on employer health and wellbeing at an all-staff level, staff communications that prioritised employee wellbeing, training sessions on mental health, and initiatives such as health and wellbeing weeks. These were reported more by interviewees in the SCR IW treatment group.

I mean the other positive thing is the wellness action plan has triggered changes at work which should have happened months and months ago but now they’ve put out a document saying that employee wellbeing is priority for this year…all that’s happened as a result of that action plan and me being ill. So, it’s all good. It’s all gone in the right direction.

SCR in-work treatment

4.5 Employing the treatment group

The planned mechanism that led to employment outcomes was primarily assistance with recruitment processes rather than job brokerage.

WMCA employers who had taken on a member of the treatment group valued the support of the trial in this, either because they had experienced challenges with recruitment (such as applications from people not interested in their sector), or due to having specific targets and objectives to support people with health conditions. None of the employers interviewed who had engaged with WW had recruited from the treatment group. Instead they had engaged to support their existing staff. However, interviewees across both trials discussed how engagement with employers had led to job outcomes.

Employment specialists facilitated the treatment group getting work through engaging with standard recruitment processes (planned mechanism).

However, this involved getting in touch about existing vacancies rather than accessing “hidden jobs” as envisaged by the IPS fidelity principles. Employers reported that applications received from the treatment group were often higher quality than other applicants, and that these candidates were well prepared, tailored their applications to the job description, and used relevant examples from daily life to counter a lack of work experience. In one instance an employer’s job adverts were now being shared with provider staff as one of their standard ‘agency’ routes. Interviewees from the treatment groups in SCR and WMCA also reported examples of their employment specialists contacting employers speculatively as well as in response to recruitment opportunities. This led to employers recruiting people from the trials, although the effect of this cannot be mapped back to the employer.

She went around prospectively looking for places, and she set it up for me. So, I’m still there now…She rang them up, and then arranged, like, an informal interview, where I went down there and spoke to them.

WMCA treatment group

Employment specialists with an established employer relationship were more able to support the treatment group to find employment opportunities (unplanned mechanism).

Good relationships between employment specialists and employers including those established pre-trial allowed access to employment directly rather than through the standard recruitment process.

Work taster opportunities were a planned mechanism for sourcing employment.

However, in some cases employers offering tasters had found the treatment group to be too far from the workplace to be considered for recruitment. One employer felt they had already done much to make their jobs appropriate prior to trial engagement, and that the impairments of some of the treatment group meant that they were not able to take on board necessary information and training about work performance, leaving the impression that some very high needs cases had been referred to them.

4.6 Summary of causal pathways

The causal pathways observed for employer outcomes were limited due to some challenges that delayed their achievement during the trial delivery and evaluation period, in particular, limitations in the design of employer engagement approaches in the trials and limited staff time and expertise, to deliver these activities (as discussed in the implementation and four month outcomes report sections 2.3.4 and 3.3.4). For future delivery of IPS in settings similar to the HLT it may be beneficial to include more system level mapping as part of the ToC design process to gain more granular insights into the contexts of different employers and their motivations for engaging. The qualitative evidence presented here suggests that engaging with employers to support existing staff was a stronger pathway. It might be beneficial in future designs of IPS services similar to those in the HLT to differentiate more between the IW and OOW group within the ToC at design stage.

It is difficult to conduct a full comparison between the two areas in how effective employer engagement was and how far employer outcomes were achieved. For example, the MI data for SCR (appendix 6.9) records that only 15% of the treatment group were offered job brokerage or job customisation. This data was not collected for WMCA. However, it is clear that employer engagement was more effective with the SCR IW group rather than the OOW group.

Successful outcomes tended to be achieved when employers had a good understanding of the trials’ purpose and offer upon engagement. The use of mental health training sessions for existing staff and line managers in SCR was a strong example of how this could be done effectively. Employers who already supported their employees’ health and wellbeing needs were more likely to engage. However, this meant that the effect of engagement was likely to be more limited than if this was a new approach. Employers who had difficulty recruiting or were recruiting regularly were also more likely to engage with the IPS services. In future delivery of IPS in contexts similar to HLT it may therefore be beneficial to further consider different employer groups and their motivations for engaging at the design stage.

When job outcomes were secured, there is limited evidence that they led to further outcomes such as increased confidence and willingness to support employees with health conditions and disabilities. There are several potential reasons why these outcomes were not in evidence, not least the limited evidence from employers. The reluctance of some in the treatment group to disclose the nature of their condition and their needs to employers was a barrier. It meant that further mechanisms of ongoing support did not occur and so opportunities to further engage employers were lost. The impact of taking on a single employee, as was the case in most examples, was also likely to lead to a limited impact on workplace behaviours.

5. Conclusion

In summary, the assumptions and causal pathways outlined in the treatment group’s ToC during the design of the trial generally proved to be predictive of the experiences of the treatment group. There was evidence of individuals reporting outcomes such as new jobs or improved wellbeing and attributing this to the support they received. This is supported by staff perceptions of the treatment group’s outcomes. Where outcomes were not evidenced, the qualitative data suggested that this was often due to individuals not being ready to work, having serious or multiple health conditions, or to external factors such as lack of suitable jobs or difficult family circumstances.

The support received by the treatment group did not consistently help them overcome these contextual factors. Shortcomings in support mechanisms included general job search rather than accessing hidden vacancies, generic rather than tailored support particularly for those who were more highly skilled, not discussing workplace adjustments, breakdowns in communication, or not enough support with tasks such as completing applications. When the needs of the treatment group were more severe than anticipated by the trials’ design, then rapid progression to job search combined with time-limited support (IPS-LITE) were constraining factors.

There is less evidence of major outcomes for employers and health systems. There are similar reasons why the assumptions and causal pathways did not work as intended in these ToCs. The design stage of the trial did not fully consider how to engage with employers and health organisations at a strategic level. This meant that there was not enough differentiation between different kinds of employers or different health organisations and too much depended on the individual actions of employment specialists. When these initial blockages were cleared then there was effective engagement. However, the effect of this was often limited because the employer and health organisations who engaged with the trial were already sympathetic to its aims. With regard to employers, the extent change was also limited to the effect of an individual employee (or manager) rather than systemic change.

The treatment group, health systems and employer ToCs are closely linked, in that changes in health systems and in employer behaviour would be expected to support the achievement of outcomes for the treatment group but also represent valuable outcomes in themselves assuming they affect all people engaged with these systems. It is therefore possible to identify points where the more limited success in achieving employer and system level outcomes may have impacted on the causal pathway for the treatment groups. It would be beneficial in future designs to further consider the points of interaction between the three ToCs. This could involve a more detailed exploration of the risks and assumptions about how employers and health organisations would interact with the trials, and the effect of this on activities within the intervention-level TOCs.

The limited success in receiving referrals from health organisations meant increased pressure to recruit from other sources such as community organisations and JCP. As referenced in section 3.4.3 of the implementation and four month outcomes report this may have meant people with higher needs than anticipated were recruited to the trials. It appears from the qualitative data, that this group were less likely to benefit from participating in the trial. Employment specialists may also have been less able to provide multi-professional support as case conferencing and co-location did not happen as fully as intended. More effective employer engagement may have meant that the treatment group could have accessed “hidden jobs” and so not experienced the barrier of not being able to find suitable work. Less extensive job brokerage may also have meant the treatment group were less likely to find good suitable work with the right adaptions that could lead to work as a therapeutic outcome.

Findings from the impact report show that in SCR the treatment group (particularly the IW group) experienced improved health and wellbeing and those in WMCA experienced improved employment outcomes. The SCR IW group were also more likely to have achieved employment outcomes than the SCR OOW group. There is some evidence from the qualitative research that this may reflect a higher level of success in engaging employers in WMCA, and a higher level of success in providing health support in SCR. There is stronger evidence that employer engagement was more effective with the SCR IW group than the OOW which is suggestive of its importance in supporting employment outcomes.

6.1 Appendices

6.1 Analysis process

The research reported here uses the terminology of context, mechanism, and outcome to interpret the ToCs, rather than to build context mechanism outcome configurations. Interviewees (recruits, staff, partners, and employers) were asked what outcomes had been achieved, and what they felt had enabled and prevented outcomes being achieved. The interview transcripts were then analysed to identify and group the contexts and mechanisms that enabled outcomes, as well as the relationship between intermediate and major outcomes. These included contexts and mechanisms identified by the treatment group as well as inferred by the research team.

6.2 WMCA management information

Introduction

This appendix provides an overview of the quantitative element of the process evaluation of the Health-led employment trial, Thrive into Work. It draws on descriptive analysis of baseline data collected through the randomisation tool, and management information data collected between May 2018 and April 2021 by the three delivery providers operating in the WMCA.

• Remploy (operating in the Birmingham South-Central Clinical Commissioning Group (CCG) footprint and the Wolverhampton CCG footprint)

• Prospects (operating in the Sandwell and West Birmingham CCG footprint), and

• Dudley & Walsall (D&W) Mental Health Partnership NHS Trust (operating in the Dudley CCG footprint)

This appendix presents analysis of the profile of the treatment group attached to the trial, the support delivered, and the outcomes achieved to date. Baseline data was collected before recruits were randomised, through the randomisation tool. This information was therefore held for all those attached to the service. Management information data was collected by delivery providers and records information on individuals in the treatment group, of which there are records for 1,773 people.

Attachments

Between May 2018 and October 2019 (end of referral period), 3,586 people had completed an initial meeting and joined TIW. Looking at baseline data collected by the randomisation tool, 1,792 people were allocated to the treatment group and 1,794 were allocated to the control group.

Table 6.1 shows the breakdown of the treatment group across providers, and by referral pathway. Across the trial as a whole, 15.8% of referrals had come from GPs, a further 4.2% of referrals had come from specialist care services and the majority, 80.0%, had come from other sources, such as self-referrals.

Table 6.1 Number of recruits in treatment group by provider and referral pathway

Provider	GP	Specialist care service	Other (for example: Pharmacist or self-referral)	Total
D&W	142	28	331	501
Prospects	76	22	473	571
Remploy BSC	46	7	318	371
Remploy Wolves	23	19	307	349
Total	287	76	1,429	1,792

Base: All recruits in treatment group (n=1792)

Source: WMCA provider MI data

Table 6.2 Number of recruits in control group by provider and referral pathway

Provider	GP	Specialist care service	Other (for example: Pharmacist or self-referral)	Total
D&W	138	36	325	499
Prospects	85	24	462	571
Remploy BSC	37	4	334	375
Remploy Wolves	20	10	319	349
Total	280	74	1,440	1,794

Base: All recruits in control group (n=1794)

Source: WMCA provider MI data

Table 6.3 Total number of recruits by provider and referral pathway

Provider	GP	Specialist care service	Other (for example: Pharmacist or self-referral)	Total
D&W	280	64	656	1,000
Prospects	161	46	935	1,142
Remploy BSC	83	11	652	746
Remploy Wolves	43	29	626	698
Total	567	150	2,869	3,586

Base: All recruits in both groups (n=3686)

Source: WMCA provider MI data

Proportionally, D&W received more referrals from GPs than other providers. 28% of D&W referrals came from GPs, 15 percentage points higher than any other provider.

Figure 6.1 Proportional breakdown of client cohort by referral pathway

Provider	GP	Specialist care service	Other, for example Pharmacist or self-referral
D&W	28%	6%	66%
Prospects	14%	4%	82%
Remploy BSC	11%	1%	87%
Remploy Wolves	6%	4%	90%

Base: All recruits in WMCA (n= 3586)

Source: WMCA provider MI data

Profile of the treatment group

Table 6.4 displays the profile of the treatment group. The cohort is evenly split between males (52%) and females (48%), and over 1 in 3 (35%) is aged 50 to 64. Recruits were mainly white (63%); a fifth (19%) Asian. Proportionally, D&W has more white recruits compared to other areas, while Birmingham West and Sandwell, where Prospects operates, has more.

Table 6.4 Proportional breakdown of each program by various demographics

	Male	Female	White	Mixed	Asian	Black	Other	Age 16 to 29	Age 30 to 49	Age 50 +
D&W	54%	46%	84%	2%	10%	3%	1%	25%	33%	43%
Prospects	51%	49%	46%	5%	27%	18%	3%	19%	41%	40%
Remploy BSC	51%	49%	63%	4%	18%	11%	2%	25%	37%	38%
Remploy Wolves	42%	48%	58%	5%	22%	14%	1%	19%	41%	40%
Total	52%	48%	63%	4%	19%	11%	2%	22%	38%	40%

Base: All recruits in treatment group (n=1792)

Source: WMCA provider MI data

Support delivered

The data in this section includes management information data, collected by all providers to record information on individuals in the treatment group, of which there are records for 1,792 people. This information should be treated with some caution, as the data is incomplete.

More than 9 in 10 (92%) of the treatment group received face-to-face support at a location outside of a home, office, or healthcare setting, which was the most common mode of support delivery. A similar proportion (95%) received telephone based support, and 61% received support via text message. Only 6% received support in their home.

The COVID-19 pandemic led to a marked shift in the mode of support delivered. Prior to the pandemic (from 16 March 2020), more than two fifths (44%) of all support offered was in a face-to-face setting. This fell sharply to two per cent during the pandemic. Consequently, telephone based support increased from 30% of all support before the pandemic, to 57%.

Table 6.5 Mode of support delivered

Format (setting) of support	Number of sessions	Number of cohort	Percentage of total cohort
Face-to-face other location	21,412	1,642	92%
Telephone	21,689	1,700	96%
Text	8,851	1,098	61%
Email	6,268	1,007	56%
Face-to-face clinical setting	1,829	273	15%
Face-to-face office or work	995	205	11%
Face-to-face other location	10,403	1,583	88%
Telephone	8,793	1,602	89%

Base: All recruits in treatment group (n=1792)

Source: WMCA provider MI data

Almost all (95%) in the treatment group attended a brief (shorter than 15 minutes) catch-up with their TIW employment specialist. Almost two-thirds (63%) attended a catch-up between 15 and 30 minutes long, and 79% have attended a support session lasting between 30 and 60 minutes. A quarter (23%) have had a support session lasting 90 minutes to 2 hours, and 6% attended a session lasting up to 3 hours. Only 2% had support for a half or full day.

The pandemic also affected the length of delivered support sessions. Prior to the pandemic, 51% of all support sessions were shorter than 15 minutes. This increased to 70% during the pandemic. Longer support sessions (lasting more than 60 minutes) fell from 19% of all support before the pandemic, to 3%.

Table 6.6 Length of support sessions used by the treatment group

Length of support session	Number of sessions	Number of cohort	Percentage of total cohort
Brief catch up (less that 15 minutes)	34,798	1,711	95%
15 to 30 minutes	6,421	1,136	63%
30 to 60 minutes	12,102	1,415	79%
60 to 90 minutes	9,298	1,314	73%
90 to 120 minutes	1,741	417	23%
120 to 180 minutes	284	109	6%
Half or full day	72	30	2%

Base: All recruits in treatment group (n=1792)

Source: WMCA provider MI data

Outcomes

757 individuals in the treatment group achieved a job outcome in the period between May 2018 to April 2021.

Table 6.7 Job outcomes achieved by provider

Provider	Number of individuals achieving job outcomes	Percentage of cohort
D&W	150	29%
Prospects	152	26%
Remploy Wolves	124	33%
Remploy BSC	124	35%
Total	550	30%

Source: WMCA provider MI data

Figure 6.2 shows the industrial sector of the job outcomes achieved. 17% were in wholesale and retail, and 14% in health and social work.

Figure 6.2 Proportional breakdown of job outcomes by sector

Wholesale and retail	17%
Health and social work	14%
Manufacturing	11%
Hotels and restaurants	8%
Other business services	6%
Transport and communication	6%
Other community services	6%
Education	5%
Construction	4%
Public administration	3%
Financial services	2%
Electricity, gas or water supply	1%
Other	16%

Base: All job outcomes (n=757)

Source: WMCA provider MI data

6.9 SCR management information

Introduction

This appendix provides an overview of the quantitative element of the process evaluation of the Health-led employment trial, WW. It draws on descriptive analysis of baseline data collected through the randomisation tool, and management information data collected between May 2018 and October 2021 by SHYA, the delivery provider operating in the SCR. It presents analysis of the profile of recruits attached to the trial, the support delivered to the treatment group, and the outcomes achieved.

Baseline data was collected before recruits were randomised, through the randomisation tool. This information is therefore held for all those attached to the trial. Management information data was collected by SYHA and records information on individuals in the treatment group, of which there are records for 2,716 people.

Attachments

Between May 2018 and October 2019 (end of referral period), 6,016 people had completed an initial meeting and joined WW. Looking at baseline data collected by the randomisation tool, 3,004 people were allocated to the treatment group and 3,012 were allocated to the control group.

Table 6.8 shows the breakdown of the referral pathway of treatment group, by area. 18.0% of referrals had come from GPs, a further 26.4% of referrals had come from specialist care services and the majority, 55.6%, had come from other sources, such as self-referrals.

Table 6.8 Number of recruits in treatment group by provider and referral pathway

Provider	GP	Specialist care service	Other (for example, Pharmacist or self-referral)	Total
Sheffield	230	284	757	1271
Rotherham	85	110	382	577
Doncaster	89	204	238	531
Bassetlaw	62	98	86	246
Barnsley	74	97	208	379
Total	540	793	1671	3004

Base: All recruits in treatment group (n=3004)

Source: SCR provider MI data

Table 6.9 Number of recruits in control group by provider and referral pathway

Provider	GP	Specialist care service	Other (for example, Pharmacist or self-referral)	Total
Sheffield	223	276	780	1279
Rotherham	90	89	400	579
Doncaster	106	203	221	530
Bassetlaw	67	82	93	242
Barnsley	73	113	196	382
Total	559	763	1690	3012

Base: All recruits in control group (n=3012)

Source: SCR provider MI data

Table 6.10 Total number of recruits by provider and referral pathway

Provider	GP	Specialist care service	Other (for example, Pharmacist or self-referral)	Total
Sheffield	453	560	1537	2550
Rotherham	175	199	782	1156
Doncaster	195	407	459	1061
Bassetlaw	129	180	179	488
Barnsley	147	210	404	761
Total	1099	1556	3361	6016

Base: All recruits treatment and control (n=6016)

Source: SCR provider MI data

Figure 6.3 Proportional breakdown of client cohort by referral pathway

	GP	Specialist care service	Other (for example, Pharmacist, self-referral)
Sheffield	18%	22%	60%
Rotherham	15%	17%	68%
Doncaster	18%	38%	43%
Bassetlaw	26%	37%	37%
Barnsley	19%	28%	53%

Base: All recruits in SCR (n=6016)

Source: SCR provider MI data

Profile of the treatment group

Table 6.11 displays the profile of the treatment group. The group was evenly split between males (51%) and females (49%), and over 1 in 3 (37%) was aged 50 to 64. In respect of diversity, 87% of the treatment group were white, and 5% were Asian.

The demographic composition of the treatment group in each location is similar, except in Sheffield, where a smaller proportion (77%) of the treatment group were white.

Table 6.11 Proportional breakdown of client cohorts by various demographics

	Male	Female	White	Mixed	Asian	Black	Other	Age 16 to 29	Age 30 to 49	Age 50 +
Barnsley	52%	48%	98%	0%	0%	1%	1%	27%	38%	35%
Bassetlaw	53%	46%	95%	1%	2%	1%	1%	22%	34%	44%
Doncaster	53%	47%	93%	1%	2%	2%	2%	21%	31%	38%
Rotherham	54%	46%	93%	1%	4%	1%	1%	26%	39%	35%
Sheffield	47%	52%	77%	4%	9%	7%	3%	23%	41%	36%
Total	51%	49%	87%	2%	5%	4%	2%	23%	40%	37%

Base: All recruits in treatment group (n=3004)

Source: SCR provider MI data

Support delivered

The data in this section includes management information data, collected by SYHA to record information on individuals in the treatment group, of which there are records for 2,716 people. This information should be treated with some caution, as the data is incomplete.

Almost all (97%) received a support action of some kind, with just under 1 in 5 (19%) receiving out of work support and 10% receiving in-work support. Just 4% of the treatment group were referred to another support service.

Table 6.12 Type of support delivered

Type of support	Number of sessions	Number of treatment group	Percentage of treatment group
Support action	42,731	2,644	97%
Out of work support	2,603	524	19%
Job application and interview	604	314	12%
In-work support	1,265	260	10%
Referral to other service	152	120	4%
Re-engagement	156	60	2%

Base: All recruits in treatment group (n=2,716)

Source: SCR provider MI data

Table 6.13 shows that face-to-face support delivered in a location outside of an office or healthcare setting was the most common mode of support experienced by the treatment group (76%). Over two thirds (71%) received telephone based support, and 6% received support via Skype or FaceTime.

The COVID-19 pandemic led to a marked shift in the mode of support delivered. Prior to the pandemic (16 March 2020), almost three quarters (71%) of all support offered was in a face-to-face setting. This fell to 27% during the pandemic. Consequently, telephone based support increased from 13% of all support before the pandemic, to 48%.

Table 6.13 Mode of support delivered

Mode of support	Number of sessions	Number of treatment group	Percentage of total cohort
Face-to-face in other location	12,557	2066	76%
Telephone	12,460	1932	71%
Face-to-face in office setting	3,646	902	33%
Text	2,569	883	33%
Email	7,402	728	27%
Face-to-face meeting	2,520	711	26%
Face-to-face in healthcare setting	2,572	604	22%
Face-to-face in customer home	919	247	9%
Skype or FaceTime	649	160	6%

Base: All recruits in treatment group (n=2,716)

Source: SCR provider MI data

More than three-quarters (81%) of the treatment group were recorded as reviewing their action plan during a support session, and two-thirds (68%) were given advice on managing health conditions. A similar proportion received job-search support (66%) and CV support (64%).

The effects of COVID-19 on the content of support were minimal, with one notable change. Advice on managing health conditions rose from 11% of all support offered prior to the pandemic, to 19%.

Table 6.14 Content of support delivered

Content of support session	Number of sessions	Number of treatment group	Percentage of total cohort
Action plan reviewed	12,323	2212	81%
Advice on managing health conditions	12,272	1860	68%
Job search support	11,819	1791	66%
CV support	5,999	1731	64%
Vocational profile reviewed	2,097	1386	51%
Advice on benefits and the impact of work on finances	3,558	1206	44%
Discussing additional needs (not employment or health)	3,694	1046	39%
Support to access work trial, work experience or volunteering	1,551	568	21%
Reasonable adjustments	874	527	19%
Workplace assessment or observation	75	486	18%
Job carving or customising job duties	1,092	414	15%
Access to work application	277	125	5%
Onward referral made	870	60	2%
Coaching (one-to-one)	7	5	0%
Other	8,396	1725	64%

Base: All recruits in treatment group (n=2,716)

Source: SCR provider MI data

Outcomes

Table 6.15 shows that 2,796 employment outcomes were achieved for the treatment group. Of these:

• 1,183 were for new jobs, achieved for 708 in the treatment group

• 1,160 were for retaining their existing employment, achieved for 691 treatment group members

• 453 were for returns to their employment, achieved for 294 in the treatment group

There were also 516 education and 330 volunteering outcomes for the treatment group.

Table 6.15 Outcomes achieved

Type of outcome	Number of outcomes
Labour outcome	2,796
Education outcome	516
Volunteering outcome	330

Base: All recruits in treatment group (n=2,716)

Source: SCR provider MI data

Table 6.16 shows that almost 3 in 4 (72%) new job outcomes were sourced by the individuals themselves.

Table 6.16 Source of new job outcome

Source of job outcome	Count	Percentage of new jobs
Customer sourced	855	72%
Advisor sourced	250	21%
Trial sourced	78	7%

Base: New jobs (n=1,183)

Source: SCR provider MI data

Figure 6.4 shows the industrial sector of the job outcomes achieved by the treatment group. 1 in 5 (18%) were in wholesale and retail, and a further 18% in health and social work.

Figure 6.4 Proportional breakdown of job outcomes by sector

1	18%
2	18%
3	10%
4	9%
5	7%
6	6%
7	6%
8	5%
9	5%
10	4%
11	3%
12	1%
13	8%

Base: All job outcomes (n=2,796)

Source: SCR provider MI data

1. Partners refers to organisations who were directly involved in the delivery of the trial, for example referral partners. Stakeholders refers to organisations who had strategic oversight either of the trial or the local context but were not directly involved in delivery. ↩

2. The area has since rebranded as South Yorkshire Mayoral Combined Authority ↩

3. More information about the analysis process is included in appendix 6.1. 4. A causal pathway is a way to explore and understand how mechanisms can lead to outcomes, and how intermediates outcomes can lead to major outcomes. ↩

4. A causal pathway is a way to explore and understand how mechanisms can lead to outcomes, and how intermediates outcomes can lead to major outcomes. ↩

5. Job brokerage is the process of working with employers based on the individual requirements of IPS recipients. Job carving refers to customising an employment opportunity to meet an individual’s needs by, for example, removing or adapting particular tasks. ↩

6. Because of the relatively small number of the interview sample entering work, the relationship between intermediate outcomes and mechanisms of support is explored for all those in the treatment group interview sample. ↩

7. Equivalent data was not available in WMCA ↩

8. The reasons why case conferencing was not more in evidence as a mechanism is explored in section 1.3.2 and in the Implementation and 4-month outcomes report ↩

9. The employers who took part in the interviews were generally larger organisations with occupational health departments. However, due to the small number of interviews conducted, it is not possible to draw any general conclusions about how the size and type of employer impacted on their engagement with the trial. ↩