Research and analysis

Summary: An evaluation of the Group Work/JOBS II evaluation

Updated 17 May 2022

Applies to England

Authors: Tim Knight, Richard Lloyd and Matt Rayment (ICF), Dr Susan Purdon and Caroline Bryson (BPSR), Christabel Downing and Siv Svanaes (IFF Research), Dr Adam Coutts (University of Cambridge), Professor Stephen McKay (University of Lincoln), and Dr Clara Mukuria (University of Sheffield).

Introduction

Group Work is the application of the JOBS II model, which is an intervention originally developed in the United States by the University of Michigan to enhance the job search skills and psychological resilience of individuals out of work.

Between January 2017 and March 2018, the Department for Work and Pensions and Department of Health and Social Care Joint Work and Health Unit conducted a Randomised Control Trial to test the effectiveness of JOBS II in a UK labour market context. Group Work is a 20-hour group-based course, comprising five four-hour sessions delivered over the course of a working week. The course focuses on job search skills but is also designed to enhance self-efficacy, self-esteem and social assertiveness of those looking for paid work. It is one of several interventions being trialled by the Work and Health Unit to build a strong evidence base on what works best to help those with health issues move into and retain work.

Recognition, uptake and participation in Group Work

The Group Work trial was targeted at claimants of Jobseeker’s Allowance, Employment Support Allowance, Universal Credit Full Service and Income Support (Lone Parents with child(ren) aged three and over) who were struggling with their job search and/or feeling low, anxious or lacking in confidence with regard to their job search. Participation was entirely voluntary.

Work Coaches in the five Jobcentre Plus districts where Group Work was trialled were responsible for recognising benefit claimants who could potentially benefit. By the end of the trial, a total of 16,193 potential beneficiaries had been recognised, against a target of 26,000. The timing of the training that Work Coaches received and other challenges associated with implementing a trial within the complex environment of local Jobcentre Plus delivery were cited as the main reasons for this shortfall.

The potential beneficiaries recognised by Work Coaches in the trial were also diverse. They included benefit claimants who were potentially struggling with job search, lacking in confidence, anxious and with likely depression, but also included others who did not exhibit these characteristics. Some Group Work provider staff felt this potentially diluted the impact of the trial, but equally there was a belief that a mix of participant characteristics was a positive for the group dynamic on the course.

A total of 11,900 benefit claimants were randomised into the Group Work group in the trial and offered the opportunity to go on the course. Of these, 5,355 (45 per cent) initially accepted this offer, 4,046 (34 per cent) attended the Initial Reception Meeting^{[footnote 1]} that preceded the course, and 2,596 (22 per cent) attended day one of the course itself. The reasons benefit claimants gave for declining to take part or withdrawing, centred around health-related factors, childcare responsibilities, and sometimes a perception that either the course would not help them or that they did not need help.

Experiences of the Group Work course

Overall, 92% of Group Work participants said they had found the course useful when they were surveyed about it six months later.

The course was facilitated by two trained Group Leaders who, as found in previous trials in other countries, played an important role in shaping participants’ experience of the course. Over three-quarters of participants thought their Group Leaders understood the challenges of finding work (79%) and had personal experience of being unemployed (76%).

The group dynamic was another feature of the course that participants and Group Leaders emphasised. Equally, how well the group dynamic worked depended partly on the mix of participants on the course. Too much commonality or too much difference was viewed as potentially less effective with the ideal being a balance between the two.

Observational research also identified 3 “active elements” which appear to lead to changes in the health, wellbeing and job search behaviour of participants:

• active participation in a group context - the combination of active participation and the group dynamic established was important
• replicating the time structure and routine of employment - the structure and routine of attending the course was an important element, alongside providing participants with constructive activities and a change to what were often described as monotonous daily routines
• Group Leader effectiveness and credibility - Group Leaders acted as a catalyst for the other active elements of the course

The impacts of Group Work

The impact evaluation was based on administrative and survey data collected at randomisation^{[footnote 2]} when benefit claimants were identified as potential beneficiaries of Group Work, and 6 and 12 months later. The impact evaluation findings reported here focus on outcomes for benefit claimants who went on to participate in Group Work compared to a matched comparison group of potential beneficiaries in the control group who received ‘business as usual’ Jobcentre Plus support.

There were no statistically significant differences between the work status of Group Work participants and the matched comparison group at the 6 and 12-month points. However, some positive but not statistically significant differences were detected, with 23% of participants being in paid work compared to 20% of the matched comparison group at 12 months. Group Work also appeared to equip participants with attributes and behaviours likely to increase their likelihood of finding work in the future. After 12 months, 57% had higher levels of job search self-efficacy^{[footnote 3]} compared to 45% of the matched comparison group, and they were more likely to be submitting a high volume of CVs to employers. Both these differences were statistically significant.

Statistically significant impacts were detected on life satisfaction, feeling life is worthwhile, happiness and loneliness after 6 months. After 12 months, only the impact on happiness remained statistically significant, with Group Work participants having a mean score of 6.5 compared to 5.8 among the matched comparison group.

There were similar positive impacts in terms of mental health, although these were smaller and not statistically significant after 12 months for any of the mental health indicators.

The impact evaluation included extensive subgroup analysis to examine the impacts of Group Work on participants with different baseline characteristics. Group Work was found to have employment and health impacts on participants:

• with lower levels of general self-efficacy at baseline^{[footnote 4]}
• with suggested case level anxiety on the GAD-7 scale at baseline^{[footnote 5]}
• with suggested case level depression on the PHQ-9 scale at baseline^{[footnote 6]}

For participants with lower levels of general self-efficacy or suggested case level anxiety, there were statistically significant positive impacts on participation in paid work. After 6 months, those with lower levels of general self-efficacy (21% versus 11%) or suggested case level anxiety (20 per cent versus 10%) were around twice as likely to be in paid work as the matched comparison group, with the proportions in paid work of 30 hours or more continuing to be significant after 12 months.

There were also statistically significant positive impacts - at both six and 12 months - on these groups’ levels of general and job-search self-efficacy. Those with lower levels of general self-efficacy or suggested case level anxiety were around twice as likely as their matched comparison groups after six months to score as having higher levels of general and job search self-efficacy. For those with lower levels of general self-efficacy and suggested case level anxiety, Group Work also appears to have a statistically significant impact on levels of mental health (as measured by the GAD-7 and World Health Organisation (WHO-5) scales) after six months, sustained after 12 months for those with lower levels of general self-efficacy but not for those with suggested case level anxiety. A similar, but more limited pattern of statistically significant impacts was detected among those with suggested case level depression at baseline.

The costs and benefits of Group Work

The total costs of Group Work were estimated at £3.3 million, of which £1.7 million was spent on delivering the intervention and £1.6 million on running the trial and evaluation. After deducting the trial and evaluation elements, the delivery costs averaged £886 per participant completing the course.

The cost benefit analysis found that, overall, the costs of delivering Group Work outweighed the value of the monetised benefits. The societal benefit cost ratio ranged from 0 (main assessment) to 0.67 (assuming a positive employment effect and lower cost estimate). However, this analysis does not reflect the statistically significant benefits to some of the mental health, wellbeing and job-search outcomes detected in the impact evaluation, which could not be valued in monetary terms.

The cost benefit analysis also assessed the benefits and costs of Group Work for different subgroups of participants. The monetised benefits were found to exceed the intervention costs for two groups, with a benefit cost ratio of 1.76 for those with suggested case level anxiety at baseline, and 1.39 for those with lower levels of general self-efficacy at baseline.

This suggests that Group Work could deliver net benefits to society if targeted at these groups. This finding relies on the assumption that the costs of delivery to these groups can be held at the average intervention cost of Group Work, and that similar levels of benefits can be maintained by participants drawn from benefit claimants, whose characteristics suggests they are at greatest distance from the labour market.

Conclusions

The evaluation concluded that for Group Work to be most effective it must be tightly targeted, and specifically with benefit claimants with lower levels of general self-efficacy and poorer mental health, for whom several statistically significant impacts and positive benefit cost ratios were detected after 12 months.

Key learning from the trial included:

• if Group Work was trialled again, consideration should be given to targeting the sub-groups found to benefit most
• if Group Work is implemented in the future, consider how Work Coaches or others can be trained and helped to accurately recognise claimants likely to benefit most and put in place measures to monitor this
• any future delivery of Group Work should seek to replicate the structure and delivery of the Group Work trial, which was found to be effective, and ensure it continues to be delivered by suitably trained and experienced facilitators
• explore how each of the ‘active elements’ of Group Work identified in the trial may be applicable to wider provision
• consider the inclusion of mental health, wellbeing and self-efficacy measures in other interventions and provision as indicators of effectiveness, in addition to routine job-related measures

1. All Group Work participants were invited to an Initial Reception Meeting (IRM), to give participants the opportunity to meet the Group Leaders who would deliver their course and learn more about what it would involve. ↩

2. Some baseline measures were collected at the point of randomisation. For others, they were collected for participants on the first day of the course and for the control group (and course decliners) in a survey collected some months after the participant baseline. ↩

3. Job search self-efficacy is the strength of an individual’s belief that they have the skills to undertake a range of job search tasks. ↩

4. General self-efficacy is the strength of an individual’s belief that they are effective in handling life situations. ↩

5. A person is described as having suggested case level anxiety if their score on the GAD-7 scale suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of anxiety would be based on a clinical interview and would take account of additional evidence, to which the GAD score may contribute. ↩

6. A person is described as having suggested case level depression if their score on the PHQ-9 scale suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnosis of depression would be based on a clinical interview and would take account of additional evidence, to which the PHQ score may contribute. ↩