Effective contracting of employment and health services

Question 1

Acknowledgements

Accepted Answer

We would like to thank numerous colleagues from the Department for Work and Pensions, in particular Simon Marlow and Michael Conn, for their continuous support and guidance during this study.

We are also very grateful to the many researchers and service providers who generously offered their time and contributed to the interviews and workshop consultations.

Our thanks go also to colleagues at RAND Europe who provided valuable feedback on the design, implementation and results of this research.

Question 2

The authors

Accepted Answer

Dr Julia Doyle
Dr Madeline Nightingale
Jessica Dawney
Michael Whitmore
Dr Chris van Stolk
Joanna Hofman

Question 3

Summary

Accepted Answer

Context

Robust, quantitative evidence to determine the costs and benefits of contracting out employment services and health assessments and comparing these models to no provision or to traditional, in-house delivery by Public Employment Services (PES), is relatively scarce. In addition, existing studies do not allow us to fully understand why contracted services performed well (or not so well), if and how this differs according to context and contract design. As a result of this, evidence is also limited on how the process of contracting out could be improved to achieve better outcomes for individual clients, while keeping the costs of service reasonable for the public purse.

This study, commissioned by the Department for Work and Pensions (DWP) and conducted by RAND Europe, synthesises findings from the existing literature and qualitative interviews with commissioners, providers and other experts, to address the question:

What are the advantages and disadvantages of contracting out public employment services and health assessments and what is the optimum model for doing so?

This main question agreed with DWP captures a number of specific issues explored in this report around contracting employment services and health assessments. These 2 types of services represent different contracting models in the UK:

(i) payment by results (PbR) contracts feature prominently in employment services (these were temporarily replaced with Cost Plus contracts introduced in response to the COVID-19 pandemic)

(ii) payment for service contracts are commonplace in health assessments. The reviewed evidence focuses primarily on contracts utilising PbR mechanisms, which are common in the commissioning of employment services in the UK and internationally. There were no international examples of commissioning health assessments in the reviewed literature. This is reflected in the research findings but where possible and relevant, other contract models are discussed.

Findings

Contracting out allows to better adjust the volume of service provision to labour market conditions, and – at least in theory – to better tailor services to the clients’ needs by using specialised service providers (empirical evidence is discussed below). We found that PbR contracts in employment services may bring benefits in terms of programme efficiency and value for money. However, there is mixed evidence on the impact of PbR contracts on programme effectiveness: some studies find that PbR contracts for employment programmes are associated with cost savings, whereas this appears not to be the case for other programmes. The main costs associated with PbR contracts are linked with undesirable behaviour from providers such as creaming and parking (i.e., focusing their efforts on job seekers who are deemed closer to the labour market). Contracting out may also weaken the accountability of public services.

We examined risks associated with PbR contracts and found that, for commissioners, these risks include overpaying for services, underpaying and market failure. For providers, the risks are high upfront costs, underperformance and reputational concerns. In addition, contracting out may incur a risk of variable service provision for users/beneficiaries.

The process of contracting out is determined by specifications issued by the commissioners. This documentation formulates expectations of the purchaser towards the service provider(s), and it defines, and governs responsibilities and rights among the parties. Such documentation can attempt to regulate every aspect of services to be provided or leave some (or most) aspects in the provider’s discretion (black box). Compared to detailed specifications, black box provision is intended to allow providers greater flexibility and facilitate innovation, but there is little empirical evidence this actually happens (see more below). Flexibility for providers results in greater service variability and may increase the risk of creaming and parking. Minimum service requirements are designed to assure basic service standards, but views are divided on the optimum level of specification.

In the UK, DWP commissions employment contracts to one or more (prime) contractors across geographical regions known as Contract Package Areas (CPAs). These prime contractors manage a network of subcontractors delivering services in the area. We found that the prime model of contracting builds long-term relationships with large contractors who can take on the risk of a PbR contract. However, there is a risk of subcontractors not being able to benefit fully from contracts. The number and size of CPAs differs across programmes, but large CPAs may mean that providers lack local expertise.

Criteria used to select providers for contracted out services often combined quality and price. In the examples identified in this evidence review, the weight of price relative to quality ranged from 20% to 40%. The quality aspects usually aim to capture relevant experience and expertise. The approach using the Price per Quality Point (PQP) metric is relatively new and it has not been discussed in the reviewed literature.

There is little evidence on how different types of contracts influence market competition in the short- and long-term. However, there is some evidence on the effect of contracting out on market competition: this evidence points to some providers being ‘pushed out’ or forced to adapt to secure contracts and deliver services. The dominance of prime contractors in the market has been well established.

Trade-offs between optimising short-term and long-term outcomes has been a process of ‘trial and error’ in the UK and internationally. One of the key challenges is finding an optimal balance between upfront fees and outcome payments to create incentives for providers and savings for commissioners. In the examples identified in this evidence review, the proportion of payments linked to outcomes varied markedly by country and by programme ranging from 30% (e.g. CareerCompass and CareerAdvance in the US) to over 90% (e.g. the WP in the UK).

While contract management is critical for ensuring the quality of service provision, there is no agreed definition of service ‘quality’ or how to measure it. Performance measurement and monitoring aim to encourage innovation and limit gaming.

Incentives to encourage collaboration or information sharing among providers are limited. There is little evidence on sharing contract resources amongst providers and other stakeholders. Expertise and specialist knowledge of providers is key and the need for professional development of staff was noted.

Innovation in employment services may mean a variety of things including working with a new client group, introducing new forms of support or training or making organisational processes more efficient. Competition intends to encourage innovation and this needs to be considered early on in the commissioning process. However, evidence for innovation taking place in PbR contracts is limited and the reviewed literature points to opportunities for innovation often not being taken up.

Conclusion

Optimum contract design will depend on the aims and objectives of the programme, meaning it is not possible to draw overarching conclusions about the ‘best’ approach. There may be tensions and trade-offs between objectives associated with different elements of contract design. Multiple external factors may also affect delivery (e.g., economic performance throughout the contract duration may make it easier or more difficult for providers to achieve targets set up at the start). While there is no single recipe for a flawless PbR contract model, the evidence base is growing. The table below outlines main features of PbR contracts, associated levers through which their features could be optimised and how these levers may lead to adverse effects elsewhere. This framework may help commissioners better understand these dependencies and allow them to make better informed choices suited for their needs.

Optimising PbR contracts to achieve the desired outcomes

Benefits, costs and risks in PbR contracting	Assumption	Lever	Associated negative effects
Benefit: improved effectiveness and cost-efficiency	Focus on outcomes drives better results (‘what gets measured, gets done’)	Increased weighting for outcome payments	Gaming behaviour by providers – – Disadvantaging specialist providers / subcontractors by primes
Cost: high transaction costs	Reducing costs of commissioning through PbR increases VFM to commissioners	Less detailed contract specifications (black box) – – Fewer minimum service requirements Longer, fewer and bigger contracts	Variability in service provision – – Deteriorating service quality – – Reducing market competition
Risk: gaming strategies	Reducing incentives to gaming minimises risks and costs to commissioners	Differentiate outcome payments by client group – – More minimum service requirements – – More detailed contract specifications (grey box)	Limiting flexibility of providers and potential for innovation
Risk: primes’ monopoly	Market competition stimulates greater effectiveness and cost savings for commissioners	Smaller CPAs or two-stage competitions within existing CPAs – – Shorter contracts – – (Financial) incentives or contractual requirements for primes to work with smaller providers	Increased costs for commissioners
Risk: quality of services	Safeguards must be in place to ensure service users receive high quality provision	More minimum service requirements – – Strengthening monitoring arrangements or introducing a licencing / accreditation system – – Strengthening enforcement or breakout clauses	Increased costs for commissioners – – Reduced incentives (increased costs) for providers

More evidence is needed on health assessments (both internationally and in the UK) to better understand the state of play in contracting out these services and possible ways forward.

Stronger evidence on the actual costs incurred and benefits achieved of different models of provision (in-house, contracted out (including PbR, Cost Plus and payment for service contracts), partnership, grant funding arrangements) is needed to be able to compare these. On that basis it would be possible to draw more informed conclusion on which works better for whom, under what conditions, and why.

Question 4

Glossary

Accepted Answer

Black box	Service provision where providers have the freedom to decide what to do and they receive financial rewards for outcomes achieved (with the aim of encouraging innovation).
Contract Package Areas	Geographical areas across England and Wales with a small number of prime contractors managing a network of subcontractors delivering services in the area.
Cost Plus contract	An agreement to reimburse a company for expenses incurred in addition to a specific amount of profit, usually stated as a percentage of the contract’s full price.
Creaming or cherry-picking	Undesirable behaviour from providers to focus on clients with good employment prospects (and to neglect those with complex needs).
Gaming	A strategy used by providers to exploit weaknesses in contracts / specifications to maximise economic gains (at cost to service quality).
Grey box	Specifying some aspects of service provision but allowing providers to tailor other elements, in contrast to a ‘black box’ in which some (or most) aspects are left in the provider’s discretion.
Live running	Execution of the contracts for delivering employment services and health assessments.
Outcome payment	Financial renumeration awarded by the commissioner to providers for each service user reaching the pre-agreed criteria for claiming an ‘outcome’.
Price cap	A limit on the price a provider can charge.
Parking	Undesirable behaviour from providers to withhold or reduce support for unemployed clients who are difficult to place.
Payment by Results (PbR)	A model of financing public services where payments depend (largely or wholly) on the results achieved by service providers, thus providing a financial incentive for providers to deliver effective (in the sense of achieving outcomes) services throughout the term of the contract.
Prime/subcontractors	Model of commissioning employment contracts in the UK to a small number of (prime) contractors per Contract Package Area (CPA); prime contractors manage a network of subcontractors delivering services in the area.
Quasi-market	A term commonly used in the literature to describe the market created by contracting out public services. In quasi-markets the recipient of the services and the purchaser are separate entities and providers do not necessarily compete for profit.
Restart Scheme	The Restart Scheme was launched in 2021 in England and Wales in 12 CPAs. It gives benefit claimants who have been out of work for at least 9 months enhanced support to find jobs in their local area^{[footnote 1]}.
Specification	Documentation stating how the service should be provided: – – Little specification means ‘black box’ provision where providers have the freedom to decide what to do (this should encourage innovation) – – Detailed specification limits the freedom of the providers (and limits the potential for innovation) – – Specifying some aspects of service provision but allowing providers to tailor other elements is referred to by DWP as ‘grey box’.
Upfront or service fee	Payment to providers for the delivery of some basic service to each user, usually early in the contract and regardless of the ‘outcome’.
Work and Health Programme	The Work and Health Programme (WHP) replaced the Work Programme (WP) and was rolled out in England and Wales between November 2017 and April 2018. It aims to help disabled people, the long-term unemployed, and some priority groups (so called early access groups) to enter into and stay in work. It is delivered in 6 CPAs and devolved to some Local Government Partners and Devolved Deal Areas^{[footnote 2]}.
Work Programme	The Work Programme (WP) was a PbR welfare-to-work programme in Great Britain in the period 2011-2017. It was delivered in 18 CPAs^{[footnote 3]}.
Workfare programmes	Social assistance programmes that require recipients to participate in work or other activities (e.g., training).

Question 5

1. 	Introduction

Accepted Answer

This report presents findings from an evidence review conducted by RAND Europe on behalf of the Department of Work and Pensions (DWP) about effective contracting of employment services and health assessments.

1.1. Contracting out public services

DWP is responsible for welfare, pensions and child maintenance policy. It is the biggest public service department in the United Kingdom (UK). It administers the State Pension and a range of working age, disability and ill health benefits to around 20 million claimants and customers^{[footnote 4]}. DWP provides publicly financed health assessments to assess eligibility for health-related benefits (which are the statutory services), and employment support (which are the discretionary services). Some employment services in the UK are contracted out by DWP to providers, including private and not-for-profit organisations, with an intention to manage by objectives^{[footnote 5]}, and improve effectiveness (extent to which results are achieved)^{[footnote 6]} and efficiency (extent to which results are delivered in an economic and timely way^{[footnote 7]})^{[footnote 8]}^{[footnote 9]}.

The evidence behind this model of service provision is growing, especially for commissioning employment services^{[footnote 10]}, but it is showing mixed results (discussed in section 2.1.2)^{[footnote 11]}^{[footnote 12]}^{[footnote 13]}. There is still little research and evidence-based guidance for commissioners on how to contract out employment and health contracts^{[footnote 14]} effectively and how to manage such markets over time.

1.2. Research objectives and questions

There is little quantitative evidence to determine, which model of providing employment services and health assessments is better: contracting out or delivering these in-house^{[footnote 15]}. Ideally, one would be able to make a direct – quantitative – comparison of effectiveness between services contracted out and those delivered by Public Employment Services (PES) for the same group of people. However, such comparisons are difficult to conduct and thus rare. In addition, such studies often do not allow to fully understand why contracted services performed well (or not so well), when compared to the business as usual, and how the process could be improved.

This study, commissioned by DWP and conducted independently by RAND Europe, looks to collate existing evidence from literature on the effectiveness and efficiency of the contracting out model. Furthermore, it employs qualitative interviews to provide additional insights on the matter to address the following main research question:

What are the advantages and disadvantages of contracting out public employment services and health assessments and what is the optimum model for doing so?

This research draws on robust impact evidence (where it exists) and provides qualitative perspectives on the processes of contracting out employment services and health assessments, both in the UK and internationally. Thus, this report helps to improve the understanding and practice of contracting out.

Specific research questions (see Annex 1) are grouped under three broad topics: (i) contract design^{[footnote 16]}, (ii) market competition and (iii) live running. These topics cover a comprehensive process of contracting out and include subsequent stages from how contracts are designed, how contractors are selected, and how the contracts are managed (Steps 1 to 5).

Steps 1 to 5. Key stages of the process of contracting out

Market engagement process informs contract design, selection and contract management during life running

Market engagement
Contract design
Competition starts: selection
Implementation starts: Contract management during life running.
Back to step 1. Contract ends.

1.3. Methods

This research is based on a targeted review of literature, 46 semi-structured interviews and a synthesis workshop. Annex 1 illustrates how these different methods contributed to answering the specific research questions.

1.3.1. Literature review

A Quick Scoping Review (QSR) was chosen due to pragmatic considerations about the timeline of the research, which took place between November 2021 and March 2022 (the review had to be completed by early February 2022 to inform the programme of interviews – see section 1.3.2).

The QSR focused on sources published in English, related to the UK or other members of the Organisation for Economic Co-operation and Development (OECD) and published between 2011 and 2021. The search strategy is outlined in Annex 2. Additional sources were identified via snowballing, and this included some highly relevant sources published prior to 2011 as well as material shared directly by some interviewees. After being assessed against inclusion/exclusion criteria, data from included sources were inputted in a data extraction template in Excel (see Annex 2), enabling systematic analysis against key themes and specific research questions.

1.3.2. Semi-structured interviews

Interviews were conducted to complement the QSR and to fill in gaps in understanding of the processes of contracting out (employment services and health assessments) and obtaining in-depth views and perspectives from a range of stakeholders around the research questions. A long list of interviewees was agreed with DWP and it included three main groups of stakeholders: (i) 11 DWP staff (employees with knowledge on the policy and strategy, commercial and contracting issues faced by commissioners), (ii) 20 employment service providers operating in the UK (primes and small organisations) with experience of delivering contracted services, (iii) 15 experts (i.e. academics and researchers who investigate the issue of contracting out in the UK and internationally). A total of 46 interviews were conducted between February and March 2022. Interviews were conducted remotely via Microsoft Teams and lasted approximately 60 minutes. Interviews were semi-structured to enable comparability whilst allowing some flexibility to tailor the discussion according to the interests and expertise of the individual. The topic guide for the interviews can be found in Annex 3. Interviews were recorded (with interviewees’ permission) and the transcripts were analysed thematically. Interviewee codes used in this report use letters ‘C’, ‘E’ or ‘P’ to indicate interviewees representing commissioners, experts and providers, respectively.

Additional insights were provided by representatives from DWP in form of comments to an early version of this report. When these comments present new or alternative perspectives, these are incorporated with the interview findings and marked as contributions from DWP representatives for clarity and transparency.

1.3.3. Synthesis workshop

The purpose of the workshop was two-fold: (i) to elicit feedback on the preliminary findings from the literature review and interviews (examine if these are plausible, identify inconsistencies and gaps) and (ii) to consider possible solutions to further improve the commissioning processes at DWP.

The online workshop (lasting 2 hours) was held on 24 March 2022 with 18 participants from different stakeholder groups: 11 commissioners of employment support and health assessments^{[footnote 17]}, 4 providers of contracted out employment services and providers’ representatives, and three experts: academics and researchers. The list of workshop participants was agreed with DWP and included some stakeholders who participated in the interviews, as well as those who were not interviewed. More information about the workshop is presented in Annex 4.

1.3.4. Strengths and limitations of the methodology

To the knowledge of the authors, this is one of the few studies that explores contracting out in such a comprehensive way, including diverse perspectives from practitioners and academics, commissioners and providers^{[footnote 18]}.

A number of limitations to this methodology should be noted. Whilst it takes a structured approach, a QSR does not follow the same levels of rigour as a systematic review or rapid evidence assessment^{[footnote 19]}. It is possible that some relevant sources were missed, particularly those published prior to 2011. The focus on English-language sources may likewise have resulted in certain findings being excluded. One major limitation to the review is the dearth of literature on contracting out health assessments specifically. Where this evidence exists, it is highlighted as text boxes in this report for greater visibility.

In addition, the study draws on a relatively limited number of in-depth interviews; different viewpoints or additional findings might have emerged if the pool of interviewees were larger. This particularly pertains to the health assessments, where interviews were conducted only with commissioners and the perspectives of providers were not included^{[footnote 20]}. Interview findings in relation to health assessments – even if representing only the views of commissioners – are included in text boxes for better visibility.

1.4. Structure of this report

The remainder of this report is structured as follows:

Section 2 presents key findings in relation to contract design (including cost and benefits, risks, specifications and other aspects associated with Payment by Results (PbR) contracts)
Section 3 offers insights on market competition (including selection criteria used and types of contracts)
Section 4 outlines live running (or execution of contracts from the commissioners’ perspectives)
Section 5 offers key conclusions and outlines implications from this research.

Question 6

2. 	Contract design

Accepted Answer

Key findings on contract design

PbR contracts may bring benefits in terms of programme efficiency and value for money. However, there is mixed evidence on the impact of PbR contracts on programme effectiveness. The main costs associated with PbR contracts are linked with undesirable behaviour from providers such as creaming and parking. Contracting out may also weaken the accountability of public services
For commissioners, risks associated with PbR contracts include overpaying, underpaying and market failure. For providers, these are high upfront costs, underperformance and reputational concerns. In addition, contracting out may incur a risk of variable service provision for users/beneficiaries
Black box provision allows providers greater flexibility and may facilitate innovation, but evidence is scarce. Flexibility results in greater service variability and may increase the risk of creaming and parking. Minimum service requirements are designed to assure basic service standards
The prime model of contracting builds long-term relationships with large contractors who can take on the risk of a PbR contract. However, there is a risk of subcontractors not being able to benefit fully from contracts
Contract Package Areas are considered by some interviewees as too large (and not conducive to horizontal integration) and successful contractors may lack local expertise, although there is limited evidence in this area
An optimal length of contracts is considered by interviewees between five and seven years, where there is a perceived need for services lasting that long.

This chapter summarises evidence on the benefits, costs and risks associated with the contracting out of employment services (and health assessments – where evidence exists). The chapter considers how these factors are shaped by elements of contract design, such as payment structures (e.g., outcome payments versus service fee), levels of specification (e.g., black box versus more prescriptive approaches), the prime/sub-prime model, contract length and geographical divisions (e.g., Contract Package Areas).

This chapter focuses primarily on the design of PbR contracts, which have dominated the contracting out of employment services in the UK in recent decades (see Box 1). Cost Plus contracts (an example of contracted out provision without PbR) are covered in section 2.4. Some arguments made about the effects of contracting out and the optimum model for doing so are not specific to PbR (even if most contracted out programmes where evidence is available include PbR elements), and where this is the case efforts are made to draw this out in the text.

Before the specific features of PbR contracts are presented, a basic description of the PbR mechanisms are outlined below:

Commissioner: usually a public authority (here: DWP)
Service provider(s): organisations entitled by the commissioner to compete for delivery of services
Results: outcomes of service provision to clients (here: job placements for employment support or established appropriate level of benefit claims for health assessments)
PbR contract: an agreement determining what services are to be provided (when, where, for whom, and how), what the expected targets are, and how are these paid for:
- Upfront or service fee – a fee for services paid to provider per customer for offering some basic service to help providers with an initial cost of setting up and providing the service until results (paid for outcomes) are expected to start to materialise
- Outcome payments – price paid to providers for results achieved; these can vary for client groups (e.g., lower price for clients easier to place into jobs and higher price for clients with complex needs) and may include fees for achieving long-term outcomes (e.g., keeping clients in work over a certain period) or exceeding the targets set for performance. There may be any number of separate outcomes which attract payments.

2.1. Costs and benefits associated with contracting out

The main potential benefits of contracting out employment services identified in the literature and interviews are increased programme efficiency (section 2.1.1) and effectiveness (section 2.1.2). Concepts such as effectiveness, efficiency, cost effectiveness, and value for money are central to understanding the costs and benefits of contracted out provision. Whilst there is no universally agreed definitions of these concepts, this is how they are used in this report:

effectiveness: the extent to which a programme achieves the desired outcomes^{[footnote 21]}
efficiency: the degree to which results are achieved in an economic and timely way^{[footnote 22]}
cost effectiveness: the effect of a programme relative to the cost^{[footnote 23]}
value for money (VFM): using public resources in a way that creates and maximises public value^{[footnote 24]}.

A programme may be effective without being efficient, for instance if it achieves the desired outcomes but at a prohibitively high cost. Conversely, programme efficiency may come at the cost of effectiveness if low-cost services are pursued to a degree that compromises quality.

The main possible costs associated with contracting out are ‘gaming’ behaviours on behalf of providers which can undermine programme quality and disadvantage job seekers who face greater challenges finding employment (section 2.1.3), as well as concerns around weakened transparency and accountability (section 2.1.4).

Whether these benefits and costs materialise in reality is another question. On many, if not all, of these points, the existing evidence is limited, with mixed results^{[footnote 25]}^{[footnote 26]}. It is difficult to reach overarching conclusions about the effect of contracting out (relative to public employment services (PES) provision or no provision) for several reasons:

limited empirical evidence, linked to methodological challenges associated with estimating the impact (whether in terms of effectiveness, efficiency, cost effectiveness etc.) of contracting out (see Annex 5 for a more detailed discussion)
challenges establishing a counterfactual from which to draw robust comparisons between different forms of provision (e.g., contracted out versus PES)
difficulties estimating the effect of contracting out as distinct from effects associated with specific elements of contract design (PbR, black box etc.)
empirical studies generally compare contracted out provision (using PbR) to PES provision (or no provision), but where differential outcomes are observed it is difficult to say whether this was due to contracting out per se or to PbR specifically (or indeed other aspects of contract design).

Theoretical arguments made in the literature and empirical evidence about the advantages and disadvantages of contracting out (see Table 1) focus on PbR, so this section focuses on such contracts. However, the available literature does not allow us to compare outcomes for PbR versus other forms of contracted out provision (such as Cost Plus contracts) or to directly compare different PbR designs (e.g., placing lesser or greater emphasis on outcome payments).

Synthesising the evidence in this area to draw overarching conclusions is complicated by the following factors:

studies relate to different contexts and use different research methodologies
studies differ in terms of the programmes they relate to and the extent to which and how these programmes incorporate different elements of contract design.

2.1.1. PbR contracts may bring benefits in terms of programme efficiency and value for money

Amongst potential benefits of contracting out employment services is the flexibility to scale up or down the provision as needed and the ability to better tailor services to specific customer groups^{[footnote 27]}. It has been also argued that contracting out employment services can improve their efficiency and cost effectiveness, reducing expenditure on activities unlikely to produce job outcomes^{[footnote 28]} and strengthening incentives for delivery organisations to pursue cost savings^{[footnote 29]}.

A desire to increase the efficiency and cost effectiveness of programmes and to achieve VFM has been a key motivating factor for implementing PbR contracts in countries such as the US (where they are sometimes described as Pay for Success (PFS) programmes)^{[footnote 30]}^{[footnote 31]} Australia, the UK, Sweden and the Netherlands^{[footnote 32]}. The empirical evidence described in this chapter is mixed: some studies find PbR contracts for employment programmes are associated with outcomes such as efficiency, cost effectiveness and VFM, whereas this appears not to be the case for other programmes.

In the UK, the impact assessment of the Work Programme (WP) assessed cost effectiveness and VFM^{[footnote 33]}. Following an Intention to Treat (ITT) approach^{[footnote 34]}, the study compared outcomes for job seekers claiming out of work benefits before and after March 2017, when referrals to the WP closed. The comparison made in this study was between those eligible for the WP and those who would have been eligible for the WP if it was still in operation. However, the authors recognise limitations such as a potential lack of external validity (generalisability of the findings) associated with focusing on participants at the end of the WP^{[footnote 35]}. The cost benefit analysis^{[footnote 36]} found that outcomes achieved by DWP resulted in in an estimated £0.83 (with a range of £0.56 to £1.10) in return for each £1 spent by DWP over the observed two-year period, but this rose to £1.76 (with a range of £1.21 to £2.32) when the findings were extrapolated over a four-year period^{[footnote 37]}. This suggests that over the medium to long-term, the WP was cost effective and achieved VFM for the taxpayer. However, the comparison drawn is with no provision. It is not possible to infer from this analysis whether the WP was more/less cost effective compared PES provision or a different form of contracting out (e.g., non-PbR). Evidence on other UK contracted out programmes is summarised in Box 1.

Box 1. Body of evidence on contracting out employment services from the UK

The New Deals were a series of workfare programmes^{[footnote 38]} implemented in the UK between 1998 and 2009 using both contracted out and in-house provision, including targeted programmes for young people (18-24), older people (50+), people with disabilities and lone parents.

the New Deal for Young People (NDYP) (1998 to 2009) was a mandatory programme for unemployed workers aged 18 to 24 who had been claiming Job Seeker Allowance (JSA) continuously for 6 months or more. An evaluation of longer-term outcomes associated with the NDYP found that participants spent on average 64 fewer days claiming JSA and other labour market benefits over a four-year period compared to a comparison group of workers who were slightly too old to participate in the programme^{[footnote 39]}.
the New Deal for Disabled People (NDDP) (1998 to 2009) was a voluntary employment programme available to people claiming incapacity benefits. Services were provided by Job Brokers comprised of a mix of public, private and third sector organisations^{[footnote 40]}. An evaluation found that more NDDP participants than non-participants left incapacity benefits and entered paid employment^{[footnote 41]}. The cost-per-placement for NDDP was estimated to be £2,000 to £3,000, rising to £4,000 to £5,000 for a sustained job outcome of six months or more^{[footnote 42]}. Cost-benefit analysis found that the Government saved money (£3.41 to £4.50 for continuing claimants and between £1.71 to £2.26 for new claimants) for each pound spent on the NDDP programme^{[footnote 43]}.

Employment Zones (EZs) (2000 to 2009) used contracted out employment services (under ‘black box’ PbR contracts – see section 2.3) to support unemployed workers in deprived areas into employment. EZ provision was originally targeted at long-term unemployed workers aged 25 and older but was later expanded to include young people and lone parents^{[footnote 44]}. A synthesis of the evidence on EZs^{[footnote 45]} concludes that they were more effective than comparative New Deal programmes in terms of job outcomes achieved. However, EZs were found to offer less VFM than comparable New Deal programmes, particularly NDYP^{[footnote 46]}.

The Flexible New Deal (FND) (2009 to 2010) was an employment programme for long-term unemployed workers^{[footnote 47]}. In addition to Jobcentre Plus (JCP), services were provided by prime providers commissioned on a PbR basis^{[footnote 48]} who were given considerable flexibility in service provision (the ‘black box’ approach)^{[footnote 49]}. An evaluation based on survey and interview data found that the proportion of young people (aged 18 to 24) participating in the programme who had secured paid work was higher in Phase 2 areas (where the FND was introduced in 2010) compared to Phase 1 areas (where the FND was introduced in 2009), providing some indication that the programme’s predecessor – NDYP – may have produced better outcomes for this group^{[footnote 50]}. This evaluation found that work outcomes achieved by people participating in the FND were largely sustainable. No impact assessment was published for the FND.

Pathways to Work (2003 to 2011) was an employment programme for people claiming incapacity benefits. Initially delivered by JCP, a provider-led model was introduced in 2008 where services were delivered by contracted private and third-sector organisations. Providers were paid for job outcomes and sustained job outcomes, and received a service fee for taking people onto their caseloads^{[footnote 51]}. An evaluation found the net impact of the provider-led Pathways had no statistically significant effect on employment, although there was a reduction in benefit claims^{[footnote 52]}. Outcomes were similar for the JCP and provider-led Pathways^{[footnote 53]}.

Work Choice (2010 to 2017) was a tailored employment programme for people with a disability or health condition^{[footnote 54]}. An evaluation of the Work Choice programme found that providers experienced challenges achieving job outcome targets^{[footnote 55]}.

Work Programme (WP) (2011 to 2017) was an employment programme for the unemployed^{[footnote 56]}. Results from the impact assessment are discussed elsewhere in this chapter.

New Enterprise Allowance (NEA) (2011 to 2022) was a programme to support the unemployed into self-employment^{[footnote 57]}. A qualitative evaluation found that the programme offered effective support to people interested in self-employment^{[footnote 58]}. No impact evaluation is available at the time of writing.

The support for the very long term unemployed trailblazer was a randomised controlled trial (RCT) of support for people reaching the end of their two years on the WP. The trial identified positive results, with those receiving support spending significantly less time on benefits and more time in employment compared to the control group^{[footnote 59]}.

The Work and Health Programme (WHP) (2017 to present) is an employment programme for people with disabilities and those long-term unemployed^{[footnote 60]}. No impact evaluation is available at the time of writing.

Intensive Personalised Employment Support (IPES) offers one-to-one support and training for people with disabilities. No impact evaluation is available at the time of writing.

Restart (2021 to present) provides support for Universal Credit and JSA Income Based claimants who have been claiming for at least 9 months (with no upper limit on the duration of claims)^{[footnote 61]}. The Restart Scheme has only been in operation since 2021 so it is too early for results from an impact assessment to be published.

Evidence from Australia^{[footnote 62]} supports the idea of improved cost effectiveness associated with PbR contracts. Summarising findings from evaluations of the Job Network, 2 sources^{[footnote 63]}^{[footnote 64]}, highlight the fact that providers achieved lower placement costs compared to previous PES provision (i.e., cost per outcome). The cost per employment outcome achieved by contracted providers was around 50 per cent lower compared to public provision^{[footnote 65]}. Whilst outcomes achieved by providers in the Job Network were similar to previous programmes delivered by the PES, these outcomes were achieved with fewer resources, suggesting improvements in efficiency^{[footnote 66]}. However, cost per job outcome achieved is an imperfect measure because there may be differences in the type of services delivered by the two provider types and/or the profile of job seekers they support (particularly in this case because the provision was at different points in time). Moreover, lower costs per job outcome may be achieved by ‘gaming’ practices such as creaming and parking (see section 2.1.3) i.e., prioritising job seekers with a stronger employment profile for whom achieving outcomes is less resource intensive.

Other studies find no evidence of improved efficiency, cost effectiveness or VFM associated with PbR contracts. An RCT looking at the effect of contracting out employment services for unemployed university graduates in Denmark (relative to PES provision) under PbR contracts found no difference in public expenditure associated with the two programmes (coupled with comparable effectiveness, suggesting similar value for money for the taxpayer)^{[footnote 67]}. An RCT comparing the impact of job search activities provided by private providers and the PES in France found that only publicly delivered services were associated with a reduction in the amount of time spent unemployed, thereby reducing expenditure on unemployment benefits (contributing to VFM)^{[footnote 68]}.

A literature review drawing on evidence from France, Germany, Denmark and Sweden found that whilst contracting out is associated with increased cost effectiveness for services that are more straightforward to contract out, such as cleaning, waste disposal and transportation, for employment services the effect is neutral or negative (i.e., a decrease in cost effectiveness)^{[footnote 69]}. This is consistent with findings from within the UK where outsourcing in employment shows mixed evidence i.e. including both increased and decreased cost effectiveness^{[footnote 70]}. However, it is not clear whether all the programmes considered in the literature have a PbR design.

Efficiencies and improvements in cost effectiveness achieved by PbR contracts should be offset against transaction costs (e.g., costs associated with writing specifications, evaluating bids and monitoring performance) in assessing VFM^{[footnote 71]}^{[footnote 72]}. Transaction costs will depend on the approach taken to contracting out and the systems put in place, but generally amount to only a small proportion of total costs. However, stakeholders in the Netherlands cited in one qualitative research study described “costly, over-complex contractual models” which detracted from the cost effectiveness of contacted out employment services^{[footnote 73]}.

In sum, evidence on the impact of PbR contracts on the efficiency, cost effectiveness and VFM of employment services is mixed. In countries such as Australia and the UK, PbR contracts have been associated with cost effectiveness and/or VFM, but in other contexts (France, Denmark) they have not. There seems to be consensus in the reviewed literature that:

i. PbR is not equally well suited for all public services (it works well in simple, transactional services like waste management but less well in more complex services, like health or education)^{[footnote 74]}

ii. it is a difficult form of contracting^{[footnote 75]}

iii. benefits (and costs) of PbR contracts in employment services depend on contract design and monitoring arrangements^{[footnote 76]}; concerns have been also raised in the literature that incentives to cut costs created by PbR contracts may result in negative consequences for staff providing services such as poorer employment conditions and lower wages^{[footnote 77]}^{[footnote 78]}.

Table 1. Empirical evidence on the benefits of contracting out employment services using PbR

Study	Country	Services contracted out	Recipient group	Contract design Research/evaluation design	Comparator group(s)	Outcome type	Specific outcome(s)	Effect of contracting out/PbR
Kay and Marlow (2020)	UK	Work Programme – employment support with flexible ‘black box’ provision	Economically inactive and unemployed. Impact assessment focused on those aged 25+ in receipt of benefits for 12+ months^{[footnote 79]}	PbR, weighted towards outcome payments differentiated by group	Impact assessment using ITT approach	No provision	Cost effectiveness^{[footnote 80]} – – VFM^{[footnote 81]} – – Effectiveness	Cost effectiveness: Return on investment^{[footnote 82]} – – VFM: Return on investment – – Effectiveness: Additional days in employment and not in receipt of benefits	+
Rehwald et al. (2015)	Denmark	Employment counselling and training	Newly registered unemployed with a university degree	PbR (operation subsidy and performance-based bonus)	RCT	PES provision	VFM^{[footnote 82]} – – Effectiveness	VFM: Net expenditure – – Effectiveness: Regular employment, employment with wage subsidies, non-benefit receipt, unemployment	0
Dockery and Stromback (2001)	Australia	Job Network – job-matching activities, job-search training and intensive assistance	Unemployed	PbR, upfront fee + payment for job outcomes, additional bonus for placing long-term unemployed	Descriptive statistics	Mixed provision (PES and private contractors)	Cost effectiveness	Net cost per outcome	+
Struyven and Steurs (2004)	Australia	Job Network – job-matching activities, job-search training and intensive assistance	Unemployed	PbR, upfront fee + payment for job outcomes, additional bonus for placing long-term unemployed	Descriptive statistics	Mixed provision (PES and private contractors)	Cost effectiveness	Net cost per outcome	+
Behaghel et al (2012)	France	Intensive job search programmes	Newly unemployed in specific geographical areas	PbR – 30% upfront fee, 70% performance-based payment	RCT	PES, no intensive support (control group)	VFM – – Effectiveness	VFM: Time spent claiming unemployment benefits – – Effectiveness: Exit rate to employment	0 relative to no intensive support; effect is + for PES relative to no intensive support – – + relative to no intensive support, but effect weaker than PES compared to no intensive support
Cockx and Baert	Belgium (Flanders)	Counselling and training	Long-term unemployed	PbR – 30% outcome-based payment	Event history analysis	PES	Effectiveness	Exit to employment	+
Pastore (2019)	Italy (Lombardy)	Dote Unica Lavoro – voucher system for employment services	Unemployed	Bonus payments linked to achieving sustained job outcomes	Descriptive statistics	PES	Effectiveness	Sustained employment/self-employment for 6 months	–
Koning and Heinrich (2010)	Netherlands	Employment services	People claiming unemployment/disability benefits	PbR – shift from partial to 100% outcome-based payments	Probit model using administrative data	100% PbR compared to lower outcome-based payments	Effectiveness	Job placement rate	+ for cohort claiming unemployment benefits 0 for cohort claiming disability benefits

Note: + means contracting out/PbR had a positive effect compared to the comparison group, - means negative effect and 0 means neutral/no effect

Findings from the interviews

It is important to note that DWP – similar to other commissioners – uses a hybrid model which combines outcome payments with service fees^{[footnote 83]}. Therefore, investigations of the benefits of PbR in this context involve determining the optimal balance of these two elements within a payment model^{[footnote 84]}.

Interviewees across all stakeholder groups expressed the opinion that PbR drives performance by focusing the attention and efforts of providers on efficient achievement of the desired outcomes^{[footnote 85]}. For this mechanism to work, clearly defined, relatively short-term and measurable outcomes are required^{[footnote 86]}. Weighting a contract towards outcome payments was also thought to incentivise quick responses to problems or unexpected changes, so providers avoid any disruption to their achievement of payable outcomes^{[footnote 87]}. Ultimately, PbR creates a ‘lever’ for the commissioner which they can use in combination with the level of specification to drive efficient and effective behaviours in providers^{[footnote 88]}.

In a contract weighted towards outcome payments, any ‘waste’ of DWP expenditure can be understood to be minimised as the commissioner will only pay for successful outcomes^{[footnote 89]}. However, this opinion assumes the outcomes would not have been achieved without the contracted service^{[footnote 90]}.

DWP representative, said:

A fundamental problem is that the outputs being paid for are rarely if ever the outcomes that we are aiming for. (…) A major concern in [value for money] considerations is deadweight – paying for outcomes which would have been achieved without the intervention – and PbR generally does nothing to reduce this.

To a certain extent, PbR presents financial risk to providers, alongside the potential of making substantial profit^{[footnote 91]}. This is explored in greater detail in section 2.2.

On the one hand, one possible effect of PbR is providers are likely to be operating a low cost, ‘lean’ business which can be viewed as a maximisation of efficiency for these organisations^{[footnote 92]}. For service users, providers’ response to PbR can result in a more focused and time-sensitive provision as wasting service user’s time or failing to create a sustainable outcome is disincentivised^{[footnote 93]}. However, the model may also incentivise gaming strategies likely to reinforce inequalities amongst service users (discussed in section 2.1.2).

On the other hand, the staggering of income produced by PbR can result in a cashflow problem for providers which impacts negatively on service quality, investment and innovation^{[footnote 94]}. Fluctuation in payable outcomes over the course of a contract can result in providers lowering or cutting investment, especially towards the end^{[footnote 95]}. Even a temporary dip in income, or a period of making a loss, can cause a ‘downward spiral’ for the provision and the health of the provider organisation^{[footnote 96]}.

Some expert and provider interviewees held the view that PbR drives innovation^{[footnote 97]}:

Expert interviewee, said:

The main driver here would be the value for money idea, this idea that you ideally only pay what you get for, but you also, in that sense, obviously spur innovation, because usually […] there should be incentives to get outcomes for as many people as possible.

This statement suggests that the incentive to achieve more or better outcomes, and to do this more efficiently, encourages the development of new approaches and problem-solving. However, there were also reservations about the potential for innovation in PbR contracts which are heavily weighted towards outcome payments, with the lack of immediate income making investment in innovation almost impossible^{[footnote 98]}.

Interviewees differed on how best to facilitate successful PbR contracts: some commissioner and provider interviewees insisted that PbR cannot work efficiently if the commissioner interferes in delivery^{[footnote 99]}, while other commissioners and providers maintained that specification was needed to maximise benefits and contain risks or costs^{[footnote 100]}.

2.1.2. Mixed evidence on the impact of PbR contracts on programme effectiveness

The contracting out of employment services is expected to improve service quality by fully utilising the knowledge and expertise of external providers^{[footnote 101]}. It has been argued in the literature that competition drives improvements in the quality and effectiveness of employment services^{[footnote 102]}. PbR contracts specifically are thought to lead to improved performance because they encourage providers to focus on the purpose of the service and to invest in effective strategies^{[footnote 103]}. PbR contracts can be used to direct efforts towards specific desired outcomes, such as higher paying and/or more sustainable employment^{[footnote 104]}^{[footnote 105]}. It matters greatly, therefore, which outcomes are linked to payments under PbR contracts.

The impact assessment of the WP in the UK compared outcomes for groups of job seekers eligible for the WP^{[footnote 106]} and those who would have been eligible for the programme if it had still been in operation after March 2017. The study estimates the WP was associated with 46 additional days in employment and not in receipt of an Out of Work benefit for participants over a two-year period^{[footnote 107]}.

Cockx and Baert (2015)^{[footnote 108]} compare mandatory counselling and training for the long-term unemployed provided by the public, private and non-profit organisations in Belgium (Flanders) using event history analysis. Their analysis estimates the likelihood of leaving unemployment for employment (relative to remaining unemployed or leaving the labour market) under different conditions. The PbR element in this contract was relatively small (30%), with a bonus of €500 paid per placement if the provider managed to attain a placement rate at least three percentage points above a target set at the regional level. The study finds that private providers were slightly more effective than PES in achieving job outcomes for the unemployed. One year after assignment, exit rates to employment were 2.8 percentage points higher for private providers compared to the PES, and the return to unemployment was postponed by 1.4 months. Private providers achieved better outcomes on average than not-for-profit organisations, but these differences were small and not statistically significant. The authors hypothesise several reasons for the better job outcomes achieved by private providers. Private providers were new to the market and had a greater incentive to develop a good reputation, particularly because the contracts were likely to be re-tendered. Private providers were larger than non-profits and may have been able to benefit from economies of scale. In addition, even though private providers/non-profits were not able to refuse referrals, there is evidence to suggest that the composition of their clientele differed (with private providers supporting job seekers with better employment prospects). This suggests that some of the differential may be due to selection bias (see Annex 5) rather than provider type.

Literature reviews identify other countries in which PbR contracts have been associated with improvements in the effectiveness of employment services relative to PES provision, including Ireland (JobPath programme) and New Zealand^{[footnote 109]}. However, research from other countries paints a more mixed picture in terms of the impact of PbR contracts on programme effectiveness, as described below.

The evidence on the impact of reforms in Germany (2003 to 2005) which introduced quasi-markets in employment services and a voucher system, are mixed: one evidence review finds that private providers tend to secure better outcomes in terms of job quality (wages, job satisfaction, fixed-term employment and employment duration) compared to the PES^{[footnote 110]} but a literature review drawing on multiple studies concludes there is no evidence that private (relative to public) provision, or the introduction of a voucher system increased the likelihood of participants entering employment^{[footnote 111]}. The voucher system was intended to drive improvements in performance through competition and consumer choice^{[footnote 112]}^{[footnote 113]}. The limited effectiveness of the voucher system in Germany (in terms of driving improvements relative to public provision) may be partly attributable to the fact that many voucher recipients did not use them^{[footnote 114]}. Job seekers – particularly those with a lower level of education – may not have the necessary information to make a meaningful choice under the voucher system, and in certain regions there may not be a sufficient number of providers to enable true competition^{[footnote 115]}. Market information was imperfect, since participants often did not have information about the performance of different providers^{[footnote 116]}. One study comparing job outcomes (defined as at least 6 months continued employment or self-employment) for private and non-profit providers and the PES under a voucher system in Italy identified better outcomes for the PES (and to a lesser extent non-profits) compared to private agencies^{[footnote 117]}.

Koning and Heinrich (2010)^{[footnote 118]} estimate the effect of private provision of employment services under PbR in the Netherlands. Over the period 2002 to 2005, the PbR element of welfare-to-work contracts was strengthened, moving to a model where 100% of payment was outcome-based. Using administrative panel data, this study finds that shifting to a full PbR model was associated with an increase in job placements observed for unemployed workers, but not for those claiming disability benefits. There was no increase in average duration of employment associated with the shift to full PbR; additional job placements were largely confirmed to temporary positions. The authors conclude:

These benefits of performance-based contracting appear to be limited to short-term outcomes, and thus, their effectiveness in increasing the long-run impact of public programs such as welfare-to-work services remains questionable^{[footnote 119]}.

An RCT evaluating the impact of the contracting out of employment services via PbR contracts for unemployed university graduates in four municipalities in Denmark^{[footnote 120]} found no evidence of differences in the effectiveness of public and private programmes. However, there was some evidence of better job outcomes secured by private providers for specific groups, namely men and those with prior work experience. This could indicate private providers’ greater investment in groups deemed easier to place (creaming and parking, see section 2.1.3) although this is not addressed in the source.

Behaghel et al. (2012)^{[footnote 121]} compare outcomes for intensive job search programmes delivered by private providers and the PES in France (comparing them to a control group who did not receive intensive support) using an RCT design. Private providers operated under a PbR framework, with a 30% upfront payment and 70% conditional on securing job outcomes (35% linked to starting a job within 6 months; the remaining 35% paid if the worker was still employed after 6 months). Due to differences in types of job seekers supported by the two programmes, comparisons could only be drawn in relation to the newly unemployed in certain geographical areas. The study finds that both programmes were effective, increasing exit rates to employment (relative to the control group who did not receive intensive job counselling). However, the impact was greater in the programme delivered by the PES compared to private contractors. After 6 months, the exit rate was 4.2% higher for the private programme relative to the control group and 9.1% higher for the public programme. The authors argue this relates to the large upfront payment for private providers, encouraging them to enrol as many job seekers as possible. In contrast, the public employment agency had to manage scarce resources and had no incentive to maximise enrolment. In recent years, the involvement of private contractors in providing employment services in France has declined and in particular services targeted at the most vulnerable job seekers have been re-integrated into the PES^{[footnote 122]}.

Findings from the interviews

Any disruption caused to outcomes due to circumstantial or environmental change could be financially debilitating, leaving providers vulnerable to economic shifts or other uncontrollable factors^{[footnote 123]}.

Commissioner interviewee, said:

I think the Work Programme was almost all payment by results, which did cause issues with cash flow, did then lead to maybe only [having] companies that could […] borrow easily, because you end up paying for that borrowing […] also, if you go too far on the PbR route […] it leaves providers very vulnerable to fluctuations in the economic context that they can’t control, and volume contexts that they can’t control.

These levels of uncertainty and financial precarity are understood to have the additional effect of forcing small and medium-sized providers out of the market^{[footnote 124]}. These effects will likely have a negative impact on programme provision, with providers focusing on risk management over delivery and specialist providers not being present to provide tailored support.

Nonetheless, a narrow focus on outcomes was seen as detrimental to the quality of the service and the experience of users by interviewees across commissioners, providers and experts alike^{[footnote 125]}. This has been seen to manifest negatively in the following ways:

an incentive for rapid outcomes, rather than sustainable ones (depending on how outcomes are defined)^{[footnote 126]}
a loss of staff or a less skilled workforce as staff feel over-pressured to reach outcome targets^{[footnote 127]}
the absence of incentives, resources and will to innovate^{[footnote 128]}
a lack of resources allocated to partnership or ‘joined up’ provision^{[footnote 129]}
the side-lining of service users who may need more costly or time-consuming support before an outcome can be claimed^{[footnote 130]}.

Some interviewees, across the three stakeholder groups, suggested that poorly designed PbR can ‘hurt’ a programme^{[footnote 131]}. For example, ‘aggressive’ PbR was thought to hinder the effectiveness of the Work Programme^{[footnote 132]}, while the cost of financing programmes^{[footnote 133]} with low service fees was said to divert funds from service users^{[footnote 134]}. Meanwhile, some alternative payment mechanisms were seen, by commissioner and provider interviewees, to be better suited to certain programmes’ design^{[footnote 135]}, enabling greater success in programmes such as the Job Entry Target Support programme (JETS)^{[footnote 136]}.

2.1.3. The main costs associated with PbR contracts are linked with undesirable behaviour from providers such as creaming and parking

One potential cost associated with contracting out employment services is an increased risk of practices such as parking and creaming (or cherry-picking), which have a detrimental effect on the prospects of jobseekers who are harder-to-place. Evidence from countries including the UK^{[footnote 137]}^{[footnote 138]}, the USA^{[footnote 139]}, Australia^{[footnote 140]}^{[footnote 141]}, Germany^{[footnote 142]}, Denmark^{[footnote 143]}, Italy^{[footnote 144]} and the Netherlands^{[footnote 145]}^{[footnote 146]}, supports the idea that contracting out is associated with gaming practices that draw resources and support away from job seekers who are most in need.

Gaming practices are concerning from an equity perspective because the most vulnerable will be left without sufficient support. These practices could also undermine VFM (since some of these outcomes would have been achieved anyway)^{[footnote 147]}^{[footnote 148]}. The term ‘gaming’ implies deliberate strategies on behalf of providers to maximise profit, but these behaviours may be inadvertent or deemed necessary to avoid making a loss. We differentiate between gaming (which has negative consequences for some customers) and rational economic practices among providers to deliver services in the most economic ways (see section 4.5).

Countries where provision is contracted out have developed and adapted their approach over time, with a particular focus on limiting gaming practices^{[footnote 149]}^{[footnote 150]}. Evidence suggests the effect of contracting out on gaming practices can be mitigated by ‘accountability levers’^{[footnote 151]} built into contract design. Contract design can be used to weaken or counteract incentives to direct support away from jobseekers who are hard-to-place.

Accountability levers to address cherry picking relate to eligibility and the degree to which providers can choose to accept or reject job seekers^{[footnote 152]}^{[footnote 153]}. Objective eligibility criteria^{[footnote 154]} and an independent referral process^{[footnote 155]}, as well as making it mandatory for providers to accept referrals are suggested as tools to address cherry picking. In Australia, private providers cannot refuse referrals, and evidence suggests this has been effective in limiting cherry picking^{[footnote 156]}. However, there is a danger that if private providers are obliged to accept all referrals, then this will result in parking^{[footnote 157]}.

A common approach to address creaming and parking is to differentiate payments for different target groups, with higher payments for placing jobseekers with poorer employment prospects^{[footnote 158]}^{[footnote 159]}. In Ireland job seekers are assigned to different payment groups according to unemployment duration^{[footnote 160]}. In Australia outcome-based payments increase with the duration of unemployment^{[footnote 161]}. In Italy the payment amount linked to a voucher for achieving a successful job outcome depends on the profile of the job seeker^{[footnote 162]}^{[footnote 163]}.

Under the WP in the UK, participants were placed into one of 10 claimant groups as a proxy for their perceived support needs^{[footnote 164]}. Higher fees were paid in response to job outcomes for claimants in groups deemed harder-to-place^{[footnote 165]}. The pricing model was influenced by the ‘Non-Intervention Rate’ (NIR) – the percentage of participants it is estimated would have achieved a successful outcome without assistance^{[footnote 166]}. Drawing on WP data and qualitative research^{[footnote 167]}, Creaming and parking practices were widespread in the early stages of the WP, something blamed on the programme’s weak accountability framework^{[footnote 168]}. The authors suggest the design of the payment structure may not have been sufficiently well aligned with the level of difficulty associated with placing job seekers (although it would be difficult, if not impossible, to create perfect alignment), weakening the effect on creaming and parking^{[footnote 169]}. This source describes how, rather than using the claimant groups as defined under the DWP payment structure, many providers used their own system of ‘triaging’ participants, for instance the red, amber, green (RAG) rating system used by some providers. Some providers confirmed at that time the RAG approach was used to determine which participants to direct the most resources to, including within payment groups. Imbalances in job outcomes achieved by the different payment groups (as evident in programme statistics) may indicate some degree of creaming and parking as well as variation in the challenges faced by different groups in finding work^{[footnote 170]}. The authors observe:

Claimant groups [payment groups in the Work Programme] were a blunt instrument orientated primarily around the prior benefit received and not necessarily coterminous with a customer’s distance from the labour market^{[footnote 171]}.

Another source focusing on WP notes that for some groups, payments may not be high enough to cover the cost of supporting them into work^{[footnote 172]}. The considerable heterogeneity within payment groups (modelling suggests greater variation within payment groups than between them) may have led providers to focus their efforts on those with the best employment prospects in each group^{[footnote 173]}. Qualitative research found that some WP providers admitted to seeing job-ready participants more frequently. However, most programme participants thought the support they received was adequate and well matched to their needs^{[footnote 174]}.

As a means of addressing this issue in future programmes, some commentators have argued for the use of ‘accelerator’ or ‘escalator’ models, where providers are paid according to the percentage of participants who achieve job outcomes in each payment group^{[footnote 175]}, see Table 3 (section 3.3).

Ayaita et al. (2021)^{[footnote 176]} find the introduction of a voucher system in Germany in 2002 reduced gaming practices and allowed unemployed workers to choose support from a range of private providers. Agencies could redeem the voucher from the public employment agency if they secured a positive job outcome^{[footnote 177]}. The voucher amount received by the provider was higher for the long-term unemployed. Private providers also received remuneration from the new employer of the placed worker. The study finds evidence of gaming practices between 1995 and 2002. In this period, long-term unemployed workers were more likely to be placed by the public employment agency than private providers, suggesting private providers were focusing their efforts on workers with higher qualifications and better employment prospects. The same trend is evident in the period 2003-2018. However, the effect is substantially larger in the earlier time period where the voucher system was not in place, suggesting that vouchers were effective at reducing creaming and parking. Moreover, there is evidence that individuals placed privately with vouchers were more likely than those placed privately without vouchers to have poorer employment prospects (e.g., lower educational qualifications, experience of unemployment)^{[footnote 178]}.

Findings from the interviews

The largest concern around PbR was indeed the possibility of incentivising gaming behaviours, including parking and creaming^{[footnote 179]}.

Expert interviewee, said:

The ones that are most easy to place into a job […the providers] cream those ones off and work hardest on getting them into a job, but the hardest ones to place, [they] park and do minimal stuff with them, and they won’t get jobs. So, from the provider’s point of view, that’s good in terms of return on their investment. From the government’s point of view, it’s probably not what you want, because the people you want getting into work are those hardest to help ones. The ones that are closer to the labour market may well have found those jobs themselves anyway, and so you’re paying for dead weight.

However, views were mixed on which types of gaming were most harmful or, indeed, if some instances could be acceptable:

Provider interviewee, said:

The problem with some payment by results systems is the parking aspect, so, to me, creaming and parking are often thrown together [but] they’re very different things: creaming can be “let’s get people who are ready for work in work quickly and then we’ve got more resources”. Parking is […] the opposite of that: it’s “let’s ignore those people who could be getting more time and more resources because we can focus on this low hanging fruit”.

Some expert and provider interviewees noted that mitigating strategies can be used to counter this tendency towards gaming in PbR contracts^{[footnote 180]}. Possible strategies that were described include increased prescription and auditing^{[footnote 181]}, the use of complementary policies to incentivise focusing on ‘harder to help’ users^{[footnote 182]}, differential or accelerator payment mechanisms^{[footnote 183]}, a reduction of the emphasis on price competition^{[footnote 184]} and the use of earning thresholds to calculate payable outcomes^{[footnote 185]}.

Alternative perspectives on PbR costs and gaming were also offered by representatives from DWP. This new material is incorporated below:

DWP representative,said:

The main cost of a PbR contract is the same as with any other type of contract – paying for deadweight or not paying enough to make a difference and the main risk is setting the thing up with performance expectations that aren’t aligned with what’s actually possible.

There is some (admittedly limited) evidence that net impact of provision can be greater for those who are closer to the labour market. So it is possible in principle that a focus on those who are easier to help might improve overall programme effectiveness in terms of total net impact – but at the same time (…) [it may have] negative consequences for equity.

2.1.4. Contracting out may weaken the accountability of public services

The degree to which contracting out services weakens accountability is contested in the literature^{[footnote 186]}. Some argue that contracting out weakens accountability, whereas others argue to the contrary that contractual agreements and minimum standards strengthen it. Commissioners remain accountable for the services they procure from external agencies^{[footnote 187]}. However, the details of these arrangements are generally commercially confidential and protected from public scrutiny^{[footnote 188]}.

Evidence related to the costs and benefits of contracting out health assessments is very limited (see Box 2).

Box 2. Health assessments: suitability of the PbR model

The reviewed literature focused primarily on contracting out employment services or took a wider perspective of commissioning public services more broadly. None of the reviewed sources discussed contracting out health assessments. Some limited evidence emerged from the interviews. It was noted that PbR may not be the right model for health assessments^{[footnote 189]}. While this payment mechanism can encourage efficiency, the combination of a limited market and the importance of correct procedure during assessments may counteract this^{[footnote 190]}.

2.2. Risks associated with contracting out

2.2.1. For commissioners, risks include overpaying, underpaying and market failure

When employment services are delivered by PES, the success (or failure) of supporting people into work falls primarily on the public sector. This situation becomes more complex when public authorities commission the services externally: the risk – or a possibility of failing to support people back to work – could arguably rest with providers (their knowledge, abilities, good will, etc. or lack of these), or with commissioners (the way the service contracts were designed and managed), or with both parties. PbR contracts are often described as transferring risk from commissioners to providers^{[footnote 191]}^{[footnote 192]}^{[footnote 193]}. However, PbR contracts (and indeed, other contracting out without PbR) are not without risk to commissioners and to an extent these risks can be mitigated by contract design. The government Sourcing Playbook notes that outsourcing complex services carries more risk for commissioners^{[footnote 194]}.

A key risk for commissioners is overpayment i.e., paying providers for outcomes that would have been achieved even without the services delivered^{[footnote 195]}^{[footnote 196]}. Provider behaviours such as cherry picking, creaming and parking (which may be exacerbated by PbR – see section 2.1.3) contribute to overpayment since they direct resources away from job seekers for whom programmes might have the greatest impact^{[footnote 197]}. Differential payments (i.e., higher payments for securing job outcomes for workers who are harder-to-place) are designed in part to address the risk of overpayment, since providers are incentivised to work with groups for whom job outcomes are less likely to be achieved without support. However, even the differential payments create a rather simplistic framework, given the diversity of the client group. In addition, linking outcome payments for placing the most disadvantaged groups in sustainable work, requires long follow up periods, and late outcome payments – which diminish their appeal to providers. There may also be a risk of fraudulent claims of outcomes by providers (e.g. claiming outcomes which have not occurred or claiming that the individual who achieved outcomes was in a higher payment group – in either case leading to overpayment), although little evidence was identified to substantiate this^{[footnote 198]}.

On the flip side, underpayment also presents a risk for providers under PbR i.e., not investing enough in the programme to secure the desired outcomes. Underpayment can result in poor provider performance associated with cost-cutting measures and market failure (see section 2.2.2).

From the commissioners’ perspective, a key risk is poor performance from providers^{[footnote 199]} and, in the extreme, market failure, where the commissioning body needs to ‘bail out’ the provider or find an alternative provider^{[footnote 200]}^{[footnote 201]}, incurring high transaction costs^{[footnote 202]}. Strategies to manage the risk of poor provider performance (other than PbR to strengthen incentives and improve performance) include setting performance targets and minimum service requirements (see section 2.3.4)^{[footnote 203]}, measuring service quality through monitoring and benchmarking (see section 4.1) and contract provisions that penalise providers for poor performance. The risk of market failure can be mitigated by having a larger number of providers rather than a few ‘primes’ (see section 2.6)^{[footnote 204]}. However, a larger number of providers will require more effort in terms of managing and monitoring them, increasing the costs to commissioners.

Findings from the interviews

It appears that risk should not be understood as fully transferred from commissioners to providers in PbR contracts, or indeed in any other type of contract^{[footnote 205]}. This can largely be attributed to the political and social responsibility held by DWP which means contracts should not be allowed to fail without contingency plans and services must be maintained unless low performance or quality demands change or termination^{[footnote 206]}.

Risks for the commissioner can be understood as a matter of trade-offs in which an assessment is made of which factors can or cannot be controlled^{[footnote 207]}.

In the discussions with interviewees about risk, considerations of financial risk were most prominent for all stakeholder groups. Overspending is one such risk; without a mechanism to set a total contract value, exceeding a programme budget may be financially damaging for the commissioner^{[footnote 208]}.

Commissioner interviewee, said:

And the risk is […] that actually it’s too easy or the provider’s too successful and we can’t afford it, they’ve performed so highly we run out of money paying them, and that becomes a problem for the Department and for Treasury who are then overspending what they plan to spend. Arguably you could say “well, if the provider’s continuing to delivery additionality, it’s well worth spending more” and that case can certainly be made, but you still need to find the money from somewhere.

This is distinct to overpaying in which the commissioner bears the risk that they may not have secured the best price for the service^{[footnote 209]} or may be paying for outcomes that would have occurred without the provision^{[footnote 210]}. This would be a scenario in which VFM is not achieved. In another, policy or circumstantial changes could mean the commissioner paying for the duration of a contract, knowing its effectiveness, and therefore its VFM, is now reduced^{[footnote 211]}.

Indeed, underspending is another risk for the commissioner, with both financial and social implications^{[footnote 212]}. With ‘money left on the table’, the commissioner might miss an opportunity to use their full financial capacity for maximum impact on their goal, namely moving jobseekers into employment. In some scenarios, the programme or its replacement could have a future budget reduced on the basis of a gap between actual and planned expenditure^{[footnote 213]}.

DWP has already taken the following actions to assess and limit risk:

using risk profiles to restrict bidders on the basis of financial assets and stability (Financial Viability Risk Assessment)^{[footnote 214]}
utilising insights from modelling and large data sets to forecast risk^{[footnote 215]}
running pilots and other trials^{[footnote 216]}
planning for legal risks or litigation^{[footnote 217]}.

The commissioner also, to some extent, bears risk on behalf of the service users, given its political and social responsibilities. This means the commissioner will aim to avoid the social risk of any reduction in service quality or any direct or indirect harm to service users^{[footnote 218]}. Thus, contracting out, while passing on operational responsibility, is not thought to transfer this risk and the commissioner must, instead, seek to limit the potential for providers to, intentionally^{[footnote 219]} or otherwise^{[footnote 220]}, under-serve jobseekers.

The commissioner also considers how the landscape of the provider market itself may pose risks to the maintenance and quality of service provision. Indeed, disproportionate financial risk transfer to providers could increase the risk of gaming and other negative behaviours as providers try to secure income^{[footnote 221]}.

Reputational risks must also be considered^{[footnote 222]}. These risks include negative public opinion and media coverage^{[footnote 223]}, the morale of DWP staff^{[footnote 224]} and the political scrutiny on public expenditure^{[footnote 225]}. Damage to the commissioner reputation could impact on the success of the programme (and indeed could spill over to other programmes)^{[footnote 226]}. If service users have a negative view of providers or DWP, their engagement and efforts within a provision may suffer^{[footnote 227]}.

Overall, some commissioner and expert interviewees suggested lessons have been learned by DWP on risk assessment and risk management^{[footnote 228]}. Some expert interviewees noted it is not always possible to price risks^{[footnote 229]} while provider and commissioner interviewees emphasised that a mature and experienced market lowers the risk involved in PbR contracts^{[footnote 230]}. Further suggested strategies to minimise or account for risk were as follows:

negotiating more openly with providers on risk to avoid uncertainty driving up price^{[footnote 231]}
carrying out risk profiles on all providers throughout supply chains^{[footnote 232]}
using a Target Cost Incentive Fee^{[footnote 233]} to set pricing but allow for agreed-upon adjustments as costs change^{[footnote 234]}
increasing marketing and communications efforts to manage reputational risk^{[footnote 235]}
strengthening minimum service guarantees and customer service standards to protect the service user^{[footnote 236]}
setting appropriate contingency plans for low-performing providers^{[footnote 237]}.

2.2.2. For providers, risks include high upfront costs, underperformance and reputational concerns

For providers, financial risk is a key consideration in taking on a PbR contract. The government Sourcing Playbook notes providers taking greater risk is usually associated with higher profit margins^{[footnote 238]}.

PbR contracts, particularly those with low upfront payments, may put some, particularly smaller organisations, at risk of cash flow issues, since payments may not be made until some time after costs are incurred^{[footnote 239]}. This risk for providers may be managed by shifting the balance from conditional, outcome-based payments to fixed upfront payments^{[footnote 240]}. However, too great an emphasis on fixed payments may increase the risk of parking^{[footnote 241]} and weaken the advantages of PbR contracts.

In a programme governed by a PbR framework, a key risk for providers is underperformance. Underperformance carries a financial risk since in PbR contracts payment is conditional on achieving a successful job outcome^{[footnote 242]}. Research from Australia has suggested providers in the Job Network invested fewer resources in activities such as training that were with a decline in certain activities or services, for instance training, which were considered riskier investments because they may not result in a job for the job seeker^{[footnote 243]}.

The risk of provider underperformance is shaped to a large extent by economic conditions, namely the number of job seekers and vacancies in the local labour market^{[footnote 244]}^{[footnote 245]}. Economic conditions can change after the terms of the contract are agreed and uncertainty in this regard is a key risk to providers, but also to commissioners, for whom provider under- and over-performance is also undesirable. The risk to both parties may be mitigated by building provisions into contracts to allow for changing circumstances and unforeseen events^{[footnote 246]} such as the possibility of contract renegotiation^{[footnote 247]}.

Participating in PbR contracts can incur some reputational risk for providers. Voluntary and/or community-based organisations may feel that PbR contracts are in tension with their organisational mission to serve certain communities in a broader sense than securing specific job outcomes, and therefore may be less likely to participate in programmes^{[footnote 248]}. Qualitative research in the UK has found that some non-profit organisations were concerned about the reputational risk of participating in the WP given the emphasis on conditionality and mandatory participation for unemployed workers^{[footnote 249]}. One may argue providers can choose contracts that suit them best – as other actors (e.g. local authorities, NHS) also contract out service provision. However, DWP is one of the main players in the area of employment.

Findings from the interviews

It is safe to assume providers understand that any economic incentive, including profit, involves risk^{[footnote 250]}. Viewing financial risk as an inevitability implies a provider should make their own judgements on whether their cashflow or reserves allow them to afford any risk attached to a contract^{[footnote 251]}.

Nonetheless, financial risk will still be the major focus for most providers^{[footnote 252]}. The following concerns were raised by interviewees:

imbalances in risk-sharing between providers and commissioner^{[footnote 253]}
financial risks involved in securing premises and hiring staff which might not be required^{[footnote 254]}
providers’ reliance on DWP for referrals^{[footnote 255]}
impact of shareholders on providers’ risk tolerance^{[footnote 256]}
uncertainty around income due to extrapolation^{[footnote 257]}
the difficulty of innovating when financial risk is high^{[footnote 258]}
possibility of going out of business^{[footnote 259]}
challenge of meeting high targets set by the commissioner to minimise overspend^{[footnote 260]}.

Providers take on risk around delivery and the implications, financial and otherwise, of delivery failure^{[footnote 261]}. This is especially the case in a contract with a payment mechanism closer to ‘pure’ PbR (which is not used, as DWP opts for a hybrid model in which outcome payments are combined with service fees)^{[footnote 262]}. While there are risks associated with mistakes or low performance, providers’ main fears will be around disruption, delays or failure caused by factors beyond their control^{[footnote 263]}. Indeed, providers may bear the risk that the policy and its programme does not deliver the results envisaged^{[footnote 264]}.

These factors could include issues with other actors involved, such as employers^{[footnote 265]} or even competition with other providers for engagement with these employers^{[footnote 266]}, but most often these factors would involve labour market changes^{[footnote 267]}. Some provider interviewees observed that even differences in the expected demographics of the service users can impact on delivery and performance, as a programme would have been planned for users with different needs^{[footnote 268]}.

Interviewees from all stakeholder groups identified risks around varying volumes of referred service users as the most disruptive to financial stability^{[footnote 269]}. This variation has most often been caused by changes in the labour market or inaccurate forecasts. The knock-on effect of changing volumes for providers is outlined below:

Provider interviewee, said:

A significant issue for us providers […] is that we will have assumed that, round numbers, 10,000 people are going to flow through our books, and we’re going to put [a certain proportion] of those into work, and therefore the revenue, or the profit from the contract will be X and Y, and we will have told banks or, in our case, shareholders, or whoever, that’s what’s going to happen, and if the volumes are half that, all those assumptions are out, and therefore all of the cost assumptions are out. I may have taken a lease on an office in London assuming that I would need X, Y and Z square footage.

But some provider interviewees also reported that competing with other parallel programmes was sometimes the cause of low referrals^{[footnote 270]}. Interviewees suggested change clauses for unprecedented events, such as the COVID-19 pandemic, should be included in contracts so renegotiations or a move to Cost Plus model (see section 2.4) can be triggered to reduce risk for providers^{[footnote 271]}.

Additionally, for subcontractors (especially small and medium providers), reliance on primes for cashflow can mean an unreasonably high level of financial risk^{[footnote 272]}. Where large prime providers can rely on their resources to flexibly respond to changes in volume or performance, this is not the case for smaller organisations or even providers with fewer resources and a large supply chain^{[footnote 273]}. However, some providers may be less vulnerable to cashflow reliance – for example, the different streams of income received by local authority subcontractors mitigate financial risk, especially since these subcontractors are not required to generate profit^{[footnote 274]}.

But other risks remain. Reputational risk^{[footnote 275]}, for example, may be particularly important to charity or local authority subcontractors who may wish to differentiate themselves from private providers^{[footnote 276]}.

Provider interviewee, said:

There is reputational risk [because…] as a charity we also have to be really careful of the customer groups that we are targeting to support. We have a broad remit of people at disadvantage, so for most of the people that we’d ever work with within DWP [they are] going to meet that […] benchmark, that’s not a problem. However, we did have discussions about the JETS variation within Work and Health Programme, [thinking] “are they really the hardest to help that we really have set up as a charity [to serve]?

Past instances of contracting failures^{[footnote 277]} in both employment services and health assessments^{[footnote 278]} have made the question of reputational risk particularly important and new entrants to the provider market may be wary of this association^{[footnote 279]}.

Providers will also consider the impact upon their organisation if a policy, programme or the contracting out process receives public critique or negative publicity^{[footnote 280]}. Risks to an organisation’s reputation within the provider market may also be assessed. For instance, providers aspiring to prime contracts will want to make positive impressions on the commissioner^{[footnote 281]} and new entrants will wish to be well-regarded not only by DWP, but possibly other government departments^{[footnote 282]}. It is likely these providers would then avoid contracts which could possibly damage industry reputations by risking low performance or delivery failure^{[footnote 283]}.

Some provider interviewees also mentioned extrapolation as a destabilising practice in which funds are reclaimed by the commissioner after incorrectly paying for an outcome^{[footnote 284]}. This is one example of the type of uncertainty providers might face which creates risk around PbR contracts^{[footnote 285]}.

Interviewees, including commissioners, experts and providers, expressed the opinion that PbR places disproportionate risk onto providers, noting this will also mean increased costs for commissioners as the providers start to price that risk into bids^{[footnote 286]}. The pressure on providers to maintain results may also limit provider flexibility, as financial rigidity makes organisations risk-averse and unable to respond to varying needs or scenarios^{[footnote 287]}. This could result in a negative impact on service users depending on the costs of tailoring services to their location or specific needs.

2.2.3. Contracting out may incur a risk of variable service provision for users/beneficiaries

Contracting out employment services (not specific to PbR contracts), particularly in combination with a flexible (black box) model of service provision, might incur a risk of variable service provision onto service users^{[footnote 288]}. On a quasi-market, inconsistencies may develop not just in the form and content of services but, crucially, in service quality^{[footnote 289]}. The risk of variable service quality may be contained to some degree by introducing minimum service requirements to ensure some consistency in delivery^{[footnote 290]}.

Findings from the interviews

In the words of one expert interviewee:

One of the shortcomings, of course, has been […] not being able to actually adequately service people who are further from the labour market. That is a major shortcoming that’s not been fixed in the UK or Australian model, so therefore it’s a risk to the jobseeker of course.

As noted in section 2.1.3, interviewees across the three stakeholder groups warned of the impact of parking upon service users^{[footnote 291]}. The deliberate avoidance of serving users with greater or more complex needs is considered to be neglectful, leaving users without the support to which they are entitled. The long-term impact of such neglect could include compounding issues keeping users from the labour market, losing the potential for earnings and other benefits of employment. The combination of different aspects of contract design, such as payment mechanism or specification, could increase or decrease the likelihood of users being parked^{[footnote 292]}.

There is also a risk that parking is made more likely by the very notion of some service users being ‘hard to help’. It may be that service users are framed as having complex needs simply because the support they need is costly^{[footnote 293]} or because progress is inevitably slower, due, for example, to language barriers or digital illiteracy^{[footnote 294]}. Furthermore, this phrasing may categorise some service users (e.g. people with disabilities) as a homogenous group^{[footnote 295]}, rather than recognise the diversity of different disabilities and associated support needs^{[footnote 296]}. Creating diverse supply chains with specialist subcontractors may counter-act these risks, dependent on the dynamics within the prime-sub model (see section 2.5)^{[footnote 297]}.

There is also a danger that service users feel their voices are not heard during their time on a programme^{[footnote 298]} and afterwards, during assessment or later policymaking^{[footnote 299]}. While some expert and provider interviewees emphasised the utility of surveys and similar feedback mechanisms^{[footnote 300]}, others had reservations^{[footnote 301]}. The impact of factors beyond a provider’s control, namely government policies, on service user views and levels of satisfaction were thought, by expert and provider interviewees, to affect customer feedback^{[footnote 302]}. Conversely, there were concerns among provider and expert interviewees as to whether there was enough available information to allow public scrutiny of provisions and their treatment of service users^{[footnote 303]}.

There is also a risk of missing the balance between maintaining consistency across a provision^{[footnote 304]} and adapting appropriately to service user needs^{[footnote 305]}. Many expert and provider interviewees suggested a lack of locally grounded and ‘joined-up’ services could result in lower quality provision for some users, as the service is less tailored to their needs and location^{[footnote 306]}. Mitigating these risks could involve greater incorporation of a ‘customer focus’ into contract design, whether through greater choice for users between services^{[footnote 307]} or using customer satisfaction in PbR or other financial incentives.^{[footnote 308]}

Evidence related to contracting our health assessments is scarce (see Box 3).

Box 3. Health assessments: risks in contracting out

The reviewed literature noted some risks in relation to contracting out health assessments. The example mentioned relates to the provider Atos^{[footnote 309]} withdrawing from conducting Work Capability Assessments (WCA) for Employment and Support Allowance (ESA) in the UK due to concerns about reputational risk^{[footnote 310]}. Atos was the sole provider for carrying out work capability assessments in 2008. After DWP identified some quality concerns and following negotiations between DWP and Atos, a settlement was reached for Atos to exit the contract in early 2014 (that is before it was due to end in August 2015^{[footnote 311]}.

Additional insights were gained from the interviews.

Negative effects on service quality from pressure on front-line staff is a particular concern for health assessment provision^{[footnote 312]}. Increasing demand for related benefits and subsequent growing caseloads could overburden staff, risking mistakes or lower quality assessments. Reputational risks for providers were also mentioned, given previous providers terminating contracts early^{[footnote 313]}.

In health assessments, the demand produced by the number of benefit claimants can be unpredictable, especially if policy on specific benefits changes during the contract^{[footnote 314]}.

In health assessments, if decision-making is not based on a high-quality assessment, this can have a negative impact on service user’s livelihood and wellbeing (if the individual’s health restrictions are under-estimated)^{[footnote 315]}^{[footnote 316]} or result in benefit over-payment (if they are over-estimated). Commissioners emphasise consistency across delivery to minimise such occurrences^{[footnote 317]}. However, interviewees suggested that such quality assurance will not protect service users if policies prove to be unhelpful or even harmful^{[footnote 318]}. Other suggestions included emphasising communication around an assessment as part of the service user experience as well as providing feedback to staff on the outcomes and customer view of the assessment^{[footnote 319]}.

2.3. The level of specification in contracted services

2.3.1. Black box provision allows providers greater flexibility and may facilitate innovation, but evidence is scarce

The process of contracting out is determined by specifications issued by the commissioners. This documentation formulates expectations of the purchaser towards the service provider, and it defines, and governs responsibilities and rights among the parties. Such documentation can attempt to regulate every aspect of services to be provided or leave some (or most) aspects in the provider’s discretion (black box). Black box refers to:

A policy model where the service providers or agents are free to define their own actions including how to personalize support for unemployed people, engage them in unique activities or provide them with training and mentoring^{[footnote 320]}.

Under the black box approach, providers are awarded discretion to determine the nature of provision other than the requirement to meet minimum service requirements (see section 2.3.2). Although there had been previous moves towards a less prescriptive approach to contracting out in the UK, the black box approach became more pronounced under the WP^{[footnote 321]}^{[footnote 322]} The flexibility afforded by the black box approach was felt to be suitable for the WP because of the need to cater to all long-term unemployed claimants – some of whom will have distinctive or complex needs – within a single employment scheme^{[footnote 323]}.

The flexibility afforded by black box contracts may drive improvements in performance because of the potential for providers to deliver individualised, tailored support. For instance, for job seekers with more complex needs and barriers to employment such as the long-term unemployed, those with disabilities or health-related barriers to employment and ex-offenders^{[footnote 324]} Black box provision is also thought to provide greater scope for innovation in programme delivery. Summarising the evidence from Australia, Finn (2012)^{[footnote 325]} notes that flexible delivery models (combination with outcome-based payments) enabled providers to tailor services to different participants, as well as to innovate, developing new practices to tackle specific barriers to employment. Under the black box approach, providers have more flexibility to adapt their services in response to the latest evidence on effectiveness and best practice^{[footnote 326]}. Another advantage of the black box approach is that it reduces administrative complexity and running costs for commissioners^{[footnote 327]}.

Under the right conditions, private providers contracted under black box conditions might generate greater innovation in service delivery compared to the PES, who are more likely be constrained by factors such as capacity, resource and institutional constraints^{[footnote 328]}^{[footnote 329]}. Empirical evidence in this area is scarce, perhaps reflecting the fact that innovation is a challenging concept to define and measure (see section 4.5).

An evaluation of the Flexible New Deal in the UK noted that the black box approach is not necessarily associated with greater innovation in service provision^{[footnote 330]}. Stakeholders from the Netherlands interviewed for one research study^{[footnote 331]} expressed concerns that contracting out had reduced innovation and resulted in greater uniformity in service provision. Stakeholders explained this lack of innovation as a result of the drive towards cost saving, benefitting from economies of scale^{[footnote 332]}. Under a PbR system, providers may be cautious about innovating and deviating from ‘tried and tested’ approaches^{[footnote 333]}.

Similarly, qualitative evidence has emerged from the UK in relation to DWP that resulted only in modest (rather than radical) service innovation: providers were reluctant to invest in expensive, specialist support services for clients and could not access (other) funding pots or co-ordinate with parallel employment support initiatives at the local level^{[footnote 334]}.

2.3.2. Flexibility in service provision results in greater service variability and may increase the risk of creaming and parking

Black box provision may create opportunities for providers to pursue cost savings at the expense of programme quality^{[footnote 335]}^{[footnote 336]} unless checks and balances are put in place to prevent this. Black box PbR contracts may increase the risk of provider behaviours such as creaming, parking and cherry-picking^{[footnote 337]}. In Australia, policy has shifted away from black box provision due to concerns about creaming and parking resulting in low levels of service for job seekers who face greater challenges finding employment^{[footnote 338]}. The black box approach in the UK WP was associated with modest changes in the self-reported behaviour of frontline employment services staff^{[footnote 339]}. Compared to before the WP, staff reported greater autonomy in their role, less reliance on standardised assessment tools and greater emphasis on tailoring and client choice. However, this study also found evidence of an increase in creaming and parking (see section 2.1.3) associated with black box provision, such a growth in the proportion of job seekers not participating in any activity and an increase in the proportion of staff saying their agency’s practice was to focus on the most capable job seekers. However, the authors recognise changes between the two time periods may also be due to other factors such as economic conditions.

Black box provision may result in variable service provision, which is concerning from the perspective of equity. Ceolta-Smith et al^{[footnote 340]} argue black box provision under the UK WP resulted in variable provision for job seekers with disabilities and health-related barriers to employment. Whilst policy documents make clear that providers should address health-related barriers and tailor provision to meet the needs of individual customers, including those with disabilities or health conditions, in practice only a minority of primes implemented health-related provision such as Personal Advisors (PAs) with specific health-related expertise, employing healthcare professionals or forming NHS partnerships. The authors note:

Individuals facing similar health-related obstacles to employment can expect to receive very different levels and types of support depending on which Primes’ programme they are assigned to join^{[footnote 341]}.

In addition to variability in service provision, black box contracts make it more difficult to gather information about activities and services delivered as part of a programme, and to classify them (e.g., job search assistance, training, subsidised employment)^{[footnote 342]}.

Findings from the interviews

Views diverged on the functionality of black box commissioning. Some provider interviewees advocated for this model^{[footnote 343]}, while others across the three stakeholder groups suggested a ‘grey box’ was a better solution^{[footnote 344]}. These contrasting views are explained further below.

The lack of restrictions involved in the black box model may allow providers necessary flexibility to tailor services to the local area^{[footnote 345]} or to labour market changes^{[footnote 346]}. Expert and provider interviewees expressed the view that, to a certain extent, black box commissioning passes risk to providers as they alone are accountable for their processes, judged only on outcomes^{[footnote 347]}. Commissioner and provider interviewees argued that the success of supposedly black box programmes has been undermined by intervention and additional specification^{[footnote 348]}. Indeed, in another study the WP was described as “a ‘black box’ with 300 pages of guidance”^{[footnote 349]}. Some providers claimed black box commissioning would work well if the commissioner’s goals were clear, and providers were able to work independently to deliver results^{[footnote 350]}.

More sceptical views of the black box model suggested the lack of restrictions of providers allows for gaming^{[footnote 351]}. One interviewee reported that concerns about gaming were the cause of Australian commissioners moving back towards a ‘grey box’ or higher specification^{[footnote 352]}.

Expert interviewee,said:

In Australia we did have a black box system at the start but it was quickly established that the providers were basically parking and creaming […] and so there was an inquiry into this in Australia and […] so the government of the day’s response was to introduce a whole swathe of requirements, mutual obligation requirements, that were intended to ensure that all citizens receive some level of service.

However, other provider interviewees suggested gaming can be avoided through other aspects of design, such as differentiated payments^{[footnote 353]}, and undesirable behaviours might be less prevalent than popularly believed^{[footnote 354]}.

Provider interviewee, said:

What you then saw, in the next set of programmes, is […] the kind of notion that there was creaming and parking, which I think, […] without ever actually being really properly tested and proven, has been received as being […] folklore [that] there was creaming and parking, [when…] I personally didn’t see examples of that.

Additionally, commissioner and provider interviewees suggested the black box model drives down prices^{[footnote 355]} and brings down the quality of service to the ‘lowest common denominator’^{[footnote 356]}.

Providers are then understood to compete for contracts based on price alone and are incentivised to deliver as cheaply as possible to maximise their profits on payable outcomes. These concerns about gaming and low service quality suggest greater specification might mitigate social and reputational risks for the commissioner^{[footnote 357]}. Indeed, the social risk might be of such great concern that a ‘completely black box’ cannot be used at all, so as to protect service users and ensure equal treatment nationally^{[footnote 358]}.

Providers, however, contend that similarity in services delivered, even under the black box model, can be attributed to existing industry knowledge on ‘what works’ in employment services^{[footnote 359]}. While this does mean that black box commissioning may not result in a variety of methods or even in significant innovation^{[footnote 360]}, perverse behaviours or cheap delivery are not inevitable.

The combination of two elements: payment structure and service requirements (in the PbR contract) is an important factor in a contract’s success. When designing a contract commissioners face trade offs between these two elements^{[footnote 361]}. For example, some interviewees suggested that a combination of low outcome payments with high level of service requirements would undermine benefits associated with PbR, as providers’ efforts would focus on ensuring compliance with detailed specifications, rather than on finding better or more efficient ways of helping people into work^{[footnote 362]}. However, some expert interviewees warned that a payment mechanism weighted heavily towards outcome payments (rather than upfront or service fees) can also limit the benefits gained from a black box model, as a scarcity of cashflow and resources may lower the likelihood of innovation as providers will be reluctant to take risks^{[footnote 363]}.

It may be that different programmes require different levels of specification, depending on their goals and target service users^{[footnote 364]}. For example, some provider interviewees reported that Work Choice was a much more prescriptive contract than most but thought to be among the most successful^{[footnote 365]}.

2.3.3. Minimum service requirements are designed to assure basic service standards

There is a trade-off between enabling flexibility and promoting innovation and the need to monitor programmes and to ensure they are being delivered as intended^{[footnote 366]}. Minimum service requirements or standards specify the type, quantity or quality of services to be delivered as part of contracted out employment programmes. They promote a certain level of quality and consistency in service provision and make it easier to monitor programmes and compare services delivered by different providers^{[footnote 367]}. Minimum service standards are used to guard against behaviours such as creaming, parking and cherry picking^{[footnote 368]}. Minimum requirements may differ across groups of service users, with additional support required for those facing particular challenges or considered further from the labour market. In New Zealand, service users considered harder-to-place must be supported at least once a week, whereas for those deemed to have better employment prospects contact may only be on a monthly basis^{[footnote 369]}.

In some programmes, for instance the WP in the UK, minimum service standards were left to be set by the provider and formed part of the contract with the commissioning body. These minimum service levels were made public so clients were able to judge whether providers delivered what they had promised. The government had the right to withdraw contracts if the minimum service requirements were not met^{[footnote 370]}. There was considerable variation in the commitments given and the level of detail of minimum service standards under the WP^{[footnote 371]}^{[footnote 372]} detracting from their ability to promote consistency. There was also variability in the degree to which minimum service standards were communicated to service users^{[footnote 373]} undermining their effect as an ‘accountability lever’^{[footnote 374]}.

Findings from the interviews

A new perspective on the black box provision was explained as follows.

DWP representative, said:

Although the combination of services delivered is in the control of the provider, that does not mean that commissioners are not allowed to see what this is. Some have suggested that ‘glass box’ might be a better term – you can see inside, but not change it.

Minimum service requirements might be considered the factor which turns a black box into a ‘grey’ box^{[footnote 375]} (not because of lack of transparency for commissioners but for their ability to affect service provision). Some interviewees, across the three stakeholder groups, emphasised the importance of these requirements^{[footnote 376]}. Minimum service requirements might be understood as a necessary guarantee in a national service^{[footnote 377]}. These requirements are a matter of social and legal responsibility in many cases, ensuring that safeguarding and due diligence are carried out appropriately^{[footnote 378]}.

The contents of minimum service requirements have been subject to review and development by the commissioner over time^{[footnote 379]}. It was thought by provider interviewees that the inclusion of these conditions in a contract drives improvement and an adherence to high standards of service^{[footnote 380]}. Expert interviewees, however, believed the current minimum service requirements are too weak and vague^{[footnote 381]}.

Ultimately, those in favour of minimum service requirements emphasised these should focus on the service provided to jobseekers and the compatibility of activities with the labour market and employer landscape^{[footnote 382]}. This could also involve including requirements related to social values and public service as well as job outcomes^{[footnote 383]}.

Other critiques were based on the opposite view: provider interviewees felt the commissioners had moved towards excessive minimum service requirements in contracts^{[footnote 384]}. From this perspective, the requirements may even specify activities that providers are already carrying out^{[footnote 385]} but, most importantly, the rigidity of these conditions places emphasis on meeting requirements rather than job seeker needs^{[footnote 386]}. Provider interviewees suggested the addition of a new requirement should be conditional on the removal of another to act as a balance against over-specification^{[footnote 387]}.

Requirements to meet service users face-to-face at specific milestones were the most objectionable to provider interviewees^{[footnote 388]}. They suggested this requirement lags behind developments in remote communication and specificity on dates could hinder support work with enthusiastic users who needed more flexibility^{[footnote 389]}.

Provider interviewee, said:

There’s just one quite irritating […] requirement stating that every month you must see every client face to face, like physically face to face, and update their action plan. […] they’ve relaxed that requirement when there are Covid restrictions in place – but if you just take Covid out of it – it just means what you’re doing is basically a mechanical every month, you’re seeing people for the sake of seeing people because you have to, not because it’s […anything] to do with the client, where they’re at in their job, in the journey […] and that’s just ridiculous because we are supposed to have a little bit of expertise in this field and that’s just a very blunt instrument.

The effect of such rigidity may also risk creating a ‘low trust’ relationship between the commissioner and providers^{[footnote 390]}. A punitive approach with contracts focused on penalising providers may have a negative effect on service quality, as well as on long-term relations with the provider market^{[footnote 391]}.

Commissioner interviewee, said:

I’m very much of the mindset that you write rules for the people who are looking to break them, not the ones who are compliant, and so I think you have to have a set of minimum standards but in saying that, […] that’s what people will revert to and so you really need to be creating incentives then, I think, to make people strive for better.

Some provider interviewees described minimum service requirements as the cause of excessive focus on process and paperwork, to the detriment of more efficient work on outcomes^{[footnote 392]}. The resources used to meet and record compliance were thought by provider interviewees to limit potential innovation, alongside the obstacles created by the specifications of the requirements themselves^{[footnote 393]}. Scepticism was expressed by expert interviewees about the effectiveness of minimum service requirements which encourage providers to innovate^{[footnote 394]}.

It was also suggested by provider interviewees that subcontractors might suffer disproportionately with the burden of meeting minimum service requirements, particularly if their prime has agreed to requirements which are unrealistic for a small or specialist service, either due to resources or the service user profile^{[footnote 395]}. Indeed, mandated referrals can limit a provider’s ability to meet requirements on contact or other activities as jobseekers may be reluctant or ill-equipped to engage^{[footnote 396]}. With cases such as these in mind, it may be better if minimum service requirements are written as adjustable for different types of jobseeker to allow for tailored services or even innovation^{[footnote 397]}.

If prescription and monitoring of compliance with minimum service standards increases, it may be that any supposed benefits of contracting out are lost, including reduced administrative costs and the utilisation of provider expertise or innovation^{[footnote 398]}. To counter such a result, greater collaboration with providers on minimum service requirements might be considered^{[footnote 399]}. This could involve consulting with providers when drafting these requirements^{[footnote 400]}, or even allowing earlier sight of the contract so providers can negotiate or at least begin necessary organisational planning^{[footnote 401]}.

Evidence related to contracting our health assessments is more limited (see Box 4).

Box 4. Health assessments: level of specifications

The reviewed literature did not consider the specific arrangements for contracting out health assessments. However, some insights were provided by interviews.

The question of specification involves different considerations and can be more straightforward in the health assessment context. While one interviewee reported that black box models have been used less and less over time, they also noted that the earlier use still involved intervention and additional specification from the commissioner eventually^{[footnote 402]}. As mentioned above, the importance of consistency and correct procedures in health assessments will likely limit any benefits gained from the flexibility of a black box model.

2.4. Cost Plus model

In contrast to PbR contracts, Cost Plus contracts reimburse contractors for agreed expenses and allow additional payments to enable them to make a profit. This approach transfers no or minimal risk to suppliers^{[footnote 403]}. The reviewed literature did not discuss this model of contracting out, but interviews provide some useful insights (including on health assessments – see Box 5).

Findings from the interviews

Interviewees across all three stakeholder groups reported that moving contracts onto Cost Plus during the pandemic was greatly appreciated by providers and praised for safeguarding the long-term capacity of the market and the maintenance of service^{[footnote 404]}. Expert interviewees also emphasised that during this period all parties demonstrated good will and commitment to the overall goals as providers continued to invest in high performance without the financial incentive of PbR^{[footnote 405]}.

For instance, it was noted by provider interviewees that, in retrospect, they would have made higher profits if contracts remained on PbR during this time, or at least once the peak of uncertainty had passed^{[footnote 406]}. Despite this, provider interviewees still expressed gratitude for DWP’s rapid response to protect and stabilise the provider market^{[footnote 407]}.

Regarding future uses of Cost Plus, some interviewees, across stakeholder groups, suggested DWP could standardise the use of Cost Plus as an emergency measure, even for scenarios which are more feasible to forecast, such as large economic downturns^{[footnote 408]}.

An alternative strategy could plan for a transfer to Cost Plus in the event of disruptive non-emergency changes. Some interviewees suggested that an introductory or concluding period on contracts which uses Cost Plus could address existing problems with cashflow or reduced investment over the lifetime of a PbR contract^{[footnote 409]}. Placing outcomes for certain client groups on a Cost Plus mechanism while others remain on PbR could also counter parking or assist providers in moving disadvantaged clients or those with complex needs closer to the labour market^{[footnote 410]}.

Reservations around future uses of Cost Plus were also expressed. One provider interviewee would only choose a Cost Plus contract if they had identified risks around failures to receive prompt and full payments from a client^{[footnote 411]}. From a commissioner perspective, Cost Plus could be seen to transfer financial risk back to DWP^{[footnote 412]}. Both commissioner and provider interviewees observed that Cost Plus models do not incentivise efficiency^{[footnote 413]}, although it may be possible to design a contract with additional efficiency or innovation incentives such as bonuses^{[footnote 414]}.

One requirement of Cost Plus models is the operation of open book accounting and cost oversight^{[footnote 415]}. This process can be burdensome to both commissioner and provider, perhaps diverting resources from work on job outcomes^{[footnote 416]}.

Commissioner interviewee, said:

I think it’s extremely difficult to run cost models without having a large amount of cost insight. So, we were only able to really run cost models in the employment space because we’d done the work to establish open book costing management, we’d done the work to understand how money flowed through those contracts, and therefore we were in a position where that insight, aligned to the structures we’d set up, meant there was very little incentive for gaming, and our view would be, very little gaming took place.

Debate was also seen to arise around which costs were counted; this process could cause combative relations not only between commissioners and providers, but also between primes and subcontractors^{[footnote 417]}. These are similar objections to those described above around the administrative burden of tracking compliance with minimum service requirements.

Box 5. Health assessments: Cost Plus model

For health assessments, some contracts had used a Cost or Cost Plus model prior to the pandemic^{[footnote 418]}. It may be that for this type of service, where oversight and commissioner involvement in delivery is preferred, a Cost Plus payment mechanism could be a better choice^{[footnote 419]}. It may be particularly suitable for services for which demand is volatile, as can be the case for Personal Independence Payment (PIP)^{[footnote 420]} claims^{[footnote 421]}. In this case, the stability of income for the provider also establishes certainty and continuity of service and investment in quality^{[footnote 422]}. It was also suggested that the upcoming introduction of new IT systems to health assessments might be paired with a period on Cost Plus as providers adjust and re-calculate their costs and procedures^{[footnote 423]}.

2.5. Prime model

2.5.1. The prime model of contracting builds long-term relationships with large contractors who can take on the risk of a PbR contract

The prime model refers to:

A system where contracts are awarded to ‘large, professional placement agencies at regional level’, which in turn ‘awards sub-contracts to smaller, specialized, local service providers^{[footnote 424]}.

Introduced in relation to the New Deals for the unemployed in 2006, the prime model featured prominently in DWP’s 2008 commissioning strategy:

The commercial opportunities we shall offer will be arranged into larger, longer lasting (subject to performance achievements) contractual packages which we expect will be delivered by top-tier providers leading and managing diverse supply chains^{[footnote 425]}.

The prime model was strengthened under the WP^{[footnote 426]} and subsequent DWP programmes. Under the WP, prime providers were required to demonstrate an annual turnover of at least £20 million as an indication of the ability to finance upfront investment and take on the risk of a PbR contract^{[footnote 427]}. The emergence of the prime model has developed in tandem with greater flexibility given to providers under the black box approach (see section 2.3.1)^{[footnote 428]}. Prime providers may or may not provide direct delivery as well as work with subcontractors^{[footnote 429]}.

The rationale behind the prime approach is to develop long-term relationships based on trust and cooperation^{[footnote 430]} with a small number of large providers who are well placed to take on the financial risk of a PbR contracts^{[footnote 431]}. The prime/sub-prime model was designed to enable DWP and providers to work together on a longer-term basis, taking a more strategic approach and building on previous delivery experience^{[footnote 432]}. Prime providers work with a network of subcontractors, enabling them to draw on the skills and capacities of smaller, specialist, community-based providers who would themselves not be able to take on the financial risk of a PbR contract. Primes may themselves use competition in their supply chains to improve performance^{[footnote 433]}.

Involving fewer contracts, the prime model reduces transaction costs (set up and management costs) for the commissioning body^{[footnote 434]}.

The prime model results in contracts being concentrated in the hands of a few, large, predominantly for-profit providers. The approach makes it more difficult for new organisations to enter the market^{[footnote 435]} and therefore the degree of competitive pressure on providers. Awarding contracts to a smaller number of providers may stifle competition and increase the risks/implications of provider failure^{[footnote 436]}. Awarding contracts to a larger number of providers also has downsides – larger transaction costs and harder to monitor and enforce minimum service standards^{[footnote 437]}. In the UK, the prime model has been associated with a decline in the number of non-profit organisations involved in service provision^{[footnote 438]}. Similar trends towards the concentration of contracts in the hands of fewer large providers have also been observed in the Netherlands, Australia and Denmark^{[footnote 439]}.

The majority of contracts in the UK WP (35 out of 40) went to private sector primes (of the remaining five, two were public sector organisations and three were third sector organisations)^{[footnote 440]}. Similarly, the majority of primes operating as part of the Restart Scheme (7 out of 8) are private sector organisations. Although large, for-profit providers may be well placed to take on financial risk of a PbR contract, there is some evidence to suggest that compared to socially driven or not-for-profit providers, private companies may be more inclined towards ‘gaming’ practices (see section 2.1.3), requiring more robust accountability levers to be built into contract design^{[footnote 441]}.

2.5.2. There is a risk of subcontractors not being able to benefit fully from contracts

The prime/sub-prime model may result in a lack of oversight and control for commissioners^{[footnote 442]}, who have less insight into the content and quality of service delivery, as well as how primes work with subcontractors^{[footnote 443]}^{[footnote 444]}^{[footnote 445]}. There is a risk of primes shifting the risks associated with PbR contracts onto their subcontractors^{[footnote 446]} and questions have been raised about whether subcontractors, particularly smaller voluntary or non-for-profit organisations are able to benefit fully from contracts under the prime model^{[footnote 447]}. There is a risk of subcontractors being used as ‘bid candy’ to secure contracts and then not receiving the expected number of referrals^{[footnote 448]}.

In the UK WP there was a requirement to have subcontractors in the bid, but not to work with them after commissioning^{[footnote 449]}. Summarising the evidence, Carter observers that under DWP smaller subcontractors received a low number of referrals^{[footnote 450]}. Qualitative research with 13 third sector organisations in Scotland acting as subcontractors in DWP found that many had received fewer referrals than expected and some had not managed to offset the costs of preparing their bids^{[footnote 451]}. Some third sector organisations interviewed as part of that study suggested this might be due to primes parking more difficult or complex cases who might otherwise have been referred to specialist organisations such as theirs^{[footnote 452]}. Organisations also reported receiving referrals for different types of clients than they had outlined at the bidding stages, creating additional work for their organisation^{[footnote 453]}.

To counteract supply chain issues, the Merlin Standard^{[footnote 454]} was introduced in 2013, as an assessment and enforcement tool to ensure compliance with DWP’s code of conduct in relation to subcontracting provision^{[footnote 455]}. Under the WP, all providers were required to achieve and maintain the Merlin Standard^{[footnote 456]}. However, there is some evidence that, despite the Merlin Standard, primes in the WP passed the risks associated with a PbR contract onto subcontractors^{[footnote 457]}, as well as, in some cases, starving subcontractors of referrals^{[footnote 458]}^{[footnote 459]}^{[footnote 460]}. The Merlin Standard was withdrawn in April 2022^{[footnote 461]} and replaced with guidance for providers on programmes such as the Work and Health Programme^{[footnote 462]} and Restart^{[footnote 463]}.

Findings from the interviews

In providing contract management services, prime providers reduce costs and increase efficiency for the commissioner^{[footnote 464]}. While some commissioner and provider interviewees valued this service as one lowering cost and administrative burdens for DWP^{[footnote 465]}, others in the same two stakeholder groups questioned this^{[footnote 466]}. The prime-sub model is attractive due to the belief that the resources of larger providers ensure consistency and reliability in service provision^{[footnote 467]}. This also presumes an ability to bear risk on behalf of both subcontractors and the commissioner^{[footnote 468]}.

Commissioner interviewee, said:

So overall […] it’s a lot more manageable, standardised and then [still] linking in more of a local flavour […] and we felt that rather than us having to manage all those individual relationships, if we manage one and allow them [to be] experts of the rest of the market […] that would give us a more cost effective and value for money model.

In the view of commissioner and expert interviewees, the prime model offers flexibility, as multiple subcontractors can be deployed for different aspects of a provision and to meet different service user needs^{[footnote 469]}. From another, as expressed by a mixture of interviewees across the three stakeholder groups, the complexity of supply chains might be seen to cause delays and inflexibility as change takes time to trickle down to all of a prime’s suppliers^{[footnote 470]}. For this reason, some subcontractors would prefer to have direct communication with DWP so as to be better informed of expectations or any upcoming changes^{[footnote 471]}. The large number of separate actors may also mean that stimulating innovation requires more effort and resources for effective coordination^{[footnote 472]}.

In the view of interviewees across all three stakeholder groups, a prime-sub model facilitates variety in the provision of services and allows for small and medium organisations to use their niche or specialist skills as part of supply chains^{[footnote 473]}. In other views, also from interviewees from all three groups, the establishment of the prime-sub model risks losing specialist knowledge among these smaller providers^{[footnote 474]}. Not all of these interviewees believed this to be inevitable, but it was suggested by both provider and commissioner interviewees that primes have not been sufficiently incentivised to prioritise subcontractors with local and specialist knowledge in their supply chain^{[footnote 475]}.

Some interviewees across the commissioners, providers and experts emphasised the importance of due diligence on prime provider’s claims about their supply chain^{[footnote 476]}. It was also suggested DWP could design selection stages to reward bidders with a history of maintaining satisfied members of a supply chain^{[footnote 477]}. While the frequency of prime providers using specialist providers as ‘bid candy’ was contested, concerns were expressed around the risks of sharing information with prime providers at the bidding stage^{[footnote 478]}. Anecdotal accounts described staff of small and medium providers feeling they were misled about their role in a bid so a prime provider could demonstrate their specialist expertise or research without hiring them^{[footnote 479]}.

Overall, the power dynamic between prime and subcontractors appears to vary depending on the relationships built. Some provider interviewees reported that they were content in the supply chain of ‘their’ prime^{[footnote 480]}. Others described unfair treatment or maintained that this was a consistent concern for subcontractors^{[footnote 481]}. The Merlin Standard was not felt to be effective by some expert and provider interviewees^{[footnote 482]}, leaving a gap into which greater DWP intervention would be welcome to offer more systematic protection for subcontractors^{[footnote 483]}.

Expert interviewee, said:

I haven’t seen any of the analysis of supply chains from the Restart providers so far […] I think it was seen as being pretty problematic under some of the former schemes and […] there were real questions as to whether [the] Merlin [Standard] is fit for purpose. I just have not seen enough evidence that it can work, in order to give the Department or other interested parties confidence that a supplier who was being [mistreated] in the supply chain […] really [has] reasonable recourse.

Other problems experienced by subcontractors were not attributed to prime providers but rather to the model itself. While some primes were thought to withhold funding from subcontractors^{[footnote 484]} to protect themselves from financial risk^{[footnote 485]}, others might simply be unable to pass down referrals due to unexpected changes to the volume or demographic of service users^{[footnote 486]}.

However, provider interviewees suggested the prime provider had a responsibility to subsidise specialist work when the prime could afford to^{[footnote 487]}, as well as to protect small subcontractors from retrospective extrapolation which could be crippling^{[footnote 488]}. Additionally, some small and medium providers in supply chains may be nervous about primes who also work as subcontractors in other areas^{[footnote 489]} or who are capable of taking work over from their supply chain if they wish to lower costs^{[footnote 490]}.

Expert interviewees suggested the solidification of a prime-sub model has left DWP beholden to prime providers because continuation of employment services is a priority^{[footnote 491]}. While this may be an exaggerated fear, provider interviewees worried the lack of commissioning on the other tiers, namely those for smaller organisations, forecast a semi-monopoly for the dominant prime providers^{[footnote 492]}.

Interviewees suggested the following alternatives which could provide similar benefits to the prime-sub model:

establishing a DWP case management IT system for all providers to use during contracts (allowing for shared benefits from economies of scale usually gained from large Contract Package Areas or CPAs) – this is in addition to the Provider Referrals and Payments (PraP) system for referrals and claiming payments for outcomes that is already in place: the new system was suggested for case management during the duration of the contract so small providers were not disadvantaged with IT costs, and primes did not have to rely on for economy of scale^{[footnote 493]}
moving to a licensing model in which providers are accredited and then free to move into and out of the provider market at any time, subject to performance standards^{[footnote 494]}.

2.6. Contract Package Areas may be too large

Contract Package Areas (CPAs) relate to the geographical division of service provision in specific contracts. The UK WP had 18 CPAs, each consisting of two or three separate contracts, with a total number of 40 contracts awarded^{[footnote 495]}. Claimants were randomly allocated to one of the primes in a CPA if they had not found work within a specified period. Competition between primes in the Work Programme operated at the CPA level. Minimum Performance Levels were set at the level of the CPA, requiring primes to deliver results at 10% higher than the estimated (regional) non-intervention rate^{[footnote 496]}. Primes who did not achieve the Minimum Performance Level for each CPA could lose their contracts. A second competitive mechanism at the CPA level was the ‘market share shift’, where higher performing primes received a greater proportion of referrals^{[footnote 497]}. While we cannot know the net impact of any one provider, we can know how they are performing relative to their competitors within the CPA. An evaluation found the market share shift was perceived by DWP staff to have a limited impact due to the relatively small proportion of referrals (5%) moved to higher performing providers in the CPA^{[footnote 498]}. Other programmes differ in the design of CPAs. The Work and Health Programme had 6 CPAs and the Restart Scheme has 12.

CPAs are intended to reflect and respond to spatial variation across the UK in the functioning of local labour markets^{[footnote 499]}. Providers bidding for contracts as part of the Work Programme were expected to adapt pricing to reflect the ease of achieving job outcomes within the region^{[footnote 500]}. There was no variation in pricing below the CPA level of geographical division.

The rationale behind introducing large CPAs in the WP was to enable large, private sector organisations to participate in the quasi-market^{[footnote 501]}. Having a smaller number of large CPAs reduces complexity and lowers transaction costs for DWP. However, some have argued that CPAs are too large and fail to capture localised differences in labour market conditions and provider performance^{[footnote 502]}. Carter^{[footnote 503]} finds little variation in the performance of primes operating in the same CPA, masking considerable variation at the level of local authority areas. This results in a lack of transparency and accountability for localised differences in provider performance:

Because the contractual agreements are sited at the level of CPAs (…) local authorities [areas] with unacceptably low job outcome performance are contractually invisible since they are diluted, concealed and deemed irrelevant within the CPA aggregate performance figures^{[footnote 504]}.

Setting competition at the CPA level might encourage providers to focus their efforts on better performing local areas, neglecting areas where the local labour market is more challenging (a regional version of creaming and parking)^{[footnote 505]}.

Questions have been raised about the degree of variability in service provision across CPAs, in light of the black box approach to contract design in the WP (see section 2.3). Focusing on health-related provision, Ceolta-Smith et al^{[footnote 506]}. highlight considerable variation across CPAs and across providers, raising concerns about inequitable service provision in terms of quality and content.

Findings from the interviews

Interviewees, across commissioners, experts and providers, expressed the view that CPAs should be smaller than they are now and have been in the recent past^{[footnote 507]}. An alternative strategy could see larger CPAs for mainstream programmes and smaller for specialist programmes which require greater integration with local economic zones and specialist support services^{[footnote 508]}.

In terms of effect on the provider market, creating fewer and larger CPAs could limit competition and exacerbate the semi-monopoly of prime providers mentioned above^{[footnote 509]}. Previous programmes in the UK and elsewhere have attempted to address this issue by issuing contracts to more than one provider in each CPA, however service user needs and lack of information limited any competitive drivers^{[footnote 510]}. It may be that the concentration of contracts among fewer providers can be remedied by promoting diverse and locally integrated supply chains^{[footnote 511]}.

Commissioner interviewee, said:

A criticism which was made of […] fairly large contract areas is that the prime contractors are insufficiently attuned to local conditions. So, they don’t have the relationships with local organisations that might be important players locally, and there’s no reason why they couldn’t be. So, a prime contractor can work with the range of subcontractors they feel appropriate, and that […] could be somebody’s a specialist, because they deal with a particular type of disability or other need, or they could be a specialist, because they happen to know everything which goes on within a particular town.

Some commissioner and provider interviewees suggested that fewer CPAs were desirable to promote simplicity and consistency across geographies^{[footnote 512]}. However, the drawing of boundaries, and subsequent size of CPAs, also need to consider the four nations’ borders^{[footnote 513]} and the complexity of devolved budgets^{[footnote 514]}. Some provider interviewees even argued that consistency and efficacy in services was hampered by the division of a national provision among different providers while still acknowledging the dangers of monopolies^{[footnote 515]}.

It was also noted that fewer and larger CPAs also result in high value contracts^{[footnote 516]}. But contracts must still be well-funded, avoiding heavy discounting, to avoid resources being spread thinly over a large geographical area^{[footnote 517]}. These high value CPAs will be attractive to bidders and so likely increase competition, driving a healthy market of providers^{[footnote 518]}. However, the maintenance of large-scale, high value contracts excludes small and mid-size providers from bidding, keeping contracts among a small number of prime providers^{[footnote 519]}. But this effect may not be inevitable; consortiums of small providers could combine their local expertise to bid for large CPAs^{[footnote 520]}.

Ultimately, the design of large CPAs depends on the logic of economics of scale. While some provider interviewees believed this was an accurate assessment^{[footnote 521]}, commissioner and expert interviewees disagreed that economies of scale were at play to a significant degree^{[footnote 522]}. Interviewees across the three stakeholder groups agreed that the existence of fewer CPAs lowers administrative commercial costs and management burdens for the commissioner^{[footnote 523]}. However, if a programme is less effective due to the design of geographical divisions, these savings may be undermined.

For example, some interviewees, across the three stakeholder groups, contended that the geographical boundaries of CPAs have been and continue to be arbitrary and not always well aligned with functional economic zones or regional labour markets^{[footnote 524]}. Advocates for smaller CPAs suggested these were better suited to tailoring a programme to local resources and restrictions^{[footnote 525]}. This includes variation between urban and rural areas or between wealthy and less prosperous areas, all contained within large CPAs^{[footnote 526]}.

While large CPAs involve transferring a great deal of administrative and operational responsibility, it is also possible that less risk is held by the commissioner if smaller CPAs are created which lowers the impact of scenarios such as delivery failure^{[footnote 527]}.

Very little evidence was found in relation to health assessments (see Box 6).

Box 6. Health assessments: CPAs

In health assessment contracts lots replace the notion of CPAs. It seems monopolies on delivery have been less of a concern, with contracting out to two suppliers being considered ‘diverse’ for this market^{[footnote 528]}. Additionally, as the devolved Scottish government will be taking over PIP in Scotland, the commissioner’s hands may be tied on some aspects of geographical division^{[footnote 529]}. While a smaller number of suppliers might be more appropriate in health assessments, compared to employment services, there have been issues in health due to the small number of primes^{[footnote 530]}.

2.7. Length of contracts ideally falls between five to seven years

Several advantages of longer contracts have been noted in the literature. Longer contracts give greater scope for providers to invest in services, resources and human capital^{[footnote 531]}, to develop their capacity and expertise^{[footnote 532]} and to invest in innovation^{[footnote 533]}. For providers, longer-term contracts are necessary to recoup their investments under PbR contracts^{[footnote 534]}. Longer contracts may reduce transaction costs for commissioners since the tendering process is less frequent. A longer-term perspective is also important in relation to the provision of employment services, where gains may take some time to emerge, for instance after poor performers have been excluded from the market^{[footnote 535]}. However, there is a balancing act since in other respects shorter contracts offer advantages. Shorter contracts may improve competition/performance because it is easier for new entrants to join the market^{[footnote 536]} and for poor performers to be excluded^{[footnote 537]}. Shorter contracts enable the commissioners to review and adapt contract terms^{[footnote 538]}. However, provisions can be built into longer-term contracts allowing commissioners and providers to renegotiate terms^{[footnote 539]}. There is also greater danger of designing contracts that are not well matched to the realities of the labour market with longer contract durations^{[footnote 540]}.

Findings from the interviews

Interviewees, across all 3 groups of stakeholders, stated that 5 years was the optimal length of a contract for employment services^{[footnote 541]}. There was variation, with some suggesting 3 years^{[footnote 542]} or ranges of 3 to 5 years^{[footnote 543]} or 5 to 7 years^{[footnote 544]}. Others suggested 7 years as ideal, if an extension on 5 years^{[footnote 545]}. Some expert interviewees suggested one-year contracts were optimal for employment services^{[footnote 546]}. Others thought contracts should be shorter than they have been^{[footnote 547]}, although many more believed increasing the typical length of contracts would be beneficial^{[footnote 548]}.

The time taken in start-up and wind down was commonly referenced as the reason for designing longer contracts, as these periods reduce the amount of time in which a contract is producing maximum outcomes^{[footnote 549]}. Furthermore, too short a contract discourages investment from the provider and can make securing premises or staff very challenging^{[footnote 550]}. Additionally, short contracts result in more frequent tender, bidding and negotiation stages, which drives up commercial transaction costs for DWP^{[footnote 551]}.

However, if contracts are too long this will ultimately limit market entry and reduce the benefit of competition by locking the commissioner in to contracts with a few providers for an extended period of time^{[footnote 552]}. This could be problematic if DWP, or indeed providers, are restricted in responding to political and economic change for which the contract or programme does not account^{[footnote 553]}.

Longer contracts also increase the importance of robust mechanisms for monitoring and managing poor performance^{[footnote 554]}. The impact of low service quality or poor performance over a long period of time could have more severe long-term implications than issues which occurred over a one-to-three-year period.

Indeed, programmes of different lengths, which may have been chosen for a specialist type of intervention, will require different types of support from the commissioner^{[footnote 555]}. These types of calculations will likely be part of the assessment of trade-offs when designing a contract and should be considered in combination with the other features discussed above, such as payment mechanisms and CPAs.

Evidence in relation to health assessments was very limited (see Box 7).

Box 7. Health assessments: length of contracts

For health assessments, longer contracts were much more typical, with some interviewees even suggesting ten years as an optimal length, citing a need for stability and to ensure the retention of skilled health professionals carrying out the assessments^{[footnote 556]}.

Question 7

3. 	Market competition

Accepted Answer

Key findings on market competition:

selection criteria often combined quality and price. The examples identified show the weight of the price ranges between 20% and 40%. The quality aspects address issues usually aiming to capture relevant experience and expertise.
there is little evidence of how different types of contracts impact market competition in the short- & long-term. However, there is some evidence on the effect of contracting out on market competition: this evidence points to some providers being ‘pushed out’ or forced to adapt to secure contracts and deliver services. The dominance of prime contractors in the market has now been well established.
trade-offs between optimising short-term and long-term outcomes has been a process of ‘trial and error’. One of the challenges is finding an optimal balance between upfront fees and outcome payments to create incentives for providers and savings for commissioners.

3.1. Selection criteria often combined quality and price

Overall, there was some evidence found on selection criteria and scoring mechanisms used in commissioning employment services within the literature reviewed. Country-specific examples are listed in Table 2.

Table 2. Selection criteria

Country	Criterion (quality or experience)	Weight	Price
Australia	Past performance	30%	No information
Australia	Achieving outcomes	30%	No information
Australia	Meeting employers’ needs	30%	No information
Australia	Governance	10%	No information
Belgium	Description of the implementation methods	50%	20%
Belgium	Expertise: past experience and competence of case workers	20%	20%
Belgium	Location of service provision (accessibility by public transport)	10%	20%
Canada	(Unspecified) qualitative criteria	70%	22%
Canada	Final stage interview	8%	22%
Ireland	Implementation	18%	40%
Ireland	Delivery	30%	40%
Ireland	Contract management and governance	12%	40%
United States	Proposed services and client journeys	30%	No information
United States	Providers experience and strength of partnerships	20%	No information
United States	Other (unspecified) qualitative aspects	50%	No information

Source: Cockx & Baert (2015)^{[footnote 557]}, Langenbucher & Vodopivec (2022).

In Denmark commissioning was decentralised, and regional PES were given freedom to decide the types of service, target groups, contract forms, and payment models, as long as at least 10% of the unemployed users were in services provided by non-public providers^{[footnote 558]}.

Germany included quality (in addition to price) in their decision to select contractors: private providers, and most of their services, had to be approved according to criteria such as capacity and personnel; certification companies themselves also had to be approved by the national accreditation body, Deutsche Akkreditierungsstelle^{[footnote 559]}^{[footnote 560]}.

The Netherlands varied the selection criteria according to the needs of particular groups, placing greater emphasis on the professional competence of contractors, rather than price, for those working with more disadvantaged groups^{[footnote 561]}.

As evident from Table 2, price is a common criterion used. However, it has both advantages and disadvantages. On the one hand, it helps to reduce cost. Proponents of contracting out argue that introducing competition for providers incentivises them to better use resources and thus increases efficiency in service delivery: competition among providers confines the provision of a service to those best suited for it and transfers some cost savings to the buyers^{[footnote 562]}. However, incomplete (or less detailed) contracts give providers scope for cutting costs and quality^{[footnote 563]}. On the other hand, price may have negative impact on quality, as providers remove expensive elements to become more competitive^{[footnote 564]}^{[footnote 565]}. Empirical evidence from the Netherlands helps to illustrate this point. The fall in price and focus on short term results for outcome payments led providers to remove long-term training for the unemployed from their offer^{[footnote 566]}.

By the same token, relevant, proven experience and expertise are frequently used as criteria but while their intent is to ensure service quality, these criteria are far from perfect. They can also inhibit access for new providers; they may be unreliable, as outcomes achieved may depend on a multitude of factors; they are also hard to use to objectively compare between providers with different capacities to generate outcomes^{[footnote 567]}^{[footnote 568]}. Another example from the Netherlands illustrates how the quality requirements can be a barrier to gain entry to the market. In a Dutch tendering process, new providers had much lower chances: the average success rate to secure the contract was 30% but this varied between new entrants and more established providers (with 14% and 43% success rates respectively)^{[footnote 569]}. Barriers to market entry (such as red tape or high costs) need to be low because risks associated with PbR contracts may discourage some providers from the tendering process and limit competition^{[footnote 570]}.

The selection process is different in voucher systems, where jobseekers obtain a voucher which entitles them to receive employment support and it is jobseekers (and not the commissioners) who select their providers^{[footnote 571]}^{[footnote 572]}. In Germany, providers can enter and exit without going through a lengthy tendering process or entering into long-term commitments on set conditions. This creates easier access for small providers (including small- and medium-sized enterprises, SMEs) but poses challenges for brand recognition^{[footnote 573]}. However, users often lack information to make informed choices about selecting the right provider for them. As such, implementing schemes that include a degree of user choice is challenging^{[footnote 574]}. While different forms of quality recognition were identified in the reviewed literature, these were contested:

Star Ratings in Australia^{[footnote 575]} – (see section 4.1) these Star Ratings were designed to aid service users in choosing between providers based on performance and satisfaction matrices^{[footnote 576]}
Certification in Germany – where PES lose direct influence over providers: providers are accountable to (private) certification companies^{[footnote 577]}

There is little evidence to support practitioners to determine what market-shaping activities they should undertake and under which circumstances^{[footnote 578]}.

Findings from the interviews

The major topic of concern regarding the selection process was an excessive emphasis on price in previous rounds of bidding^{[footnote 579]}. Some interviewees did suggest this has improved^{[footnote 580]} but, generally, many interviewees were critical of the short- and long-term impact of damaging practices around discounting^{[footnote 581]}.

Provider interviewee, said:

I think there was a concern, at one point, around the price scoring, around the amount of discount which […] if they hadn’t […] changed that [for Restart], it probably would have meant […] kind of a race to the bottom, so we were really pleased when that was changed.

Commissioner interviewee, said:

It’s just really important that we don’t [buy] this stuff too cheaply, because the people who are going to suffer are the people that we’re trying to support. If the service levels and the quality of the service ends up getting cut in order to achieve that cheaper price, it doesn’t serve anybody well.

Additional insights were offered by representatives from DWP who commented on the price criterion and noted that:

DWP representative, said:

For FAS [Financial Assistance Scheme] we are using the UK Government “Bid Evaluation guidance note^{[footnote 582]}” which advocates using a Price per Quality Point [PQP] metric.

Indeed, PQP is a novel approach and it has not been discussed in the reviewed literature. A PQP is calculated by dividing the bid price by the quality score to give an output price per quality point: the lowest PQP is the most economically advantageous tender^{[footnote 583]}.

The selection criteria are known to seek indicators of strong performance^{[footnote 584]}. However, there were calls for a greater emphasis on assessing track record – even outside this field (i.e. employment support) – and rewarding this in selection^{[footnote 585]}.

Some interviewees across the commissioners, providers and experts alike regretted the exclusion of smaller providers based on reserve or turnover requirements^{[footnote 586]} or the resources needed to produce bids^{[footnote 587]}. In addition, some expressed concerns about the treatment of smaller providers by primes during the bid stage (discussed in section 2.5)^{[footnote 588]}. One interviewee suggested that contracts with heavy weighting to outcome payments only attracts large providers to bids^{[footnote 589]}.

A past tendency for bids to over-promise on performance was noted^{[footnote 590]}. One interviewee suggested that DWP management of performance offers during the Restart bidding phase was good example of course-correction^{[footnote 591]}. This greater intervention by the commissioner used existing experience, knowledge and research to set parameters for the expected pricing and feasible performance levels for Restart bids. This then allowed both for sharing evidence with bidders on a realistic performance offer and limiting the attraction of low-cost bids.

There was some reporting of previous provider involvement with developing the tender, largely in market engagement^{[footnote 592]}. Nonetheless, more collaboration between DWP and providers on the programme, contract and selection question was recommended^{[footnote 593]}. In an example of good practice, some interviewees praised DWP’s previous use of ‘local actors’ to formulate qualitative selection questions which would limit surface-level answers from bidders^{[footnote 594]}. This involved consulting smaller organisations and local, supporting, actors in the employment services industry, while formulating questions that should be asked to prime providers in their bid.

While, overall, provider interviewees reported positive relations with DWP, some noted a few points of dissatisfaction. Some provider interviewees felt there was a gap between the commissioning strategy they discussed with DWP staff (mostly in the policy team) in informal conversations and the strategy that was seen in the tender announcement – in terms of the scope for innovation, minimum service requirements and the weight of price criterion in the selection process (attributed by interviewees to the commercial team)^{[footnote 595]}. Others reported feeling that DWP due diligence on their bid found inaccuracies which they believed could have been resolved or disproven, if dialogue was facilitated to a greater extent in this stage of the process^{[footnote 596]}.

In general, some providers thought the outcomes of selection processes were unpredictable^{[footnote 597]}. This was thought to be a negative characteristic as existing connections or embeddedness in local areas were not thought to be taken into account appropriately, meaning newcomers to an area would win a competition and start from scratch. Some provider interviewees also felt that confident bids with overly optimistic claims were rewarded over those which expressed uncertainty or caution on volumes or performance^{[footnote 598]}. Others felt that local geographical embeddedness not sufficiently rewarded, also resulting in a waste of the provider’s local investment from previous programmes^{[footnote 599]}.

Suggestions for changes to the selection process included:

continuing to collect feedback from bidders to further inform future competitions^{[footnote 600]}
choosing providers through a licensing system to satisfy the need for proven experience and expertise^{[footnote 601]}
shifting to a social partnership model in which private providers, government and relevant third sector organisations voluntarily work together in the design and implementation of commissioning public services^{[footnote 602]}
encouraging non-profits and charities to include margins to provide for re-investment or risk-management in their financial bids even when they are not-for-profit organisations^{[footnote 603]}.

It appears that, despite some critique, the methods of selection model are acceptable to some interviewees, with the caveat that the current model was not thought to promote innovation or facilitate a ‘market disruptor’^{[footnote 604]}.

3.2. There is little evidence of how different types of contracts impact market competition in the short- & long-term

Beyond discussing the selection criteria in contracting out employment services (section 3.1), little empirical evidence was found in the reviewed literature on how the type of contract affects market competition, or even, more broadly, on how the performance could be improved in the short- and long-term^{[footnote 605]}.

Experiences documented in the literature include examples from a few countries but these focus on the effect of contracting out on competition (rather than the effect of a specific type of contracts):

in the UK, DWP sought to animate ongoing competition by using ‘market share shift’ to reallocate a portion of the caseload to better performing providers. In practice, this has been a weak tool^{[footnote 606]}
rapid use of contracting out in the Netherlands led to the emergence of a ‘pluriform market’^{[footnote 607]}, referring to the complexity of an intensified tender mechanism with many purchasers and limited co-ordination between them, with no evidence of increased efficiency nor equity^{[footnote 608]}.
Denmark first saw a rapid expansion and then contraction of contracted out provision due to poorly designed financial reward systems^{[footnote 609]}.

Former state monopoly systems transitioned in some countries to one characterized by the co-existence of the public and private actors: e.g. Germany^{[footnote 610]}, France^{[footnote 611]} and Israel^{[footnote 612]}. While the situation in each country has likely changed over time, snapshots were captured in the reviewed literature that provide some insight into the shape of the market at the time the studies were carried out:

in France, in 2007 to 2008, there were 11 private providers including (i) job agencies supplying temporary workers, (ii) consultancies specialised in the placement of workers after mass layoffs, and (iii) international firms specialised in the placement of foreign jobseekers^{[footnote 613]}
the number of further education providers in Germany in 2002 was estimated between 5,000 and 30,000 providers, but PES still had a strong position and exerted considerable influence over the quality and price on the market^{[footnote 614]}
looking at the commissioners in 2002, Australia and the Netherlands were the 2 extremes: Australia had only one purchaser, while in the Netherlands there were a number of buyers and providers^{[footnote 615]}. There were 44 large for-profit providers operating in Australia and thousands of providers in the Netherlands^{[footnote 616]}.

Some studies focused on the impact of contracting out on specific market players (i.e. not-for-profit organisations). According to an empirical, longitudinal study, the WP in the UK changed the landscape contracting the share of the market held by (smaller) third sector organisations and expanding the large private sector presence in the delivery of employment support leading to a loss of diversity at the market^{[footnote 617]}. However, the third sector organisations examined in the study were able to adapt and find alternative funding. Those which stayed in the market had to work in a similar way to private sector providers. In Germany, some not-for-profit providers were ‘pushed out’ but many others were able to secure contracts, improve performance, develop and deliver services in line with their values and missions^{[footnote 618]}. Some authors point to a risk of declining diversity of providers leading to the homogenisation of employment services^{[footnote 619]}^{[footnote 620]}.

Findings from the interviews

Commissioner, expert and provider interviewees characterised the landscape of the provider market as one of ‘feast and famine’^{[footnote 621]}. This refers to the risk that after a long programme, with substantial funding, there will be a ‘cliff edge’^{[footnote 622]} for providers until the next is commissioned, assuming they are even successful in the next round of bids. For those that are successful, there remains a need to scale up or down unpredictably in response to the programmes commissioned by DWP^{[footnote 623]}. This lack of long-term stability restricts planning and worsens financial risk for providers, sometimes to the extent that smaller contractors cannot afford to participate^{[footnote 624]}.

Expert interviewee, said:

You saw these massive changes in the contracting framework […] like, you had the Flexible New Deal: it was then completely scrapped, and all the supply chains torn up, and you moved onto the Work Programme, and then the Work Programme ends, you do feel, well, “what happens to all of these providers?”. I think I’ve seen surveys where they ask providers, […] “what’s going to happen if you don’t get Work Programme funding, or what’s going to happen when the Work Programme ends?”. And obviously, they always say, we might close, but, then, we don’t know that that’s true, and I think there’s a tendency for everyone to always give that pessimistic view.

Furthermore, a combination of large-scale PbR contracts has firmly established the dominance of prime contractors in the market^{[footnote 625]}. These may be corporate entities who also operate beyond this market, allowing overall revenue and reserves to off-set the risk or fluctuations of PbR and the ‘feast-famine’ effect.

These knock-on effects are also thought to stifle entry to and exit from the market^{[footnote 626]}:

Provider interviewee, said:

I think, potentially, because of the volume of risk and the subject matter expertise that people build up over a period, your market is constrained. There’s a high barrier to entry, for new entrants, and the provisions […] of the contract will, in part, be driving that

Interviewees across all three groups were concerned with the prospect of small providers with specialist skill sets being pushed out or lost^{[footnote 627]}. Even meeting the requirements for putting bids together can be beyond the resources of smaller providers^{[footnote 628]}. Some provider interviewees also questioned the efficiency and fairness of the logistics or timing of the tender and bidding stages^{[footnote 629]}.

Other factors in instability for this market include the risk of variation in volumes and the production of contracts based on discounted prices and unrealistic performance^{[footnote 630]}. This latter phenomenon was described as creating a ‘vicious cycle of poor results’ which has a negative impact for commissioner, clients and providers^{[footnote 631]}.

Alongside well-designed contracts, some commissioner and provider interviewees emphasised the importance of maintaining good relationships between commissioner and providers for the sustainability of the market long-term^{[footnote 632]}. For example, the commissioner needs to communicate clearly with the market about its short- and long-term goals^{[footnote 633]}. Indeed, due to uncertainty around commissioner strategy or behaviour, some providers may have decided to price risk into their bids^{[footnote 634]}. This is a cost that could be avoided.

Suggestions for sustaining a healthy and competitive market included:

running competition within CPAs^{[footnote 635]}
outsourcing Jobcentre Plus^{[footnote 636]}
changing to a licensing model^{[footnote 637]}
continuing to include local authorities among contractors^{[footnote 638]}
creating greater continuity of geographies for selected providers^{[footnote 639]}
building financial insight to understand drivers of market behaviour^{[footnote 640]}.

While some provider interviewees suggested market share shift mechanisms^{[footnote 641]} for the maintenance of competition within the market, commissioners critiqued prior use^{[footnote 642]}.

3.3. Trade-offs between optimising short-term and long-term outcomes has been a process of ‘trial and error’

As noted above, price is an important and delicate element in the market competition. Commissioners need to set the price in such a way as to:

attract providers to participate in tenders, to ensure that payment models create the right incentives to achieve the desired outcomes, as well as to formulate performance expectations to achieve value for money^{[footnote 643]}.

While the ways in which prices are set (and their values) may vary and are usually not made public, there is some information on the balance between upfront fees and outcome payments which is crucial to ensuring that contracting out employment services brings mutual benefits for commissioners and providers^{[footnote 644]}^{[footnote 645]}^{[footnote 646]}. The process of contracting out has been through ‘trial and error’ in most countries^{[footnote 647]} and continuous learning and adjustment constitute the path forward.

The most recent review and analysis of PbR in contracting out employment services^{[footnote 648]} identifies a range of practices from different countries on setting the outcome payments and their overall weighting in the contracts (Table 3).

Table 3. Structure of outcomes payments and weights of outcome payments as % of the total

Country	Outcome payments	Weight of outcome payments*
Australia (Jobactive – Stream B)	Payable after 4 weeks, 12 weeks, and 26 weeks	89%
Canada (Employment Services Transformation)	Payable at months 1 and 12	40 to 60%
France (Private Placement Operators scheme)	Payable upon entering employment (35% of maximum contract value), and at month 6 if client remains employed (further 35%)	70%
Ireland (JobPath)	Payable 13, 26, 39, and 52 weeks	65% or more
Italy (Reintegration vouchers)	Payments vary according to type of employment contract and client characteristics (variation between permanent contracts, fixed-term over 6 months, and fixed-term between 3-6 months)	Over 90%
Korea (National Employment Support Programme)	Outcome payments vary depending on: – – Employment competency of individual – – Income in the job – – Speed of finding employment	58 to 73%
New Zealand (no name)	Payment triggered after 0, 3, 6, 9, and 12 months of employment	73 to 78%
Sweden (KROM)	At entry of employment or education/training programme	70 to 80%
Sweden (STOM)	Employment outcome payment at 4 months (minimum) – – Education and training outcomes at 20 weeks (minimum)	50 to 65%
United Kingdom (WP)	Payments after 13 weeks or 26 weeks, and then for every additional 4 weeks in employment (each dependent on payment group)	75 to 100%†
United Kingdom (Restart)	Payment after individual has earned 6 months of cumulated earnings at 16hrs times National Living Wage within 18 months of starting the programme	70%
United States (CareerCompass and CareerAdvance)	Payment triggered based on sustained employed after 1, 6, and 12 months	30 to 40%
United States (Wisconsin Works)	Payment at 31 days and 91 days, alongside payment (respectively) for job placements of individuals who have received 2 years minimum of social assistance benefits and placement into high-wage jobs Approx.	60%

Note: * The remainder of the payment is formed by the upfront service fees. † The delivery fee declined over the course of the contract to 0. Over the whole seven years it was effectively under 80%

Source: Langenbucher & Vodopivec (2022), DWP

There have been calls for a better understanding that sustainable outcomes require long-term investment and long-term monitoring (which also requires stable contracting out frameworks)^{[footnote 649]}^{[footnote 650]}^{[footnote 651]}. As indicated in Table 3, several countries do incentivise sustainability of employment in their contracted out programmes.

Carter proposes a more nuanced approach to payment groups and PbR, in which modelled likelihoods of moving closer to work can be measured as part of provider performance and outcomes^{[footnote 652]}. Indeed, a similar approach has been used in some of the countries and programmes examined by the OECD (e.g. Canada)^{[footnote 653]}. Rather than provider performance being measured solely in terms of the extent to which the final desired (employment) outcomes are achieved, it is also possible to measure achievement of intermediate outcomes which are known to represent movement towards work; and to add rigour to this by explicitly modelling the extent to which those intermediate outcomes increase the likelihood of finding work.

Findings from the interviews

Interviewees across all three categories emphasised that lower costs should not be equated with VFM^{[footnote 654]}. Some commissioners and providers believed a balance of VFM and provider goodwill or capacity has been achieved so far^{[footnote 655]}. Some of them even reported how this has been demonstrated in past impact assessments^{[footnote 656]}.

Others reported outstanding issues in the market which were detrimental to sustaining positive short-term and long-term outcomes for all parties. Firstly, according to provider interviewees, the management of risk around volumes may disproportionately protect the commissioner^{[footnote 657]}. Additionally, some commissioner and expert interviewees mentioned the co-existing risks and potential associated with re-negotiation and change schedules^{[footnote 658]}.

Both commissioner and provider interviewees suggested the approach to contracting out should direct fewer punitive measures towards providers and, instead, facilitate greater incentives for desired behaviours^{[footnote 659]}.

Commissioner interviewee, said:

There’s a perception we don’t want government to be seen to be giving bonuses, while I do think personally there should be more of an incentive structure […] If they deliver, why aren’t we paying them bonuses, you know what I mean? […] the opposite of […] you deliver late, and you get a late charge […], we don’t do the early side of it, and I think that’s because we’ve got limited funding […and we don’t] want to be seen to be paying all this additional money out from a public perception point of view.

DWP representative, said:

If the system penalises those who fall short of standards/targets and does not reward those who exceed them, that providers will just build this into their pricing

As mentioned above, providers’ dissatisfaction with the timing of contracts and the ‘feast-famine’ effect^{[footnote 660]} might be seen as a further hindrance to sustained quality delivery. Additional areas of contention include the following:

the role of commercial versus policy priorities^{[footnote 661]}
opinions on whether the market has matured and stabilised^{[footnote 662]}
the belief that VFM has previously been achieved (New Deal) but since lost^{[footnote 663]}
the question of whether market freedom is preferable to strong stewardship^{[footnote 664]}.

Despite these observations, many reported good relations between DWP and their providers^{[footnote 665]}. It was felt that DWP has learned lessons while contracting out^{[footnote 666]}. For example, one commissioner interview stated DWP understood that previous strategies had driven providers to offer unrealistic bids and that DWP had realised that course-correction during a programme is sometimes necessary^{[footnote 667]}. Expert interviewees also observed that they have seen DWP experimenting over the years with a commendable intention to learn and improve^{[footnote 668]}.

Expert interviewee, said:

I think it’s important that we find space to acknowledge where there has been learning from experimentation and this kind of incremental improvement […] and I definitely think you can see some of that learning coming through in Restart in particular.

Provider interviewees suggested DWP management of entry into the market and costing per service user is now ‘broadly’^{[footnote 669]} or ‘about’ right, improving on past approaches^{[footnote 670]}. It was thought that DWP has recognised previous issues and worked hard, in particular, on the Commercial Agreement for Employment and Health Related Services (CAEHRS) framework^{[footnote 671]}.

Of course, some critiques of the approach remain^{[footnote 672]}; namely, the need for DWP to use the knowledge generated so far to better define their overall objectives, plan these with long-term goals and then map their desired outcomes to these^{[footnote 673]}.

Question 8

4. 	Live running

Accepted Answer

Key findings on management of contracts during life running

while contract management is critical for ensuring the quality of service provision, there is no agreed definition of service ‘quality’ or how to measure it. Performance measurement and monitoring aim to encourage innovation, improve performance and limit gaming
incentives to encourage collaboration or information sharing among providers are limited. There is also little evidence on sharing contract resources amongst providers and other stakeholders
expertise and specialist knowledge of providers’ staff is key and need for professional development of staff was noted
innovation in employment services may mean a new client group or new skills training for jobseeker or new ways to provide existing training. It may include making organisational processes more efficient. Competition intends to encourage innovation and this needs to be considered early on. However, evidence for innovation taking place in PbR contracts is limited.

4.1. Measuring service quality

4.1.1. There is no agreed definition of service ‘quality’ or how to measure it

Managing contracts, and monitoring the quality of services during the live running (or contract delivery), is important as this forms one of the ways commissioners can mitigate the risk of undesired behaviours (such as creaming or parking) among the providers^{[footnote 674]}.

There is no universally applicable definition of quality in employment services^{[footnote 675]}^{[footnote 676]}^{[footnote 677]}^{[footnote 678]}^{[footnote 679]}. What it means to provide ‘high quality’ services will depend on the aims and objectives of the policy, for instance, whether the service is geared towards enabling sustained and/or higher paying job outcomes and/or job outcomes for specific groups of job seekers.

Another difficulty is setting targets for the said outcomes, as these need to anticipate the situation on the labour market at least 6 months or even for the length of the contract i.e. a couple of years ahead (if the economy is doing better than expected, the targets will have been set too; conversely, if the economy is not doing well, then the target will have been set too high). As such, the actual performance is more difficult to interpret, in that providers who exceed their target may in fact achieve little impact (and have high deadweight); by the same token, providers who fail to achieve their targets may actually achieve net impact. This is important as their performance affects the cost effectiveness of the overall programme, even if this is difficult to measure.

Since setting a good job outcomes target is difficult, it is important to define other measures of quality. Contracted out services will be expected to also include specifications for minimum service standards, which represent a baseline level of quality that all providers should achieve (see section 2.3.3). Service quality may also be understood more broadly than the terms of a PbR contract, in terms of wider factors such as customer experience and satisfaction or desirable outcomes such as innovation that do not tend to feature as criteria for outcome-based payments.

Findings from the interviews

Numerous and varied definitions of service quality emerged from the interviews. Some interviewees thought outcomes were the only clear demonstration of service quality^{[footnote 680]}. Others distinguished outcomes as ‘performance’, with ‘quality’ referring to the customer experience^{[footnote 681]}.

Provider interviewee, said:

I suppose the performance is about, […] just get people into jobs […] but the quality would be the customer experience so from when they’re coming into the centre, how the centre looks, how they’re looked after, is it an environment that’s going to give them confidence? [Are they]] going to get the support that they need? Are they getting a good quality experience through whether it’s CVs, application, job searching etc.? […] and how they’re treated, because you can be very focused on performance but not give that quality of delivery to the customer.

Others tied service quality to the processes used to reach positive outcomes^{[footnote 682]}. Combinations of the factors listed above were also used when defining quality. These included: outcomes plus (customer) service^{[footnote 683]}, outcomes plus processes ^{[footnote 684]} and outcomes plus ethics^{[footnote 685]}. The latter refers to balancing outcomes against basic ethical and moral safeguarding of the service users’ best interests and wellbeing in any assessment of quality.

For some, ‘fidelity’, referring to the alignment of delivery with bid promises and contractual requirements, was the true indicator of quality^{[footnote 686]}. The use of quality assurance certification^{[footnote 687]} and the assessment of a service’s social value^{[footnote 688]} were also suggested.

Concerns were raised about over-specification or specification of misguided proxies for quality^{[footnote 689]}. This was suggested to mistake standardised processes with quality service delivery, to the detriment of providers’ capacity to maximise value for each client.

However, commissioner, expert and provider interviewees still asserted that aspects of quality assurance or control should be specified in contracts^{[footnote 690]}. Others also emphasised that the selection process could better seek out provider suggestions to improve quality or even use these proposals as the basis of live running quality measurement during contract delivery^{[footnote 691]}.

Regulations and professional standards were seen to be underutilised, where they exist^{[footnote 692]}. Some interviewees suggested the introduction of ‘kite-marks’^{[footnote 693]} while others proposed that an industry equivalent to Ofsted is established^{[footnote 694]} or a similar model to the Star Ratings, used in Australia, or other metrics which could even be visible to service users are adopted^{[footnote 695]}.

At the time of writing this report there was little publicly available evidence on measuring service delivery in relation to health assessments (see Box 8). In late summer 2022 some information became available on the UK Parliament website through written questions, answers and statements^{[footnote 696]}.

Box 8. Health assessments: service quality

The quality concerns surrounding the contract with Atos (see Box 3) were raised and the need for improvements was identified following ‘a series of reviews’^{[footnote 697]}. According to DWP, lessons have been learnt: the negotiated agreement that covered the remaining term was ‘more robust, with an agreed performance regime that gives [DWP] confidence delivery goals will be achieved. It is that same commercial rigour that will underpin the new procurement for these services’^{[footnote 698]}.

According to DWP, provider performance on Personal Independence Payment (PIP) assessments is measured against quality, performance delivery targets and customer experience^{[footnote 699]}. The monthly performance measures against contractual targets in February 2022 show among others:

quality of assessments derived from the audit of all assessment reports at 94.4% (where the target was defined as 95% or more reports in each month deemed as acceptable or acceptable without significant learning points)
claimant Satisfaction rate at 97.8% (92% is the annual target)^{[footnote 700]}.

Additional evidence in relation to health assessments was gained from the interviews. Clinical governance standards are a central part of the assessment process, making quality at once simpler and more complex^{[footnote 701]}. Compliance with specified processes is of great importance during these assessments, while outcomes are not likely to be subject to targets^{[footnote 702]}. Caseloads^{[footnote 703]} and number of complaints^{[footnote 704]} are used as indicators in quality assessment. Observations of service delivery have been carried out in some health assessment contracts^{[footnote 705]}.

Challenges around representing user voices were also noted given that the decision, which is not taken by the provider, will likely dominate a user’s impression of the service^{[footnote 706]}.

4.1.2. Performance measurement and monitoring aims to encourage innovation and limit gaming

Contracting out and the introduction of competition requires strong monitoring and measurement of quality and overall performance^{[footnote 707]}. As noted by Langenbucher & Vodopivec, performance management in PbR contracts incorporates two aspects: monitoring quality of services delivered to clients and examining providers’ performance against (outcome) targets^{[footnote 708]}. These aspects are usually set out in contract specifications through minimum service requirements (which regulate processes to reduce possibility for gaming) and performance targets (that incentivise innovation). Naturally, there is a tension between these two and balancing these is a challenge for commissioners^{[footnote 709]}.

According to the OECD, performance measurement targets in PbR employment contracts include supporting clients (back) into work, reducing the time clients are without work, reducing the risk of long-term or recurring unemployment and securing client satisfaction from services received^{[footnote 710]}. These targets can be measured through:

monitoring providers (e.g. inspections, site visits) including accreditation, certification or licencing systems
collating client feedback (e.g. conducting surveys or monitoring client complaints)
measuring quantifiable achievements (e.g. referrals versus placements, sustainable employment outcomes, etc.)^{[footnote 711]}.

A number of existing practices illustrate the diversity in approaches taken by commissioners in addressing this:

Austria is monitoring contract performance using a Balanced Scorecard which includes levels of customer and staff satisfaction^{[footnote 712]}.
From 2022, Australia is moving to a new licensing system but it is not clear if the Star Ratings model, which was widely discussed in the reviewed literature^{[footnote 713]}, continue^{[footnote 714]}. The plans for the new system require providers to apply for a licence, which after (successful) assessment according to the Provider Performance Framework would be granted to selected providers for three years. This licence would be extended for high performing providers after an annual review, while licences for low performers would be revoked and made available to others – as the number of licences would be capped. The application process can be seen in example below^{[footnote 715]}. Examples of performance measures in the Provider Performance Framework include:
- achieving employment outcomes for job seekers
- progressing job seekers towards employment (e.g. successfully completing designated milestones, such as training, to increase employability skills)
- quality of service given to job seekers and employers
- job seeker and employer satisfaction
- achieving results for disadvantaged cohorts^{[footnote 716]}.

Example of an organisation interacting with the Australian licensing system

Step 1: Request for proposal

Provider X applies to deliver services in 4 Employment Regions:

Adelaide North
Ballarat
Gold Coast
Sydney East Metro

Step 2: Panel membership

Provider X is assessed as scheduled in 3 Employment Regions, and is offered membership on the national panel. Provder X accepts and signs the Deed of Standing Offer. They join the sub-panels for the 3 regions they are found suitable for (along with several other organisations):

Adelaide North
Ballarat
Gold Coast

Step 3: Issuing of licences

Provider X is offered a licence in 2 Employment Regions (along with several other panel members)

Adelaide North
Ballarat

Step 4: Future opportunities

In the future, Provider X may:

be offered a licence in Gold Coast, due to already being on the sub-panel.
have the opportunity as part of a panel refresh to join the sub-panel in Sydney East Metro, or other regions

Source: Department of Education Skills and Employment (DESE, n.d.). Proposed licensing system for the New Employment Services Model

Findings from the interviews

The potential for user feedback to better inform the monitoring of performance and service quality was emphasised across all three groups of interviewees^{[footnote 717]}.

Service user voices were described as ‘chronically underused’^{[footnote 718]}, with one interviewee reporting that ‘review systems’ have been seen to facilitate transparency and dialogue between users and providers^{[footnote 719]}. Another speculated that service users with disadvantages or complex needs might provide more useful feedback on quality and performance, given the standardisation of provision for ‘mainstream’ users^{[footnote 720]}.

It may be that overarching or political contexts also impact on service user feedback in the employment services setting. For instance, changes in the profile of service users during Restart or the introduction of stricter sanction regimes over time could shift the focus of user feedback from the provision to other aspects of their experience^{[footnote 721]}.

Observations of service delivery have been carried out in some contexts^{[footnote 722]}. These might be a complementary tool to user feedback for the purposes of establishing multiple monitoring perspectives.

There is evident frustration with quality monitoring that focuses exclusively on compliance. These practices were characterised as ‘tick-box’ activities or ‘paperwork’ which distract from seemingly more meaningful work^{[footnote 723]}. Examples of such activities include insistence on ‘wet signatures’^{[footnote 724]} or set numbers of in-person meetings^{[footnote 725]}.

However, some interviewees held the view that the minimum service requirements used so far have been neither sufficient nor accompanied by an appropriate level of monitoring^{[footnote 726]}. In comparison, commissioners in Australia are thought to have sophisticated methods for logging each of a provider’s interactions with a job seeker^{[footnote 727]}. This large data set can then be combined with providers’ qualitative reports on quality and progress to assess performance^{[footnote 728]}.

Accreditation and professional standards bodies could also play a role in monitoring quality and driving improvement:

Provider interviewee, said:

Making sure that every […] provider’s advice and guidance is matrix accredited would be a good start in showing that people are professionally qualified. Or [if] there was professional systems in place, or ensuring people are all members of ERSA^{[footnote 729]} or a trade body or something like that where there was good practice sharing […] I know not all prime contractors are a member of [ERSA], but it just seems that there’s opportunities for […] not self-policing of quality, but […] the dispersal and spreading of quality. Then also […making] the measurements […] more objective, because there is no Ofsted for employment services.

Accountability measures are evidently an important aspect of monitoring quality. Determining which measures should be used is less clear. One possibility involves withholding payment for low quality or poor performance^{[footnote 730]}. However, this leans hard on the punitive lever, without considering incentives or the role of clear and shared definitions of quality in facilitating improvement.

Beyond specifications and monitoring, it may be that sustained investment (mostly in longer contracts) is an effective tool in improving focus on quality and facilitating high performance^{[footnote 731]}.

Providers operating in a quasi-market may experience both incentives and disincentives to collaborate and share information with other providers. Networking and relationship building with other providers helps to build trust and foster collaboration between providers^{[footnote 732]}. This is particularly important where customers with complex needs interact with multiple providers who need to coordinate with each other^{[footnote 733]}^{[footnote 734]}^{[footnote 735]}^{[footnote 736]}. Collaboration across providers has the potential to improve the efficiency and effectiveness of employment services if providers are able to learn from one another by sharing examples of good practice and innovation^{[footnote 737]}^{[footnote 738]}. At the same time, there may be a reluctance to share information and to collaborate with organisations with which one competes for resources (for instance, if the number of referrals is linked to performance) or when bidding for future contracts^{[footnote 739]}. This reluctance or difficulty is further exacerbated by a lack of transparency, commercial sensitivities and contract confidentiality^{[footnote 740]}.

Collaborating (or even merging) with other providers may make the market more stable or predictable and weaken competition^{[footnote 741]}, although from the commissioner’s perspective, competition is crucial to the effective functioning of quasi-markets^{[footnote 742]}^{[footnote 743]}. What appears from one perspective to be collaboration designed to improve efficiency for all providers might appear to others as collusion, countering the competitive market forces are fundamental to driving quality under a PbR framework^{[footnote 744]}^{[footnote 745]}.

Commissioners may take steps to incentivise or facilitate collaboration between providers whilst maintaining healthy competition in a quasi-market. This may occur at the bidding stage, for instance by adapting bid and tender requirements to include SMEs or partnerships, or to establish relationships with other (public service) stakeholders like the NHS^{[footnote 746]}. Commissioners may also remove barriers to information-sharing to incentivise collaboration, for example, through improving systems and efficiency for sharing and changing current legal/commercial restrictions^{[footnote 747]}^{[footnote 748]}.

Findings from the interviews

Some commissioner, expert and provider interviewees reported they had seen no evidence of collaboration^{[footnote 749]}. These and others also suggested incentives for collaboration were non-existent or insufficient^{[footnote 750]}.

It may be that the narrow focus on claimable outcomes promoted by PbR discourages collaboration^{[footnote 751]} or, indeed, that the competition of the quasi-market itself means collaboration is not a commercially viable practice^{[footnote 752]}. Some expert interviewees spoke of instances of staff poaching as an example of how collaboration is disincentivised^{[footnote 753]}.

Other disincentives include barriers to data-sharing, whether due to the requirements of contracts or concerns around intellectual property^{[footnote 754]}. Indeed, for some health assessment contracts, exclusivity or the absence of other providers made collaboration impossible^{[footnote 755]}.

Nevertheless, there are examples of limited collaboration between providers^{[footnote 756]}. These largely take the form of roundtables and forums for sharing success stories or good practice^{[footnote 757]}. Other, more practical, instances include sharing points of contact with employers for efficiency^{[footnote 758]}.

Interviewees across all three stakeholder groups emphasised that collaboration is the product of sustained and mutual relationship building^{[footnote 759]}. This may be informal and could be the result of commercial staff moving between providers and building networks and goodwill.

The following incentives to promote collaboration were suggested:

introducing contractual obligations around collaboration, either requiring primes to share best practice with their own supply chain^{[footnote 760]} or upgrading existing ‘exhortation’ and ‘encouragement’^{[footnote 761]} for prime-to-prime collaboration to compulsory requirements which would be monitored along with other standards^{[footnote 762]}
launching a collective target or collaboration bonus^{[footnote 763]}
facilitating shared problem-solving (e.g., for a case management system)^{[footnote 764]}.

While the commissioner’s ‘encouragement’ to collaborate was noted^{[footnote 765]}, collaboration is thought to require resources and perhaps greater facilitation from DWP^{[footnote 766]}.

Commissioner interviewee, said:

People can share information and this sort of thing and therefore we can pay for […providers] sharing bits and pieces. So, I am aware of those items that are in the contract […] they’re in the set up – are they used effectively in life is a different question and I’m not sure they are currently.

Expert interviewee, said:

I think […] you also have to create the space for providers to be able to share with each other. They are a fairly – below the kind of very high-level company to company, […] they are a fairly collaborative bunch, but in the old days there used to be a lot more networking, a lot more events, a lot more activity, and a lot of that has got squeezed out because the contracts have been much tighter, there’s been less funding in the system and because the [DWP] doesn’t support it.

The ReAct partnership was named as a good example of a collaboration scheme which brings together providers delivering the Restart programme, led and funded by six of the prime providers for the purpose of sharing evidence and good practice^{[footnote 767]}. However, while the potential of the ReAct partnership was noted, it was believed a lack of financial resources was the cause of two providers opting out of this instance of collaboration^{[footnote 768]}.

In addition to sharing information and examples of good practice with other delivery organisations, providers may share contract resources, either with providers operating within the same programme or a wider set of stakeholders. Resource sharing (as opposed to information sharing) was mentioned infrequently in the literature, making it difficult to draw conclusions about the extent to which this occurs and the circumstances in which it may be facilitated. Resource-sharing is referenced more frequently in the literature in relation to the PES compared to contracted provision^{[footnote 769]}.

Concerns around the commercial sensitivity of contract resources may deter providers from sharing these with other organisations. Sharing contract resources amongst providers and other stakeholders is heavily dependent on trust and likely driven by the desire to help service users^{[footnote 770]}. For instance, Innovation Fund projects in the UK have been described as cooperative efforts between parties to achieve shared social goals based on relationships of trust^{[footnote 771]}.

Findings from the interviews

Interviewees across all three stakeholder groups suggested deliberate DWP facilitation is required to enable any sharing of contract resources beyond their remit^{[footnote 772]}. This may be due to legal restrictions or the logistics of liaising with other public services with whom DWP should already have contacts. In fact, interviewees, from commissioner, expert and provider groups, reported they had never seen this type of resource sharing^{[footnote 773]}. There seems to be a distinct lack of incentives for collaboration of this kind^{[footnote 774]}.

A few examples of resource sharing amongst providers and other stakeholders involved ‘joined-up’ service provision which aimed to prioritise holistic support for service users. Interviewees reported on examples of this type of collaboration occurring within the cities of Manchester^{[footnote 775]} and Sheffield^{[footnote 776]}. The Talent Match programme was also named as an instance of ‘joined-up’ operation being facilitated^{[footnote 777]}. Other instances include the development of employer networks, mentioned above, which could be shared by providers and other similar initiatives^{[footnote 778]}.

Despite some commissioner and provider interviewees indicating a lack of evidence for this type of collaboration, they claimed they would expect providers to initiate this behaviour^{[footnote 779]}. The commissioner interviewees suggested plans for sharing resources with other stakeholders, especially public services, should be requested as part of providers’ bids^{[footnote 780]}.

Another factor hindering this type of resource-sharing may be a scarcity of resources to share:

Expert interviewee, said:

Given the relatively low level of funding that the providers had, you know their financial incentive was to implement relatively low-cost measures or to persuade other organisations to fund an intervention where they could. Of course, then that impacted on other organisation’s budgets. If you try and do things on the cheap, you can’t then really expect too much.

This would come under a broader problem of provider capability in which the investment in outreach and collaboration required exceeds the current financial model and skills basis^{[footnote 781]}. Competing for funds or risking confusion as to the ‘ownership’ of outcomes are further disincentives linked to this issue^{[footnote 782]}.

There was very little evidence on sharing information and collaboration amongst providers in relation to health assessments (see Box 9). This can be partly explained by the fact that the number of providers offering these services is much smaller, compared to employment support^{[footnote 783]}.

In the health assessment context, interviewees believed resource sharing could and did occur but had to be requested^{[footnote 784]}. However, given the sensitivities and different expectations of the health assessment tasks, this was considered appropriate and understandable^{[footnote 785]}.

4.4. Expertise and specialist knowledge of providers’ staff is key and need for professional development of staff was noted

Outsourcing employment services can enable access to specific skills, knowledge and expertise held by providers, potentially reducing costs and improving service quality^{[footnote 786]}^{[footnote 787]}. Compared to the PES, providers may have more in-depth knowledge of how to support specific groups of service users, particularly those with complex needs (also discussed in relation to the prime/sub model in section 2.5), and/or a better understanding of local labour markets^{[footnote 788]}. This specialist expertise may enable providers to deliver more efficient or higher quality services compared to the PES^{[footnote 789]}. Providers – particularly private sector organisations – may be better placed than the PES to investment into new and innovative technologies^{[footnote 790]}, which could lead to gains in efficiency and/or effectiveness. Private sector providers may also have strengths in managerial efficiency and entrepreneurism, which have benefits for the quality of employment services^{[footnote 791]}.

A lack of capacity and knowledge can impede on the delivery of services. Rees et al. (2014) describe an example of how job advisors in one prime provider lacked the knowledge of where to refer individuals with complex needs (for specialist support) effectively delaying needed support (and parking such clients)^{[footnote 792]}. There is also a possibility that where providers do have relevant knowledge and expertise, these may be used to cream more effectively in PbR models^{[footnote 793]}. Freedom to use own skills and capabilities encourages flexibility, but risks providers working in their own best interests^{[footnote 794]}.

Findings from the interviews

Some interviewees across all three stakeholder groups emphasised the benefits of provider agility and efficiency and, in the case of prime contractors, a unique capacity to adapt and react in operational management^{[footnote 795]}.

In regard to expertise in the delivery of employment (support) services, some expert and provider interviewees reported there was no ‘secret knowledge’ on ‘what works’ being withheld by providers from the commissioner^{[footnote 796]}. With most methods of engagement with jobseekers being established and shared across providers, this view again suggests that operational expertise is the true ‘added value’ brought by providers to the delivery of this service^{[footnote 797]}. Provider interviewees expressed pride in their support and management of specialist subcontractors as a crucial facilitation of connecting expertise to the delivery of employment services^{[footnote 798]}.

Discussion of provider expertise mostly emphasised the specific and local expertise of small, charitable and specialist providers who make up the supply chain managed by a prime contractor^{[footnote 799]}. However, this also raised potential difficulties associated with operating as a smaller or charitable provider in a prime-sub and PbR model^{[footnote 800]}. Concern was expressed by expert and provider interviewees around the survival of organisations in this environment and the subsequent loss of expertise as a result^{[footnote 801]}. With this expertise most likely related to more specific categories of ‘harder-to-help’ or disadvantaged jobseekers, this would also negatively impact service users^{[footnote 802]}. Some expert and provider interviewees also believed existing specialist organisation’s full potential was not being met, largely due to the financial constraints of PbR and a lack of connected services^{[footnote 803]}.

Indeed, it appears contracts have lacked financial incentives for providers to fully utilise their expertise or that of their supply chains. This was attributed by expert and provider interviewees both to a tendency for PbR to narrow focus onto faster outcomes^{[footnote 804]} and the constraints of specification around customer journeys, or even who could be referred to the programme^{[footnote 805]}.

Provider interviewee, said:

We tend to talk about programmes as being open programmes where anybody can refer onto it [or] closed programmes [which] we deliver for DWP – but I think […] one of our specialisms is that over the years we’ve created really good partnerships with other organisations so they may very well be supporting unemployed people and they very well be a better referral pathway for that person than DWP, but that’s not allowed. So I think they don’t allow us to use that specialism of “we’ve got really good relations with other organisations” and I think that’s a really missed opportunity.

The expertise of staff who are interacting with service users was also emphasised as crucial^{[footnote 806]}. Providers need the motivation and resources to recruit, train and retain skilled staff^{[footnote 807]}. One interviewee also emphasised that greater involvement of ground-level staff in bids and perhaps even contract design could better utilise this expertise in ‘hands-on’ service delivery^{[footnote 808]}.

Several interviewees across commissioners, providers and experts suggested DWP could learn more from provider expertise^{[footnote 809]}. From a commissioner perspective, such knowledge could be utilised in future policymaking, or even to choose to take certain services back in-house^{[footnote 810]}. In the view of some providers and experts, better utilisation of provider expertise could shape improved selection processes and contract design, arguably benefitting all parties^{[footnote 811]}.

It is worth noting that the historical and continued expertise of DWP was also raised. This referred to the Department’s unique access to data and analytical insights as the commissioner, as well as institutional learning over many decades^{[footnote 812]}.

Some evidence specific to health assessments can be found in Box 10.

Box 10. Health assessments: expertise and specialist knowledge

The question of provider expertise at a staff level is more complicated in the health assessment context^{[footnote 813]}:

Commissioner interviewee, said:

We also have to make sure that they don’t become too specialist, because specifically in our contract, […] they bring specialist knowledge in terms of how you might deal with different disabilities or environments, so there’s a really strong clinical and continuous learning presence […] But again […] I’m not buying assessors who deliver assessments specifically for certain types of people, as it were. So, therefore, it’s really important that they use that expertise in a broad sense.

There are concerns around maintaining a generalist approach to health assessments, with fears that a little knowledge could be more dangerous than none^{[footnote 814]}. This could manifest in staff holding assumptions around health conditions or disabilities which impact their assessment of the service user’s claims and capacities. It could also lead to confusion around the role of the assessor for the service user, risking a belief they should have received a ‘specialist’ assessor to match their health condition or disability^{[footnote 815]}.

4.5. Encouraging innovation through PbR

4.5.1. Innovation in employment support services can mean different things

Before findings on if and how PbR contracts encourage innovation are presented, this section outlines what innovation in employment services may entail. Thijs and Staes (2012) detail numerous types of service innovations adapted from Hartley (2005)^{[footnote 816]}:

product innovation, including specific skills training for the long-term unemployed
service innovation, developing new ways for how services can be provided and includes tailored coaching for jobseekers
process innovation, developing new ways in which organisational processes are designed; examples of this type include ‘lean approaches’ looking into processes to make them as efficient as possible
position innovation, looking for new contexts or ‘customers’ with examples of adapting the PES to changing contexts of individuals needing services
strategic innovation through new goals/purposes for the organisation; this includes privatisation and outsourcing additional services (e.g., counselling)
governance innovation looking at new forms of citizen engagement, including further decentralisation of job centre functions
rhetorical innovation of new language and concepts.

4.5.2. Competition intends to encourage innovation

Competition is one of the key reasons to outsource services in the first place, as there is belief it can act as a spur for innovation^{[footnote 817]}. Competition in service provision can stimulate innovation, for example, in the education and health care sectors^{[footnote 818]}. Evidence suggests the payoffs to innovation are small when competition is between public entities^{[footnote 819]}.

Decentralisation, in relation to innovation, allows for local flexibility, thus promoting upwards competition^{[footnote 820]}. Competition can result in more testing of innovations, which in turn improves service quality and cost effectiveness^{[footnote 821]}.

4.5.3. Encouraging innovation needs to be considered early on

For the outsourcing of contacts, engagement in innovative thinking needs to occur from the start^{[footnote 822]}. This can be through early dialogue with potential suppliers and increased understanding of new technologies^{[footnote 823]}. There is also encouragement to consider innovation-based procedures, and inviting the market to suggest novel and innovative solutions to problems^{[footnote 824]}. Partnerships and collaboration can also enable experimentation and testing of new approaches^{[footnote 825]}.

The literature suggests there is a need within contracts for flexibility to encourage and allow for innovation^{[footnote 826]}^{[footnote 827]}. Delivery bodies need flexibility within their contacts to innovate^{[footnote 828]}. Therefore, the challenge is to design pricing, processes and outcome measures in ways that one the one hand providers have the flexibility (which enables innovation) whilst on the other hand the service quality for the users is protected^{[footnote 829]}. Flexibility in programme design can be achieved, for example, through design waivers, and allowance for special budget allocations for innovative local programs^{[footnote 830]}. An example in the Netherlands highlighted that a too strict monitoring regime will impose a high administrative burden on both the department and providers, with the potential to reduce flexibility and deter innovation^{[footnote 831]}.

There is, however, a challenge of encouraging innovation early on and allowing for flexibility. Sultana and Watts (2005) identify a key challenge for the PES to confront. This is seeking to find the right balance between, on the one hand, encouraging innovative, flexible, and context-sensitive responses in service delivery between clients and labour markets demands, and on the other, maintaining standards across all providers^{[footnote 832]}. This aims to ensure service users have guaranteed access to the same quality service, irrespective of their geographical or social location^{[footnote 833]}. Increased flexibility and freedom can act as a ‘double edged sword’^{[footnote 834]}, as it creates the opportunity for reduced costs and potential for innovation, but also gives the opportunity for providers to act in their own best interest^{[footnote 835]}.

Notably, research suggests that certain regions and areas are more likely to innovate than others^{[footnote 836]}^{[footnote 837]}. Carter (2018) commented that the local/city region system is most innovative compared to the central government. Leading city-regions can articulate a progressive vision of locally integrated employment support^{[footnote 838]}^{[footnote 839]}. Integration is key for providers to be able to respond sensitively to context-specific needs and individual needs, which requires connections across advice, skills, housing, among other support^{[footnote 840]}. Some findings from the UK suggest in certain contexts, and under certain contractual agreements, improve outcomes for particular groups and encourage innovation to service delivery^{[footnote 841]}.

4.5.4. There is limited evidence for innovation taking place

Opportunities for innovation are often not taken^{[footnote 842]}. Examining prospectuses of the Work Programme, Ceolta-Smith et al. (2015) comment that some prime providers showed promise of designing innovative interventions, but it was not clear if these would have been effective or cost effective^{[footnote 843]}. Indeed, the authors questioned whether a black box commissioning can effectively stimulate innovation for health-related approaches^{[footnote 844]}. Carter (2018) notes that providers appeared to use increased freedom and flexibility to maximise profits and deprioritise services to those more difficult to move into work^{[footnote 845]}.

As noted earlier (section 3.1), price competition and emphasis on reduced costs within programme design can act to stifle innovation, resulting in reduced quality and the provision of a “bare bones” service^{[footnote 846]}. Innovation is also often expensive, and consequently “low hanging fruit’ innovations are selected when handling the balance between service quality and cost trade off^{[footnote 847]}.

A too strict monitoring regime, or full specified contract, imposes high administrative burden on both the department and providers, and has the potential to reduce flexibility and deter innovation^{[footnote 848]}. Therefore, over-monitoring can quash innovation.

Decentralisation can also weaken the capacity of central management to such an extent that it becomes difficult to manage an innovative project on a large, national scale^{[footnote 849]}. An example provided from Lindsay and McQuaid (2009) highlights that in the Netherlands, government representatives acknowledged disappointment at the lack of innovative practice, despite increases of flexibility in this market^{[footnote 850]}.

The reviewed literature also provided several examples of how innovation is encouraged or promoted, primarily focusing on the PES, rather than contracted out services. This may be due to the fact that if innovation occurs through black box, or even grey box commissioning, it is more difficult to capture, document and share more broadly due to the issues around transparency, information sharing and collaboration discussed above.

Findings from the interviews

Overall, risk aversion appears to stifle innovation by providers; this means that any rewards of experimenting in service delivery are not often anticipated or, indeed, attempts at innovation would risk breaking with the specifications of the contracts^{[footnote 851]}.

Specifically, interviewees across all three stakeholder groups suggested innovation was not incentivised by PbR in its current form^{[footnote 852]}. The risk of failing to claim payments after investing in innovative methods appears to be too great for providers to take on. Furthermore, a basic lack of cash-flow to spend on innovative schemes is also seen as a consequence of the PbR model, if up-front investment or service fees are low^{[footnote 853]}. Some also claimed that innovation was restricted by specifications or compliance^{[footnote 854]}. This restriction was either a matter of specifications not permitting other activities or of compliance-related activities taking up resources which could then not be used to innovate.

However, these interviewees and others also suggested there is a consensus within the market, and its advisors, on tried and test methods in employment support services^{[footnote 855]}. This means that, even when given the opportunity to innovate - via black box specification or other models - providers are not motivated to deviate from established procedures. In fact, some commissioner and provider interviewees believed this to be a wise decision, with some scepticism expressed around a search for ‘magic dust’ or ‘reinventing the wheel’^{[footnote 856]}. Instead, there were suggestions that innovation be focused on service users known to require additional support^{[footnote 857]} or proposals aiming to meaningfully shift long-term (un)employment trends^{[footnote 858]}.

A distinction between incremental and disruptive or radical innovation was also suggested^{[footnote 859]}:

Expert interviewee, said:

There are sort of two types of innovation […] there’s radical innovation and then there’s incremental innovation and radical innovation is […] doing something completely new and completely innovative and experimental, and then incremental innovation is tweaking […] and I think what we have seen is that from payment by results […] they are good on incremental innovation and adaptation […] so I think the argument for the incremental bit is pretty good […] but not the radical innovation, so doing something completely new is just not something that’s done in […] payment by results because you’re on the hook with the financial risk so why would you do something completely new?

In relation to employment, it may be that the quasi-market structure does not generate sufficient competitive drivers to motivate providers to radical innovation^{[footnote 860]}. Small shifts and improvements within a provider organisation may, however, generate enough incremental innovation to improve their performance, relative to other competitors^{[footnote 861]}. In contrast, one provider interviewee suggested that a competitive drive to agree to low price points means shifting provider focus from improvement and innovation towards survival and sustainability^{[footnote 862]}.

Suggestions for better promotion of innovation in the future included increasing focus on innovation at the tender and bidding stages^{[footnote 863]}, reducing barriers to data-sharing^{[footnote 864]} and using different payment structures or pilots to better facilitate innovation without the risk or resource-scarcity described here^{[footnote 865]}.

Some evidence specific to health assessments can be found in Box 11.

Box 11. Health assessments: innovation

One interviewee observed that the health assessments contracts have seen incremental innovation, with changes proposed and then evaluated^{[footnote 866]}. This is likely to be expected due to the nature of the work commissioned and the different identification of ‘outcomes’ in comparison to outsourced employment services.

Question 9

5. 	Conclusions

Accepted Answer

5.1. The aims of this research

The contracting out approaches promise effectiveness and efficiency gains for the commissioners and society at large, and financial rewards for those involved in service delivery. There is a long tradition of contracting out employment services and health assessments in the UK. Despite this, there are still gaps in understanding of how the processes of commissioning could be further improved. Addressing these gaps would help DWP (and other commissioners), as well as providers and clients to reap fuller benefits from different models of contracting.

This study reviews available evidence from the literature and held by key stakeholders: commissioners, providers and experts in the field. It takes a comprehensive approach to the contracting out process: from the point of designing contracts and its different aspects (such as associated costs, benefits and risks, level of specifications, etc.) through to issues rising during market competitions and live running of the awarded contracts.

The main research question this review responds to is about the advantages and disadvantages of contracting out public employment services and health assessments and the optimum model(s) for commissioning these.

5.2. Summary of findings and key conclusions

5.2.1. Contracting models: key considerations and trade-offs

The reviewed evidence focuses primarily on contracts utilising PbR mechanisms, which are common in the commissioning of employment services in the UK and internationally. There were no international examples of commissioning of health assessments in the reviewed literature.

PbR contracts are associated with potential advantages (for instance, a better use of specialist knowledge and cost reductions in service delivery) but also with possible drawbacks (such as gaming practices, disadvantaging small, specialist providers and non-profits). Whether the benefits outweigh the costs largely depends on contracts:

Unless contracts are well specified and payments well structured, negative incentive effects will seriously detract from social outcomes^{[footnote 867]}.

Some of these undesirable costs and risks can be addressed by contract design or specification. However, addressing one characteristic of PbR contracts often has implications for other features, as summarised below. Commissioners may face tensions or trade-offs between:

supporting the largest number of people into work and effectively reaching job seekers with the most significant barriers to employment
the number of job outcomes achieved and the degree to which these are high-quality and sustainable
creating scope for innovation and adaptation (e.g., to the local labour market) whilst ensuring a baseline level of quality is met by all providers
allowing providers flexibility to tailor services to meet the needs of service users without resulting in highly uneven or variable service provision
building long-term relationships with providers with the capacity to take on the risk of PbR contracts and creating a diverse marketplace where smaller, more specialist providers can flourish.

Optimum contract design will depend on the aims and objectives of the programme, meaning it is not possible to draw overarching conclusions about the ‘best’ approach. The design of PbR contracts will be informed by the unique circumstances of the country, region, client group, and labour market^{[footnote 868]}^{[footnote 869]}. While there is no single recipe for a flawless PbR contract model, the evidence base is growing, allowing commissioners to make better informed choices suited for their needs.

Effective contracting out requires successfully establishing a quasi-market^{[footnote 870]}^{[footnote 871]}. Again, the evidence and guidance available for commissioners in the area of market stewardship is scarce but growing numbers of successful and unsuccessful experiences and examples provide some indication for future actions. These lessons are further emphasised below.

5.2.2. Contract design: optimal contract model to achieve the desired outcomes

Table 4 lists the main features of PbR contracts, outlines levers through which they could be optimised and flags how these levers may lead to adverse effects elsewhere. However, these need to be considered with caution as other factors – also discussed in this report may affect delivery (for example, the way in which the targets are set versus how the economy will do during contract duration which may give an inaccurate impression of actual impact/effectiveness).

Table 4. Optimising PbR contracts to achieve the desired outcomes

Benefits, costs and risks in PbR contracting	Assumption	Lever	Associated negative effects
Benefit: improved effectiveness and cost-efficiency	Focus on outcomes drives better results (‘what gets measured, gets done’)	Increased weighting for outcome payments	Gaming behaviour by providers – – Disadvantaging specialist providers / subcontractors by primes
Cost: high transaction costs	Reducing costs of commissioning through PbR increases VFM to commissioners	Less detailed contract specifications (black box) – – Fewer minimum service requirements – – Longer, fewer and bigger contracts	Variability in service provision – – Deteriorating service quality – – Reducing market competition
Risk: gaming strategies	Reducing incentives to gaming minimises risks and costs to commissioners	Differentiate outcome payments by client group – – More minimum service requirements – – More detailed contract specifications (grey box)	Limiting flexibility of providers and potential for innovation
Risk: primes’ monopoly	Market competition stimulates greater effectiveness and cost savings for commissioners	Smaller CPAs or two-stage competitions within existing CPAs – – Shorter contracts – – (Financial) incentives or contractual requirements for primes to work with smaller providers	Increased costs for commissioners
Risk: quality of services	Safeguards must be in place to ensure service users receive high quality provision	More minimum service requirements – – Strengthening monitoring arrangements or introduce a licencing / accreditation system – – Strengthening enforcement or breakout clauses	Increased costs for commissioners – – Reduced incentives (increased costs) for providers

A key consideration in commissioning employment services is minimising the risk of creaming and parking, enabling job seekers with the greatest barriers to employment to benefit fully from programmes. Creaming and parking undermine the equity of programmes and – when these result in deadweight – weaken value for money for the commissioner. However, mechanisms to address creaming and parking (minimum service standards, greater specification in contracts and differentiated outcome payments) may have undesirable effects (see Table 4). Greater consistency in provision may help to ensure job seekers with the greatest barriers to employment receive support, but the flip side of this may be less innovative, tailored services. It is also extremely difficult to differentiate outcome payments in a way that accurately reflects job seekers’ distance from the labour market. This issue taps into fundamental questions about who employment programmes are for and what they hope to achieve; there can be no ‘one size all’ solution.

5.2.3. Market stewardship and service quality

Findings in this report show a number of lessons learned by commissioners in contracting out employment services in different contexts. These include:

developing a long-term strategy for the market and engagement with providers in the commissioning process so that numerous programmes are informed by the current situation on the market and commissioned continuously, with some overlaps, to avoid ‘feast and famine’ cycles feared by providers
using selection criteria carefully, especially since too much emphasis on price may harm service quality and drive some providers away, thus stifling competition (and innovation)
considering implications for the altered composition of the market and a more dominant role of providers that can operate at scale but offer relatively similar services, as opposed to smaller and specialist providers, including not-for-profits and SMEs. Evidence shows that contracting out has so far led to this result but it is unclear if (and how) this affects the service users or the market in the long term.

It is challenging to find consensus on definitions of service quality in the provision of the services considered in this report. While the ‘success’ of a programme may speak through itself through outcomes such as sustained employment placements, achieving strong impact and VFM, the maintenance of consistent, fair and tailored engagement with users is also of concern. Similarly, strong provider expertise can manifest in high-quality provision and strong outcomes, but commissioners might also fear that this expertise instead translates into more effective gaming behaviours.

Some perspectives would conceptually separate performance from quality, with the latter encompassing indicators such as positive customer experience or compliance with minimum service requirements. It may be that standardisation across programmes, and thus the industry, could help align stakeholders on the question of quality. This standardisation could involve the introduction of regulatory and professional standards with observations by an inspection body similar to Ofsted. The findings from this report on measuring and monitoring quality identified the following obstacles and opportunities:

service user feedback has so far been under-utilised but has certain limitations as a metric of quality measurement, with users’ views on an overall policy or other factors beyond provider control potentially shaping their assessment of delivery
emphasis on fidelity and compliance is seen as an important safeguard for service users and guard against many risks for the commissioner, although providers can feel limited in their delivery of high-quality and tailored provision by ‘tick-box’ exercises
accountability measures such as withholding payment for poor performance can be an important lever in contracts but it may be that a punitive approach reduces opportunities to improve quality through sustained investment and mutual learning
cashflow restrictions and a narrow focus on outcomes is seen by some as an obstacle to the use of provider expertise which could improve the quality of service, especially for disadvantaged users or those with complex needs or a greater ‘distance’ to travel to employment.

5.2.4. Collaboration

Providers operating in a quasi-market may experience both incentives and disincentives to collaborate and share information and resources, both with other providers and with external stakeholders. Collaboration may foster mutual learning and the dissemination of effective and innovative practice, enabling providers to learn from the specialist expertise of other organisations. From the providers’ perspective, however, there is a tension between the benefits of collaboration and the potential risks of sharing commercially sensitive information with competitors in a quasi-market. The sharing of information and resources is heavily dependent on trust, and in the context of employment services may be driven by a shared social mission to produce the best possible outcomes for service users. It is therefore plausible that the level of collaboration amongst providers in various contracting models vary significantly: while it seems limited in PbR contracts, it may be more common in other models (e.g. partnership or grant agreements).

Commissioners may seek to drive improvements in performance by encouraging and incentivising collaboration, removing barriers to information-sharing through improving systems and changing current legal/commercial restrictions. However, commissioners will be mindful of the need to maintain healthy competition in a quasi-market and to ensure that collaboration does not enter into the territory of collusion.

5.2.5. Innovation

A key motivation for introducing PbR contracts, particularly in combination with flexible (black box) provision, has been encouraging innovation in service provision. This review finds no evidence of PbR contracts leading to break-through innovations in effectively supporting clients into jobs. To an extent, this may reflect the fact that innovation is hard to define and measure. Whilst a theoretical argument is often made in the literature that PbR contracts encourage innovation, empirical evidence on this point is scarce. However, incremental changes in how, when or where services are provided may make small differences for providers to operate more economically, while maintaining the focus on outcomes. The evidence to support this is found in examples of cost savings achieved in some countries and testimonies from a number of interviewees. With this in mind, the debate about whether the black box approach is superior to grey box in stimulating (breakthrough) innovation is constrained by a lack of evidence. Given that possibilities to save costs through incremental innovation cannot be exercised endlessly, it is also conceivable that the scope for this may be declining. Moreover, elements of PbR contracts thought to encourage innovation, for instance flexible provision, may have disadvantages such as greater scope for creaming and parking and variable service provision. PbR contracts may not always be the most appropriate vehicle to stimulate innovation in commissioning employment services and other forms of contracting may be considered and explored (such as small-scale grants, competitions)^{[footnote 872]}.

5.3. Remaining evidence gaps and future research

One of the most evident gaps relates to contracting out health assessments. The evidence of how these services are organised in other countries, as well as perspectives of the UK current, past and prospective providers (including their healthcare and administrative staff) would help better understand both the state of play in contracting out these services and possible ways forward.

Most of the evidence reviewed and collated in this report presents perceived costs and benefits of PbR contracts and how different levers within this contracting model can mitigate some risks but potentially increase others. Some of those risks are not unique to PbR but may be applicable to contracting out more broadly. Stronger evidence on the actual costs incurred and benefits achieved of different models of provision (in-house, contracted out (including PbR, Cost Plus and payment for service contracts), partnership, grant funding arrangements) is needed to be able to compare these. On that basis it would be possible to draw more informed conclusion on which works better for whom, under what conditions, and why.

Question 10

References

Accepted Answer

Adams, L., Oldfield, K., Riley, C. & Nightingale, M. (2011). Trial incapacity benefits reassessment: customer and staff views and experiences. DWP.

Adams, L., Oldfield, K., Riley, C., Vegeris, S., Husain, F., Bertram, C., … & Vowden, K. (2011). Jobseekers Regime and Flexible New Deal Evaluation: findings from longitudinal customer surveys and qualitative research (Vol. 767). A report of research carried out by Policy Studies Institute on behalf of the Department for Work and Pensions.

Albertson, K., Fox, C., O’Leary, C., Painter, G., Bailey, K., & Labarbera, J. (2018). Payment by Results and Social Impact Bonds. University of Chicago Press Economics Books.

Andersson, F., Jordahl, H., and Josephson, J. (2019). Outsourcing Public Services: Contractability, Cost, and Quality. IZA Discussion Paper Series. No 12401.

Australia Productivity Commission (2002). “Independent Review of the Job Network”, Report No.21, AusInfo, Canberra.

Ayaita, A. et al, (2021). ‘Job Placement via Private vs. Public Employment Agencies: Investigating Selection Effects and Job Match Quality in Germany’, IZA Discussion Paper No.14024

Bach-Mortensen, A. & Barlow, J. (2021). Outsourced austerity or improved services? A systematic review and thematic synthesis of the experiences of social care providers and commissioners in quasi-markets. Social Science & Medicine. 276

Beale, I, Bloss, C and Thomas, A. (2008). The longer-term impact of the New Deal for Young People. Department for Work and Pensions Working Paper No 23.

Behaghel, L. et al. (2012). ‘Private and Public Provision of Counselling to Job-Seekers: Evidence from a Large Controlled Experiment’, IZA Discussion Paper No.6518

Bruttel, O., (2005). ‘Delivering active labour market policy through vouchers: experiences with training vouchers in Germany’, International Review of Administrative Sciences 71(3), pp.391-404

Benjamin, O. (2020). Mothers’ negotiation of welfare support: the emergence of privatized entitlement, International Journal of Sociology and Social Policy, 40(11/12), pp.1439-1454.

Carey, G. et al. (2020). ‘Quasi-market shaping, stewarding and steering in personalization: the need for practice-orientated empirical evidence’, Policy Design and Practice 3(1), pp.30-44

Carter, E. (2018a). ‘Making Markets in Employment Support: Promises and Pitfalls in the Work Programme’s Private Power Market’. PhD Thesis

Carter, E. (2021). ‘More than marketised? Exploring the governance and accountability mechanisms at play in Social Impact Bonds’, Journal of Economic Policy Reform 24:1, pp.78-94

Carter E. and Whitworth A. (2015). Creaming and parking in quasi-marketised welfare-to-work schemes: Designed out of or designed into the UK Work Programme? Journal of Social Policy 44(2): 277–296.

Ceolta‐Smith et al. (2015). citing Weishaupt, J. T. (2011). Social Partners and the Governance of Public Employment Services: Trends and Experiences from Western Europe, Geneva: International Labour Organization Page 11.

Ceolta‐Smith, J., Salway, S., & Tod, A. M. (2015). A Review of Health‐related Support Provision within the UK Work Programme–What’s on the Menu?. Social Policy & Administration, 49(2), 254-276.

Child, C., Gibbs, B. G., & Rowley, K. J. (2016). Paying for success: An appraisal of social impact bonds. Global Economics and Management Review, 21(1-2), 36-45.

Cockx, B., & Baert, S. (2015). Contracting Out Mandatory Counselling and Training for Long-Term Unemployed. Private For-Profit or Non-Profit, or Keep it Public? IZA DP No. 9459

Cole, A., Cubi-Molla, P., Elliott, R., Feast, A., Hocking, L., Lorgelly, P., Payne, K., Peek, N., Sim, D., Sussex, J., Zhang, K and Steuten, L. (2021). Making Outcome-Based Payment a Reality in the NHS. Phase 2: Practical Considerations; van Stolk, C., Bjornsson, G., and Goshev, S. (2010). Provider Incentives in Social Protection and Health: A selection of case studies from OECD countries. RAND Corporation

Collins, A.M., Coughlin, D., Miller, J., Kirk, S. (2015). The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide.

Conolly, A., Day, N., Hales, J., Kinsella, R. and O’Connor, W. (2010). The operation of choice in Multiple Provider Employment Zones. Department for Work and Pensions research report no. 650.

Considine, M., O’Sullivan, S., & Nguyen, P. (2018). The policymaker’s dilemma: The risks and benefits of a ‘black box’ approach to commissioning active labour market programmes. Social Policy & Administration, 52(1), 229-251.

Davern, E., Nunn, A. and Scopetta, A. (2021). Stakeholder conference “The power of PES partnerships” 20-21-22 April 2021 Synthesis paper

Department for Transport (2015). Value for Money Framework.

Department for Work and Pensions (2020). ‘The Work Programme – A quantitative impact assessment.’ As of 24.11.2021: https://www.gov.uk/government/publications/the-work-programme-impact-assessment

DESE (2021). As of 19/08/2022 : https://www.dese.gov.au/new-employment-services-model/resources/new-employment-services-update-and-proposed-licensing-system-webinars-frequently-asked-questions

DESE (n.d.). Proposed licensing system for the New Employment Services Model: Discussion paper. As of 03/05/2022: https://www.dese.gov.au/new-employment-services-model/proposed-licensing-system-nesm-discussion-paper

Dockery, A. M. & Stromback, T. (2001). ‘Devolving public employment services: preliminary assessment of the Australian experiment’, International Labour Review, 140(4), pp.429-451

DWP (2002). As of 12/05/2022: https://www.gov.uk/government/organisations/department-for-work-pensions/about DWP (2008). Quoted in DWP (2015), Review of the Merlin Standard. Department for Work and Pensions research report no. 907, pp. 15.

DWP (2013). Evaluation of the Work Choice Specialist Disability Employment Programme: Findings from the 2011 Early Implementation and 2012 Steady State Waves of the research. See: rrep846.pdf (publishing.service.gov.uk)

DWP (2013). New Enterprise Allowance: Qualitative evaluation. See: rrep836.pdf (publishing.service.gov.uk)

DWP (2013). Support for the very long term unemployed trailblazer: longer term analysis of benefit impacts

DWP (2014). “Maximus appointed to carry out Health assessments for the Department of Work and Pensions”, As of 17/06/2022: https://www.gov.uk/government/news/maximus-appointed-to-carry-out-health-assessments-for-the-department-for-work-and-pensions

Egdell, V. et al. (2016). ‘Third Sector Experiences of Work Programme Delivery ‘, Journal of Social Policy 45(4), pp.729-746

Ejler, N. and Sidelmann, P. (2016). Application of Process Efficiency Techniques in PES. European Commission Analytical Paper.

Finn, D. (2010). Outsourcing Employment Programmes: Contract Design and Differential Prices

Finn, D. (2011). Sub-contracting in Public Employment Services: Review of research findings and literature on recent trends and business models. The European Commission Mutual Learning Programme for Public Employment Services. DG Employment, Social Affairs and Inclusion

Finn, D. (2012). Subcontracting in public employment services: the design and delivery of ‘outcome based’ and ‘black box’ contracts: analytical paper. Centre for European and International Studies Research.

FitzGerald, C. et al., (2019). ‘Walking the contractual tightrope: a transaction cost economics perspective on social impact bonds’, Public Money and Management 39:7, pp.458-467

Foster et al (2014). Work Programme Evaluation: Operation of the commissioning model, finance and programme delivery. See: Work-programme-evaluation-operation-of-the-commissioning-model-finance-and-programme-delivery-summary.pdf

Fuertes, V., Jantz, B., Klenk, T. and McQuaid, R. (2014). ‘Between cooperation and competition: the organisation of employment service delivery in the UK and Germany’, International Journal of Social Welfare, 23: S1, S71–S86.

Gash, T., Panchamia, N., Sims, S., and Hotson, L. (2012). Making public service markets work: Professionalising government’s approach to commissioning and market stewardship. Institute for Government

Gash, T. and Panchamia, N. (2012). When to contract: Which service features affect the ease of government contracting? Greenberg, D. H., & Davis, A. (2007). Evaluation of the New Deal for Disabled People: The cost and cost-benefit analyses. Department for Work and Pensions.

Griffiths, R and Durkin, S. (2007). Synthesising the evidence on Employment Zones. Department for Work and Pensions Research Report 449

Hartley (2005) in: Thijs, N. and Staes, P. (2012). Organisational development, improvement and innovation management in Public Employment Services. European Commission.

HM Government (2021). ‘The Sourcing Playbook. Government guidance on service delivery, including outsourcing, insourcing, mixed economy sourcing and contracting.’ As of 24.11.2021:

HoPES – Working Group (2013). PES Efficiency Working Group. Final report. European Commission.

Jacobi, L. & Kluve, J., (2006). ‘Before and After the Hartz Reforms: The Performance of Active Labour Market Policy in Germany’, IZA Discussion Paper No.2100

Jantz, B. et al. (2018). ‘Marketization and Varieties of Accountability Relationships in Employment Services: Comparing Denmark, Germany, and Great Britain’, Administration and Society 50(3), pp.321-345

Kay, J. & Marlow, S. (2020). ‘The Work Programme: A Quantitative Impact Assessment’, UK Department for Work and Pensions Knight, G et al. (2013). Provider-led Pathways to Work: Net impacts on employment and benefits Department for Work and Pensions Working Paper No 113

Koning, P. & Heinrich, C. J., (2010). “Cream-Skimming, Parking and Other Intended and Unintended Effects of Performance-Based Contracting in Social Welfare Services”, IZA DP No. 4801

Langenbucher, K. & Vodopivec, M. (2022). Paying for results: Contracting out employment services through outcome based payment schemes in OECD countries. OECD Social, Employment and Migration Working Papers No. 267

Lindsay, C. & McQuaid, R. W. (2009). ‘New Governance and the Case of Activation Policies: Comparing Experiences in Denmark and the Netherlands’, Social Policy and Administration 43(5), pp.445-463. Page 458.

Lødemel, I., & Trickey, H. (Eds.). (2001). ‘An offer you can’t refuse’: workfare in international perspective. Policy Press.

Manoudi, A. et al. (2014). ‘Small Scale Study on PES Business Models’, European Commission

Mosley, H., G., (2011). ‘Decentralisation of Public Employment Services’, European Commission Mutual Learning Programme for Public Employment Services

Mulgan, R. (2006). Government Accountability for Outsourced Services

Mulheirn, I and Menne, V. (2008). The Flexible New Deal: Making it work. Social Market Foundation. As of 16.06.22: https://www.smf.co.uk/wp-content/uploads/2008/09/Publication-The-Flexible-New-Deal-Making-it-work.pdf

NAO (2015). Outcome-based payment schemes: government’s use of payment by results.

NESTA (2018). Funding innovation: A practice Guide. Making money work harder. As of 18/05/2022: https://media.nesta.org.uk/documents/Funding-Innovation-Nov-18.pdf

Newton, B., Meager, N, and Sainsbury, R. (2014). Work Programme evaluation: the participant experience report Department for Work and Pensions research report 892.

O’Halloran, D. et al, (2022). ‘The development of the Australian Unemployed Workers Union Rating Scale (AUWURS) of employment service providers’, Australian Journal of Public Adminitration, (Early View), p.3.

OECD (1997). Contracting out government services: Best Practice Guidelines and Case Studies. Occasional papers No. 20.

OECD (2006); in: Lindsay & McQuaid (2009).

OECD (2021). Applying Evaluation Criteria Thoughtfully.

Pastore, F. (2019). The quasi-market of employment services in Italy. IZA DP No. 12662

Poutvaara, P. and Jordahl, H. (2020). Public sector outsourcing. The desirability of outsourcing the provision of public services depends on their characteristics and market conditions. IZA World of Labor. 65 (2).p1-11.

Rees, J. et al. (2014). ‘Support for All in the UK Work Programme? Differential Payments, Same Old Problem’, Social Policy & Administration 48(2), pp.221-239

Rehwald, K., Rosholm, M., and Svarer, M. (2015). Are Public or Private Providers of Employment Services More Effective? Evidence from a Randomized Experiment. IZA Discussion Paper Series. No. 9365.

Sasse, T., Guerin, B., Nickson, S., O’Brien, M., Pope, T. and Davies, N. (2019). ‘Government outsourcing: What has worked and what needs reform?’ Institute for Government. As of 24.11.2021: https://www.instituteforgovernment.org.uk/sites/default/files/publications/government-outsourcing-reform-WEB.pdf

Scharle, A. (2011). New Developments, Good Practices and Lessons Learnt. One year of the PES to PES Dialogue: Discussions at the Dissemination Conference. DG Employment, Social Affairs and Inclusion.

Scottish Government (2018). ‘Report – to date – of the Strategic Public Social Partnership (PSP) Model in Scotland.

Shutes, I. & Taylor, R. (2014). Conditionality and the financing of Employment Services - Implications for the Social Divisions of Work and Welfare. Social Policy & Administration. 48 (2), p204-220.

Sienkiewicz, L. (in collaboration with ICF) (2016). Measuring Customer Satisfaction with PES. Increasing PES effectiveness by meeting customer needs. Employment, Social Affairs and Inclusion.

Stafford, B., (2012). Supporting moves into work: New Deal for Disabled People findings. Scandinavian Journal of Disability Research, 14(2), pp.165–176. DOI: http://doi.org/10.1080/15017419.2011.558235

Stephan, G. (2016). Public or private job placement services: Are private ones more effective?. IZA World of Labor 2016: 285 doi: 10.15185/izawol.285

Struyven, L. (2004). Design choices in market competition for employment services for the long-term unemployed. OECD Social, Employment and Migration working papers, no. 21.

Struyven, L. & Steurs, G. (2004). ‘Quasi-market reforms in employment and training services: first experiences and evaluation results’ in Evaluation of Systems and Programmes: Third report on vocational training research in Europe, Luxembourg: Office for Official Publications of the European Communities, (Cedefop Reference Series 57)

Sultana and Watts. (2005). Career Guidance in Europe’s Public Employment Services

Tubb, H., (2012). ‘Activation and Integration: Working with Individual Action Plans’, European Commission Mutual Learning Programme for Public Employment Services

van Gestel, N. et al. (2019). From quasi‐markets to public–private networks: Employers’ engagement in public employment services. Policy Administration. 53, pp.434-448.

Vegeris et al (2011). Flexible New Deal evaluation: Customer survey and qualitative research findings. Department for Work and Pensions.

Warren, J., Garthwaite, K.. & Bambra, C., (2014). ‘After Atos Healthcare: is the Employment and Support Allowance fit for purpose and does the Work Capability Assessment have a future?’, Disability and Society, 29:8, pp.1319-1323

Weishaupt, T.J. (2014). Central Steering and Local Autonomy in Public Employment Services. PES to PES Dialogue.

Welsh Government (2021). ‘Draft Social Partnership and Public Procurement (Wales) Bill

Whitworth, A., & Carter, E. (2018). ‘Rescaling employment support accountability: From negative national neoliberalism to positively integrated city-region ecosystems’, Environment and Planning C: Politics and Space 36(2), pp.274-289

Question 11

Annex 1: Research framework

Accepted Answer

Black box	Service provision where providers have the freedom to decide what to do and they receive financial rewards for outcomes achieved (with the aim of encouraging innovation).
Contract Package Areas	Geographical areas across England and Wales with a small number of prime contractors managing a network of subcontractors delivering services in the area.
Cost Plus contract	An agreement to reimburse a company for expenses incurred in addition to a specific amount of profit, usually stated as a percentage of the contract’s full price.
Creaming or cherry-picking	Undesirable behaviour from providers to focus on clients with good employment prospects (and to neglect those with complex needs).
Gaming	A strategy used by providers to exploit weaknesses in contracts / specifications to maximise economic gains (at cost to service quality).
Grey box	Specifying some aspects of service provision but allowing providers to tailor other elements, in contrast to a ‘black box’ in which some (or most) aspects are left in the provider’s discretion.
Live running	Execution of the contracts for delivering employment services and health assessments.
Outcome payment	Financial renumeration awarded by the commissioner to providers for each service user reaching the pre-agreed criteria for claiming an ‘outcome’.
Price cap	A limit on the price a provider can charge.
Parking	Undesirable behaviour from providers to withhold or reduce support for unemployed clients who are difficult to place.
Payment by Results (PbR)	A model of financing public services where payments depend (largely or wholly) on the results achieved by service providers, thus providing a financial incentive for providers to deliver effective (in the sense of achieving outcomes) services throughout the term of the contract.
Prime/subcontractors	Model of commissioning employment contracts in the UK to a small number of (prime) contractors per Contract Package Area (CPA); prime contractors manage a network of subcontractors delivering services in the area.
Quasi-market	A term commonly used in the literature to describe the market created by contracting out public services. In quasi-markets the recipient of the services and the purchaser are separate entities and providers do not necessarily compete for profit.
Restart Scheme	The Restart Scheme was launched in 2021 in England and Wales in 12 CPAs. It gives benefit claimants who have been out of work for at least 9 months enhanced support to find jobs in their local area^{[footnote 1]}.
Specification	Documentation stating how the service should be provided: – – Little specification means ‘black box’ provision where providers have the freedom to decide what to do (this should encourage innovation) – – Detailed specification limits the freedom of the providers (and limits the potential for innovation) – – Specifying some aspects of service provision but allowing providers to tailor other elements is referred to by DWP as ‘grey box’.
Upfront or service fee	Payment to providers for the delivery of some basic service to each user, usually early in the contract and regardless of the ‘outcome’.
Work and Health Programme	The Work and Health Programme (WHP) replaced the Work Programme (WP) and was rolled out in England and Wales between November 2017 and April 2018. It aims to help disabled people, the long-term unemployed, and some priority groups (so called early access groups) to enter into and stay in work. It is delivered in 6 CPAs and devolved to some Local Government Partners and Devolved Deal Areas^{[footnote 2]}.
Work Programme	The Work Programme (WP) was a PbR welfare-to-work programme in Great Britain in the period 2011-2017. It was delivered in 18 CPAs^{[footnote 3]}.
Workfare programmes	Social assistance programmes that require recipients to participate in work or other activities (e.g., training).

Topic or Question: Market competition	Interpretation/How we plan to respond to the question	Review	Interviews	Workshop
5. What questions, criteria and scoring mechanisms are used in commissioning services and what effect do these have on market competition and (dis)incentives for efficacy and efficiency?	– – We will review the literature to establish what was tried and pros/cons identified. – – We will seek opinions from interviewees of how these could be improved.	Yes	Yes	No
6. – 6.1. How does the type of contracts affect the dynamics of market competition in the short- and long-term?	We will review the literature to find evidence on market entry/adaptation/exit, on provider goodwill or capacity (and the role of a price cap on competition). – – We will seek views from service providers representatives about their experiences.	Yes	Yes	No
6.2. When designing contracts and running competitions, what are the trade-offs between optimising short-term and long-term outcomes?	It seems unlikely that this information will be available in the literature. – We will seek views from service commissioners and experts on the trade-offs between factors such as ‘value-for-money’ in the short term and maintaining provider capacity or goodwill, in the long term, as well as other such examples	?	Yes	No

Topic or Question: Live running	Interpretation/How we plan to respond to the question	Review	Interviews	Workshop
7. How is service quality measured and what customer service standards are used?	We will review the literature to establish what was tried and pros/cons identified. – – We will seek opinions from interviewees of how quality could be measured and what elements customer service standards should include.	Yes	Yes	No
8. What incentives are used to encourage collaboration or sharing information among contractors?	We expect most evidence will come from interviews with service providers representatives or other commissioners, but we will look out for it in the review too.	?	Yes	Yes
9. To what extent do service providers share contract resources with each other or other lines of business– and does this benefit the department?	We expect evidence will come from interviews only. We will ask interviewees for their views and opinions on the extent of resource-sharing by contractors and on its potential benefits or costs to the DWP.	No	Yes	Yes
10. Do providers bring their own knowledge and expertise to the area – and to what extent (if at all) does the DWP allow them to use it?	We expect evidence will come from interviews only.	No	Yes	No
11. How (if at all) does contracting out encourage innovation in provision of services?	We will review the literature to establish what worked or not and why. – – We will seek opinions from interviewees of how service innovation could be better facilitated when contracting out.	Yes	Yes	Yes

Question 12

Annex 2: Search protocol

Accepted Answer

Databases

The search was carried out on:

Web of Science (screening all results) and
Google Scholar (screening the first 30 results) in order to identify relevant literature.

Search terms

We used a Boolean search string to ensure the relevance of results where the search functionality supported it^{[footnote 879]} (see below). We have drawn upon the search strings suggested in our proposal, refined in consultation with our expert team, and through testing.

Web of Science^{[footnote 880]}

TS= ((((“contract” OR “outsourc” OR “commission” OR “payment by results”) AND (“employment”)) AND (“advantage” OR “disadvantage” OR “benefit” OR “drawback” OR “cost”)) AND (“UK” OR “United Kingdom” OR “England” OR “Scotland” OR “Wales” OR “Northern Ireland” OR “OECD”))^{[footnote 881]}

Google Scholar advanced search

With all of the words: employment services

With at least one of the words: contract commission outsourc

Where my words occur: anywhere in the article (about 1,220,000 results)

Where my words occur: in the title of the article (53 results)

Return articles dated: 2001 – (onwards)

Search results were downloaded and combined with the literature identified in the preliminary search. After removal of duplicate studies, the titles and abstracts were screened. Full papers of potentially relevant studies were retrieved and a final judgement on eligibility will be made by RAND Europe researchers.

Data from relevant studies were extracted into a standardised template (see below).

All search results were screened by members of the research team against the inclusion/exclusion criteria (see below). Screeners consulted with each other during the early stages of the review to ensure consistency in the process.

Inclusion and exclusion criteria

Included in the review	Excluded from the review
Academic articles and grey literature, including government documents or reports	Commentaries, editorials and features
Published in English	Published in languages other than English
Articles published after 2001^{[footnote 882]}	Published prior to 2001
Research about the UK or other OECD country	Research about non-OECD countries

Snowball search

The bibliographies of studies identified via the search of databases that met the inclusion criteria and retained for full-text review were searched for potentially relevant sources. RAND Europe researchers compiled a list of all potentially relevant sources and took the same steps as those for the results of the database search, i.e. application of the above inclusion and exclusion criteria, title and abstract screen, full-text review and data extraction.

Targeted hand search

A targeted search included resources cited in the tender specifications that set out the principles for Government contracting^{[footnote 883]} and review Government outsourcing practices and results^{[footnote 884]}, including in employment contracts^{[footnote 885]}, as well as international evidence^{[footnote 886]}. In addition, RAND Europe researchers searched websites of the Institute of Labour Economics (IZA)^{[footnote 887]}, Organisation for Economic Co-operation and Development (OECD)^{[footnote 888]}, and Knowledge Centre of Public Employment Services (PES)^{[footnote 889]} for resources on contracting out (employment and health) services. The above inclusion/exclusion criteria were also applied in the targeted hand search.

Search results

Database	Number of results
Web of Science	271
Google Scholar	53
Snowballing	40
Tender specifications	4
IZA, OECD, PES Knowledge Centre	59
Total	427
Total number of results selected for screening after the removal of duplicates	424
Total number of studies retained after title and abstract screening	78
Total number of studies included in the review	78

Data extraction template

Category: Corresponding specific question	Description	Source 1
General information	Database	[Enter data]
General information	Short reference	[Enter data]
General information	Full reference	[Enter data]
General information	Abstract	[Enter data]
General information	Countries covered	[Enter data]
General information	Years covered	[Enter data]
Contract design: 1(3)	Benefits of PbR (incentives to maintain quality, achieve outcomes)	[Enter data]
Contract design: 1(3)	Costs of PbR (incl. increased overhead, perverse incentives, loss of goodwill)	[Enter data]
Contract design: 1(2)	Managing risks in contracts (types of risks, ways of assessing risks, transfer of risk)	[Enter data]
Contract design: 1(2)	Impact of uncertainty on volume and performance (incl. setting prices and targets)	[Enter data]
Contract design: 1(4)	Level of detail in contract specification (‘black box’, ‘grey box’, cost effectiveness, opportunity for innovation)	[Enter data]
Contract design: 2	Implications of using a prime/sub model	[Enter data]
Contract design: 3	Implications of geographical division (incl. competition, economies of scale)	[Enter data]
Contract design: 4	Length of contracts	[Enter data]
Market competition: 5	Selecting contractors (questions, criteria and scoring)
Market competition: 5	Effects of contracting on the market (competition, entry/adaptation/exit, efficacy, efficiency)	[Enter data]
Market competition: 6(1)	Effects of contract type (incl. a pricing cap) on market competition in the short- and long-term	[Enter data]
Market competition: 6(2)	Trade-offs between optimising short- and long-term outcomes	[Enter data]
Live running: 7	Measuring service quality and customers standards	[Enter data]
Live running: 8	Incentives for collaboration and sharing resources among contractors	[Enter data]
Live running: 9	Sharing resources with other lines of business	[Enter data]
Live running: 10	Contractors’ use of knowledge and expertise	[Enter data]
Live running: 11	Facilitating service innovation	[Enter data]
Other	[Enter data]	[Enter data]

Question 13

Annex 3: Interview topic guide

Accepted Answer

Introduction/background

1. Could you please tell me about your organisation and your role?

2. Could you please briefly tell me about your experience with contracting (public) services?

The discussion today will focus on different models of commissioning public services. These are referred to as pay for performance schemes, payment by results, payment by outcomes.

3. Which model of (PbR) contracting are you most familiar with?

4. Which health assessment contracts are you familiar with?

Contract design

5. In your experience, what benefits are associated with different PbR contracts?

6. Based on your research, what costs are associated with different PbR contracts?

7. What other implications are associated with different PbR contracts?

8. What risks are associated with different PbR contracts?

9. Drawing on your experience, how do commissioners assess the risks in different PbR competitions?

10. How (if at all) did the uncertainty on (labour) market outlooks and expected volume of clients affect provisions in this contract?

11. How (if at all) did the uncertainty on contractors’ performance affect provisions in this contract?

12. To what extent are pricing caps used in different PbR contracts?

13. Contract specification can detail closely how the service should be provided, or it can leave some or no detail at all (‘black box’ provision). What in your view is the optimal level of specifications in PbR contracts?

14. What in your view is the optimal length of a PbR contract?

15. CostPlus employment contracts were introduced during the pandemic. If you have any experience or knowledge of these, what aspects of these contracts would be useful for new contract designs in the future?

16. Services are often contracted out using a prime/sub model. What, in your opinion, are the positive or negative implications of using this model?

17. In the UK, providers have submitted bids for Contract Package Areas which divide service provision up geographically. What is your knowledge or experience with this strategy of geographical division?

Market competition

18. A variety of criteria, scoring mechanisms and questions can be used at the tendering and bidding stages to select contractors. For this contract competition, what was involved in this selection process?

19. How did this contract’s provisions affect service providers over time?

20. This competition and contract provisions aimed to strike a balance between securing the ‘value-for-money’ for the commissioner in the short term and maintaining providers’ capacity or goodwill in the long term. How well (if at all) has this balanced been achieved, in your view?

Live running

21. Based on your experience, how is service quality measured in different contract models?

22. What (if any) incentives does/did this contract contain to collaborate or share information with other contractors?

23. In your experience or knowledge, have you been aware of contractors sharing resources with each other?

24. Have you come across contractors sharing resources with other stakeholders in public service provision?

25. In your view, what specialist knowledge and expertise do contractors bring to the provision of these services, if any?

26. Commissioners may look to balance necessary specification with an aim to incentivise innovation and new approaches to the provision of services. How, if at all, was/is innovation encouraged in this contract/PbR models?

Conclusion, thanks and wrap up

27. Is there anything else that you would like to add, that we haven’t mentioned so far?

28. Is there anyone you recommend that we should also try to talk to as part of this study?

29. Would you be available to participate in a workshop which we will be organising?

30. Do you have any questions for us?

Thank you very much for your time!

Question 14

Annex 4: Synthesis workshop

Accepted Answer

List of workshop participants

Providers (4) – names withheld to ensure anonymity

Academics and researchers (3) – names withheld to ensure anonymity

Commissioners and other DWP staff (11) – names withheld to ensure anonymity

RAND Europe (6):

Dr Chris van Stolk
Michael Whitmore
Dr Julia Doyle
Jessica Dawney
Dr Madeline Nightingale
Joanna Hofman

Agenda

09.30 to 09.40: Welcome and purpose of the workshop
09.40 to 10.05: Presentation of the study and early findings
10.05 to 10.30: Breakout groups
10.30 to 10.35: Break
10.35 to 11.20: Plenary: summary and discussion
11.20 to 11.30: Wrap up, next steps and thanks to participants

Breakout groups topics and facilitation questions

Group 1: Costs and benefits associated with PbR contracts:

Is there a need for more/different research to better understand the benefits & costs of contracting out and the trade-offs involved?
How can competitions/contracts be designed to enable DWP to benefit more fully from providers’ specialist expertise and knowledge?
What provisions in future competitions/contracts would reasonably limit costs for DWP or providers? What trade-off would these require?
How can we help small/specialist providers better access the market within a prime model?

Group 2: Level of specification: balancing the ‘black box’ with minimum service requirements:

How – if at all – do minimum service requirements affect: (i) innovation? (ii) costs and pricing?
What risks do minimum service requirements bear for providers?
How are the minimum service requirements examined and how this can be improved?
How are the minimum service requirements linked to performance management?

Group 3: Incentives to encourage collaboration, sharing information or resources amongst providers:

What incentives for collaboration could be trialled? How would these work, what are risks and costs involved?
What strategies for incentivising collaboration have worked elsewhere (also in other contexts)?
What are the different impacts of collaboration groups/schemes if they are optional or compulsory?
What barriers have there been to sharing resources to best serve the clients (how can they be overcome)?
How do we ensure that collaboration/sharing is fair and mutually beneficial?

Question 15

Annex 5: Methodological challenges associated with estimating the costs and benefits associated with PbR contracts

Accepted Answer

Empirical analyses of the effects (costs/benefits) of contracting out provision face certain methodological challenges. The primary challenge is establishing a meaningful counterfactual to illustrate what would have occurred to participants if they have not participated in the programme, or if they had participated in a different programme^{[footnote 890]}.

Some studies seek to estimate differences in outcomes (e.g., cost effectiveness, job outcomes) achieved by the PES and contracted providers. However, programmes delivered by the PES and contracted providers may not be fully comparable. Differences in outcomes may reflect variation in programme design and delivery rather than the type of provider (public, private, non-profit). The profile of participants may also differ, further complicating comparisons.

Even if the programmes are comparable, it is difficult to establish a counterfactual due to the risk of unobserved heterogeneity/selection bias^{[footnote 891]}^{[footnote 892]}^{[footnote 893]}. There may be differences in the unobserved characteristics of jobseekers who participate in programmes delivered by public/private (or non-profit) providers. In addition, contracting out may be more common in sectors or regions that have particular characteristics that affect programme cost and quality, confounding comparisons.

A number of studies use randomisation to ensure comparability between programmes. However, randomisation does not necessarily eliminate the risk of selection bias. Behaghel et al (2012)^{[footnote 894]} describe an RCT where randomisation was implemented in terms of which programme was offered (public or private intensive job assistance versus a control group). Participation was voluntary, uptake relatively low (less than half of those assigned to a programme entered it) and higher for private agencies than the PES, meaning that there could be differences in the composition of the two groups of job seekers stemming from their willingness to sign up.

Methods used in observational studies to address the risk of unobserved heterogeneity include instrumental variable models, regression discontinuity design, or difference-in-difference estimates^{[footnote 895]}. Other studies use propensity score matching to maximise the comparability of the groups whose job outcomes are being compared^{[footnote 896]}. Selection bias can be addressed to some degree by using panel data to control for unobserved heterogeneity^{[footnote 897]}^{[footnote 898]}.

See more on the Restart Scheme ↩
See more on the WHP ↩
See more on the Work Programme ↩
DWP (2002). As of 12/05/2022: https://www.gov.uk/government/organisations/department-for-work-pensions/about ↩
Management by objectives is a management system that directs activities toward the effective and efficient achievement of organisational objectives through participation in decision making, goal setting, and objective feedback. See: Rodgers, R., and Hunter, J. E. (1992). A foundation of good management practice in government: Management by objectives. Public Administration Review, 27-39. ↩
OECD (2021). Applying Evaluation Criteria Thoughtfully. ↩
OECD (2021). ↩
Finn, D. (2011). Sub-contracting in Public Employment Services: Review of research findings and literature on recent trends and business models. The European Commission Mutual Learning Programme for Public Employment Services. DG Employment, Social Affairs and Inclusion ↩
OECD (1997). Contracting out government services: Best Practice Guidelines and Case Studies. Occasional papers No. 20. ↩
For contracting out health services (but not specifically health assessments) see: Cole, A., Cubi-Molla, P., Elliott, R., Feast, A., Hocking, L., Lorgelly, P., Payne, K., Peek, N., Sim, D., Sussex, J., Zhang, K and Steuten, L. (2021). Making Outcome-Based Payment a Reality in the NHS. Phase 2: Practical Considerations; van Stolk, C., Bjornsson, G., and Goshev, S. (2010). Provider Incentives in Social Protection and Health: A selection of case studies from OECD countries. RAND Corporation ↩
Gash, T., Panchamia, N., Sims, S., and Hotson, L. (2012). Making public service markets work: Professionalising government’s approach to commissioning and market stewardship. Institute for Government ↩
Langenbucher, K. and Vodopivec, M. (2022). Paying for results: Contracting out employment services through outcome based payment schemes in OECD countries. OECD Social, Employment and Migration Working Papers No. 267 ↩
Stephan, G. (2016). Public or private job placement services: Are private ones more effective?. IZA World of Labor 2016: 285 doi: 10.15185/izawol.285 ↩
See, for example: Gash, T. and Panchamia, N. (2012). When to contract: Which service features affect the ease of government contracting? ↩
Other delivery models include partnership and grant funding arrangements. ↩
This area includes benefits and costs of contracting out. ↩
This number includes all representatives from the DWP. However, not all of them have been responsible for contracting out services. ↩
Others include Langenbucher and Vodopivec (2022) and Whitworth and Carter (2018). ↩
Collins, A.M., Coughlin, D., Miller, J., Kirk, S. (2015). The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide. ↩
This was due to contract negotiations in health assessments that coincided with the research ↩
OECD (2021). ↩
OECD (2021). ↩
IDinsight evidence toolkit. As of 16/06/2022: https://www.evidencetoolkit.org/cost-effectiveness ↩
Department for Transport (2015). Value for Money Framework. ↩
Langenbucher and Vodopivec (2022). ↩
Stephan (2016). ↩
Langenbucher and Vodopivec (2022). ↩
Finn, D. (2012). ‘Subcontracting in Public Employment Services: The Design and Delivery of “Outcome Based” and “Black Box” Contracts’, European Commission ↩
Andersson, F., Jordahl, H., and Josephson, J. (2019). Outsourcing Public Services: Contractability, Cost, and Quality. IZA Discussion Paper Series. No 12401. ↩
Albertson, K., Fox, C., O’Leary, C., Painter, G., Bailey, K., and Labarbera, J. (2018). Payment by Results and Social Impact Bonds. University of Chicago Press Economics Books. ↩
Child, C., Gibbs, B. G., and Rowley, K. J. (2016). Paying for success: An appraisal of social impact bonds. Global Economics and Management Review, 21(1-2), 36-45. ↩
Struyven, L. and Steurs, G. (2004). ‘Quasi-market reforms in employment and training services: first experiences and evaluation results’ in Evalution of Systems and Programmes: Third report on vocational training research in Europe, Luxembourg: Office for Official Publications of the European Communities, (Cedefop Reference Series 57) ↩
Kay, J and Marlow, S. (2020). ‘The Work Programme: A Quantitative Impact Assessment’, UK Department for Work and Pensions ↩
The Intention to Treat (ITT) approach in randomised experiments includes all participants in the statistical analysis according to the group they were originally assigned regardless of what treatment (if any) they received. ↩
The payment model for WP differed across the contact. To begin with, the contract included an upfront service fee, but this was later abolished, moving to 100% outcome payments. ↩
Cost benefit analysis for this impact assessment was conducted using the DWP Social Cost-Benefit Analysis Framework. See: Kay and Marlow (2020). ↩
Kay and Marlow (2020). ↩
Workfare is a term used to describe social assistance programmes that require recipients to participate in work or other activities (e.g., training). See: Lødemel, I., and Trickey, H. (Eds.). (2001). ‘An offer you can’t refuse’: workfare in international perspective. Policy Press. ↩
Beale, I, Bloss, C and Thomas, A. (2008). The longer-term impact of the New Deal for Young People. Department for Work and Pensions Working Paper No 23. ↩
Greenberg, D. H., and Davis, A. (2007). Evaluation of the New Deal for Disabled People: The cost and cost-benefit analyses. Department for Work and Pensions. ↩
Stafford, B., (2012). Supporting moves into work: New Deal for Disabled People findings. Scandinavian Journal of Disability Research, 14(2), pp.165–176. DOI: http://doi.org/10.1080/15017419.2011.558235 ↩
Greenberg and Davis (2007). ↩
Greenberg and Davis (2007). ↩
Griffiths, R and Durkin, S. (2007). Synthesising the evidence on Employment Zones. Department for Work and Pensions Research Report 449 ↩
Griffiths and Durkin (2007). ↩
Griffiths and Durkin (2007). ↩
Job seekers who had been claiming Job Seeker’s Allowance (JSA) for 12 months or more. ↩
Mulheirn, I and Menne, V. (2008). The Flexible New Deal: Making it work. Social Market Foundation. As of 16.06.22: https://www.smf.co.uk/wp-content/uploads/2008/09/Publication-The-Flexible-New-Deal-Making-it-work.pdf ↩
Vegeris et al (2011). Flexible New Deal evaluation: Customer survey and qualitative research findings. Department for Work and Pensions. ↩
Adams, L., Oldfield, K., Riley, C., Vegeris, S., Husain, F., Bertram, C., … and Vowden, K. (2011). Jobseekers Regime and Flexible New Deal Evaluation: findings from longitudinal customer surveys and qualitative research (Vol. 767). A report of research carried out by Policy Studies Institute on behalf of the Department for Work and Pensions. ↩
Knight, G et al. (2013). Provider-led Pathways to Work: Net impacts on employment and benefits Department for Work and Pensions Working Paper No 113 ↩
Knight, G et al. (2013). ↩
Knight, G et al. (2013). ↩
A total of 210,330 individuals were referred to Work Choice between 2010 and 2017 and 158,420 started the programme. Of those who started the Work Choice programme, 35% achieved a short-term job outcome and 23% achieved a sustained job outcome. See: Work Choice: referrals, starts and job outcomes to October 2019 – GOV.UK (www.gov.uk) ↩
DWP (2013). See Evaluation of the Work Choice Specialist Disability Employment Programme: Findings from the 2011 Early Implementation and 2012 Steady State Waves of the research (publishing.service.gov.uk) ↩
Almost 2 million individuals were referred to the WP between 2010 and 2017. Around a third (32%) of claimants referred to the programme achieved job outcomes. 39 out of 40 providers on the WP achieved minimum performance levels as set out in the contracts. See Work Programme statistical summary: data to June 2020 – GOV.UK (www.gov.uk) ↩
Almost 300,000 individuals took part in the New Enterprise Allowance. As of December 2021, 161,000 businesses were set up through the NEA programme by 157,000 individuals. See: New Enterprise Allowance statistics: April 2011 to December 2021 – GOV.UK (www.gov.uk) ↩
DWP (2013). See New Enterprise Allowance: Qualitative evaluation.(publishing.service.gov.uk) ↩
DWP (2013). Support for the very long term unemployed trailblazer: longer term analysis of benefit impacts ↩
As of February 2022, 310,000 individuals have been referred to the WHP and 210,000 have started the programme. 78% were from the disability group, 10% from the long-term unemployed group and 12% from the Early Access group. See: Work and Health Programme statistics to February 2022 - GOV.UK (www.gov.uk) ↩
Prior to January 2022, the Restart Scheme was focused on individuals who had been claiming Universal Credit and part of the Intensive Worksearch Regime for between 12 and 18 months. ↩
Dockery, A. M. and Stromback, T. (2001). ‘Devolving public employment services: preliminary assessment of the Australian experiment’, International Labour Review, 140(4), pp.429-451 ↩
Dockery and Stromback (2001). ↩
Struyven and Steurs (2004). ↩
Dockery and Stromback (2001). ↩
Struyven and Steurs (2004). ↩
Rehwald, K., Rosholm, M., and Svarer, M. (2015). Are Public or Private Providers of Employment Services More Effective? Evidence from a Randomized Experiment. IZA Discussion Paper Series. No. 9365. ↩
Behaghel, L. et al. (2012). ‘Private and Public Provision of Counselling to Job-Seekers: Evidence from a Large Controlled Experiment’, IZA Discussion Paper No.6518 ↩
Andersson, F., Jordahl, H., and Josephson, J. (2019). Outsourcing Public Services: Contractability, Cost, and Quality. IZA Discussion Paper Series. No 12401. ↩
Sasse, T., Guerin, B., Nickson, S., O’Brien, M., Pope, T. and Davies, N. (2019). Government outsourcing: What has worked and what needs reform? Institute for Government. ↩
Cockx, B., and Baert, S. (2015). Contracting Out Mandatory Counselling and Training for Long-Term Unemployed. Private For-Profit or Non-Profit, or Keep it Public? IZA DP No. 9459 ↩
Rehwald et al. (2015). ↩
Lindsay, C. and McQuaid, R. W. (2009). ‘New Governance and the Case of Activation Policies: Comparing Experiences in Denmark and the Netherlands’, Social Policy and Administration 43(5), pp.445-463. Page 458. ↩
Gash et al. (2012); NAO (2015). Outcome-based payment schemes: government’s use of payment by results. ↩
Sasse et al. (2019). ↩
Langenbucher and Vodopivec (2022); Stephan (2016); NAO (2015). Outcome-based payment schemes: government’s use of payment by results. ↩
Bach-Mortensen, A. and Barlow, J. (2021). Outsourced austerity or improved services? A systematic review and thematic synthesis of the experiences of social care providers and commissioners in quasi-markets. Social Science and Medicine. 276 ↩
Stephan (2016). ↩
Payment group 2 – the largest payment group in the WP ↩
In the medium-long term ↩
In the medium-long term ↩
For participants, DWP, the Exchequer and society ↩ ↩²
C1, E4, P1, P6, P18, P20. ↩
E4, P1, P6, P20. ↩
C1, C2, C4, C7, C8, C9, C10, E2, E9, E11, P4, P8, P10, P11, P12, P14, P16, P17, P18, P20. ↩
C7, C8, E4, P6. ↩
E11. ↩
C3. ↩
C10, E5, E7, P1, P7, P14, P16, P18. ↩
C8, E9, E15, P3, P13. ↩
C1, C3, C5, E3, P1, P2, P7, P8, P20. ↩
E11, P12. ↩
C8, C9, E4, P2, P14. ↩
C1, E1, E4, E5, E8, E9, P4, P6, P13, P14, P16. ↩
C1, P3, P4, P6, P7, P14, P20. ↩
C3, E4, E7, E11, P6, P13, P17. ↩
C4, E3, E7, E13, P9, P11, P18. ↩
C1, P3, P4, P6, P7, P14, P18, P20. ↩
C2, P2, P7, P11. ↩
C8, P8. ↩
HoPES – Working Group (2013). PES Efficiency Working Group. Final report. European Commission. ↩
Finn (2012). ↩
Finn (2012). ↩
Finn (2012). ↩
Shutes, I. and Taylor, R. (2014). Conditionality and the financing of Employment Services - Implications for the Social Divisions of Work and Welfare. Social Policy and Administration. 48 (2), p204-220. ↩
The impact assessment of the Work Programme focused one of the payment groups (Payment Group 2), comprised of individuals aged 25 years or over and in receipt of Jobseeker’s Allowance or Universal Credit who had been in the Searching for Work conditionality group for 12+ months. ↩
Kay and Marlow (2020). ↩
Cockx, B., and Baert, S. (2015). Contracting Out Mandatory Counselling and Training for Long-Term Unemployed. Private For-Profit or Non-Profit, or Keep it Public? IZA DP No. 9459 ↩
Langenbucher and Vodopivec (2022). ↩
Ayaita, A. et al, (2021). ‘Job Placement via Private vs. Public Employment Agencies: Investigating Selection Effects and Job Match Quality in Germany’, IZA Discussion Paper No.14024 ↩
Jacobi, L. and Kluve, J., (2006). ‘Before and After the Hartz Reforms: The Performance of Active Labour Market Policy in Germany’, IZA Discussion Paper No.2100 ↩
Jantz, B. et al., (2018). ‘Marketization and Varieties of Accountability Relationships in Employment Services: Comparing Denmark, Germany, and Great Britain’, Administration and Society 50(3), pp.321-345 ↩
Bruttel, O., (2005). ‘Delivering active labour market policy through vouchers: experiences with training vouchers in Germany’, International Review of Administrative Sciences 71(3), pp.391-404 ↩
Jacobi and Kluve (2006) ↩
Bruttel (2005) ↩
Jantz et al. (2018) ↩
Pastore, F. (2019). The quasi-market of employment services in Italy. IZA DP No. 12662 ↩
Koning, P. and Heinrich, C. J., (2010). “Cream-Skimming, Parking and Other Intended and Unintended Effects of Performance-Based Contracting in Social Welfare Services”, IZA DP No. 4801 ↩
Koning and Heinrich (2010). ↩
Rehwald et al. (2015). ↩
Behaghel, L. et al. (2012). ‘Private and Public Provision of Counselling to Job-Seekers: Evidence from a Large Controlled Experiment’, IZA Discussion Paper No.6518 ↩
Manoudi, A. et al. (2014). ‘Small Scale Study on PES Business Models’, European Commission ↩
C1, C2, C6, C10, E8, P7, P13. ↩
E5, E8, P2, P4, P7, P8, P14, P16. ↩
C8, C9, E1, E2, E4, E5, E9, E11, E15, P3, P5, P8, P13, P17, P20. ↩
E4. ↩
C9. ↩
E1. ↩
E2. ↩
P3, P5, P8. ↩
C10, E4, P2, P6. ↩
E4. ↩
This refers to the ‘cost of money’ in regard to loans and interest incurred by providers. ↩
P2. ↩
C3, C10, P15. ↩
C10. ↩
Rees, J. et al. (2014). ‘Support for All in the UK Work Programme? Differential Payments, Same Old Problem’, Social Policy and Administration 48(2), pp.221-239 ↩
Carter (2018a). ↩
Finn, D. (2010). Outsourcing Employment Programmes: Contract Design and Differential Prices ↩
Dockery and Stromback (2001). ↩
Finn (2012). ↩
Ayaita, A. et al. (2021). ‘Job Placement via Private vs. Public Employment Agencies: Investigating Selection Effects and Job Match Quality in Germany’, IZA Discussion Paper No.14024 ↩
Lindsay. and McQuaid (2009). ↩
Pastore, F. (2019). The quasi-market of employment services in Italy. IZA DP No. 12662 ↩
Finn (2010). ↩
Koning, P. and Heinrich, C. J. (2010). “Cream-Skimming, Parking and Other Intended and Unintended Effects of Performance-Based Contracting in Social Welfare Services”, IZA DP No. 4801 ↩
Carter, E. (2018a). ‘Making Markets in Employment Support: Promises and Pitfalls in the Work Programme’s Private ower Market’. PhD Thesis ↩
Dockery and Stromback (2001). ↩
Finn (2010). ↩
Finn (2011). ↩
Carter, E. (2021). ↩
Carter (2021). ↩
Finn (2011). ↩
FitzGerald, C. et al., (2019). ‘Walking the contractual tightrope: a transaction cost economics perspective on social impact bonds’, Public Money and Management 39:7, pp.458-467 ↩
FitzGerald et al. (2019). ↩
Dockery and Stromback (2001). ↩
Ayaita et al. (2021). ↩
Finn (2011). ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Pastore, F. (2019). The quasi-market of employment services in Italy. IZA DP No. 12662 ↩
Kay and Marlow (2020). ↩
Carter (2021). ↩
Finn (2012). ↩
Rees et al. (2014). ↩
Whitworth, A., and Carter, E. (2018). ‘Rescaling employment support accountability: From negative national neoliberalism to positively integrated city-region ecosystems’, Environment and Planning C: Politics and Space 36(2), pp.274-289 ↩
The authors note that their findings are consistent with the presence of creaming and parking in the Work Programme, but do not prove that these took place. ↩
Rees et al. (2014). ↩
Rees et al. (2014). ↩
Carter E. and Whitworth A. (2015). Creaming and parking in quasi-marketised welfare-to-work schemes: Designed out of or designed into the UK Work Programme? Journal of Social Policy 44(2): 277–296. ↩
Carter and Whitworth (2015). ↩
Newton, B., Meager, N, and Sainsbury, R. (2014). Work Programme evaluation: the participant experience report Department for Work and Pensions research report 892. ↩
Langenbucher and Vodopivec (2022). ↩
Ayaita et al. (2021). ↩
This was defined as a socially insured job of at least 15 hours per week with a duration of at least three months. ↩
Ayaita et al. (2021). ↩
C1, C7, C8, C9, E1, E4, E7, E9, E11, E13, E13, P1, P2, P5, P8, P11, P17. ↩
E8, E12, P6. ↩
E8. ↩
E8. ↩
E12. ↩
E12. ↩
P6. ↩
Mulgan, R. (2006). Government Accountability for Outsourced Services ↩
Mulgan (2006). ↩
Mulgan (2006). ↩
C3, C8. ↩
C3. ↩
Considine, M., O’Sullivan, S., and Nguyen, P. (2018). The policymaker’s dilemma: The risks and benefits of a ‘black box’ approach to commissioning active labour market programmes. Social Policy and Administration, 52(1), 229-251. ↩
Finn (2012). ↩
HoPES – Working Group (2013). PES Efficiency Working Group. Final report. European Commission. ↩
HM Government (2021). The Sourcing Playbook Government guidance on service delivery, including outsourcing, insourcing, mixed economy sourcing and contracting’, Open Government licence v3.0. ↩
FitzGerald et al. (2019). ↩
FitzGerald et al. (2019). ↩
Carter (2018a). ↩
Finn (2012). ↩
Langenbucher and Vodopivec (2022). ↩
Finn (2011). ↩
HM Government (2021). ‘The Sourcing Playbook: Government guidance on service delivery, including outsourcing, insourcing, mixed economy sourcing and contracting’, Open Government licence v3.0 ↩
Poutvaara, P. and Jordahl, H. (2020). Public sector outsourcing. The desirability of outsourcing the provision of public services depends on their characteristics and market conditions. IZA World of Labor. 65 (2).p1-11. ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
C1, C3, C4, C6, C7, E4, P2, P15, P20. ↩
C1, C3, E11, E13, P15. ↩
C3, C5. ↩
C5, C6, C8, C10, C11, E4, P18. ↩
C8, P2, P6. ↩
C5, C8, E13. ↩
C4, C8, C11, E4, E11, E13, P9. ↩
C6, C8. ↩
C8. ↩
C4. ↩
C4. ↩
C5. ↩
E4, P9. ↩
C3, C7, E4, P6. ↩
C3, C7, E4. ↩
C10. ↩
E2. ↩
C8, E3, E4, E5, E14, P15, P16, P17, P20. ↩
E3. ↩
C8. ↩
P3. ↩
DWP representative. ↩
C8, P8, P15, P20. ↩
C2, C3, C4, E13. ↩
E11, E13. ↩
C4, C7, P11. ↩
C7, C10, P15. ↩
C4. ↩
When a Target Cost Incentive Fee is used, allowable costs estimated at the time the price is agreed are used to determine a target cost that is multiplied by the contract profit rate to calculate the contract price. This is then subject to change, based on the difference between actual and estimated allowable costs, according to terms set out in the contract. (see SSRO (2014) Glossary of terms used in report templates: contract reports (including contract pricing statement)). ↩
C4. ↩
P15. ↩
E4. ↩
E14 ↩
HM Government (2021) The Sourcing Playbook. ↩
Finn (2011). ↩
Behaghel, L. et al., (2012). ‘Private and Public Provision of Counselling to Job-Seekers: Evidence from a Large Controlled Experiment’, IZA Discussion Paper No.6518 ↩
Behaghel (2012). ↩
Shutes, I. and Taylor, R. (2014). Conditionality and the financing of Employment Services - Implications for the Social Divisions of Work and Welfare. Social Policy and Administration. 48 (2), p204-220. ↩
Struyven and Steurs (2004). ↩
Finn (2011). ↩
Finn (2011). ↩
FitzGerald et al. (2019). ↩
Finn (2011). ↩
Shutes and Taylor (2014). ↩
Shutes and Taylor (2014). ↩
C10, E3, E4, P6. ↩
E3, P6. ↩
C7, E4, E11, E13, P12, P15, P16, P17. ↩
C1, E1, E3, E4, P2, P11, P15, P20. ↩
P2, P5, P6, P16, P19. ↩
P10, P13, P15, P19. ↩
E2, P6. ↩
P16, P17. ↩
C4, E1. ↩
P12, P13. ↩
E1. ↩
C5, C7, E11, E13, P10, P17. ↩
E7, E13. ↩
E3. ↩
P10. ↩
E3. ↩
E3. ↩
C7, E1, E14, P6, P8, P9, P14, P15. ↩
P18. ↩
C1, C3, C4, C7, C8, C10, C11, E1, E3, E4, E7, E9, E12, E13, E14, P2, P3, P6, P8, P9, P10, P11, P13, P14, P15, P17, P18, P19. ↩
P13, P15. ↩
C7, E12, P9, P16, P18. ↩
P10. ↩
C5, E4, P5. ↩
C4, C8, E11. ↩
C4, C8, E11. ↩
P16, P17, P20. ↩
See for instance Sasse et al. (2019). ↩
C4, C8. ↩
E14, P8. ↩
E3, E5. ↩
P16. ↩
P8. ↩
P8, P17. ↩
P16, P17. ↩
C8, E7, E11, P7, P13. ↩
C7, E4, E12, P5, P8, P11, P17. ↩
C6. ↩
Lindsay and McQuaid (2009). ↩
Lindsay and McQuaid (2009). ↩
Mosley, H., G., (2011). ‘Decentralisation of Public Employment Services’, European Commission Mutual Learning Programme for Public Employment Services ↩
C6, C7, C8, E3, E4, E8, E13, P11. ↩
C3, E13. ↩
C7. ↩
E7. ↩
E9, P1. ↩
E15. ↩
C7, E2, E15, P2, P7, P10. ↩
C9, E14. ↩
P1. ↩
E4, E9, E12, P11. ↩
C9, E5, E8, E9, P1, P11, P14. ↩
E5, E9, P1, P11. ↩
P1, P11, E4. ↩
C9, E8, P14. ↩
P11, P14. ↩
E8, E13, P1, P5, P10. ↩
E3, E4, E7, E12, P7. ↩
E2, P8. ↩
See https://atos.net/en-gb/united-kingdom ↩
Warren, J., Garthwaite, K.. and Bambra, C., (2014). After Atos Healthcare: is the Employment and Support Allowance fit for purpose and does the Work Capability Assessment have a future?, Disability and Society, 29:8, pp.1319-1323 ↩
See https://publications.parliament.uk/pa/cm201314/cmhansrd/cm140327/wmstext/140327m0002.htm ↩
C9. ↩
C4, C8. ↩
C3. ↩
Adams, L., Oldfield, K., Riley, C. and Nightingale, M. (2011). Trial incapacity benefits reassessment: customer and staff views and experiences. DWP. ↩
E14. ↩
C9, E2, E14 ↩
E5, E14, P1. ↩
E14. ↩
Considine et al. (2018). ↩
Considine et al. (2018). ↩
Kay and Marlow (2020). ↩
Rees et al. (2014). ↩
Tubb, H., (2012). ‘Activation and Integration: Working with Individual Action Plans’, European Commission Mutual Learning Programme for Public Employment Services ↩
Finn (2012). ↩
Finn (2012). ↩
Considine et al. (2018). ↩
Finn (2011). ↩
Langenbucher and Vodopivec (2022). ↩
Vegeris et al (2011). ↩
Lindsay. and McQuaid (2009). ↩
Lindsay. and McQuaid (2009). ↩
Lindsay. and McQuaid (2009). ↩
Gash et al. (2012). ↩
Rehwald et al. (2015). ↩
Struyven and Steurs (2004). ↩
Langenbucher and Vodopivec (2022). ↩
Finn Finn (2011). ↩
Considine et al. (2018). ↩
Ceolta‐Smith, J., Salway, S., and Tod, A. M. (2015). A Review of Health‐related Support Provision within the UK Work Programme–What’s on the Menu?. Social Policy and Administration, 49(2), 254-276. ↩
Ceolta‐Smith et al. (2015). ↩
Kay and Marlow (2020). ↩
P6, P11, P16. ↩
C6, E4, E13, P14, P18. ↩
P10, P16. ↩
P2. ↩
E5, P6. ↩
C2, P6, P9, P15, P20, ↩
Gash et al. (2012). ↩
P13, P14, P16. ↩
E4, E8, P2, P9, P11. ↩
E3. ↩
P11. ↩
P3. ↩
C8. ↩
P1. ↩
E4. ↩
E11, P8. ↩
P13, P17. ↩
E5, E9, P1. ↩
P2, P6, P20. ↩
C2, E13, P2, P9, P20. ↩
E2. ↩
E4, P5. ↩
P7. ↩
Carter (2018a). ↩
Langenbucher and Vodopivec (2022). ↩
Finn (2012). ↩
Langenbucher and Vodopivec (2022). ↩
Kay and Marlow (2020). ↩
Finn (2012). ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Carter, E., (2021). ‘More than marketised? Exploring the governance and accountability mechanisms at play in Social Impact Bonds’, Journal of Economic Policy Reform 24:1, p.88. ↩
C10. ↩
C10, E3, E4, E11, E13, P19. ↩
C3, P3, P17. ↩
E11, P15. ↩
E4, E11, E12, P20. ↩
P19. ↩
E4. ↩
C2, C3, C8, C11, E3, E15. ↩
E5. ↩
P6, P9, P18, P20. ↩
P17. ↩
E3, P1, P3, P9, P11, P12, P15, P18, P20. ↩
P3. ↩
P6, P9, P15, P20. ↩
P12, P15. ↩
E7, E8, P9, P11, P15. ↩
P9. ↩
P3, P4, P11, P15. ↩
P12, P20. ↩
E5, E7. ↩
P13, P16. ↩
P16. ↩
E7, P16. ↩
P4, P6, P15. ↩
P7, P15. ↩
E12, P7. ↩
P11. ↩
C2. ↩
HM Government (2021). ‘The Sourcing Playbook: Government guidance on service delivery, including outsourcing, insourcing, mixed economy sourcing and contracting’, Open Government licence v3.0 ↩
C1, C3, C4, C5, C8, C10, E12, E13, P3, P4, P5, P10, P11, P12, P13, P14, P16, P17, P18, P20. ↩
E13. ↩
P2, P6, P12, P17, P20. ↩
P4, P10, P11, P12, P13, P14, P16, P17, P18, P20. ↩
C5, E12, P2, P5, P18, P20. ↩
C3, P2, P14. ↩
P9, P13. ↩
P8. ↩
C1, P6. ↩
C1, C8, P2, P6, P8, P9, P17. ↩
C3. ↩
C1, C2, E12, P5, P10, P12. ↩
P3, P8, P9, P12, P14, P20. ↩
C5, P6, P8, P9, P11. ↩
C1, C2. ↩
C2. ↩
More information on PIP is available: here. ↩
C3, C8. ↩
C8. ↩
P18. ↩
Ceolta‐Smith et al. (2015). citing Weishaupt, J. T. (2011). Social Partners and the Governance of Public Employment Services: Trends and Experiences from Western Europe, Geneva: International Labour Organization Page 11. ↩
DWP (2008). ↩
Ceolta‐Smith et al. (2015). ↩
Carter, E., (2018a). ‘Making Markets in Employment Support: Promises and Pitfalls in the Work Programme’s Private Power Market’. PhD Thesis ↩
Finn (2012). ↩
Finn (2012). ↩
Finn (2012). ↩
HM Government (2021). The Sourcing Playbook. ↩
DWP (2008). ↩
Finn (2012). ↩
Finn (2011). ↩
Carey, G. et al. (2020). ‘Quasi-market shaping, stewarding and steering in personalization: the need for practice-orientated empirical evidence’, Policy Design and Practice 3(1), pp.30 to 44 ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Finn (2012). ↩
Egdell, V. et al. (2016). ‘Third Sector Experiences of Work Programme Delivery ‘, Journal of Social Policy 4(4), pp.729 to 746 ↩
Rees et al. (2014). ↩
Carter (2021). ↩
Finn (2011). ↩
Ceolta‐Smith et al. (2015). ↩
Considine et al. (2018). ↩
Finn (2011). ↩
Finn (2011). ↩
Considine et al. (2018). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Carter, E. (2021). ‘More than marketised? Exploring the governance and accountability mechanisms at play in Social Impact Bonds’, Journal of Economic Policy Reform 24:1, pp.78-94 ↩
Egdell, V. et al. (2016). ‘Third Sector Experiences of Work Programme Delivery’, Journal of Social Policy 45(4), pp.729 to 746 ↩
Egdell et al. (2016). ↩
Egdell et al. (2016). ↩
See https://www.gov.uk/government/publications/the-merlin-standard-guide-for-dwp-providers/the-merlin-standard-a-guide-for-dwp-providers ↩
Finn (2012). ↩
Finn (2012). ↩
Whitworth and Carter (2018). ↩
Whitworth and Carter (2018). ↩
Carter (2021). ↩
Egdell et al. (2016). ↩
More information on the Merlin Standard. Last accessed 17.05.22. ↩
More information on the Work and Health Programme last accessed 17.05.22: Work and Health Programme including JETS provider guidance – GOV.UK (www.gov.uk)] ↩
Last accessed 17.05.22: Restart Scheme provider guidance – GOV.UK (www.gov.uk) ↩
C1, C2, C4, C5, C6, C7, C8, C10, E2, E4, E8, P3, P6, P7, P9, P12, P17. ↩
C1, C4, P17. ↩
C8, P6, P7. ↩
C2, C3, C4, C6, C7, E9, E15, P3, P8, P9, P15. ↩
C7, C6. ↩
C3, E12. ↩
C3, C5, E2, P3, P13. ↩
P3, P13. ↩
C4, E2. ↩
C3, C9, C10, E9, P4, P6, P11, P14, P15, P16, P19. ↩
C3, C8, E1, E5, E12, P7, P7, P10, P11. ↩
C3, C7, P11. ↩
C4, E4, E9, E13, P3, P7, P17, P18. ↩
P18. ↩
E13, P1, P11, P16 ↩
P16, P18. ↩
P3, P4, P8, P10. ↩
C3, E13, P1, P6, P8, P9, P10, P11, P15, P16, P17, P18. ↩
E13, P1, P9 . ↩
P1, P6. ↩
C6, C8, E1, E4, P3, P4, P16, P20. ↩
P20. ↩
P16. ↩
P3. ↩
P20. ↩
E1, P16, P17, P19. ↩
P16, P17. ↩
E6. ↩
P16, P20. ↩
C8. ↩
E3. Also: Australia Productivity Commission, 2002, “Independent Review of the Job Network”, Report No.21, AusInfo, Canberra, p.xx,, p.11.1, p.11.4-11.6, p.11.7. Please also see Figure 2 for further explanation of a licensing model. ↩
Kay and Marlow (2020). ↩
Carter (2018a). ↩
Kay and Marlow (2020). ↩
Foster et al (2014). See Work Programme Evaluation: Operation of the commissioning model, finance and programme delivery. ↩
Carter (2018a). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Ceolta‐Smith et al. (2015). ↩
C8, C10, E2, E4, E13, P1, P2, P9, P12, P13, P14. ↩
E4, P1. ↩
C1, P8, P10, P12, P13, P18. ↩
C5, E9, E13, P10, P18. ↩
P18. ↩
C1, C9, P4, P6, P11. ↩
C1, E1, P14, P16. ↩
E12, P13. ↩
P6. ↩
C1, C2, C11, P7, P12, P13, P14, P20. ↩
P7, P20. ↩
C11, P18. ↩
P8, P10, P13. ↩
P12. ↩
P8, P9, P11, P20. ↩
C8, E1. ↩
C8, E1, E13, P7, P13, P14, P18. ↩
C8, E1, E5, E13, P1, P12, P16, P17, P19. ↩
C7, C10, E1, E2, E4, E5, E9, E13, E15, P1, P3, P4, P5, P7, P8, P10, P13, P14, P17, P19. ↩
E2, P17. ↩
C11, P11, P14. ↩
C3. ↩
C3. ↩
DWP representative. ↩
Finn, D. (2012). ‘Subcontracting in Public Employment Services: The Design and Delivery of “Outcome Based” and “Black Box” Contracts’, European Commission ↩
Kay, J and Marlow, S., (2020). ‘The Work Programme: A Quantitative Impact Assessment’, UK Department for Work and Pensions ↩
Kay and Marlow (2020). ↩
Struyven, L. and Steurs, G., (2004). ‘Quasi-market reforms in employment and training services: first experiences and evaluation results’ in Evaluation of Systems and Programmes: Third report on vocational training research in Europe, Luxembourg: Office for Official Publications of the European Communities, (Cedefop Reference Series 57) ↩
Stephan, G. (2016). Public or private job placement services - Are private ones more effective? IZA World of Labour. 285, p1-10. ↩
Struyven and Steurs (2004). ↩
Finn, D, (2011). “Subcontracting in public employment services: review of research findings and literature on recent trends and business models”, European Commission ↩
Finn (2011). ↩
Finn (2011). ↩
Langenbucher and Vodopivec (2022). ↩
C1, C4, C6, C8, C10, E4, E11, E12, P2, P3, P6, P8, P9, P10, P11, P13, P14, P19. ↩
C7, C11, E1, E7, P1, P12, P16, P17, P18, P19. ↩
P4, P5, P12. ↩
P20. ↩
C1, C4, C10, E6, P7, P12. ↩
E7. ↩
C4, C6, E7, P9, P16. ↩
C4, C8, C9, C10, E2, E3, E5, E8, E9, E12, E13, P5, P12, P13, P17, P18, P19. ↩
C4, C7, C8, C9, C10, C11, E2, E3, P1, P2, P4, P10, P11, P14, P17. ↩
C4, C6, E8, E9, E12, P2, P6, P8, P7, P12, P17, P18, P19. ↩
E8, E12. ↩
C4, C7, E2, E13, P11, P12, P18. ↩
C2, C3, C8, E11, P12, P13, P16. ↩
C10, E5, E7, P18. ↩
C3, E1. ↩
C1, C2. ↩
Cockx, B., and Baert, S. (2015). Contracting Out Mandatory Counselling and Training for Long-Term Unemployed. Private For-Profit or Non-Profit, or Keep it Public? IZA DP No. 9459 ↩
Jantz, B. et al. (2018). ‘Marketization and Varieties of Accountability Relationships in Employment Services: Comparing Denmark, Germany, and Great Britain’, Administration and Society 50(3), pp.321-345 ↩
See https://www.dakks.de/en/home-en.html ↩
Jantz et al. (2018). ↩
Finn (2012). ↩
Andersson et al. (2019). ↩
Stephan (2016). ↩
Jantz et al. (2018) ↩
OECD (2006); in: Lindsay and McQuaid (2009). ↩
Finn (2011). ↩
Struyven and Steurs (2004). ↩
Stephan (2016). ↩
Struyven and Steurs (2004). ↩
Stephan (2016). ↩
Bruttel, O., (2005). ‘Delivering active labour market policy through vouchers: experiences with training vouchers in Germany’, International Review of Administrative Sciences 71(3), pp.391-404 ↩
Jantz et al. (2018) ↩
Bruttel (2005). ↩
Conolly et al., (2010). ↩
Struyven and Steurs (2004). ↩
O’Halloran, D. et al, (2022). ‘The development of the Australian Unemployed Workers Union Rating Scale (AUWURS) of employment service providers’, Australian Journal of Public Adminitration, (Early View), p.3. ↩
Jantz et al. (2018). ↩
Carey, G. et al. (2020). ‘Quasi-market shaping, stewarding and steering in personalization: the need for practice-orientated empirical evidence’, Policy Design and Practice 3(1), pp.30-44 ↩
C1, C6, C8, E4, E12, E15, P2, P18, P19, P20. ↩
P9, P14. ↩
E4, E13, P2, P15, P18, P19, P20. ↩
As of 15/07/2022: Bid evaluation guidance ↩
As of 15/07/2022: Bid evaluation guidance ↩
C2, E7, P5, P6, P16, P18, P20. ↩
P16, P18. ↩
C8, E4, E5, P1, P4. ↩
P1, P7, P10, P12. ↩
P1, P16, P17. ↩
E4. ↩
C8, E1, E4, P3, P16. ↩
P9. ↩
C5, C11, P4, P6, ↩
C8, E7, P4, P6, P8, P10, P11, P13. ↩
E7, P7. ↩
P6, P7, P8. ↩
P2. ↩
P2, P6, P16. ↩
P2, C8, ↩
P2, P5, P11, P16, P19 ↩
C1, E9. ↩
E1. ↩
E3. See also: Welsh Government (2021). ‘Draft Social Partnership and Public Procurement (Wales) Bill; Scottish Government (2018). ‘Report – to date – of the Strategic Public Social Partnership (PSP) Model in Scotland. ↩
C4. ↩
E1, P6, P7, P8, P9, P14. ↩
Carey et al. (2020). ↩
OECD (2014). ↩
OECD (2006); in: Lindsay and McQuaid (2009). ↩
Struyven and Steurs (2004). ↩
Lindsay and McQuaid (2009). ↩
Ayaita et al. (2021). ↩
Behaghel et al. (2012). ↩
Benjamin (2020). ↩
Behaghel et al. (2012). ↩
Bruttel (2005). ↩
Struyven and Steurs (2004). ↩
Langenbucher and Vodopivec (2022). ↩
Egdell et al. (2016). ↩
Finn (2011). ↩
Egdell et al. (2016). ↩
Fuertes, V., Jantz, B., Klenk, T. and McQuaid, R. (2014). ‘Between cooperation and competition: the organisation of employment service delivery in the UK and Germany’, International Journal of Social Welfare, 23: S1, S71–S86. ↩
C1, C4, C10, E2, E4, E12, P2, P16, P17, P18, P20. ↩
P2, P17. ↩
C1, C4, P12, P13. ↩
C7, C8, E8, E12, E5, P2, P4, P7, P11, P16. ↩
E4, E13, P5, P10, P12. ↩
C4, E4, E7, E8, E13, E15, P6, P7, P8, P11, P15, P18. ↩
C7, C8, E8, E12, E5, P2, P4, P7, P11, P16. ↩
E8, E13, E15, P7, P11, P15. ↩
P1. ↩
E1, E4, E8, E14, P8, P13. ↩
E1. ↩
C9, P1, P14. ↩
C1, C9, P4, P5, P13, P17, P18. ↩
C10. ↩
C11. ↩
P18. ↩
P18. ↩
P17. ↩
E8. ↩
C1. ↩
P18. ↩
C4. ↩
Langenbucher and Vodopivec (2022). ↩
Bach-Mortensen, A. and Barlow, J. (2021). Outsourced austerity or improved services? A systematic review and thematic synthesis of the experiences of social care providers and commissioners in quasi-markets. Social Science and Medicine. 276. ↩
van Gestel, N. et al. (2019). From quasi‐markets to public–private networks: Employers’ engagement in public employment services. Policy Administration. 53, pp.434-448. ↩
Struyven and Steurs (2004). ↩
Behaghel, L. et al. (2012). ‘Private and Public Provision of Counselling to Job-Seekers: Evidence from a Large Controlled Experiment’, IZA Discussion Paper No.6518. ↩
Langenbucher and Vodopivec (2022). ↩
The proposal centres on the multivariate binary logistic regression models and variables used are grouped as follows: (i) benefit type and history, (ii) health and disability characteristics, (iii) demographics, (iv) employment and household characteristics, and (v) local context. Across these variables there are statistically significant and sizeable effects that lead the author to conclude that improved understanding of the likely achievement of employment outcomes could help design an alternate payment system to better calibrate payment levels. See Carter (2018a). ↩
Egdell et al. (2016). ↩
Koning, P. and Heinrich, C. J., (2010). “Cream-Skimming, Parking and Other Intended and Unintended Effects of Performance-Based Contracting in Social Welfare Services”, IZA DP No. 4801 ↩
Carter (2018a). ↩
Langenbucher and Vodopivec (2022). ↩
C6, C7, C8, E2, E4, E13, P12, P13, P17. ↩
C2, C10, P12. ↩
C10, P12. ↩
C7, P6, P7, P8, P9, P11, P12, P14. ↩
C3, C7, E4, E14, P11. ↩
C4, C7, P2, P5. ↩
C6, E12, P5, P18, P20. ↩
C1, P1, P2. ↩
C3. ↩
P16. ↩
C4, E13, P15. ↩
C9, C10, P2, P5, P6, P7, P8, P14. ↩
C8, E4, E13, P6, P15, P19. ↩
C8. ↩
E4, E13. ↩
P6. ↩
P19. ↩
P15. ↩
P1, P20. ↩
C7, E2, E14, P1, P2, P9, P13, P17, P18, P20. ↩
Langenbucher and Vodopivec (2022). ↩
Bach-Mortensen, A. and Barlow, J. (2021). Outsourced austerity or improved services? A systematic review and thematic synthesis of the experiences of social care providers and commissioners in quasi-markets. Social Science and Medicine. 276. ↩
Considine, M., O’Sullivan, S., and Nguyen, P. (2018). The policymaker’s dilemma: The risks and benefits of a ‘black box’ approach to commissioning active labour market programmes. Social Policy and Administration, 52(1), 229-251. ↩
PES – Working Group (2013). PES Efficiency Working Group. Final report. European Commission. ↩
Sienkiewicz, L. (in collaboration with ICF) (2016). Measuring Customer Satisfaction with PES. Increasing PES effectiveness by meeting customer needs. Employment, Social Affairs and Inclusion. ↩
Stephan, G. (2016). Public or private job placement services - Are private ones more effective? IZA World of Labour. 285, p1-10. ↩
E9, E11, P2, P5, P8, P11. ↩
P10, P14. ↩
P7, P17. ↩
E5, P18. ↩
P1. ↩
P8. ↩
P13, P5. ↩
E7, P1, P2, P17. ↩
E1, P20. ↩
C4, C10, E15, P7, P11, P14, P15, P16. ↩
C4, C6, C8, E1, E15, P19, P20. ↩
C6, P8, P9. ↩
C2, E1, E7, E15, P1, P7, P17, P19. ↩
P1. ↩
E1, E4, P2, P13. ↩
C11, E3, E4, E7, P1, P2, P18. ↩
For example: Personal Independence Payment; Work Capability Assessment; Work Capability Assessment: Standards; Work Capability Assessment:Standards ↩
Atos ↩
Atos ↩
For a complete set of metrics please see: Personal Independence Payment ↩
See https://qna.files.parliament.uk/qna-attachments/1489210/original/SLAs 2018 2022.pdf ↩
C2. ↩
C2, C3, C9. ↩
P14. ↩
C9. ↩
C3, C9, E4, P5, P14, P17. ↩
C2, C3, E14, P11. ↩
Andersson et al. (2019) ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
HoPES – Working Group (2013). PES Efficiency Working Group. Final report. European Commission. ↩
Langenbucher and Vodopivec (2022). ↩
DESE (2021). As of 19/08/2022 : https://www.dese.gov.au/new-employment-services-model/resources/new-employment-services-update-and-proposed-licensing-system-webinars-frequently-asked-questions ↩
DESE (n.d.). Proposed licensing system for the New Employment Services Model: Discussion paper – as of 03/05/2022 ↩
DESE (n.d.). ↩
C3, C9, C11, E2, E12, E13, E14, P4, P9, P11, P18. ↩
E13. ↩
C11. ↩
E2. ↩
P9. ↩
C3, C9, E4, P5, P14, P17. ↩
E3, E13, P1, P4, P8, P11, P12, P13, P14. ↩
P4, P16. ↩
P11, P15, P16, P18. ↩
E3, E4, E5, E12, E13, E14. ↩
E3. ↩
E3. ↩
Employment Related Services Association. ↩
C2, E4, E7, E15, P18. ↩
E2, E13. ↩
Struyven and Steurs (2004). ↩
Carter, E. (2021). ‘More than arketized? Exploring the governance and accountability mechanisms at play in Social Impact Bonds’, Journal of Economic Policy Reform 24:1, pp.78-94 ↩
Carter (2018a). ↩
Whitworth and Carter (2018). ↩
Van Geste et al. (2019). ↩
Ejler, N. and Sidelmann, P. (2016). Application of Process Efficiency Techniques in PES. European Commission Analytical Paper. ↩
Davern, E., Nunn, A. and Scopetta, A. (2021). Stakeholder conference “The power of PES partnerships” 20-21-22 April 2021 Synthesis paper ↩
Langenbucher and Vodopivec (2022). ↩
Struyven and Steurs (2004). ↩
Struyven and Steurs (2004). ↩
Bruttel, O., (2005). ‘Delivering active labour market policy through vouchers: experiences with training vouchers in Germany’, International Review of Administrative Sciences 71(3), pp.391-404 ↩
Timo Weishaupt, J. (2014). Central Steering and Local Autonomy in Public Employment Services. PES to PES Dialogue. ↩
Bruttel (2005). ↩
Timo Weishaupt, J. (2014). Central Steering and Local Autonomy in Public Employment Services. PES to PES Dialogue. ↩
Ceolta‐Smith et al. (2015). ↩
Scharle, A. (2011). New Developments, Good Practices and Lessons Learnt. One year of the PES to PES Dialogue: Discussions at the Dissemination Conference. DG Employment, Social Affairs and Inclusion. ↩
Struyven and Steurs (2004). ↩
C7, E7, E8, E13, P1, P6, P7, P8, P13. ↩
C3, C7, C9, E1, E2, E5, E13, P1, P2, P4, P7, P9, P12, P14, P17. ↩
E2, P4. ↩
C1, C2, C7, C10, E1, E5, E7, E9, E12, E14, P1, P2, P4, P7, P9, P13, P15, P16, P17, P20. ↩
E9, E12. ↩
C3, E13, P6, P10, P11, P12, P15, P16, P18. ↩
C2. ↩
C3, C8, C9, C11, E9, E12. ↩
C9, C11, E2, E3, E7, P1, P8, P9, P10, P11, P14, P15, P16, P18, P19. ↩
E12, P16. ↩
C1, C3, C8, E1, E2, E11, E12, P1, P5, P9, P10, P11, P12, P14, P15, P19, P20. ↩
E4. ↩
E12, P15. ↩
E12, P9, P11, P15, P17, P18. ↩
C1. ↩
P2. ↩
C3, C5, C10, E4, E8, P8, P15, P19. ↩
E1, E2, P4, P5, P16, P20. ↩
See the ReAct website here for more information, ↩
P16, P20. ↩
Sultana and Watts. (2005). Career Guidance in Europe’s Public Employment Services ↩
Carter, E., (2021). ‘More than marketised? Exploring the governance and accountability mechanisms at play in Social Impact Bonds’, Journal of Economic Policy Reform 24:1, pp.78-94 ↩
Carter (2021). ↩
C3, C7, E4, E5, E13, P1, P11, P18, P20. ↩
C3, C7, C11, E8, P1, P6, P8, P10, P11, P13, P16, P17, P20. ↩
C7, C11, P2, P11, P17. ↩
E4, E13. ↩
E4. ↩
E2. ↩
C11, P2, P17, P19. ↩
C8, P2, P20. ↩
C8. ↩
C11, E2, E14. ↩
E5, E8, E15, P13. ↩
C3. ↩
C3. ↩
C3. ↩
Sasse et al. (2019). ↩
Ejler and Sidelmann (2016). ↩
Dockery and Stromback (2001). ↩
Sasse et al. (2019). ↩
Sasse et al. (2019). ↩
van Gestel et al. (2019). ↩
Rees et al. (2014). ↩
Egdell et al. (2016). ↩
Rehwald et al. (2015). ↩
C5, C9, E5, E13, P2, P16, P18. ↩
E1, E2, E5, P1. ↩
P7. ↩
P19. ↩
C8, E2, E12, E13, E15, P5, P7, P10. ↩
E12. ↩
E4, E12, P10. ↩
E12, P12. ↩
E2, E13, P7, P13. ↩
E2, P4, P7, P13. ↩
P7, P11, P17. ↩
E3, E4, E8, E9, P1, P6, P14. ↩
E3, E9, P6. ↩
P10. ↩
C5, E2, P2, P6, P10, P13, P16, P19. ↩
C5. ↩
E2, P2, P6, P10, P13, P16, P19. ↩
C8, E1. ↩
C2, E14. ↩
C2, E14. ↩
C2, E14. ↩
Hartley (2005) in: Thijs, N. and Staes, P. (2012). Organisational development, improvement and innovation management in Public Employment Services. European Commission. ↩
Sasse et al. (2019). ↩
Poutvaara and Jordahl (2020). ↩
Poutvaara and Jordahl (2020). ↩
Timo Weishaupt, J. (2014). Central Steering and Local Autonomy in Public Employment Services. PES to PES Dialogue. ↩
Dockery and Stromback (2001). ↩
HM Government (2021). ‘The Sourcing Playbook: Government guidance on service delivery, including outsourcing, insourcing, mixed economy sourcing and contracting’, Open Government licence v3.0 ↩
HM Government (2021). ↩
HM Government (2021). ↩
Davern et.al. (2021). ↩
Carter (2021). ↩
Mosley (2011). ↩
Carter (2021). ↩
Finn. (2010). ↩
Mosley (2011). ↩
Struyven and Steurs (2004). ↩
Sultana and Watts. (2005). Career Guidance in Europe’s Public Employment Services. ↩
Sultana and Watts. (2005). ↩
Rehwald et al. (2015). ↩
Rehwald et al. (2015). ↩
Carter (2018a). ↩
Finn (2011). ↩
Whitworth and Carter (2018). ↩
Carter (2018a). ↩
Carter (2018a). ↩
Finn (2011). ↩
Carter (2018a). ↩
Ceolta‐Smith et al. (2015). ↩
Ceolta‐Smith et al. (2015). ↩
Carter (2018a). ↩
Finn (2011). ↩
Considine et al. (2018). ↩
Struyven and Steurs (2004). ↩
Sultana and Watts. (2005). Career Guidance in Europe’s Public Employment Services ↩
Lindsay, C. and McQuaid, R. W., (2009). ‘New Governance and the Case of Activation Policies: Comparing Experiences in Denmark and the Netherlands’, Social Policy and Administration 43(5), pp.445-463 ↩
C1, C4, C5, E2, E8, E9, E12, P13, P16, P20. ↩
C1, C4, C9, C11, E5, E8, E9, E11, E13, P2, P4, P5, P17, P18, P20. ↩
C4, C11, E2, E5, E9, P17. ↩
E8, E13, E15, P2, P9, P14, P16, P17. ↩
C4, C7, E1, E4, E9, E13, P5, P7, P12, P17. ↩
C4, E2, P12. ↩
E5. ↩
P8. ↩
E11. ↩
E1, E11, P18. ↩
E7, E11, P12. ↩
P18. ↩
C7, P7, P8, P11. ↩
C5, E15, P9, P18. ↩
C4, C5, C11, E1, E2, E3, E4, P2, P5, P13, P17, P20. ↩
C2. ↩
Dockery and Stromback (2001). ↩
Stephan (2016). ↩
Struyven (2004). ↩
Stephan (2016). ↩
Langenbucher and Vodopivec (2022). ↩
See for example: NESTA (2018) – Funding innovation: A practice Guide. Making money work harder – as of 18/05/2022. ↩
Different contracts have different risks – and the greater the risk, the higher the price. For example, a high level of uncertainty about the volumes (targets) for the contract requires to price that risk; if volumes (targets) are fairly certain then the risk is low and does not have to be priced. ↩
A price cap sets a limit on the prices that a provider can charge. ↩
PbR is a model of financing public services where payments depend on the results achieved by service providers, thus providing a financial incentive for providers to deliver good services throughout the term of the contract (see Cabinet Office 2011 Open Public Services White Paper. Cabinet Office, London) ↩
Specification here refers to what extent the way(s) how the service should be provided is detailed. No or little specification means ‘black box’ provision where providers can do whatever they want as long as they achieve outcomes – this should encourage innovation. Specifying every detail of the provision (e.g. in implementing evidence-based interventions) limits innovation but reduces risks to providers allowing the price to be based primarily on costs. Specifying some aspects of service provision but allowing providers to tailor other elements to customers according to evidence base is referred to by DWP as ‘grey box’. ↩
A cost-plus contract is an agreement to reimburse a company for expenses incurred plus a specific amount of profit, usually stated as a percentage of the contract’s full price (taken from: https://www.investopedia.com/terms/c/cost-plus-contract.asp). ↩
Contract Package Areas are the geographical areas across England and Wales with a single prime contractor managing a network of subcontractors delivering services in the area. ↩
The search terms differ due to the different functionalities that each search engine allows. Google Scholar allows far fewer terms to be included and does not support a Boolean search. ↩
https://www.webofscience.com/wos/woscc/summary/ce3a4387-8977-46aa-a2a3-62030bb4d6a3-19453c10/relevance/1 ↩
The inclusion of “OECD” facilities the inclusion of literature on international contracting out without risking inclusion of conflict, post-conflict or otherwise fragile governments which are not relevant for comparison. ↩
Could potentially be changed to 2011, depending on number and relevancy of search results. ↩
HM Government (2021). ‘The Sourcing Playbook. Government guidance on service delivery, including outsourcing, insourcing, mixed economy sourcing and contracting.’ – as of 24.11.2021: ↩
Sasse, T., Guerin, B., Nickson, S., O’Brien, M., Pope, T. and Davies, N. (2019). ‘Government outsourcing: What has worked and what needs reform?’. Institute for Government – as of 24.11.2021. ↩
Department for Work and Pensions (2020). ‘The Work Programme – A quantitative impact assessment.’ – as of 24.11.2021. ↩
Stephan, G. (2016). Public or private job placement services—Are private ones more effective?. IZA World of Labor: 285 doi: 10.15185/izawol.285 ↩
https://www.iza.org/ ↩
https://www.oecd.org/ ↩
https://ec.europa.eu/social/main.jsp? ↩
Langenbucher and Vodopivec (2022). ↩
Andersson, F., Jordahl, H., and Josephson, J. (2019). Outsourcing Public Services: Contractability, Cost, and Quality. IZA Discussion Paper Series. No 12401. ↩
Ayaita, A. et al, 2021, ‘Job Placement via Private vs. Public Employment Agencies: Investigating Selection Effects and Job Match Quality in Germany’, IZA Discussion Paper No.14024 ↩
Cockx, B., and Baert, S. (2015). Contracting Out Mandatory Counselling and Training for Long-Term Unemployed. Private For-Profit or Non-Profit, or Keep it Public? IZA DP No. 9459 ↩
Behaghel, L. et al, 2012, ‘Private and Public Provision of Counselling to Job-Seekers: Evidence from a Large Controlled Experiment’, IZA Discussion Paper No.6518 ↩
Langenbucher and Vodopivec (2022). ↩
Langenbucher and Vodopivec (2022). ↩
Andersson, F., Jordahl, H., and Josephson, J. (2019). Outsourcing Public Services: Contractability, Cost, and Quality. IZA Discussion Paper Series. No 12401. ↩
Koning, P. and Heinrich, C. J., (2010), “Cream-Skimming, Parking and Other Intended and Unintended Effects of Performance-Based Contracting in Social Welfare Services”, IZA DP No. 4801 ↩

Topic or Question: Contract design	Interpretation/How we plan to respond to the question	Review	Interviews	Workshop
1. – 1.1. What is the best payment model for achieving the outcomes that DWP is seeking to achieve, when contracting out employment and health assessment services?	Q1(1) is the main question in the “Contract design” group and will draw on evidence found in relation to Q1(2-5) to answer it. – – It is unlikely that a single model will be identified as the best solution. It is likely that models will have different advantages and disadvantages. – – We will identify possible models through the evidence review and interviews. – – We will establish key parameters of these models (risks, benefits, costs, implications, etc.), their advantages and disadvantages through evidence review and interviews. – –We will develop a set of criteria to score each method. – – We will analyse and compare different models and aim to identify an optimum one during the workshop. – – We will identify ways to improve how employment and health services are contracted out in workshop discussions.	Yes	Yes	Yes
1.2. How do we assess and, if possible, price risk^{[footnote 873]} and how should that inform the contracting model?	We will seek evidence on the types of risk associated with different types of contracts, including the impact of factors such as uncertainty on volume and performance, or a pricing cap^{[footnote 874]}. – – We will seek evidence on managing risk when contracting out services and on the extent of transfer of risk to providers through the review and interviews.	Yes	Yes	No
1.3. What are the costs and benefits of payment by result (PbR)^{[footnote 875]}?	We will document costs (e.g. increased overhead, perverse incentives, loss of goodwill) and benefits of PbR models identified through evidence review and interviews.	Yes	Yes	No
1.4. How much specification^{[footnote 876]} should be used in an employment contract?	It seems unlikely the level of detail in contract specifications will be discussed in the literature, but we will look out for it. – – We will ask views and opinions of interviewees on the essential, optional and redundant elements of employment contracts. – – We will ask views and opinions of interviewees on the optimum extent and detail of specification in contracts to balance effectiveness with opportunity for innovation.	?	Yes	Yes
1.5. What lessons were learnt from using Cost Plus^{[footnote 877]} contracts during the pandemic?	It not likely this information will be available in the literature. – – We will ask views and opinions of interviewees on the subject, informed by a review of Cost Plus contracts, if available and accessible to us.	?	Yes	No
2. What are the implications of using a prime/sub model to support clients with complex needs?	It seems unlikely that this information will be available in the literature, but we will look out for it. – – We will ask views and opinions of interviewees on this subject and, particularly, the risks of this model failing to connect specialist small providers with people with complex needs
3. What are the implications of different strategies for geographical division for Contract Package Areas^{[footnote 878]}?	We do not anticipate that there will be a large among of information in the literature, but we will look out for any analysis of Contract Package Areas. – – We will ask interviewees for their views on geographical division and particularly its implications for competition and (the limits of) economies of scale.	?	Yes	No
4. Are there optimal lengths for contracts?	It seems unlikely that this information will be available in the literature, but we will look out for it. – – We will ask views and opinions of interviewees on the subject.	?	Yes	No

Cookies on GOV.UK

Acknowledgements

The authors

Summary

Context

Findings

Conclusion

Optimising PbR contracts to achieve the desired outcomes

Glossary

1. Introduction

1.1. Contracting out public services

1.2. Research objectives and questions

Steps 1 to 5. Key stages of the process of contracting out

1.3. Methods

1.3.1. Literature review

1.3.2. Semi-structured interviews

1.3.3. Synthesis workshop

1.3.4. Strengths and limitations of the methodology

1.4. Structure of this report

2. Contract design

2.1. Costs and benefits associated with contracting out

2.1.1. PbR contracts may bring benefits in terms of programme efficiency and value for money

Box 1. Body of evidence on contracting out employment services from the UK

Table 1. Empirical evidence on the benefits of contracting out employment services using PbR

Findings from the interviews

2.1.2. Mixed evidence on the impact of PbR contracts on programme effectiveness

Findings from the interviews

2.1.3. The main costs associated with PbR contracts are linked with undesirable behaviour from providers such as creaming and parking

Findings from the interviews

2.1.4. Contracting out may weaken the accountability of public services

Box 2. Health assessments: suitability of the PbR model

2.2. Risks associated with contracting out

2.2.1. For commissioners, risks include overpaying, underpaying and market failure

Findings from the interviews

2.2.2. For providers, risks include high upfront costs, underperformance and reputational concerns

Findings from the interviews

2.2.3. Contracting out may incur a risk of variable service provision for users/beneficiaries

Findings from the interviews

Box 3. Health assessments: risks in contracting out

2.3. The level of specification in contracted services

2.3.1. Black box provision allows providers greater flexibility and may facilitate innovation, but evidence is scarce

2.3.2. Flexibility in service provision results in greater service variability and may increase the risk of creaming and parking

Findings from the interviews

2.3.3. Minimum service requirements are designed to assure basic service standards

Findings from the interviews

Box 4. Health assessments: level of specifications

2.4. Cost Plus model

Findings from the interviews

Box 5. Health assessments: Cost Plus model

2.5. Prime model

2.5.1. The prime model of contracting builds long-term relationships with large contractors who can take on the risk of a PbR contract

2.5.2. There is a risk of subcontractors not being able to benefit fully from contracts

Findings from the interviews

2.6. Contract Package Areas may be too large

Findings from the interviews

Box 6. Health assessments: CPAs

2.7. Length of contracts ideally falls between five to seven years

Findings from the interviews

Box 7. Health assessments: length of contracts

3. Market competition

3.1. Selection criteria often combined quality and price

Table 2. Selection criteria

Findings from the interviews

3.2. There is little evidence of how different types of contracts impact market competition in the short- & long-term

Findings from the interviews

3.3. Trade-offs between optimising short-term and long-term outcomes has been a process of ‘trial and error’

Table 3. Structure of outcomes payments and weights of outcome payments as % of the total

Findings from the interviews

4. Live running

4.1. Measuring service quality

4.1.1. There is no agreed definition of service ‘quality’ or how to measure it

Findings from the interviews

Box 8. Health assessments: service quality

4.1.2. Performance measurement and monitoring aims to encourage innovation and limit gaming

Findings from the interviews

4.2. Incentives to encourage collaboration or information sharing among providers are limited

Findings from the interviews

4.3. There is little evidence on sharing contract resources amongst providers and other stakeholders

Findings from the interviews

Box 9. Health assessments: collaboration and sharing resources

Web of Science^{[footnote 880]}