Research and analysis

Life Chances Fund Evaluation - second report on Kirklees Better Outcomes Partnership

Published 31 August 2023

Applies to England

Executive Summary

This is the second interim evaluation report on the Kirklees Better Outcomes Partnership (KBOP) social impact bond (SIB). You can read more about KBOP and SIBs on the Government Outcomes Lab website. This study is part of a series of evaluations on SIBs, investigating the impact of commissioning services through a SIB instead of other commissioning approaches. The KBOP SIB receives additional funding from the Department for Culture, Media & Sport’s (DCMS) Life Chance Fund (LCF). Read more about the LCF

Aim of the impact bond partnership

The KBOP SIB service seeks to improve outcomes for adults with housing-related support needs in:

• education, training and employment (ETE)
• accommodation
• health and wellbeing

Structure

In the KBOP SIB, payment is based on achieved outcomes (defined in a pre-agreed rate card). Service delivery is managed by an investor-owned social prime contractor. Kirklees Council holds the social outcomes contract with the social prime. The social prime holds bi-lateral fee-for-service contracts with eight social sector providers.

The KBOP evaluation compares this SIB with the previous commissioning approach, a fee-for-service model.^{[footnote 1]} Both services have been delivered by the same providers, offering a valuable evaluation opportunity. The KBOP SIB service is a dynamic and adaptive system, and the research team understands that practice may have evolved since data was collected for this report.

Figure E1: Key differences between the fee-for-service arrangement and SIB model

Hypotheses

This evaluation examines four hypotheses developed in the first interim evaluation of the KBOP SIB. These capture the mechanisms potentially underpinning SIB delivery.

The four mechanisms are:

1. enhanced market stewardship

2. enhanced performance management

3. enhanced collaboration

4. enhanced flexibility and personalisation

Enhanced market stewardship

The KBOP SIB model creates a dedicated team for developing service insights and managing provider performance. A hypothesis developed in the first stage of the evaluation is that the SIB would respond to limited ability for Kirklees Council to shape services or support a thriving set of service providers by more proactively stewarding the market.

Under the SIB arrangements, we found that council staff set the vision of a high functioning, person-centred and outcome-oriented service and were able to identify opportunities to reduce system barriers.

The KBOP SIB model resulted in:

• The council team being spread less thinly over a large number of contracts.
• Expanded and more granular data on service participants and service outcomes. The outcomes contract has adopted a data-led performance management approach. Service providers are encouraged to develop service pilots, address gaps in provision and build-up co-working practices.
• Improved data availability and case management tools allow for more efficient referral allocation, a quicker response to provider performance issues, and more targeted provider development.

I think if we had that many staff, we would probably have been able to manage the relationship [i.e. the service provider contracts]. The problem …was that we had nowhere near that resource to be able to focus that much on performance and quality management at all. So, it’s part of that infrastructure question as well, isn’t it? … Well, does that infrastructure add value?

– Senior council contract manager

Figure E2: Facilitators of enhanced market stewardship and delivery implications

Enhanced performance management

The KBOP SIB introduces a more intensive and data-led approach to performance management and benchmarking compared to the fee-for-service predecessor. A hypothesis developed in the first stage of the evaluation is that the SIB would respond to misaligned and inconsistent performance metrics and a process-driven performance management approach.

We found that the KBOP SIB introduced a person-level set of pre-defined payment metrics, provided a dedicated resource for more engaged performance management and secured a central intelligence system.

I think services are definitely much more accountable. There’s no hiding place. You can’t hide within this contract because everything you do, [the Social Prime Data and Operations Analyst] knows what I’m doing. There’s nowhere to hide. There are no tricks, it’s just there in numbers they can see what we’re doing and they can see in conversations and how things get written in CDPSoft [central intelligence system], conversations that people have.

– Service manager

This meant that:

• In contrast to the fee-for-service contracts which experienced misaligned and inconsistent metrics for tracking performance, the SIB’s payment-for-outcomes mechanism has a formal outcome verification process with clearly defined payment metrics and evidence requirements.
• Service providers saw increased administrative burden. However, data collection became easier over time, with improved service intelligence facilitated through a central data management system.
• Providers are able to respond more swiftly to performance issues, and there is improved transparency and accountability for success.

Figure E3: Facilitators of enhanced performance management and delivery implications

Enhanced collaboration

The KBOP SIB is associated with enhanced collaboration between service providers. A hypothesis developed in the first stage of the evaluation is that the SIB would respond to the lack of co-working practice and perceived competitive pressures through an improved collaborative infrastructure and a shared outcomes framework.

This more intentional approach to cross-provider collaboration is demonstrated in a number of ways:

• The SIB features a greater capacity for co-working through the creation and facilitation of a collaborative infrastructure by the social prime.
• The overarching outcomes framework created a shared mission across providers and a sense of collective success that seems to dilute competitive pressures.
• Although there is a greater sharing of knowledge, best practice and resources, some hesitance remains from the perceived competitive pressures in benchmarking providers’ key performance indicators.

I can see that we are working more consistently as a group of providers, [which] I think is a benefit. Because it helps with a benchmarking and an expectation around what we’re delivering. And that helps with a consistency of the service and the level of service and the quality that we might expect. Whereas I don’t think that there was any mechanism for that with the group of contracts previously.

– Provider senior operations manager

Figure E4: Facilitators of enhanced collaboration and delivery implications

Enhanced flexibility and personalisation

The KBOP SIB allows for greater flexibility and personalisation. A hypothesis developed in the first stage of the evaluation is that the SIB would respond to limited flexibility and personalisation in delivery through reducing service specifications, while ensuring accountability for outcomes.

At the frontline, this created both opportunities and challenges:

• While the previous model allowed for limited flexibility or personalised support in service provision, the KBOP SIB’s outcomes contract and provider contracts have light-touch specifications.
• The SIB’s ‘strengths-based approach’ to frontline provision encourages staff to offer flexible, personalised support and supports innovation in service provision.
• A key tension between the outcomes-focused and person-centred approach, experienced by some providers, was found in the significantly increased caseload, alongside a decreasing percentage of staff time spent with service users in an average week. However, this is not straightforward to interpret, as the caseload estimate does not account for a shift towards longer-term support, with variation in intensity depending on user need at a given time.
• The highly flexible funding and developmental leadership approach nurtures frontline staff’s innovative capacity.
• However, high caseloads sometimes impede person-centred delivery and, along with a focus on longer-term outcomes, require service managers to allocate case work more strategically to achieve a balanced caseload of intensive and light-touch support.

But it is refreshing for people to say ‘We are not focused on how you achieve these outcomes, just do what you need to do and if you want to talk to us about something, that’s fine. If you’ve got a new idea, that’s fine. Even if you think it might cost money, if it will get some of these outcomes again, let’s have that conversation.’ That’s something you don’t get with other funders as much.

– Provider Service Director

Figure 5: Facilitators of enhanced flexibility and personalisation and delivery implications

In addition to these four hypotheses, this evaluation also found that the KBOP SIB model resulted in a ‘spillover’ on the wider local delivery network:

• The KBOP project director led in building cross-sector collaboration, which extends beyond the immediate KBOP delivery network to overcome siloed working and service fragmentation. For example, the KBOP director jointly developed a pilot between the council and justice system to improve support to ex-offenders in accessing accommodation.
• There was more focused communication of frontline issues to policy-makers.
• The long-term contract duration allowed time to build sustained relationships.

If KBOP is going to work, we can’t just deliver our own service. We have to go out and change the way all these other services interact with the people we’re trying to help.

– Investment Fund Director, Bridges Fund Management

These interim findings suggest that, in contrast to the previous fee-for-service model, the KBOP SIB has led to enhanced market stewardship, performance management, collaboration, flexibility, and personalisation.

Simultaneously, it is important to acknowledge that while the SIB is associated with a variety of beneficial changes to public management practice, the research also suggests a heightened administrative burden, linked to enhanced reporting requirements and management meetings, and an increased caseload. It is also important to note that the research team is aware that, at the time of concluding the report, the KBOP social prime was trying to mitigate some of these issues.

1. The Life Chances Fund evaluation

1.1 About the Life Chances Fund

The Life Chances Fund (LCF) is a £70m fund launched by the UK government to support the growth and development of outcomes-based commissioning through the use of social impact bonds (SIBs), commissioned by local public sector organisations in England.^{[footnote 2]} Here, central government applies outcomes-based commissioning as a public service reform tool with the objective to foster co-payment by different commissioners.

LCF projects aim to tackle complex social problems across policy areas like child and family welfare, homelessness, health and wellbeing, employment and training, criminal justice, and education and early years. Following three application rounds, funding was made available for multi-year SIB projects, as the LCF runs for nine years from July 2016 to March 2025. The first LCF projects began service delivery in 2018, with the bulk of projects launching between 2019 and 2020. Whilst all projects will receive the last of their LCF funding by March 2025, and most will be finishing delivery before this, some projects are planning to continue delivery under a SIB beyond this time, backed by a local commissioner. The LCF is administered by The National Lottery Community Fund (The Community Fund, formerly known as the Big Lottery Fund) on behalf of the Public Sector Commissioning Team (formerly the Centre for Social Impact Bonds) at the Department for Culture, Media and Sport (DCMS).

The LCF has the following objectives:

• increasing the number and scale of SIBs in England
• making it easier and quicker to set up a SIB
• generating public sector efficiencies by delivering better outcomes and using this to understand how and whether cashable savings can be achieved
• increasing social innovation and building a clear evidence base of what works
• increasing the amount of capital available to a wider range of voluntary, community and social enterprise (VCSE) sector providers to enable them to compete for public sector contracts
• providing better evidence of the effectiveness of the SIB mechanism and the savings that are being accrued
• growing the scale of the social investment market.

Detailed information (including visualisations) on individual LCF projects can be found on the Government Outcomes Lab’s website.

1.1.1 What are social impact bonds?

Social Impact Bonds are a subset of outcomes-based contracting (OBC). The UK government has been experimenting with OBC as a commissioning approach through which to improve the outcomes of public services, by linking the payment made to non-government service providers to pre-agreed, measurable outcome achievements.

In its most basic form, a SIB is a tripartite relationship between:

• a government commissioner (often central or local government) who defines social outcomes and expresses a willingness to pay for them
• a service provider (usually from the VCSE sector) who delivers an intervention or programme of support with the people using services
• an investor (typically social or philanthropic), who covers the up-front costs of the intervention in order to achieve social impact and make a financial return on their investment if payable outcomes are successfully achieved (Disley et al., 2011; Fraser et al., 2018).

1.1.2 What is the Life Chances Fund evaluation?

A key contribution of the LCF evaluation is to clarify whether, where, and how SIBs add value when compared to more conventional public service commissioning arrangements. Although a series of SIB evaluations have been carried out previously, most of these evaluations have focused on the implementation or efficacy of specific interventions (i.e. the particular service funded by the SIB), often without robust quantitative impact evaluation (Carter et al., 2018; see also Fox & Morris, 2019). As part of a unique partnership between DCMS and GO Lab, the LCF is an opportunity to undertake collaborative, robust evaluation to help improve future policy and practice.

The Government Outcomes Lab is responsible for the project-level strand of the LCF evaluation, which evaluates the impact, process and value for money of LCF SIBs and compares the SIB model to alternative commissioning approaches.^{[footnote 3]} Our research aims to respond to current evidence gaps by focusing specifically on SIBs as a tool for public service delivery and reform rather than centering only on the intervention effect. The ambition is to assess ‘the SIB effect’ – that is, the influence of this commissioning model on social outcomes. In pursuing this research, the GO Lab and DCMS hope to offer crucial thought leadership in the outcomes-based-commissioning landscape.

1.2 The LCF supplementary evaluation and this report

To gain an understanding of the impacts of services commissioned through a SIB, compared to alternative commissioning approaches, the Government Outcomes Lab research team conducts a series of a longitudinal in-depth analyses with a small number of LCF SIB projects (referred to as the ‘supplementary evaluation’).

The Kirklees Better Outcomes Partnership (KBOP) evaluation is nested within the ‘supplementary evaluation’ stream. This evaluation site has been identified as a particularly valuable learning opportunity, since Kirklees Council previously commissioned the set of service provider organisations who are currently operating under the SIB to deliver comparable provision under a fee-for-service contracting arrangement. The online glossary contains a definition for this and other key terms.

Specifically, the evaluation of the KBOP SIB focuses on three research questions:

1. What is the quantitative impact of services commissioned by the KBOP SIB on the targeted social outcomes?

2. Through what mechanisms do specific aspects of the KBOP SIB contribute to these impacts?

3. Do the benefits of the KBOP SIB approach outweigh any additional costs associated with this model, when compared to legacy contracting arrangements? And, if possible, what is the cost benefit analysis of the SIB?

This is the second report of a longitudinal process evaluation which seeks to investigate the ‘SIB-mechanism’ (research question two). The SIB itself is studied as a complex intervention, with its own theory of change. The overall evaluation compares the two intervention approaches, i.e. the SIB and fee-for-service commissioning approach. The first report focused solely on the legacy fee-for-service contract. Research conducted prior to the adoption of the SIB in 2019 provided an in-depth analysis of the implications of a fee-for-service contract on service management and delivery. In addition, this first research phase was used to develop a preliminary set of hypotheses through which the SIB model might shift management approaches and practices by the council and providers and influence frontline service delivery. This second report uses the hypotheses from the first report to explore the ‘mechanisms’ of the KBOP SIB.

The remainder of the report is structured across six overarching sections:

• Section 2 sets out the research method
• Section 3 describes the KBOP SIB service and its ‘counterfactual’, the preceding fee-for-service contract
• Section 4 summarises the KBOP SIB’s contractual framework
• Section 5 outlines the KBOP SIB’s governance
• Section 6 presents four hypotheses through which the SIB model reforms and shapes management and frontline delivery practice
• Section 7 examines the effect of the KBOP SIB on the wider Kirklees service ecosystem
• Section 8 offers concluding remarks, recommendations for policy and outlines future research within the KBOP SIB evaluation

2. Research method

This report is the second written output within a mixed-method longitudinal research programme. The analysis that underpins this report is informed by a process evaluation which investigates service development and key changes that have occurred subsequent to the adoption of a SIB commissioning arrangement in Kirklees.

2.1 Data collection

For the portion of the evaluation described in this report, 38 semi-structured expert interviews^{[footnote 4]} were conducted between October 2021 and January 2022^{[footnote 5]}. Participants were selected using purposive sampling to ensure the involvement of experts from across the SIB partnership and to allow for in-depth insights regarding the design, management and delivery of the SIB service. Alongside this, snowball sampling was used to ensure representation from a similar number of stakeholders across different types of organisations involved in the KBOP SIB. Interview participants consisted of representatives from the social investment fund manager (4); the social prime ‘KBOP’ (9), service provider (19) and council contract managers (3); the Chair of the SIB governance board, an independent consultant to the council and a pro-bono legal advisor to the investment fund manager.^{[footnote 6]} The table in Appendix A provides an account of the organisational affiliation and role of the interviewees.

A similar interview protocol was used for the different stakeholder groups. The question design was informed by the initial set of hypotheses derived from the first evaluation report. The focus area of the protocols varied depending on the specialist expertise of research participants. The interview guides included the following seven themes:

• contract and rate card design
• governance arrangements
• contract and performance management
• relationship development
• experiences of SIB service delivery
• systems change
• forward view on SIB development

Interviews were either conducted remotely (31) or face-to-face (7). All interviews were recorded and transcribed. Interviews lasted on average 69 minutes. The research is endorsed by the University’s ethics review process and further details are available in Annex C.

In addition, the research draws on an extensive analysis of 154 documentary data items sourced from all SIB partner organisations, the LCF’s central administrative data portal and the external consultant to the council. Key documentary data include the social outcomes contract (council-KBOP), provider contracts (8) (KBOP-service providers), different versions of the rate card (7) and associated background documents (2) as well as documents linked to service management (7), e.g. audit and operational manuals.^{[footnote 7]} Analysis includes key meeting minutes and presentations by the social prime, including the monthly performance review meetings (70) from the SIB launch until the completion of data collection in March 2022. Service providers shared examples of performance reports and service reviews (9). KBOP’s governance framework and the job advertisements for the KBOP management team (5) supported the analysis of the SIB’s governance model.

2.2 Data analysis

Qualitative data analysis was conducted using ATLAS-ti software. Data coding was conducted using a thematic analysis (Ryan & Bernard, 2003) to unfold the central features of the SIB model, i.e. the ‘SIB mechanisms’, shaping frontline delivery and the wider service ecosystem in Kirklees. Data coding followed the Miles et al. (2014) two-phased coding approach:

Phase 1: During the first data analysis cycle, a deductive coding approach was applied. Codes developed from the initial set of hypotheses on the ‘SIB mechanisms’ from the first evaluation report were used to break down the data into discrete parts. Complementary, structural coding was applied to categorise major themes not included in the hypotheses (Saldaña, 2021).

Phase 2: In the second cycle, an inductive coding approach was applied to expand the initial top-level codes with a list of more granular sub-codes. The sub-codes were generated using either descriptive or axial coding (Saldaña, 2021); the latter method describes a code’s properties and dimensions and enables an exploration of how the code and its sub-codes relate to each other.

2.3 Limitations

The study is not without limitations. First, the findings are specific to the KBOP SIB and not all findings are generalisable to other SIB projects. Second, some research participants might have a vested interest in seeing the partnership continue after expiration of the LCF programme and this may encourage a positive response bias. The incorporation of the perspectives of local government commissioners and documentary analysis mitigates this. Third, four out of the total of 20 provider interview participants had not been involved in the pre-SIB contract (i.e. they were not involved in the delivery of services in Kirklees before September 2019). While these research participants were able to reflect on their SIB delivery experience with reference to other block contracts, they were not able to specifically compare the KBOP SIB with the preceding fee-for-activity floating support service in Kirklees. Finally, this report does not include interviews with frontline delivery staff or service participants.^{[footnote 8]}

3. Introduction: The KBOP SIB and its counterfactual

3.1 The KBOP SIB ‘counterfactual’: the previous fee-for-service contract in Kirklees

In Kirklees, the provision of services for adults with housing-related support needs has previously been commissioned as a housing Floating Support service under the umbrella of the Supporting People programme, a national grant programme launched in 2003. It was expected to function as a preventative service by supporting participants to sustain independent living and avoid tenancy issues. Importantly, these legacy contracts did not explicitly set out to support participants into training or employment.^{[footnote 9]}

The Floating Support service sat alongside accommodation-based services which delivered interventions for people who are homeless. Support was delivered on a 1:1 basis for a specified number of hours per week and support intensity was adjusted to participants being ‘low, medium or high risk’. The intervention duration was limited to 12 months (initially 24 months) due to funding cuts. In early 2019, the services were delivered by the same nine voluntary sector provider organisations, which then became delivery partners in the KBOP SIB.

Before the launch of the SIB, the Floating Support service in Kirklees involved 15 individual contracts managed by three council contract managers. The payment to providers was made monthly in advance as a block fee. There was no central intelligence system and limited standardisation in referral processes or case management. There was no standard definition or evidence required for the independent living outcome. Providers were only required to record this in the Support Plans which were subject to occasional file auditing. Likewise, the sustainment of the outcome was not part of the contracts’ key performance indicators.

3.2 The Kirklees Better Outcomes Partnership SIB service

The KBOP SIB seeks to improve accommodation, employment, stability and wellbeing outcomes for vulnerable adults who are in need of support to live independently.^{[footnote 10]} Participants may face multiple challenges, including homelessness or the immediate risk of becoming homeless, mental health or substance misuse issues, experience of domestic violence and offending.

The service is commissioned by Kirklees Council, who initially defined the outcome measures that feature as payment triggers in the KBOP SIB. The upfront capital for service provision is sourced as social investment and is managed by Bridges Fund Management, a private investment fund management company. Ongoing funding is generated from the outcome payments. The investor-owned social prime (henceforth, the Kirklees Better Outcomes Partnership (KBOP) social prime) is a newly constituted organisation, responsible for the overall coordination of the delivery arrangement.^{[footnote 11]} Kirklees Council holds a contract (referred to as a ‘social outcomes contract’) with the KBOP social prime, and this contract defines the conditions and outcome measures that direct payment to the social prime. The payment is based on a pre-defined sum for each outcome achievement, measured at the level of individual programme participants.

Following the introduction of the SIB, the KBOP social prime holds bi-lateral contracts with eight (formerly nine) provider organisations. These contracts feature key performance indicators that are tailored to each provider. Payment to the service providers is based on a monthly fee, paid in arrears.^{[footnote 12]} The initial distribution of the contract volumes (i.e. the number of service participants engaging with each delivery organisation) was based on the preceding fee-for-service contract; two of the eight providers share a significantly higher contract volume.^{[footnote 13]}

Within the KBOP SIB, the service is delivered by the same voluntary sector organisations that were involved in the provision of the pre-SIB Floating Support service. However, one provider organisation dropped out of SIB service delivery after nine months; the contract was terminated by mutual consent between the provider and the social prime. The majority of the provider organisations deliver general housing-related support, while one provider offers specialist support for mental health and another offers specialist support for people experiencing domestic abuse (details of the participating providers are available in Annex B). The legal structure of the delivery partners features housing associations (2), charities (4) and not-for-profit organisations (2); the organisations operate on a local (3), regional (3) and national (2) scale.

The KBOP SIB service launched on 1st September 2019 and will end on 31st March 2024. The total estimated maximum outcomes payments are £22.30 million. Kirklees Council receives a partial contribution of 30 % of the total contract value towards the outcomes payments from central government through co-funding via the Life Chances Fund.

Under the KBOP SIB, participants are allocated to service providers through a central referral hub, managed by the social prime. Personalisation is a key element to service provision, which is based on a strengths-based approach seeking to transfer greater power to participants.^{[footnote 14]} The ambition (from both commissioners and the KBOP team) is to disrupt a perceived deficit culture of ‘fixing’ by shifting the focus from participants’ deficiencies to their strengths. Providers are granted greater flexibility in the mode of support provision compared to the legacy contract. There is no prescribed length or frequency of support. The case is closed once the participant has achieved all relevant outcomes; after case closure the participant can still re-access the service. However, outcomes can only be claimed once for each participant. Alongside the floating support service, KBOP offers a triage service for vulnerable people who only require a one-off support.

Participant data, including outcome achievements, referral assessment and support plan are saved on a central intelligence system (CDPSoft), administered by the council and granting full accessibility to the KBOP social prime; whereas providers can only access their own data.^{[footnote 15]} The outcome claims and verification process involves two steps: providers upload the evidence for outcomes – under the supervision of the KBOP social prime - into the CDPSoft system. Evidence requirements for the outcomes are defined in the rate card, an annex to the outcomes contract. Then, the council team verifies the provided evidence and pays the pre-defined outcome payment to the social prime. The council has the right to withhold the payment in situations where the evidence is considered insufficient.

Table 1: Comparison of key contract features

Contract Features	Fee-For-Service Contracts	Social Outcomes Contract
Contract parties	Kirklees Council and provider organisations	Kirklees Council and KBOP social prime (investor-owned social purpose vehicle)
Contract management responsibility	Kirklees Council	Kirklees Council
Payment mechanism	Monthly advance block payment	Monthly outcomes payment (i.e. payment is contingent on achieved outcome number and type)
Key performance indicators (KPIs)	Service Utilisation, Throughput, Independent Living	Accommodation, Education, Training & Employment, Health & Wellbeing, Financial Resilience
KPIs require sustainment of outcome achievements	No	Yes
Auditing	No pre-defined evidence requirements; spot checks of qualitative evidence (e.g. workbooks)	Pre-defined evidence requirements; council audits every outcome
Contract duration	Max. 2 years	5 years

Figure 1: Stakeholders’ responsibilities in the Kirklees Better Outcomes Partnership SIB

Figure 2: The managerial structure of the KBOP social prime

4. The SIB’s contractual framework

The KBOP SIB features two contract types: the social outcomes contract between the Kirklees Council and the KBOP social prime; and the bilateral service delivery contracts between the social prime and the individual provider organisations. This section draws on documentary analysis (especially contracts) to offer a description of the different contractual phases of the social outcomes contract. Next, it provides an overview of the rate card and rate card development, as this is a key element for the social outcomes contract. This is followed by an exploration of three areas:

1. The contractual levers for performance management

2. The role of the social outcomes contract in facilitating enhanced performance management, collaboration and flexibility

3. The social outcomes contract as a potential example of a relational contract.

4.1 The contractual phases

The social outcomes contract divides the SIB programme into a mobilisation and an operational period. Due to Covid-19, the initial programme had to be adjusted. The following section provides a description of the initial programme structure, illustrating the envisioned transformation of services from the fee-for-service to the SIB commissioning model.

Contract mobilisation started six months prior to the service launch in September 2019. It included the development of an operational infrastructure and the negotiation of the provider contracts. Alongside this, the investment fund managers prepared the providers for SIB delivery. Training was provided on technical aspects such as the new KPIs and outcome evidence requirements, aiming to help providers understand the value of these additional requirements. Providers appreciated their early involvement, as a Regional Head of Operations noted:^{[footnote 16]}

The experience in developing the SIB model has been extraordinarily positive, in the respect that the management team involved in the SIB at Bridges have been very collaborative in their approach.

The “Operational Period” of the outcomes contract marks the beginning of the SIB service delivery. In the first year of the contract, the KBOP social prime and the providers were shielded from the implementation of a performance improvement plan (referred to as a ‘formal action plan’) and contract termination emanating from service failure or negative outcomes assessment.^{[footnote 17]} The investment fund manager described this as a “grace period” to align service delivery to the new contractual structures.

4.2 The rate card

A key element of the social outcomes contract is the rate card. A rate card is a schedule of payments for specific, pre-agreed outcomes that an outcome payer (in the KBOP SIB, Kirklees Council and LCF) is willing to make for each participant, cohort or specified improvement that verifiably achieves each outcome. The following section provides an overview of the outcomes and associated evidence requirements of the KBOP SIB. It also outlines the process for the design and re-design of the rate card and the rationale for the changes.

The outcome measures in the rate card provide an overarching set of shared success indicators for all providers of generic housing-related support.^{[footnote 18]} A different rate card is used for a specialist provider for domestic violence (Pennine Domestic Abuse Partnership, PDAP).

The KBOP SIB seeks to improve participants’ outcomes in the following fields:

• Wellbeing
• Accommodation
• Education, Training and Employment (ETE)
• Emotional and Mental Health
• Drug and Alcohol Misuse
• Domestic Violence

To incentivise providers to work with service participants towards a long-term change, outcome payments are split between the initial achievement of the outcome (e.g. entering accommodation) and sustaining the outcome over a specific period of time (e.g. sustaining accommodation over six months). The outcome payment level increases the longer the outcome is sustained (e.g. £500 for ‘entry into employment’, £2,200 for ‘26 weeks of sustained employment’), to align incentives between the financial payment mechanism and the achievement of long-term outcomes. Evidence requirements vary and include self-certification forms (see Appendix I).^{[footnote 19]} The self-certification of outcome measures allows providers to ask service users to declare the achievement of given outcomes; for instance, for the outcome ‘entering into employment’, service users are allowed to self-evidence the employment by providing signed forms instead of an employment contract or payslips.^{[footnote 20]}

Table 2: Rate card outcomes and outcome metrics (at the time of research completion)^{[footnote 21]}

Long-term improvement for service user	Outcomes
Wellbeing	1st Wellbeing assessment
	2nd Wellbeing assessment
	3rd Wellbeing assessment
	Wellbeing improvement - 1st to 2nd assessment
	Wellbeing improvement - 1st to 3rd assessment
Managing Money	Financial resilience outcomes
Emotional & Mental Health; Drug & Alcohol Misuse	Accessing services
	Mental health sustained engagement with services
	Drugs/ alcohol sustained engagement with services
Accommodation	Prevention / relief / entry into suitable accommodation
	3 months accommodation outcomes
	6 months accommodation outcomes
	12 months accommodation outcomes
	18 months accommodation outcomes
Education, Training & Employment (ETE)	Entry into education and employment
	Part completion of Ofqual approved qualification
	Completion of full Ofqual approved qualification
	Entry into employment
	6.5 weeks equivalent employment F/T
	13 weeks equivalent employment F/T
	26 weeks equivalent employment F/T
	Entry into volunteering
	6 weeks volunteering
Prevention of Domestic Abuse	Reduction in risk of domestic abuse
	Accessing rights to legal protection
	Empowering and promoting independence

Source: Adapted from KBOP social prime internal document

The rate card was designed in two phases. An initial rate card was developed prior to the SIB launch in 2019 and used until September 2020. Interview participants explained that the initial rate card was developed by drawing on the experience of previous housing-related social impact bonds, such as projects supported by the Fair Chance Fund.^{[footnote 22]}

The outcomes contract allows for adaptions to the rate card. The underlying rationale was - according to the investment fund manager - a lack of data to build an assured rate card. This means that the rate card can be adjusted without the need for renegotiation of the contract. During the early mobilisation phase, the current rate card was developed. This adaptation happened during live running, and KBOP management showed a strong interest in ensuring the inclusiveness of the re-design process. The KBOP project director hosted ‘change panels’ with provider managers and frontline staff to get a comprehensive appreciation of the issues providers experienced with the initial rate card.

The final rate card includes the same headline outcomes as the initial rate card. However, outcomes are no longer split across short and long-term outcomes (see figure 4). Also, as mentioned previously, evidence requirements were softened, allowing for self-certification. The rationale for the changes were twofold: i) frontline staff did not perceive the short-term outcome measures as meaningfully contributing to the long-term outcomes; and ii) onerous evidence requirements created a significant administrative burden for frontline staff, and some of the requirements were intrusive to participants’ privacy.

Figure 3: Rate Card Development^{[footnote 23]}

Source: Adapted from Kirklees Council

4.3 Contractual levers for performance management

4.3.1 Formal levers

In terms of contractual levers, it is important to differentiate between levers the council can apply in relation to the social prime and levers the social prime can apply in relation to the providers. The social outcomes contract contains a performance plan procedure, to be enacted if ongoing performance levels are not sufficient to fulfil contractual outcome requirements. The key contractual lever is that payment to the social prime is based on the amount and type of outcome achievement. The council contract manager highlighted:

If they’re not getting the providers to achieve the outcomes, they don’t get paid.

In contrast to the outcomes contract, the provider contracts (KBOP social prime -service providers) specify the levers applicable in situations of provider underperformance. The provider contracts contain provisions for remedial intervention, with different intensities in the case of under-performance:

1. The provider can be asked to draft a formal performance improvement plan if there is a service failure ^{[footnote 24]} or a negative outcome assessment.^{[footnote 25]}
2. Alternatively, the social prime can withhold payments for providers’ central overhead fees, in case of the failure to achieve any of the downside case KPIs. It is important to note that the direct costs of frontline staff associated with the SIB service will still be covered.
3. The social prime can terminate the contract if a provider continues to fail on the delivery of any of the downside KPIs for more than three months after the agreement on a performance improvement plan.

As discussed in the broader SIB literature, it is the investor and not the provider that principally carries the financial risk of under-performance. This is unlike more conventional payment-by-results arrangements. In the KBOP SIB, the social prime shields the providers from financial risk. It is the social prime that will default on re-payment of working capital to the investor if outcomes are not achieved. This was also stressed by the KBOP project director:

The whole point is that we take the risk. It should never be that partners feel that they have the pressure of individual risk in terms of delivery or financially, which creates the opportunity for innovation.

However, the contractual levers illustrate that providers still carry implementation risk alongside the social prime. Providers reported that these levers - specifically the right to withhold payments – had been a major concern when entering the SIB partnership:

I think the main kind of concerns that we had organisationally were around the penalties that we might incur, and kind of the stringent nature of the monitoring and whether that was going to lead to financial loss for the organisation based on the performance management element of the contract… and the fact that the overheads could be withheld if we weren’t performing at a certain level. And I think a lot of that was because we didn’t know necessarily what our key performance indicators were going to be or how likely it was that we were going to be able to achieve those. We don’t have any other contracts where financial penalties are issued for potential underperformance.^{[footnote 26]}

The next section explores the extent to which the social prime used the contractual levers in practice.

4.3.2 Application in practice

Findings suggest that the relationship between the KBOP social prime and the providers was based on ‘relational norms’ rather than on the use of formal contractual mechanisms. The KBOP project director referred to these (i.e. the formal contractual remedies) as ‘pinch points’ which exist as a framework that would only be instigated if something “has gone seriously wrong”. Similarly, the investment fund director noted that, even in cases where providers underperform, KBOP would not “automatically point to the contract”. Performance issues have been addressed in a flexible way through (what the KBOP team refer to as) ‘informal action plans’ which sit as a preliminary stage to a formal action plan.

The Chair of the KBOP board described the potential procedure as such:

If they’re not achieving their targets, we will require them to come up with some sort of performance improvement plan, that can become a formal performance improvement plan. If they fail to achieve an improvement in their performance on the back of that, then we might take volumes away from them, we might move volumes between different service providers, we might review the resources that they get, and ultimately, we might remove them from the service altogether.

In practice, the investment fund manager describes a relational approach in dealing with under-performance. To August 2022 (when fieldwork was conducted), the KBOP social prime had only issued two formal performance improvement plans; one to a provider with whom the contract was eventually terminated based on mutual agreement. However, according to the KBOP leadership, prior to applying a formal action plan, KBOP attempted to resolve performance issues with the provider on an informal basis. KBOP management only considers the use of formal contractual measures as the last resort. The KBOP project director explained:

You should never get to that point where a provider isn’t able to meet their KPIs because you should be working with them, you should be looking at what their model is, where it could be improved, collaborating to problem solve and overcome any challenges identified, always looking at how you can support them […].’ Contractual clauses for performance improvement signal to providers how seriously outcomes achievement is taken.^{[footnote 27]}

The KBOP leadership emphasised that, in principle, their approach to performance management is characterised by flexibility, trust and collaboration, rather than formal contractual performance management processes.

4.4 The outcomes contract as an emergent example of a formal relational contract

A key theme emerging from interviews across all interviewees is the importance of relationships of trust beyond formal contract terms. The Investment Fund Director explained:

The contract gives us […] a framework to work within, but actually, we’re not going to […] point to it every week. This less formal approach only works if you have really good relationships. […] I think it needs to be both parties going in on the understanding that the most important bit to this is the relationship and the contracts provide a framework for how that works

Understanding the KBOP social outcomes contract as a “framework” to enable “relationships” aligns with the notions of formal relational contracts. Formal relational contracts have been advocated as a contractual model for complex private sector contracting, such as supply chains (Frydliner et al. 2019). In contrast to ‘traditional’ transactional contracts, formal relational contracts are designed with the intention of aligning interests across parties, and creating a partnership culture which fosters a “vested interest in each other’s success” (Frydlinger et al. 2019). Formal relational contract design “recognise[s] that a higher level of trust between the partners should help to make sure partnerships run as intended”. Thus, these contracts often make “the building and maintenance of trust explicit” (Ball and Gibson 2022, p. 9). In this vein, the KBOP social outcomes contract requires the parties to “develop a close working relationship […] on all appropriate levels, based on openness and trust […]’ (KBOP social outcomes contract, Section 2).

However, the evaluation of the KBOP SIB suggests that, here, relational practice incorporates a blend of formal structures (i.e. a governance framework and a performance management system) and informal relational practice (e.g. collaborative leadership). The KBOP service manager reflected:

I think the way the [social outcomes] contract was introduced and launched is very different to how it looks now. The language, the operations manual, the rate card has changed significantly, so the experience of what it was like in that initial start-up phase is very different to how it feels now.

Findings suggest that the outcomes contract offers a malleable space within which stakeholders can build relationships and develop a collaborative environment to address complex challenges of public service delivery. In the following, the report explores how stakeholders shaped and expanded this contractual space through practice.

5. The SIB’s governance

Governance refers to all mechanisms ensuring the overall direction, control and accountability of the organisation (Cornforth & Chambers 2010). In addition to its strategic and control functions, governance is about managing relationships (Zahra & Pearce, 1989). SIBs often involve governance and oversight structures to coordinate the actions of diverse actors, ensure alignment over the course of the programme, course correct where needed, and for performance management (Burand, 2019). Hence, governance provides an important lens to analyse how the different stakeholder interests are reconciled or prioritised in a SIB and thus shaping frontline staff practice. This section investigates the structural and process characteristics of the KBOP SIB governance. A full discussion of the governance arrangements is provided in Annex C2

5.1 Formal governance structures

With the transition to the KBOP SIB model, the governance of the service was re-designed and significantly expanded. To convene the different stakeholders a series of new meeting forums have been created (see table 3). These forums either serve strategic purposes – such as the KBOP board meeting and the council meeting – or operational purposes. The KBOP social prime, an entity which has been specifically created to manage the contracts and relationships between the different SIB stakeholders, is a central facilitator to these forums. While the social outcomes contract outlines some governance structures and associated reporting obligations, governance arrangements have evolved over the course of the project and the redesign of the governance arrangements is largely left to the discretion of the KBOP project director.

Table 3: Governance structure of the KBOP partnership^{[footnote 28]}

Strategic Meetings

Meeting description	Key function	Members	Frequency
KBOP board	Ensuring compliance towards social investors Subject: investment management agreement	KBOP Chair, director of social investment fund, investment director of social investment fund, KBOP project director	Monthly
Council meeting	Ensuring compliance towards the commissioner (council) Subject: social outcomes contract	Investment director of social investment fund, council contract managers, KBOP project director, KBOP service manager, KBOP data and operations analyst, LCF project officer	Monthly

Operational Meetings

Meeting description	Key function	Members	Frequency
Individual provider performance review	Ensuring compliance towards the social prime (KBOP) & capacity-building Subject: individual bi-lateral provider contracts	KBOP service manager, KBOP data & operations analyst, provider service manager	Monthly
Individual provider performance review (senior team)	Ensuring compliance towards the social prime (KBOP) Subject: individual bi-lateral provider contracts	KBOP project director, senior provider service manager	Quarterly
Collective provider performance review	Ensuring compliance towards the social prime (KBOP) Subject: all provider contracts	KBOP project director, KBOP data & operations analyst, most senior provider leads	Quarterly
Operational change management	Provider empowerment: advancing service delivery Provider collaboration: facilitating social interaction and sharing of best practice	KBOP project director, KBOP service manager, KBOP data & operations analyst, provider service managers, provider team leaders	Monthly
Personalisation working group	Provider empowerment advancing person-led service delivery	KBOP project director, KBOP service manager, KBOP data & operations analyst, mix of provider staff (personalisation champions)	Monthly
Co-production forum	Integration of people with lived experience in service design	KBOP service manager, KBOP engagement coordinator, people with lived experience from Kirklees services	Monthly

5.2 Governance process: collaborative leadership

A key aspiration of the KBOP SIB was to foster better collaboration across the delivery partners by dedicating a particular resource, namely the KBOP social prime, to network management. This section examines the different roles practised by the KBOP operational management (i.e. the project director and service manager) to lead the partnership, using a collaborative leadership framework from the academic literature (Ansell et al., 2012).

The framework distinguishes three collaborative leadership roles: the steward, the mediator and the catalyst. The KBOP operational management applies components from all three leadership roles. These roles are not clear-cut, and so the same process might fulfil multiple functions.

5.2.1 The stewardship role

A steward is ‘someone who facilitates the collaborative process by establishing and protecting the integrity of the collaborative process itself’ (Ansell et al., 2012, p.6). The role of a steward (distinct to the function of market stewardship, outlined above) involves lending reputation and social capital to convene the process. The professional background of the KBOP project director and service manager in frontline management in the same policy field as the KBOP SIB, combined with their previous SIB delivery experience and their knowledge of the local commissioning context, were central to unlocking legitimacy from the delivery organisations. The interim project manager, in contrast, did not possess these reputational attributes and thus struggled to gain legitimacy.^{[footnote 29]} The interim manager was less familiar with the local context. Moreover, interviewees indicated that in the early stages of mobilisation, leadership was associated with a narrow performance focus. This was positioned in contrast to the current KBOP management team’s focus on service users and asset-based practices.

The current project director’s social capital is understood as critical to develop and advance the KBOP SIB partnership. Because of their professional background, the project director was aware of the issues VCSE organisations were encountering in public service delivery and understood the likely challenges in implementing change using only pre-existing governance processes. In addition, the project director’s own SIB management experience, and access to the peer network for SIB project managers facilitated by the investment fund manager, allowed the project director to transfer organisational learning. The project director highlighted the importance of social capital:

At KBOP we’re working with charities and third sector organisations so it’s important to me that I maintain my networks because that enables me to have the balance of still having practical knowledge of the sector and expertise leading on behalf of the investor. I think the strength of KBOP is that we as a team have practical experience of how it works in charities and public organisations, how their structures and processes operate and how their funding mechanisms function. We have experienced the frustrations they might have in terms of the rigidity around organisational governance and contract specifications and that helps me to balance and adapt how I co-design or mobilise changes and implement our approach so it meets their requirements.

Further, the role of a steward involves ensuring the inclusiveness and transparency of the governance process. The project director described a structured approach to ensure inclusiveness by creating various meeting forums with representation from different organisational levels (see section 5.1), which are expected to serve as ‘collaborative problem-solving’ spaces and create a sense of shared ownership. While the governance structure is inclusive in the sense that providers are also involved in the strategic meetings, i.e. the KBOP board meeting and the council meeting, it is questionable to which extent these meetings offer a genuine space for providers to influence the decision-making process, as providers only attend on rotation on a bi-annual basis and only participate in a reporting function in the first part of the meeting. Findings from interviews create a mixed picture: while some provider interviewees experienced the meetings as an opportunity to have constructive discussion around operational challenges, others considered it very much a “questions & answers sort of thing”, indicating a superficial dialogue between stakeholders.^{[footnote 30]} However, it is important to note that there are other forums, such as the operational change management forums, which offer a feedback venue for providers.

The project director stressed the practice of open communication from KBOP management with the providers:

Although there is a very clear contract management process in place… where there’s very clear accountability and reporting and governance, it is all done in a very open, honest and collaborative way. So, it’s very much “no surprises” that we play under. So, because there is such open communication, and such strong relationships, and contact with all of our partners, it should never be that anything happens which is unexpected for them or for us.

Findings from provider interviews generally uphold this description of openness. However, one participant indicated a missing feedback loop, meaning that the KBOP management does not consistently communicate how providers’ views have been incorporated into strategic management decisions. Another participant indicated that KBOP management informs providers at a late stage about planned operational changes.

Finally, the role of a steward involves managing the identity of the collaboration. The introduction of a person-led strengths-based approach articulates a guiding set of values that underpin support and to which all stakeholders could relate. In addition, the project director used the meetings to direct attention to the ‘small wins’. Acknowledging successes allowed providers to realise the link between their organisational progress and the wider purpose of the social outcomes contract.

5.2.2 The mediator role

A mediator ‘helps to arbitrate and nurture relationships between stakeholders’ (Ansell et al., 2012, p. 6). The KBOP operational management acts as a crucial interlocutor and mediator in governing the collaboration across all stakeholder groups (council, fund manager, and service providers). In relation to the board and the council, the KBOP management team advocates for the concerns of the providers. For instance, they have sought to secure the support of the council in reducing systemic barriers such as addressing the shortage of subsidised housing. Providers experienced the KBOP project director and service manager as proactive and reliable, thus acting as an advocate for their concerns with the council and the investment fund managers. A service manager remarks:^{[footnote 31]}

When we talk collectively as an operational team or as just as a strategic group, we’re all encouraged to provide that feedback, and we’re all encouraged to come up with ideas and share best practice. [KBOP project director] and the team will go away and influence what they need to in order to enable us to do what we’re suggesting.

KBOP operational management is responsible for developing a constructive collaboration across the provider partners. Our research identifies two key processes through which this is developed: the structural framework for collaboration and the process of ‘inspirational motivation’.

The structural framework entails creating a venue for the social interaction of providers to facilitate trust-building and operational co-operation. As outlined in table 3 the formal governance structure of KBOP includes various provider group meetings, involving stakeholders from different organisational levels. The process of ‘inspirational motivation’ involves communicating a stimulating vision and raising awareness on the importance of organisational values and outcomes (van Wart 2003). The following statements illustrate deliberate use of ‘inspirational motivation’ by the KBOP project director:

My vision for KBOP is around the complete collaboration. All of us are working towards the shared goal of enabling the individuals we work with to overcome current challenges and achieve anything they want to.

We tried to make sure that we had the shared vision in terms of what we were doing, why we were doing it and how we were doing it after identifying that there was a real disparity across different teams in their understanding of outcomes contracts. We developed an operating framework ensuring everybody understood the ambitions for the programme, the operating models within it and our social purpose.

KBOP’s vision is to enable the service participants to live up to their individual aspirations. While service providers could easily relate to this vision, KBOP management needed to secure buy-in for the pre-defined service outcomes and their evidence requirements. As such the collaborative re-design of the rate card helped to build ownership across providers. The asset-based approach is a fulcrum and articulates a shared set of values, and thus created a motivation, in particular across frontline staff, to support the delivery of the outcomes contract.

5.2.3 The catalyst role

A third distinct leadership role exercised by the KBOP operational management is that of a catalyst. The catalyst is ‘someone who helps stakeholders to identify and realise value-creating opportunities’ (Ansell et al., 2012, p. 6). KBOP management use meetings with providers to jointly generate new ideas to improve service management and provision. Securing financial resources from the KBOP board, providing technical implementation support and building relevant external partnerships, are further measures taken by KBOP management to support providers’ service development ambitions.

KBOP management proactively stimulates and encourages providers to re-frame their delivery approach. For example, providers have been stretched to consider the inclusion of education, training and employment (ETE) outcomes and the compatibility of person-led support provision with an outcomes-based contract. This balance of encouragement and challenge is reflected in the following statement from the KBOP project director:

It’s very much working with each partner at their pace, understanding what they want to achieve and how we can get them there, while at the same time improving quality and impact or questioning as appropriate. Part of our role is gently challenging the way that people think, services or how systems operate to support ambitions for growth. We have conversations with them when we identify opportunities, ‘you’re so strong in this area. Do you think this is an area where you would like to do more?’

While provider interview subjects frequently described the social prime’s operational leadership as ‘supportive’ and valued the opportunity for organisational growth, the process does not always grant providers full autonomy. At times KBOP management could be described as quite prescriptive in driving change in organisational practice. Hence, in acting as a catalyst, KBOP management ensures that provider development ultimately benefits the partnership.

5.3 Governance process: control versus empowerment

To analyse governance in cross-sector public service delivery, the academic literature frequently applies agency and stewardship theory (e.g. Van Puyvelde et al., 2012; Van Slyke, 2007). Agency theory assumes goal divergence on the part of the contracted agent and stresses the use of control-oriented processes. In contrast, stewardship theory presumes convergence due to shared collective interests with the contracted steward and emphasises the importance of empowerment-oriented processes (Van Slyke, 2007). This section examines to which extent the KBOP management uses control-oriented processes versus empowerment-oriented processes, or a complementary use of both processes, in the SIB governance.

The KBOP contract management features a strong monitoring practice reflected in performance review meetings and providers’ reporting obligations see section 6.2.2. However, the description of these meetings by interviewees also revealed more of a partnership approach, where the interaction was based on trust, rather than a transactional principal-agent dynamic. Providers felt that they could have honest conversations with the social prime’s operational management addressing concerns and areas of underperformance. They described the meetings as ‘informal’ and highlighted a ‘willingness to engage in dialogue’ by KBOP management. Ultimately, the social prime did not seem to use sanctions, but a supportive, conversational approach to improve organisational performance. A service manager explained:^{[footnote 32]}

I don’t feel that I go into a meeting dreading it. I don’t feel like come out of it deflated, where we have had issues underperforming. Where we’ve struggled with certain KPIs, KBOP was very supportive around that.

Empowerment-oriented governance practice is identified in some of the interview data. The project director considers the service providers as the ‘core’ of KBOP partnership explaining:

What fundamentally underpins the management approach within KBOP is that relationship, collaboration and trust is very much ‘we are a partnership’. We may be… the contract holders within this, but KBOP is nothing without our partners….

The delivery organisations described the social prime’s managerial behaviour as ‘listening’ and ‘being accessible’. This behaviour supported the trust-building between providers and KBOP management. KBOP management practised a bottom-up and collaborative approach to the re-design of the rate card, by consulting frontline and managerial provider staff over an extended timeframe. The participatory nature of the process was also acknowledged by the providers:

There is very much a willingness to work together, to listen to the experience from the frontline, and from the first line managers as to what actually is the real experience out there.^{[footnote 33]}

Alongside this, KBOP management was seen to create possibilities for providers to influence strategic decision-making: KBOP management transferred responsibility to providers by supporting their ambitions for organisational growth.

However, it remains unclear to what extent the partnership governance features genuine power-sharing between KBOP management and the providers. Conceptually, we would expect a partnership among equals to be detected via two governance processes – consensual decision-making on issues concerning the whole partnership and autonomous decision-making for providers on issues concerning their individual organisation. While findings indicate that providers’ concerns do inform decision-making, it is important to note that there is a power imbalance between the providers and the social prime as the contractor. At times there is a perception that the social prime might prioritise its interest to improve the SIB service whilst overlooking providers’ wider organisational imperatives. For example, this tension was reflected in an instance when a provider was required to re-structure its service management:^{[footnote 34]}

I think one of the things that we’re still working on is how much control we have over our own service delivery. We have had to make some immense changes. We are merging the housing support delivery and the learning delivery teams, but that is purely for the benefit of the KBOP contract. It’s something that KBOP contract managers started and the board itself have pushed us very firmly in the direction of doing. It really just benefits KBOP, but actually it doesn’t necessarily benefit other things that we do, and it’s had an impact right across the organisation…

In terms of providers’ operational decision-making autonomy, the KBOP operational management team are understood to exercise a far more hands-on contract management approach than the council in the preceding fee-for-service contract. A senior operational manager noticed:^{[footnote 35]}

And there are daily conversations and daily communications from the KBOP hub, to those teams who are working on the ground around performance, around data, around ways of working, new initiatives, policy implementation. It’s a very hands-on form of management and that brings its own challenges to us as organisations.

Organisational size and contractual performance emerged as potential moderators which influence providers’ ability to navigate the interactions with the social prime’s operational management. The proactive intervention by the social prime in providers’ individual operational management is paradoxical to the implementation autonomy granted by the contract and the developmental culture promoted by KBOP management. Overall, KBOP management apply a blended approach borrowing from both control and empowerment-oriented processes.^{[footnote 36]}

6. The causal mechanisms of the KBOP SIB: four hypotheses

This section seeks to explore whether the four hypotheses developed in the first interim evaluation report lead to a shift in contract management and frontline delivery. The four hypotheses underpinning the SIB mechanism are an enhanced practice of:

1. market stewardship

2. performance management

3. collaboration across providers

4. flexibility in implementation.

The analysis considers positive and negative changes, compared to the preceding fee-for-service contract. The first interim evaluation report entails a detailed analysis of the implications of the fee-for-service contract on management and delivery.^{[footnote 37]}

The analysis of the individual mechanisms concludes with a figure summarising the two commissioning arrangements. This includes a description of the issues associated with the fee-for-service model and supporting evidence; and a description of the facilitators (i.e. the mechanisms) in the SIB model, supporting evidence and drawbacks. Importantly, all findings need to be validated in a further, final research wave and so these findings are preliminary.

6.1 Enhanced practice of market stewardship

Public service commissioners are expected to create the conditions for an effective market of providers. Following the introduction of the outcomes contract, a marked shift in market stewardship appears to have taken place. The ‘rules of the game’ have been reoriented by the commissioner to place greater emphasis on person-centredness and long-term outcomes. The availability of the Life Chances Fund top-up funding and the involvement of the fund manager is understood to have enabled a more substantial and dedicated team to augment the practice of market stewardship. As a social prime, KBOP takes on parts of the market management work: developing market intelligence and service insights and guiding the alliance of delivery organisations with a range of formal market management interventions (managed provider exit; market share shift) and developmental support (provider training and development around asset-based working).

Local government staff tend to use the term “market sustainability” to refer to this process of managing provider entry, service quality, access, suitable options for service users and (in some cases) provider exit (personal communication between research team and Kirklees Council staff). The task of market stewardship can broadly be summarised under two headings (Broadhurst & Landau, 2017):

1. Market intelligence and service insights – activities that seek to understand the needs, objectives and enablers of successful delivery and provide data on the market, such as provider stability, relative performance, and the ‘health’ of the provider market

2. Market influencing – the activities that influence the current and future range of care and support available, or what is sometimes referred to as the “rules of the game” (Gash et al., 2013, p. 6). This refers not only to local care and support provision, but also to commissioning and social work practices including brokerage, funding, accountability mechanisms and communication between the local authority, partner agencies and individuals in the wider market (Broadhurst & Landau, 2017).

These two concepts of ‘market intelligence’ and ‘market influencing’ are used to structure themes that emerged during participant interviews and observation, particularly to describe the key shifts that have occurred.

6.1.1 Market intelligence and service insights

Following the introduction of the outcomes contract, the work of gathering service intelligence and influencing the provider market is now shared between the council and KBOP social prime.

There are both technical and relational elements of the market and service intelligence function that have shifted since 2019. Most obviously, in terms of technical infrastructure, the CDPSoft system is used by the council for monitoring and validating outcome payments; by KBOP for managing referrals of potential programme participants and underpinning performance management data on relative provider performance; and by providers themselves for collecting and retaining key case management information.^{[footnote 38]}

The establishment of CDPSoft and the KBOP referral hub are understood by both council staff and KBOP management team to have substantively improved the quality of insights available. The council contract team use CDPSoft regularly and flexibly, not just for the validation and administration of outcome achievements, but also to investigate and understand other facets of service quality. This software effectively makes visible (in stark contrast to the previous fee-for-service contract) granular service performance data on a variety of indicators that are understood as quality markers by the council. Indicators used by the council to understand the service include:

• time taken between initial referral and support commencing with a named provider
• waiting lists
• characteristics of KBOP participants, including an understanding of those who have been declined access to the service
• duration and frequency of support
• outcomes achievement

Council contract managers confidently use query functions to produce specific reports in response to requests for insights – for example, investigating all participants who have been referred to the KBOP service multiple times.

The KBOP project director describes the benefits of this intelligence:

[We are] using our evidence and data to strengthen what we already know on an operational basis, therefore influencing the commissioning and the systems change.

Speaking with reference to one delivery organisation:

they can look at the trends coming through in terms of the women they’re working with, to invest in long-term outcomes for those individuals to prevent victimisation. We can gather the evidence, tracking the impact and tracking the benefits of working with women on a longer-term basis.

Under the SIB commissioning arrangement, the KBOP social prime takes on the role of referral hub and in this guise also functions as an important conduit, both generating and responding to service intelligence. The KBOP service manager explains:

We’re also monitoring capacity with all of the delivery partners to make sure that people can get the support that they need as quickly as possible…We constantly look at caseloads and how many new starts have commenced with each of the delivery partners, so that we can send the referral to the service that they’ll get help from the quickest.

When reflecting on the differences between the KBOP SIB arrangements and the preceding bilateral fee-for-service contracts, the council team commented on the considerable resource and ‘infrastructure’ that now goes in to supporting the intelligence and market management function. This features a broader scope than the previous council contract management. The senior council contract manager notes:

I think if we had that many staff, we would probably have been able to manage the relationship [i.e. the service provider contracts]. The problem …was that we had nowhere near that resource to be able to focus that much on performance and quality management at all. So, it’s part of that infrastructure question as well, isn’t it? … Well, does that infrastructure add value?

Providers also report that intelligence is used proactively by the social prime to improve the match between service participants and their respective providers. In terms of service selection, the KBOP service does not explicitly offer service participants a ‘choice’ of provider. Staff members who facilitate the motivational interview that functions as a referral assessment note that they accommodate participant preferences to work with/avoid specific delivery teams. The referral conversation may also reveal information that is used to match participants to provider specialisms. This is particularly the case for people who are overcoming domestic abuse and dual diagnosis.

The Chair of the KBOP board also acknowledges that the service delivery network may benefit from the development of further specialism:

I’d really like to see, as we go on and mature, an understanding of the different strengths of the different organisations so that when someone comes in and we triage them, they are directed to the organisation best placed to provide them with a response to their needs.

This brings us to the consideration of how intelligence is then used for market influencing and shaping. Here the work of shaping the network of providers appears to be shared between the KBOP social prime and the council. The KBOP management team are understood to take a more hands-on role in facilitating the introduction of new providers or service components; active and constructive competition between providers; and orderly exit of service providers. The council is understood to retain an ongoing responsibility for assuring quality and access, and is also involved – through the contract review meetings (see section 5.1) – in considering potential shaping activities. It is the council’s vision of a high functioning, person-centred and outcome-oriented service that has set the backdrop for more proactive service shaping by the KBOP social prime team.

To date, the KBOP delivery network has not introduced any wholly ‘new’ service providers. Interviewees acknowledged that the procurement for an outcomes contract delivery team could have introduced a single, large, ‘outside of area’ delivery organisation, such as a national charity. But there was general support for the continued involvement of the pre-existing ‘alliance’ of organisations who had a history of delivering this service in Kirklees. The KBOP project director explained:

I think at the moment, eight partners are enough. And I think we’re focused on our partnership, achieving that very strong collaboration and working as a unit..

Rather than introducing new delivery organisations, the KBOP partnership have spent time developing delivery pilots and service innovations (Appendix G). The emphasis on asset-based working (discussed in sections 6.4.1 and 6.4.2) is a key aspect of provider development that has been promoted under KBOP.

Dealing with sustained performance issues

Managed exit has occurred since the introduction of the KBOP social prime. The Chair of the KBOP board described this process:

We have in the last 14 months removed one of the service providers who was underperforming and couldn’t turn it round, wouldn’t turn it round, wouldn’t engage. And it was mutually decided that they should exit and we reallocated the volumes elsewhere.

Interviewees and documentation indicate that the escalation of performance concerns occurred incrementally, through dialogue and informal support over several months, although the research team were not able to recruit a representative from the departing delivery organisation to participate in the research. The stages leading up to exit are outlined in an interview with the Chair of the KBOP board:

Their performance just got worse and worse. We worked with them to identify what it is that they could do to fix it. ‘You need to have a KBOP Operations Manager, stop having the person in that position, employ three people in those positions. We’re going to stop sending in new volumes, so the caseloads get down to a manageable level.’… Ultimately, we reached the point where - with their senior leadership - we said ‘You’re just not going to make this work. You don’t seem to want to make it work’ and they go ‘No, we don’t think that we can. We think we’re better, if we just step away from it’.

The task of managing a large and diverse set of provider organisations was understood to constitute a considerable commitment for the KBOP management team. This was associated with some initial nervousness on the part of delivery organisations:

… We thought ‘no commercial organisation is going to want to come on and carry on working with nine delivery partners, their agenda must be to come in and cut us down to three or four quite quickly… So they’re going to be looking at these clauses and finding ways to pick us off, or to get us fighting among ourselves, or something, this must be their agenda.’ That isn’t what’s happened.^{[footnote 39]}

Amongst delivery organisations who had been involved in other SIB projects where there was a more direct relationship with fund managers, there was reflection on the magnitude of the stewardship task:

It was always going to be incredibly difficult because I think it’s probably too big. I think if there were three providers, it would probably be a lot easier. I think there would be more effective working. I think for the KBOP contract managers themselves it must be incredibly difficult trying to coordinate eight very different providers and a huge – millions and millions of pounds worth – of investment at risk.^{[footnote 40]}

Active but constructive competition and ‘coopetition’ between providers is discussed more fully in the section 6.3 on provider collaboration, suggesting greater provider collaboration, but also competitive pressures through enhanced scrutiny on performance. In addition to this, the KBOP social prime adopted a form of ‘market share shift’. When share shift has been operationalised in previous multi-provider payment-by-results programmes in the UK (such as the Work Programme), fixed differential performance levels have been agreed in advance and a higher proportion of new referrals are directed to the better performing provider(s) within a contract geography (Carter, 2018). Within the KBOP partnership, the allocation of potential programme participants is revised iteratively and in consultation with provider organisations. This brings flexibility to connect larger numbers of programme participants to service providers who are understood to be better performing at a particular point in time, but also entails a level of unpredictability for delivery organisations. This allocation process is a potential mechanism through which KBOP drives enhanced performance. One head of services explained:^{[footnote 41]}

…we’re looking to grow that, but you just don’t know what comes along with KBOP. At the beginning of year two they asked us to expand and in October we expanded again slightly by introducing a new senior support coach role, which is a new tier again… You think you’ve settled, and you know what you’re doing and [then] there’s something else [that] comes along. We’ve learned to expect change.

6.1.2 Reflections on the changing practice of market stewardship under KBOP arrangements

Reflections on the introduction of the social prime’s management arrangements indicated some concern about its mediator position, and this distancing was expressed as a potential worry, both by service providers and council staff. Some providers lamented the lack of direct contact with council commissioners, compared to previous direct contracting in the fee-for-service arrangement:

… there’s not actually much interaction with us providers, with either the board or with the council, not in the way that we would have done previously. We don’t directly liaise with, like the commissioners in the council, all that function comes through KBOP now.^{[footnote 42]}

Mirroring this, council staff commented:

I think, informally, we see each other. Usually when you bump into [service provider staff] somewhere random. But I do feel that there is a bit of a disconnect. Because everything gets filtered through KBOP then, doesn’t it? So, they tell us what they want us to know.^{[footnote 43]}

The revised arrangements, including the rotational attendance of providers in the contract review meeting between the council and the social prime, can in part be understood as an opportunity to bring greater coherence to this joint stewardship role for the council and KBOP.

Nonetheless, it was the scale – both geographically and in terms of the level of ambition for change – that was also associated with the potential for enhanced market management to reform and improve services across the Kirklees footprint:

…Let’s say you’re doing something that is spread across different geographical areas, or different service areas, which is more the case in Kirklees. How do you create a contract that works across the whole of that service landscape, which is going to deliver for a commissioner? Because they might want to have a contract that they can both drive specific improvements and encourage integration between different services. I think in that case, something like Bridges or KBOP can play a role in getting services to work in a way that becomes more integrated, in an outcomes focused way, that would be hard to do otherwise.^{[footnote 44]}

6.1.3 Kirklees Council as market steward - shifting the ‘rules of the game’

In pursuing an outcomes-based contracting arrangement and an alliance of well-coordinated service delivery organisations, Kirklees Council staff work with an acknowledgement that the local authority does not directly control the way that services are delivered. Instead, since before the enactment of the revised contract, council commissioners understood that participant outcomes are determined by the interaction of service providers, service users and the imperative or emphasis of the market itself – including overall resource, stability, and incentives.

Analysis of interview material reveals a compelling vision for the ability for an outcomes contract to shift the rules of the game. The KBOP project director articulated a clear expectation that frontline practice is shaped by the commissioning landscape:

There was a recognition from our commissioner that the services were operating with a short-term focus, it was very “crisis resolution”. Prioritising fixing the immediate need, rather than looking at the long-term ambition for that individual… .

Siloed commissioning and short-term commissioning are the two biggest barriers to the success of any programme across the public sector. And that’s the way it has been for the entirety of my career. But until that changes you will never see the long-term systemic changes or social impact within communities because there’s just not enough time to do what you need to do, or services inadvertently work against each other. And then frequently when programmes close, what happens to the individuals when you’ve been working with them during that period? Without that long term planning and integration into the community. They’re the ones that are left.

A question remains as to whether the overall funding available for the KBOP service delivery alliance is sufficient to achieve government’s objectives, namely to overcome service fragmentation; reduce the need for cost-intensive crisis support through the expansion of preventative services; and foster service innovation. Prior to the launch of the SIB, the array of floating support providers had navigated a series of funding cuts and, although the LCF top up funding brings additional financial resource into the system, the current research study is not positioned to respond to overall considerations of resource sufficiency.^{[footnote 45]}

Table 4 below summarises the evidence underpinning data and performance management in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim evaluation report. The KBOP SIB is associated with a more intensive practice of performance management.

Table 4: Summary of KBOP SIB hypothesis 1: ‘Enhanced practice of market stewardship’^{[footnote 46]}

Fee-for-service contract

Challenges	Implications for service delivery	Evidence
Multiple bi-lateral contracts	Over-stretched council’s contract management capacity	Continuous contract roll-over without adjustment of contract terms High-level and infrequent contract auditing No pro-active fostering of service innovations
Short term contracts	Reduced ability to address long-term participant outcomes Uncertainty over funding diverted resources from support provision to service reduction/termination	Outcomes reporting in provider workbooks Participants re-entering services Providers’ observation
Limited data transparency	Inefficient referral allocation (multiple waiting lists, referral self-selection by providers) Limited availability to identify and nurture quality service provision Gaps and duplication in service delivery	Lack of a central referral hub Participants sitting on multiple waiting lists No change in delivery organisations over multiple years

Social impact bond

Facilitators	Implications for service delivery	Evidence
Single outcomes contract	More effective use of council’s contract management capacity Limitations: Limited direct sight of delivery issues	Council has only one monthly performance review meeting with the social prime Council focuses on reducing system barriers and performance verification and review
Long term contract	Enhanced ability to address long-term participant outcomes Enhanced ability to advance referral pathways and support provision Enhanced ability to develop providers’ co-working capacity	Outcomes framework entails long-term participant objectives Changes to delivery practice to support the achievement of long-term outcomes Emerging co-working practices
Improved data transparency	Enhanced efficiency and quality in referral allocation Targeted development of providers (growth and exit) Swifter course-correction ability to address performance issues	Central intelligence system provides live data on referral numbers and providers’ capacity Participant needs to explain its service history only once Central intelligence system displays individual providers’ outcomes achievements Provider’s annual contract volume re-negotiation responsive to performance Provider contract exit due to continuous under-performance

6.2 Enhanced performance management

Under the fee-for-service arrangement, previous research in Kirklees detected limited performance management of providers by the council. This was driven by capacity limits, due to resource-constrained staff managing multiple bilateral contracts and a lack of well-defined success measures. Reflecting on the contract management practice of the legacy contract, a service manager stated:^{[footnote 47]}

The delivery partners all came together from a previous service model that was based on a payment for service model where you were guaranteed your income no matter what quality of service was provided. In the last few years of that service model, I believe the quality of service provision had deteriorated. Kirklees Council’s monitoring of services under that model wasn’t very effective and they used the old Supporting People workbooks which were a pretty basic measure to assess service quality.

Research from other impact bonds reveals that they may introduce a more engaged and disciplined form of performance management, and this may be undertaken by a third party ‘intermediary’ or performance management specialist (e.g. Fraser et al., 2018). In the KBOP SIB, the contract management role for the provider contracts is transferred from the council to the KBOP social prime.

The dedicated resource of the KBOP team created a greater focus on performance management. Further, the payment-for-outcomes mechanism and the introduction of a central intelligence system for participant data and outcomes achievements led to greater transparency on the intervention success. While the tighter performance management made the service impact more visible to providers, they also reported a heightened administrative burden.

There are also counter-arguments to a more “managerialist” approach in public service delivery, such as being too complex and costly, and providing an incomplete picture of service delivery due to ill-defined metrics (Cutler & Waine, 1997). Previous academic studies of SIBs have noted that the target focus diverts attention from the projects’ social goals (Fraser et al., 2019; Joy & Shields, 2013). When providers are granted greater flexibility over implementation whilst their income is dependent on results – termed as ‘financialised discretion’ in the academic literature (Considine et al. 2020) – it can lead to reductions in service quality if pressures ‘to hit targets and minimise costs’ intensify processes of standardisation (Fuertes & Lindsay, 2016).^{[footnote 48]}

Crucial to the introduction of KBOP’s enhanced performance management regime is the establishment of useable management data. The section starts with a description of the rationalisation of the referral approach. This is seen as a pre-requisite for establishing a single, harmonised approach for collecting data on service participants and the achievement of individual outcomes. Next, it examines the new reporting and evidencing requirements. This is followed by an investigation of adaptive management practices of the social prime to tailor different organisational interests. The section concludes by discussing challenges in attempting to create greater accountability through measuring outcomes.

6.2.1 Referral and caseload management

The launch of the SIB model saw a major re-design of the referral process. Under the legacy arrangements, referral agencies referred service users to individual service providers. The referral allocation was often dependent on informal relationships between the referral agent and the frontline worker. Service providers were also able to identify their own referrals and receive self-referrals directly to their specific organisation. The simultaneous referral to multiple providers contributed to long waiting lists. Relatedly, the council was unable to comprehensively understand the demand for the service and it was difficult to assess the number of service participants re-entering the service, though anecdotally there were known issues with a ‘revolving door’ experience.

With the launch of the SIB, the KBOP social prime became the central referral hub. Alongside this, the council introduced a central referral and case management system – the ‘Customer Data Platform Software’ (CDPSoft). Though the software was procured by the council, the perspective and preferences of the KBOP management team were influential in designing the system’s individual features for the service. The system can be accessed by the council, the social prime and the providers.^{[footnote 49]} All participant data is saved centrally in the system. A key referral agency is Kirklees Council Housing Solutions (16% of referrals). Other frequently used referral pathways are custody services (10% of referrals), the KBOP providers themselves referring into the service (11% of referrals) and self-referral (10% of referrals).

Figure 4: Referral agencies who connect people to the KBOP service (September 2019-November 2022) as % of total^{[footnote 50]}

Unknown Most Early Project Referral	24%
Local Authority Housing	16%
KBOP Delivery Partner	11%
Custody Services	10%
Self Referral	10%
Drug and Alcohol Services	7%
DWP	7%
Local Authority Adult Services	5%
Mental Health Services	4%
Children’s Services	3%
VCSE	2%
Education	1%

Source: KBOP social prime

In the initial stages of the SIB programme, providers were still able to self-select service participants. However, the council and the social prime observed practices by some providers which pointed to a potential ‘cherry picking’ practice, as some service users with more complex circumstances tended to remain on the waiting list. Therefore, the social prime, in agreement with the council, decided to take ownership of the referral allocation. Central referral allocation by a social prime is intended to reduce the risk of providers ‘cherry picking’ easier to support individuals from within a wider pool of those eligible and ensure appropriate allocation of participants to provider specialisms. The council team’s verification of referrals suggests an oversight function, as the team are able to monitor and identify specific referral selection patterns (e.g. if participants from particular referral actors are repeatedly declined from the service).

Referral allocation is conducted by the social prime’s ‘triage team’, consisting of an engagement worker, programme administrator and referral assessor. Referrals are allocated according to providers’ expertise, capacity and the complexity of service users’ needs. Providers reported that the quality and efficiency of the referral process improved since the social prime began acting as the central referral hub. This is illustrated in the following quote by a regional head of operations:^{[footnote 51]}

I don’t think that we find ourselves in that competitive arena that perhaps we would have done previously, because you’d be pursuing the same avenues for referrals, because it wasn’t coordinated. You’d have your own relationships with potential referring agencies and it was very much based on what you know and who you know [and] the quality of the relationships that you’ve had. That’s being taken away with that centralised referral function and with KBOP screening those referrals, they will let us know which are the best referrals for us and we will pick up those referrals in line with our KPIs and capacity.

Figure 5: Referral process in the fee-for-service contract and in the SIB

Source: Bridges Fund Management

Service users who participated in the preceding Floating Support service were ‘rolled over’ into the SIB service. For these ‘historic’ users, a slightly different outcome payment was agreed: the council only reimbursed the KBOP social prime for long-term outcomes, as the council had already paid providers under the fee-for-service contract for the achievement of short-term outcomes with these service users.

Besides acting as a referral allocation hub, the social prime had to resolve the issue of a long waiting list of referred service users. Research participants indicated that there is a strong demand for the service. The anticipated number of participants was not clear in advance of the introduction of the KBOP SIB, yet sufficient service utilisation would be critical for the financial viability of the service. The KBOP board chair explains that Kirklees Council was responsible to “provide a certain level of referrals”. This would serve as a “guarantee mechanism […] if they aren’t able to guarantee a certain level that there will be some way that we [KBOP] could be sort of financially compensated for that”.

The implementation of the new central information system allowed the social prime and the council to have a comprehensive understanding of the ‘waiting list’ problem, as now they had a clear sense of numbers of participants across delivery partners. The key mitigation strategy introduced by the social prime was a new model of frontline support provision, splitting support into an ‘active’ and a ‘monitoring’ phase.^{[footnote 52]} In the active phase each service participant engages in more intensive support, whereas during the monitoring phase support is only delivered on an ad-hoc basis. If required, the service user could be re-allocated into the active support phase. The KBOP director explains that this arrangement enables flexible support for a larger number of participants: many remain on the programme for ad-hoc support as their journey towards independence improves. This arrangement also connects to the outcomes payment claim process. The social prime would not be able to claim for payments regarding service users who were re-allocated from the monitoring phase into active support having previously achieved interim outcomes.

Providers worked closely with the KBOP service manager and the KBOP data and operations analyst, who used data from the central intelligence system to identify capacity in frontline staff caseloads and individual support intensity requirements of service users. Providers had discretion on the implementation of the dual phase model. Some providers split support worker roles (i.e. some staff are dedicated to intensive support while other staff – typically with larger caseloads – offer a form of sustainment and follow-on support), while other provider staff applied different intensities of support within their caseload. The key objective of the model was to enable staff to use their time more effectively and to tailor the support intensity to individual circumstances and journeys. While the KBOP management described increasing staff resources by adding 21 caseworkers, 3 ETE workers and 8 further specialist roles, providers still described a lack of staff capacity. A recurring complaint was that “caseload has increased significantly across the board, but the resource hasn’t increased as significantly.”^{[footnote 53]} Likewise, the KBOP referral assessor acknowledged the issue of high demand in relation to service capacity:

We are very busy at the moment and a little break on referrals would be nice, however being this busy does reflect that our service is needed. The delivery partners do an absolutely fabulous job however, and the one thing I would change is having more support workers, recruitment regarding support workers can be challenging.

A new model for responding to high demand for the KBOP service

The following two examples illustrate a new model, introduced by the KBOP social prime, in dealing with high caseloads more effectively.

• Provider A created the role of an ‘engagement worker’ (later labelled as ‘engagement coach’) in addition to the role of a ‘support coach’ (previously labelled as ‘intervention worker’). The engagement worker provides support in the ‘monitoring phase’, i.e. once participants had achieved their initial ambitions as defined in the wellbeing assessments and support plans. The main role of the engagement worker was to follow-up on the sustainment of longer-term outcomes. Additionally, the engagement worker was responsible for re-engaging service users who had disengaged from the service. This was sometimes seen as offering a welcoming invitation back into the service, as participants who had missed appointments or sessions did not always feel comfortable re-engaging. Later changes included the introduction of a ‘senior support coach’, splitting practitioner and managerial responsibilities.

• Provider H reduced the number of service managers as some of the managerial functions were now executed by the social prime; instead, this provider increased the number of delivery staff. However, this resulted in a lack of caseload supervisors:

We had a KPI reduction because of the staffing reductions. During year one into year two, we increased our staff and now we’ve just reduced them back again. One of the reasons for that is because we’ve had to take so much out of the management - the non-delivery staff - to place into delivery staff, there isn’t enough capacity in terms of caseload management. We were falling over because we can’t caseload manage that many members of staff on the ground.^{[footnote 54]}

6.2.2 Reporting and evidencing – the adoption of new data requirements

Under SIB-backed delivery, reporting requirements for providers are understood to have significantly increased. Unlike the legacy arrangements, the reporting by providers is submitted to and saved in the central intelligence system. Monthly reporting obligations for the providers include budget planning detailing, for instance, the necessary support workers in post in that given period.

To evidence the achievements of outcomes, providers have to upload documentary evidence as outlined in the rate card onto the central intelligence system. After the social prime has approved the uploaded evidence, the council verifies and approves each individual outcome claim made by the social prime. Also, 10% of the outcomes data are subject to an additional internal council audit.

Additional service efforts which go beyond the support to achieve the outcome targets are captured in an ‘added value’ form. These are outcomes for which payment cannot be claimed, but which are understood to demonstrate wider impact to the council.

Providers frequently mentioned the additional administrative burden associated with the reporting and evidence requirements for outcome achievements. Some of the evidence requirements were considered difficult to obtain (e.g. proof of housing) or intrusive to participants’ privacy (e.g. wage slip).^{[footnote 55]} Easing the evidence requirements was a major part of the re-design of the rate card; for instance, through the inclusion of self-certification forms as evidence for different outcome measures (see Annex D). Likewise, providers struggled with the monthly reporting obligations. However, providers also recognised the benefit of responsiveness to delivery issues, due to the availability of more timely performance data. Alongside the tightened reporting requirements, some of the providers struggled with the intelligence software despite prior training from KBOP. Some providers were using multiple IT systems simultaneously, since they maintained the use of organisational IT software in addition to the KBOP-specific CDPSoft. While some providers struggled with the application of the new IT software, the council highly appreciated the software for its varied reporting functions.

While service providers readily described the increased administrative burden in their work, research participants also recognised the value in having greater visibility of their achievements. A regional head of operations noted:^{[footnote 56]}

I’ll refer back to the employment, education and training; whilst that was an element of what we did previously, we were never monitored on that…Now we do a monthly report. We explain what’s going well, what’s not gone well, we provide case studies too - I suppose – [to] put the meat on the bones of the data.

Figure 6: Changes in contract management related to referral, support and monitoring

6.2.3 Adaptive management

In the early implementation stage of the KBOP SIB, high caseloads and perceived onerous evidence requirements conveyed a feeling of the SIB delivery being solely target-driven across provider staff. To avoid staff disengagement, the social prime promoted practices that align with the concept of ‘organisational adaption’. There are two key changes associated with the ‘organisational adaption’ undertaken by the social prime: first, making personalised service provision the key objective in service delivery (see section 6.4). Second, incorporating providers’ feedback into changes to the rate card to better align it with the person-centred delivery approach.

In response to providers’ feedback, the social prime made three critical changes to the rate card:

• Easing the evidence requirements. The new rate card included self-certification to reduce the reporting burden.
• Removing the separation between short and long-term outcomes. This initial separation significantly contributed to the feeling of ‘being target-driven’ in the early stages of the SIB, as staff did not perceive the ultimate contribution of the short-term outcomes to the long-term outcomes. In the new rate card, outcomes are grouped around two key themes: ‘meaningful use of time’ (e.g. entry into education and training) and ‘sustained outcomes’ (e.g. sustaining employment for 6.5 weeks, 13 weeks, and 26 weeks). While the overarching outcomes remained the same, statements from the investment director and the KBOP project director suggest that the presentation of outcomes was fundamental to mitigate staff’s bias (and at times, discomfort) in delivering a payment-by-results contract. The investment fund manager explained:

You really need to think about how the presentation of the outcomes will be interpreted by the frontline staff. The way we had done it previously, the way we’d communicated it, created this short- vs. long-term separation. This idea that… I won’t say target-driven, because eventually it is, but in the eyes of frontline staff too much emphasis was placed on these kind of numbers, because of the way it was presented to them, and not actually enough on those long-term outcomes, and the achievement of those targets. […] I don’t think that we had really realised how big an impact that had on how the frontline teams engaged with those targets.

• Removal of needs classification. The initial emphasis on ‘need’ and ‘complexity’ was described as undermining the asset-based approach by the KBOP director. In the reformulated rate card, needs classifications are no longer used, enabling participants to aim for any outcome achievement, irrespective of need.

The organisational preparation and transition for providers was underpinned by capacity-building measures relating to performance management. All provider staff had to participate in training on strengths-based service provision.^{[footnote 57]} Training on data-driven decision-making was conducted to create greater understanding and support for measurement and reporting in an outcomes contract across managerial provider staff (topics covered include ‘identifying key questions using data’, ‘gathering data’, ‘analysing data’, and ‘making data-based decisions’). These capacity building measures reflect the objective of the social prime to create a culture which is person-centred ‘and’ (instead of ‘but’) outcomes focused. This tension will be explored in more detail in section 6.4.2.

6.2.4 Accountability and its challenges

In the KBOP SIB, the enhanced scrutiny on providers’ performance dramatically enhanced the experience of data-driven accountability for providers, which was positioned in stark contrast to their previous contract management experience by the council. A service manager reflected:^{[footnote 58]}

I think services are definitely much more accountable. There’s no hiding place. You can’t hide within this contract because everything you do, [the Social Prime Data and Operations Analyst] knows what I’m doing. There’s nowhere to hide. There are no tricks, it’s just there in numbers they can see what we’re doing and they can see in conversations and how things get written in CDPSoft, conversations that people have.

Again, providers appreciated being better able to demonstrate the impact of their work with service users. A data analysis manager explained:^{[footnote 59]}

… but I do completely understand the merit of being able to report and capture the data and for want of a better term, to prove our worth, to prove how well we are working with those that are accessing our services and to have a bit more accountability there.

Despite a perception of greater visibility on progress and achievement, other interviewees questioned the extent to which intervention success is fully measurable. An external consultant who advised the council in the early stages of the rate card design reflected:

People in different parts of the service will often be facing a range of issues… mental health, domestic abuse and so on. Do you really want to measure your success for those groups solely in terms of whether they get employment and whether they sustain accommodation? …We’re quite often going into uncharted territory in terms of anybody ever having measured how successful you could expect an intervention in some of those areas to be.

Table 5 below summarises the evidence underpinning data and performance management in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim evaluation report. The KBOP SIB is associated with a more intensive practice of performance management compared to the preceding fee-for-service contract.

Table 5: Summary of KBOP SIB hypothesis 2: ‘Enhanced performance management’^{[footnote 60]}

Fee-for-service contract

Challenges	Implications for service delivery	Evidence
Misaligned metrics	Key performance indicator (KPI) did not induce longer term participant outcomes such as entry into employment	A focus on the KPI ‘service utilisation’, ensuring that the service was used to capacity
Lack of consistent metrics	Contributed potentially to the high number of participants re-entering the service	The KPI ‘independent living’ allowed for flexible success interpretation by providers Auditing based on self-certified evidence
Capacity constraints for service monitoring	Providers were able to convey the impression of quality against tick boxes	Process-driven quality assessment Infrequent performance reviews

Social impact bond

Facilitators	Implications for service delivery	Evidence
Payment-by-results mechanism	Formal outcome verification process with clearly defined payment metrics and evidence requirements over sustained period of time Transparency and accountability for intervention success Limitations: Administrative burden for frontline staff to secure required evidence Occasionally required evidence could constrain the frontline staff - participant relationship Risk of diluting the transparency of the outcome verification process through the partial introduction of an outcome self-certification process	Pre-agreed rate card defining service outcomes, payment metrics and evidence requirements Council audits every claimed outcome achievement and spot-checks evidence with other council data
Dedicated resources	Social prime providing dedicated resources for performance management Enhanced focus on performance and swifter response to delivery issues Limitations: Administrative burden of meeting and reporting obligations Resource inefficiencies due to partial duplication in reporting requirements	Frequent performance review meetings with the social prime team Introduction of a central intelligence system for data and performance management Re-design of the providers’ staffing model
Data management infrastructure	Introduction of a central intelligence system for data and performance management Transparency on intervention success Enhanced participant data support improvements to service quality Limitations: Administrative burden for providers whose internal data management system differs from the KBOP system	Providers’ monthly performance reports On-going changes to service delivery

6.3 Enhanced provider collaboration

In the preceding fee-for-service contract^{[footnote 61]}, research identified the disconnected or ‘siloed’ delivery of each service provider, with very limited joint working or coordination. A service manager reflected:^{[footnote 62]}

I think prior to KBOP, I think we all tended to work in our own individual silos as organisations and we didn’t have that much collaboration.

This siloed working was described as partly influenced by contractual levers. The KPI ‘utilisation’, in conjunction with a decentralised referral system, induced  competition for service users. A further barrier was the limited infrastructure for co-working. This section examines the development of cross-provider collaboration under the KBOP SIB model and provides some initial insights on the implications of the KBOP arrangement for provider join-up. Providers reported that collaboration improved under the SIB and felt that the social prime brought a “collaborative ethos” to the partnership.^{[footnote 63]}

A facilitator to collaboration is mission alignment through a shared outcomes framework, creating a sense of mutual accountability across providers. A further facilitator is the reduction of systems and process barriers, i.e. reducing competition for funding through the set-up of a personalisation fund and preventing competition for referrals through the centralised referral allocation. Finally, the KBOP social prime created and facilitates a collaborative infrastructure, with dedicated convenings and resource to improve cross-provider collaboration.

6.3.1 The development of collaboration

Two features of the KBOP SIB model emerge as critical to developing collaborative relationships between providers: the social prime and the shared outcomes framework. While evidence consistently endorses the benefit of creating a central coordinator role for the partnership, findings suggest a tension between the adoption of a shared accountability for outcomes through the social outcomes contract and individual contractual KPIs which are set at the provider-level and which may intensify competition, partially diluting the collaborative ambitions of the social prime. Despite shared accountability for outcomes through the social outcomes contract, the benchmarking through the individual KPIs fostered providers’ competitive spirit. Importantly, this did not seem to impede collaboration. However, since providers remain competitors for other local tenders, they were occasionally hesitant to share best practices.

The role of the social prime

A major shift that coincides with the adoption of the outcomes contract is the creation of the KBOP social prime, a separate entity dedicated to managing the providers and setting up a collaborative infrastructure.

To improve collaboration across providers, findings suggest two central elements – leadership and infrastructure. A collaborative leadership approach (see section 5.3) is described in interviews with service providers and other stakeholders. The project director of the social prime stressed that a key focus of KBOP is “fostering a partnership and getting that very strong collaboration and working as a unit”. The director of the investment fund recognised that building collaborative practice required leadership capable of nurturing positive relationships so that provider staff are willing to contribute to the SIB project:

Otherwise, if you start getting loads of negativity that ultimately results in worse services delivered to the frontline.

Secondly, building collaborative infrastructure to increase capacity for co-working involved regular operational meetings which sought to build trust as well as collectively address barriers and define partnership goals^{[footnote 64]}. In addition, regular training facilitated by the social prime offered an informal opportunity for provider exchange and collaboration. While the majority of providers acknowledged the importance of regular meetings, findings suggest that the frequency of meetings created an additional administrative burden which may potentially divert resources from actually implementing collaborative work.^{[footnote 65]} An additional impediment to providers’ effective joined-up working were the high caseloads and perceived insufficient staff resources; a service manager explained:^{[footnote 66]}

I think a lot of the reasons [that] it probably hasn’t transpired is because the caseloads are so high… There isn’t really much breathing space or flexibility to be able to do [joint work]. There’s not enough resource, it doesn’t feel like to be able to [do] joint work, because everyone is quite at capacity.

The role of a shared outcomes framework

A common set of outcomes for all providers is another key point of difference between the SIB and the fee-for-service arrangement. Findings suggest that a shared set of measurable outcomes enhanced provider collaboration through goal alignment and by creating a sense of collective success. A regional head of operations explained:^{[footnote 67]}

If one partner does well, we all do well. I’ve always thought this partnership will succeed if we all succeed.

This finding corresponds to the academic literature which underscores the importance of goal alignment to foster collaboration among partners, foremost in a context characterised by differences in organisational experience and culture (Caldwell et al. 2017). Further, interviewees recognised the benefit of mutual accountability across the provider group associated with a regular performance monitoring of the outcomes. This was reflected in the following statement by a provider’s senior operations manager:^{[footnote 68]}

I can see that we are working more consistently as a group of providers, [which] I think is a benefit. Because it helps with a benchmarking and an expectation around what we’re delivering. And that helps with a consistency of the service and the level of service and the quality that we might expect. Whereas I don’t think that there was any mechanism for that with the group of contracts previously.

Moreover, an operations manager remarked that the shared outcomes incentivised providers to develop their specialisms and thus allowed for “far greater successes than we would have [had as] individual organisations”.^{[footnote 69]}

Competitive pressures

While the social outcomes contract, held by the social prime, introduces a harmonised set of outcome measures for all providers organisations, each provider organisation is still subject to a bespoke set of KPIs in their service delivery contracts. The performance of the individual provider on the KPIs is benchmarked against the performance of the other providers in the monthly meetings with the council and in the quarterly performance review meetings with senior provider leads. While there is evidence in the literature indicating that reputational effects of benchmarking through public reporting can improve performance (Bevan et al., 2019), our findings suggest that the benchmarking also creates competitive pressures and thus might impede collaborative practice in some instances. A head of services noted:^{[footnote 70]}

Because the nature of this type of project is you’re very aware of everybody else’s performance. Performance is something that’s presented to you in the meetings, so you’re very concerned about your performance against that of other providers and I think that that can have a negative effect on the collaboration between providers. I don’t see it as a motivator. We are already motivated to do well.

This comment mirrors a wider debate in the academic literature on SIBs which considers the tight performance management as an inhibitor to collaboration (Tse & Warner, 2020). KBOP attempted to mitigate internal competitive pressures on multiple fronts, for example, reducing competition for referrals by introducing  centralised referral allocation and reducing competition for funding through the establishment of a personalisation fund.^{[footnote 71]} Nonetheless, beyond the KBOP contract, some of these delivery organisations are large national charities and housing associations and thus remain competitors for contracts beyond the KBOP SIB. The funding insecurity for the KBOP service beyond its five-year contract duration intensified the perception of a competitive operational environment linked to open tendering. This implies that there was still a hesitance to share information and best practice which would be considered by providers to create a competitive advantage.

6.3.2 The implications of collaboration

Although it is difficult to evaluate the precise implications of collaboration on service delivery at this point, not least given the lack of granular historical data, overall interviewees felt that the KBOP partnership addressed some of the most pressing barriers to collaboration that characterised the fee-for-service model.

Increased collaboration supported the sharing of knowledge, best practice and resources. A provider’s senior operations manager highlighted:^{[footnote 72]}

The very nature of those relationships just means that there should be added benefits to working more in partnership, like maybe we can access some of their other services, or like we can tap into some of the training that they may deliver, or, you know, those are the kind of the added benefits that those partnership working arrangements should bring.

The partnership work was facilitated through cross-provider operational meetings. In addition, the social prime’s project director purposefully matched providers – using the data from the central intelligence system – to disseminate best practice through the replication of staffing models and/or delivery practice. Increased collaboration resulted in providers complementing their individual support expertise by supplementing this with the specialism and approach of other delivery organisations. According to providers, this enhanced the quality of support provision, allowing them to deliver holistic wrap-around support for each individual service participant.

Increased collaboration also helped to address systemic contextual challenges such as barriers to employment and housing. For instance, all providers had access to a KBOP provider’s Tenancy Deposit Scheme, enhancing access to accommodation for the whole cohort. Another example is the co-delivery of training sessions to improve provider capacity in achieving ETE outcomes.

Finally, increased collaboration fostered a sense of a shared mission between providers. The following statement by a regional head of operations is reflective of mission alignment:

It was introduced as one contract e.g. ‘if one provider fails, the rest of us do’. Because it’s a partnership, we’re all working towards that same common goal, which we should be anyway and we are anyway.^{[footnote 73]}

This chart summarises the evidence underpinning the collaborative practice across service providers in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim evaluation report. The KBOP SIB signals an enhanced collaborative practice across providers compared to the preceding fee-for-service contract; a final assessment of the degree to which this change is attributable to the SIB as a commissioning tool is to be explored in the final process evaluation report.

Table 6: Summary of KBOP SIB hypothesis 3: ‘Enhanced collaboration’^{[footnote 74]}

Fee-for-service contract

Challenges	Implications for service delivery	Evidence
Contractual and system barriers	Key performance indicator ‘service utilisation’(ensuring the use of service to capacity) and the decentralised referral system induced competition for referrals Limited development of co-working culture due to initial grant conditions impeding co-working Diversion of resources to relationship building/maintenance with referral agents to achieve KPI targets on ‘service utilisation’	Long provider waiting lists with participants sitting on multiple lists Providers’ observation
Under-developed collaborative infrastructure	Co-working took place on an ad-hoc basis and was dependent on individual frontline workers’ relationships	Lack of a streamlined approach to sharing of knowledge and best practice Infrequent provider group meetings

Social impact bond

Facilitators	Implications for service delivery	Evidence
Shared mission	The overarching outcomes framework applies to all providers Goal alignment induces a sense of collective success and accountability, enhancing commitment to joint working Limitations: Perceived competitive pressures in benchmarking providers’ KPIs	Stronger sense of identification with the partnership
Reduction of system barriers	Introduction of flexible funding for personalised support provision (Personalisation Fund) and centralised referral allocation Frees up resources for collaborative support provision	Providers’ observation
Enhanced capacity for co-working	Creation and facilitation of a collaborative governance infrastructure by the social prime Enhances person-centred support provision (e.g. more service offers) Expands skill set of frontline staff Limitations: Administrative burden associated with meeting frequently occasionally diverts resources from the implementation of collaborative projects	Introduction of regular cross-provider operational meetings Established routines for knowledge and resource sharing, as well as joint service advancement

6.4 Enhanced flexibility in service implementation

Under the fee-for-service contracting arrangements, providers found the requirements for service delivery to be highly prescriptive. There was a predefined frequency of contact and length of support. Austerity constrained resources for tailored intervention support and any additional personalised support was subject to bureaucratic, lengthy application processes. The legacy contract did not encourage the pursuit of long-term outcomes beyond housing-related support such as employment. A director explained:^{[footnote 75]}

There wasn’t really any scope for thinking well ‘Why don’t we have unemployment workers as part of this team?’ because that wasn’t seen as housing related support…it was very prescriptive what kind of outcomes you could achieve with people… There was a time … when it would have really been frowned upon if you included ‘find a job’ in the support. It just wasn’t the scope there for those conversations and the quality assessment framework almost told you how often you saw people, what you did when you saw them, where you saw them. To comply with the QAF our organisation, as many others as well, built up these huge packs of policies that tell you how to do your job…’^{[footnote 76]}

This section seeks to explore the process by which delivery under a SIB commissioning arrangement offers greater implementation autonomy and flexibility to providers, fostering the personalisation of services. The section first reviews the extent to which the SIB service allows for greater personalisation, then briefly describes the implications for service participants and staff. It finishes with an analysis of enablers and barriers to greater implementation autonomy.

Personalisation has become a mainstream approach to service reform and captures the intention for services to be tailored to individual needs (Fuertes & Lindsay, 2016, p. 526) and wishes (Sainsbury, 2017, p.57), ultimately resulting in stronger user choice (Cutler et al. 2007). Toerien et al. (2013) distinguish two dimensions of personalisation:

• procedural personalisation, describes the tone (or ‘how’) of support worker–participant interactions;
• substantive personalisation, relates to the ‘what’ of service provision and the extent of variation in service content according to different needs (Rice, 2017, p. 471).

Under the KBOP SIB arrangements provider staff reported greater flexibility in service delivery compared to the previous fee-for-service contract. Findings suggest three enablers: light touch service specifications, availability of flexible funding and a developmental leadership approach. While the latter was highly valued by providers, they sometimes struggled with the highly adaptive management approach as their organisational procedures require a longer time to embed the change.

6.4.1 Procedural personalisation

The introduction of a strengths-based, person-led approach to service provision provided the ‘blueprint’ for KBOP’s procedural personalisation.^{[footnote 77]} While the introduction of this approach was initiated by the KBOP project director, providers were supportive of it. To build providers’ capacity, all provider staff members participated in mandatory training provided by the Mayday Trust - a charity leading on person-led service provision.

A central feature of the strengths-based approach is a shift in focus from a participant’s deficits to their strengths.^{[footnote 78]} This entails a move away from a culture of ‘fixing someone’s needs’ to genuinely supporting someone to achieve their aspirations. The promise of a strengths-based approach is that each participant feels a greater ownership in the support process. As the strengths-based approach seeks greater empowerment of the people participating in services, the foundation of the support is a different dynamic between the intervention worker and the participant. Critical for a ‘strength-based’ approach is not the professional’s judgement, but the participant’s aspiration.

To ensure that the interaction between frontline staff and participants features the strengths-based approach, the KBOP management directly aligned major service processes and terminology to this model. More specifically, this implied a language audit in the CDPSoft data, in particular the referral assessment (e.g. removal of risk categorisations for participants) and the wider service documentation such as support plans, removing deficit-based language. This process was supported by the expertise of lived experience participants. Outcome evidence requirements were relaxed as part of the redesign of the rate card, allowing staff to respect the participant’s privacy, in turn creating a more equal power balance. Finally, KBOP management also sought to integrate the strengths-based approach into performance management by using the Homelessness Outcomes Star^{[footnote 79]} visually accessible tool to measure participants’ achievements (see figure 7).^{[footnote 80]} Likewise, integrating the assessment of individual staff caseloads into the monthly KBOP-provider performance review meetings is reflective of the person-led ambition of the service.

Figure 7: Example of Homelessness Outcomes Star use in outcome measurement

Source: KBOP social prime

While the framework for frontline-participant interaction was centrally prescribed – i.e. the KBOP operational team have sought to directly expand procedural personalisation - providers were granted discretion in the implementation of the strengths-based approach. This involves leaving the service specifications to the provider organisations. Frontline staff have discretion in determining - in agreement with the participant - the type of support, including whether it is virtual or face-to-face, the duration and location of their sessions and the frequency of support. Also, the dual phase model in service intensity (see section 6.2.1) enables tailoring to participant preferences as people can re-engage in more intensive support if a point of crisis (re-)emerges. In line with the strengths-based approach, this flexibility brings a greater choice in the mode of support provision to the participant. The director of the social investment fund emphasised the ambition to devolve power to frontline staff:

You can devolve responsibility to the frontline for quite a lot of activity. And that’s what the Mayday PTS approach [reference to personalised, strengths-based approach to service provision] hopefully does… Within reason you [reference to frontline staff] can do whatever you think is best for this person - and you’ve got a budget to do that. You can draw on the personalisation funds to buy someone a fridge or a bike. If there are things you think that would make this person’s life a lot better, you have the authority to just go and do that. And come and tell us about it, if you think it worked really well, and other people will benefit from that. But you’ve got the power to make some of those decisions. So, there’s the frontline personalisation.

6.4.2 Substantive personalisation

Proxy measures for substantive personalisation include caseworker-to-participant ratios; the variation and complementarity in service provision and the extent to which the participant’s choice influences service decisions (Considine et al. 2020, p. 882). The present report is not able to draw firm conclusions on the degree to which the KBOP service allows for substantive personalisation as this phase of research does not yet include detailed qualitative research with frontline staff or service participants.

From a service provider perspective, interviewees indicate that delivery teams have increased autonomy in piloting new interventions (see Appendix G), including the creation of new staff roles, which allow the service to be more responsive to participant characteristics and journeys. However, a greater variation in intervention offers does not directly imply that the participant is able to influence the nature or design of the service. Importantly, there is a question as to the degree to which each participant has agency in defining the goals of their service provision, given the pre-specified menu of outcome indicators set out in the contract. While the strengths-based approach is advocated alongside the use of outcome metrics by KBOP and the council, some provider participants experienced a tension between the person-centred approach and the outcomes-based approach. This tension was articulated in discussions of perceived high caseload in the KBOP SIB.

Results from a longitudinal KBOP frontline staff survey, comparing delivery experiences under the legacy arrangement with the SIB arrangement, show that the caseload for each frontline staff member has increased statistically significantly from an average caseload of 13.5 people in the fee-for-service contract to 18.6 in the SIB (an increase of 38%) see figure 8.^{[footnote 81]} Simultaneously, the percentage of time spent with individual service users in an average work week has decreased from 55 % in the legacy arrangement to 44 % under the SIB contract. Fund managers note that the duration over which service users engage with the service has been expanded but the survey data does not explicitly track this.

The compilation of data to understand caseloads and work intensity is not straightforward. Caseloads for frontline staff were not typically collected as part of the data collection processes in the pre-SIB fee-for-service contracting arrangements, and so the survey data are the most reliable, standardised measure to estimate caseloads over time. Following the introduction of the outcomes contract, there are also challenges in estimating caseloads because of joint casework.

The survey offers a self-reported figure from frontline staff, which is collected using the same survey instrument at two time points. However, there are also limitations to the survey-based data collection approach. The caseload estimate does not account for a change in provision which encourages longer-term support to achieve sustainable outcomes with users. This might result in frontline staff supporting more service users, but with a variation in support intensity.

Figure 8: Caseload shift from the fee-for-service contract to the SIB

Source: Researchers’ interpretation of longitudinal survey data on KBOP frontline staff service delivery experience.

The ‘education, training or employment’ (ETE) outcome is illustrative of the potentially contradictory dynamic between pre-defined outcomes and individual aspirations. Several provider interviewees voiced their scepticism as to whether ETE outcomes match with participants’ aspirations and abilities. Research participants’ views on whether an outcomes-based contract allows for (substantive) personalisation diverged. One stance is that the outcomes-based and personalised approaches are synergistic, i.e. the person-led support provision naturally leads to the outcome achievements. As the director of a provider organisation explained:^{[footnote 82]}

We are still very outcomes-focused in terms of the managers and monitoring the outcomes; and the support workers still having to evidence that they’ve got the outcomes. But in the support sessions, there isn’t the same focus on the outcomes as there used to be. The focus is on the relationship with the person. And then the outcomes come anyway. In fact, they’ve increased.

The opposite stance is that the high caseloads in the KBOP SIB preclude flexible, tailored support:

… because it’s all to do with how frequently you’re seeing them, isn’t it? And how frequently, what is the intensity of that support? Because actually you might only support two people, but you’re actually spending 20 hours a week with both of them. And this is why I’m not quite sure it’s always quite as person-centred as they make out, because they’ve got targets as to support this number of people per staff.^{[footnote 83]}

A head of services elaborates on the tension:^{[footnote 84]}

Where I’ve seen this [outcomes-based] approach work well before, is where you have this fixed cohort of individuals to work with, so you can achieve things where you’re not under pressure to work with new people constantly. You can invest in the individuals that you’re working with and give them completely tailored support to their needs that can flex according to when needs increase, when things decrease. I think the throughput issue and fairly high caseloads make that [strengths-based way of working] quite a difficult balance. We have staff who are completely invested in having a client-centred approach and it’s why they do the jobs they do…but it’s very, very difficult when there are pressures with high caseloads and with high turnover.

The above quotes provide an example of the difficulties some providers experience in marrying the asset-based and outcome-based approaches. The academic literature suggests that caseloads exceeding a certain threshold impede person-centred delivery (van Berkel & Knies, 2016).

The interim nature of the current report within a longer-term evaluation means that we are not yet able to draw firm conclusions on the topic of substantive personalisation. This requires further quantitative and qualitative analysis of frontline staff experiences, and validation with the service user perspective. However, it is important to stress that providers seem to have a greater autonomy compared to the legacy contract with regard to procedural personalisation.

6.4.3 Implications of personalisation

Research participants noted a positive association between the introduction of a strengths-based approach and the successful achievement of outcomes. However, this finding needs to be treated with caution as it is not yet possible to assess the impact of the strengths-based approach in isolation from other contextual factors. A service manager explained:^{[footnote 85]}

You can see the difference in the support plans in terms of like the language use and how quickly clients are able to achieve those outcomes versus how they do in some of the other services that I’ve seen. And the feedback from clients is positive. I think our team are able to engage quite hard to reach clients. We often work with the clients who have been cycling around in many different services. We’re often the service that do manage to get them hooked in and support them for a longer period of time.

Additionally, research participants stressed that the introduction of the strengths-based approach generally enhanced staff morale, allowing for greater ownership and value alignment with the KBOP partnership. Nevertheless, some members of staff also experienced difficulties in adjusting, eventually leading to staff retention issues.

6.4.4 Enablers to personalisation

There is concern in the payment-by-results literature that the flexible ‘black box’ approach is not consistently associated with flexibility and personalisation but in fact can conceal standardisation (Fuertes and Lindsay, 2016). For KBOP, this raises a question as to whether and how the outcomes-based nature of the social-outcomes-contract facilitates greater implementation flexibility. Our analysis identified the following mechanisms as routes that enable personalisation in the KBOP SIB service:

The outcomes-based nature of the contract: Some interview participants referred to the OBC as an enabler. According to the investment fund director, increased accountability for results allies with a greater implementation autonomy:

For us the main reason, or the main benefit of shifting to an outcomes-focus, it’s not really the specific outcome definitions or prices that you shift to, it’s the elements that you can then remove from the normal contracts. So, if you move to commissioning or procuring based on a set of progress metrics (instead of a defined list of inputs and activities), then what you can do is you can completely remove the service specification and the delivery budget, all this stuff that kills innovation in normal contracts. And what that then means is that that frees everyone up to redesign the service.

Nevertheless, this finding is partial as the concerns around high caseloads and a high administrative burden in the KBOP SIB are also considered as a potential barrier to providing tailored support.^{[footnote 86]}

The personalisation fund: Interviewees commended the facilitative role of highly flexible funding. The near immediate availability of funding was seen in stark contrast to the bureaucratic and lengthy processes in the resource-constrained public sector, where funding is dependent on fixed spending cycles and where strict rules limit changes to the initial budget allocation. A senior service manager explains:^{[footnote 87]}

… the advantages of being able to be more flexible with funding, to be able to bring forward funding, not having to wait for local authority to make decisions, or to pay next year’s money in order to deliver something, has been a real advantage.

The KBOP social prime set up a personalisation fund which provides additional funding to support any potential enhancement in service delivery, as determined by frontline staff. There is no maximum spend to the fund; to date funding has varied from £10 to £1,800. The KBOP project director gave examples of the creative use of the fund, for instance for IT equipment, licences and other practical tools to facilitate the achievement of ETE outcomes. The social prime’s management regularly reviews whether the spend is appropriate to the outcomes. In addition, the fund is used to finance pilot interventions and associated staff roles. This requires providers to submit a business case outlining how the intervention is intended to improve contract performance. Provider managers have discretion to spend up to £200, while the KBOP board holds the decision-making authority for larger investment decisions. The professional judgement of the KBOP project director and service manager are critical and they assess likely impacts on intended outcome measures to prioritise delivery pilots. Overall, providers perceived the social prime as very supportive in unlocking additional funding.

The developmental culture: A further facilitator to greater implementation autonomy was the social prime’s developmental leadership approach. There was a strong commitment from KBOP leadership to foster service innovation. The KBOP project director termed this work as “ongoing service design”. The investment fund director highlighted the culture shift towards a more experimental approach to service implementation as a key priority:

But the main goal in the first year is more psychological - it’s getting the right culture and the right approach. Because if this is all about trying a set of experiments and delivery pilots, you need those pilots to be bubbling up from the front line, from everybody. And so you need everybody to appreciate that they have the opportunity to contribute to improving the service and redesigning it. And everyone should be coming up with ideas for what to do differently… this isn’t just a fixed service, if there are things that you don’t like about it, or things that you think are not working, then we can probably change them.

The strong focus on innovation is also reflected in the recruitment process for service managers, which explicitly states a requirement for service innovation. Providers remarked on the strong shift towards a more developmental culture, compared to delivery under the preceding fee-for-service arrangement. The following two statements illustrate the way providers perceived the culture shift:

But it is refreshing for people to say ‘We are not focused on how you achieve these outcomes, just do what you need to do and if you want to talk to us about something, that’s fine. If you’ve got a new idea, that’s fine. Even if you think it might cost money, if it will get some of these outcomes again, let’s have that conversation.’ That’s something you don’t get with other funders as much.^{[footnote 88]}

We went out there and just had the freedom to just to go and explore really what was available and that was something that we’ve never done before.^{[footnote 89]}

However, other participants from different delivery organisations also pointed to limitations to the developmental approach. Ideally service innovation is beneficial to the whole partnership, but innovative approaches would sometimes come “from without rather than within”, suggesting a lack of provider influence in developing new approaches.^{[footnote 90]} While providers experienced the developmental approach as empowering, they also recognised that it resulted in a lack of stability. As such, the frequent changes to implementation require considerable managerial resources. Likewise, the pace of change does not always allow new approaches to be thoroughly embedded. Linked to this, the investment fund director acknowledged that the introduction of multiple change processes simultaneously made it difficult to attribute the causal impact of the service innovation.

Table 7 below summarises the evidence underpinning flexibility and personalisation of the service in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim evaluation report. The KBOP SIB is associated with enhanced flexibility in service delivery compared to the preceding fee-for-service contract.

Table 7: Summary of KBOP hypothesis 4: ‘Enhanced flexibility and personalisation’^{[footnote 91]}

Fee-for-service contract

Challenges	Implications for service delivery	Evidence
Limited availability of additional funding and bureaucratic application process	Impeded more tailored support, especially for participants with more complex needs	Service specifications narrowly defining support intensity and length Process-driven auditing
Limited availability of additional funding and bureaucratic application process	Impeded frontline innovation	Providers’ observation

Social impact bond

Facilitators	Implications for service delivery	Evidence
Light touch service specifications	Enhances tailored support, as frontline staff have greater leeway in determining frequency, length and manner of support Limitations: Heightened caseload for individual frontline staff	Provider contracts Case studies of participants’ journeys
Flexible funding	Nurtures innovative capacity of staff Allows to enhance tailored support provision	Innovations to service delivery (new staff roles, support offers and referral pathways) Providers’ observation Creation of a Personalisation Fund, providing funding for tailored intervention support Personalisation Fund spend of £148,000
Developmental leadership approach	Nurtures innovative staff capacity Limitations: High pace of change occasionally challenged providers’ ability to sustainably integrate adaptations into their operations	Innovations to service delivery (new staff roles, support offers and referral pathways)

7. Spillovers of the SIB

This section examines the effect of the KBOP SIB on the wider Kirklees service ecosystem. In particular, we analyse the role of the KBOP leadership in acting as an ‘ecosystem orchestrator’. The research team offer this as a useful organising concept, since an ecosystem orchestrator is an organisation which takes the lead in identifying joint goals and coordinating action with a diverse range of actors (Ebrahim 2019). ‘System orchestrators’ or ‘boundary spanners’ (Battalina & Kimsey 2017) are understood to be particularly important when trying to tackle “wicked problems” – societal issues that are characterised by social complexity and multifaceted interdependencies.

7.1 The evolution of KBOP’s ecosystem orchestrator role

From the start of the SIB programme, KBOP stakeholders articulated a strong ambition to foster cross-sector collaboration. The requirement for cross-sector collaboration is outlined in the contract. More importantly though, the KBOP project director showed a strong intrinsic motivation to foster cross-sector collaboration within and beyond the immediate KBOP delivery network.

The KBOP project director took active leadership in building cross-sector collaboration in Kirklees. To build relationships with local key players relevant to the SIB service, the director conducted a mapping exercise to identify relevant local stakeholders and services and invests considerable time in developing and extending these connections. From the outset, the KBOP project director worked to ensure “strong relationships with everybody from all the different parts of public and private sector that any of our participants would touch” and to embed KBOP in the wider local system of public service delivery.

The relationship-building process was supported by a number of facilitators. First, KBOP’s credibility is seen to be enhanced by acting as a representative for multiple floating support providers. This consolidating role was further enhanced by the central referral and IT system (CDPSoft), which brings more holistic information about the individual participants, tracking of longitudinal outcomes over time, and also (via KBOP) a system-level framing of the problem. Second, the long-term duration of the contract provides the time to build relationships and collaborative structures. This long-term contract was seen to justify the allocation of dedicated resources with key skills and experience in partnership management. A council contract manager explained:

They’ve got the time to do it, haven’t they? So, they are building up links that we probably wouldn’t have done as a contracts team before. And providers on their own didn’t have the capacity to do that.

Finally, Covid-19 was seen to have accelerated the position of the KBOP social prime as a local ecosystem orchestrator. The KBOP partnership was the first service in Kirklees to set up a support helpline in the early stage of the pandemic. This resulted in the KBOP partnership being the first point of contact for requests from individual users, as well as for referral agencies and social sector organisations beyond its target group. This supported the social prime in building knowledge of the local service landscape. Likewise, the helpline raised awareness of the KBOP service, thus broadening the network of referral agencies which would later make referrals to the service. During Covid-19, the KBOP partnership was understood to have become increasingly well-known and embedded as a relevant local service delivery partner.

After the initial consolidation of the KBOP SIB in Kirklees, the KBOP project director and service manager became members of relevant local and regional boards to support KBOP’s strategic network-building. A diary-based exercise with the evaluation team revealed the extent of this network.

At a local level KBOP managers are involved in:

• the Homelessness Forum
• Kirklees Reducing Re-offending Group
• the Health and Wellbeing Board.

At a regional level, KBOP managers are involved in:

• Housing partnership for West Yorkshire
• West Yorkshire Improving Women’s Lives Network
• West Yorkshire Probation Accommodation Link Meeting
• Greater Manchester Homes
• Criminal Justice and Mental Health Forum.

Alongside its local and regional involvement, the social prime pursues active engagement with central government departments such as the Ministry of Housing, Community and Local Government (now the Department for Levelling Up, Housing and Communities) and the Ministry of Justice, which have provided additional funding for system-level collaborations such as the Prison Leavers Pilot.

According to council staff, the social prime is much more engaged in relationship-building than other local partners:

[KBOP] seem to have really good networks, or a lot better than the previous providers had. They seem to have got their name out there a lot faster than the other organisations who’ve been here for years and years.^{[footnote 92]}

This resulted in the KBOP social prime becoming the central network coordinator in Kirklees and thus supporting the council in extending its partner network, as a Kirklees Council contract manager explained:

And we’re having much better links, because there’s only the one place to go, everything goes through to KBOP so that’s working, I would say really well. And I can only see that getting better as time goes on, to be honest.

While the initial network orchestration has been driven by the KBOP leadership, the vision is that increasing formal collaboration pathways will make cross-sector collaboration independent from personal relationships.

7.2 The rationale for KBOP’s ecosystem orchestrator role

The social prime’s rationale for establishing itself as an ecosystem orchestrator was twofold. First, there was a belief in the need for ‘systems strengthening’ by the KBOP leadership:

So […] for real systems change, you need to collaborate and work across all organisations. For us to take this further at KBOP, it’d be great to have more contact with health services and justice services and the employment services.^{[footnote 93]}

Second, there was a ‘business interest’, since this role is believed to support the success of the KBOP partnership. The investment fund director emphasised:

if KBOP is going to work, we can’t just deliver our own service. We have to go out and change the way all these other services interact with the people we’re trying to help.

KBOP management also identified strong partner relationships as vital for future projects beyond the current outcomes contract. The KBOP project director explained:

It’s the strength of relationship that then builds that community collaboration for future projects. The growing number of stakeholders seeking to establish partnerships with KBOP is an indicator that KBOP is successful in anchoring the organisation in the wider support ecosystem.

Strong partner relationships also enabled the KBOP social prime to access further funding through collective bids. According to the KBOP project director, the quality of proposals and thus their success rate improved through the strengthened relationships.

7.3 The Implications of KBOP’s ecosystem orchestrator role

The implications of the social prime’s ecosystem orchestrator role are twofold. Internally, the added intelligence is used align KBOP’s strategy with local and regional policies. The KBOP project director emphasised:

there’s local decision making, there’s regional decision making, so I need to be sure that whatever we do fits with local and regional priorities.

Equally, the social prime ensured that providers’ operational challenges are fed to relevant policymakers to advance the wider service system at the local and regional level. A head of services explained:^{[footnote 94]}

There is a mechanism whereby we can feed through issues that are systemic […]. You know we can push through those issues and say: “look, the context means that we’re not able to do this.” [W]here there are systemic blockages that can be removed or worked upon, […] we worked together as a strategic group of managers to affect that, and then higher up through the KBOP partnership. So those mechanisms exist, and that communication is great. And that works quite well.

Alongside the benefit of added intelligence through improved feedback loops, the social prime’s ‘boundary-spanning’ activities led to more formalised collaborations in service provision. A prime example is the Prison Leavers Pilot, implemented in collaboration with local authorities and the justice system. The aim of this initiative is to facilitate access to accommodation for prison leavers and reduce the risk of re-offending. A further example, described by the KBOP project director, constitutes the ‘Education, Training & Employment Forum’. This new partnership approach between KBOP, DWP, JobCentre Plus and local employment partners supports the sharing of information and best practice to facilitate access to employment and training opportunities for participants.

8. Conclusion

In Kirklees, the introduction of the KBOP SIB has responded to a series of challenges in the landscape of support for people with multiple and complex needs.

This report examines four mechanisms underpinning SIB delivery:

1. enhanced market stewardship

2. strengthened and data-led performance management

3. cultivation of cross-provider collaboration

4. enhanced flexibility and personalisation of frontline services.

Each of these mechanisms has broader implications for the delivery of complex, person-centred public services.

Market stewardship Findings: The growth and exit of providers are indicators of active market stewardship by the social prime. This is facilitated by a dedicated and well-resourced management entity (KBOP) and improved market intelligence. There is a symbiotic relationship between the payment-by-results mechanism and a consistently used, granular data management system. The long-term and outcomes-oriented contract supports the pursuit of long-term outcomes, increasing co-working between providers and the development of new service pilots.

Key Learning

Effective market stewardship requires adequate resourcing, a well-designed data management infrastructure and provision for data collection and monitoring. Services also need to be commissioned on a longer-term basis to mitigate the cycling of users in the system and allow for relationship-building across delivery partners. Likewise, to strengthen person-centred support provision, frontline staff need a balanced caseload. Co-designing evidence requirements with frontline staff might be supportive in reducing the administrative burden in reporting.

Performance management

Findings: Additional capacity for performance management allows challenges in service provision to be addressed swiftly and increases providers’ awareness of the impact of their services. A central data management system allows for a greater transparency on the demand for, and performance of, services. More granular and accessible data can bring efficiency so that participants do not need to repeatedly ‘re-tell’ their story.

Key Learning

Impact-oriented performance management requires the co-working of staff skilled in quantitative data analysis and strategic operational service management. The latter calls for qualitative data analysis skills and an in-depth service knowledge. The introduction of a central data management system requires adequate resourcing for infrastructure investment and staff who are willing to adapt to new data-led approaches. Co-design and an iterative approach to the user interface can facilitate the adoption of granular data management tools. The design of evidence requirements for outcome achievements needs to be sensitive to the interaction between service users and frontline staff.

Collaboration

Findings: A dedicated coordinator role enables service-focused interactions and more cohesion between providers. Regular provider meeting forums and matching of providers with complementary expertise are key in developing cross-provider collaboration. The overarching outcomes framework in a rate card facilitates goal alignment across providers.

Key Learning

Introducing greater cross-provider interactions requires adequate funding for a coordinator with strong relational leadership and service-specific network knowledge.

Flexibility and personalisation

Findings: Light-touch service specifications (known as a ‘black box’ outcomes contract), a developmental leadership approach and the availability of flexible funding allows for greater flexibility (and personalisation) in service provision.

Key Learning

Greater flexibility in service specifications requires commissioners to be more proactive in considering different, more impact-oriented accountability mechanisms such as outcomes-based contracting in their commissioning arrangements. A developmental leadership approach requires the recruitment of staff with experience in change management across social service delivery practice.

The KBOP SIB is also associated with some unanticipated spillovers. This is particularly seen with the broader ‘ecosystem orchestration’ approach to fostering collaboration beyond the immediate service delivery network, improving data-led intelligence for local and regional policy-makers and practitioners. This work galvanises a broader response to implementation problems.

Next steps

This is an interim report and a final assessment on the association between features of the KBOP SIB and changed public management practices will be made in the concluding qualitative research report due in 2025. At this stage, while the SIB is associated with seemingly positive shifts across the provider network and at the frontline, the research suggests a heightened administrative burden, linked to enhanced reporting requirements and management meetings, and an increased caseload. The KBOP SIB service is a dynamic and adaptive system and the research team understands that practice may have evolved since data was collected for this report.

The final stage of qualitative research (to be conducted in 2023-2024) will be complemented by quantitative and economic analysis to explore shifts in the achievement of outcomes and assess value-for-money.

Table 8: KBOP SIB evaluation strands

Type of Evaluation	Research Question
Process Evaluation (subject to 1st and 2nd interim report)	Through what mechanisms do specific aspects of the KBOP SIB contribute to the achievement of targeted social outcomes?
Impact Evaluation	What is the quantitative impact of services commissioned by the KBOP SIB on the targeted social outcomes?
Cost Benefit Evaluation	Do the benefits of the KBOP SIB approach outweigh any additional costs associated with this model, when compared to the legacy contracting arrangements? And if possible, what is the cost benefit analysis of the SIB?

Acknowledgements

The authors gratefully acknowledge the support of colleagues at the Department for Culture, Media and Sport. We are particularly grateful to staff in Kirklees who participated in interviews and provided access to documents. We would like to thank specific staff at Kirklees Council – who wish to remain anonymous – but who provide on-going support to the research project. Equally, we are extremely thankful to the Kirklees Better Outcomes Partnership team for being so engaged in our research projects; a special thanks to the partnership’s project director and service manager for allowing the authors to observe meetings, access documents, repeatedly participate in shaping the research, help to build connections with the delivery partners and provide detailed comments on a draft of the report. We thank Bridges Fund Management for their continuous support as research participants, granting access to contracts and meetings and providing detailed comments on a draft of the report. We are also grateful to the team at The National Lottery Community Fund for their ongoing support. Finally, the research would not have been possible without the participation of all KBOP delivery partners, who revealed in-depth insights into their SIB delivery experience and were extremely generous with their time. We would also like to thank our GO Lab colleagues, Michael Gibson and Jessica Reedy who provided editorial support.

Contribution Statement

Franziska Rosenbach developed the conceptual report design and oversaw data collection, and analysis. Franziska led on drafting, design of visuals and reviewed and co-edited the report.

Dr Felix van Lier conducted data collection and analysis and drafted section 4.1, 4.3, 4.4, 6.3 and 7 of the report.

Dr Fernando Domingos conducted data collection and analysis and drafted section 4.2 and 6.2 of the report and prepared figure 8.

Dr Eleanor Carter designed the overall research strategy, supported and conducted data collection and analysis, drafted introductory material and section 6.1 of the report. Eleanor co-edited the report and is responsible for overall research quality.

Recommended citation for this report

Rosenbach, F., van Lier, F-A., Domingos, F.D. and Carter, E., (2023) The Kirklees Better Outcomes Partnership: The second report of a longitudinal evaluation of a Life Chances Fund Impact Bond. Government Outcomes Lab, Blavatnik School of Government, University of Oxford.

About the Government Outcomes Lab

The Government Outcomes Lab (GO Lab) is a research and policy centre based in the Blavatnik School of Government, University of Oxford. It was created as a partnership between the School and the UK government and is funded by a range of organisations. Using qualitative, quantitative and economic analysis, it investigates how governments partner with the private and social sectors to improve social outcomes.

The GO Lab team of multi-disciplinary researchers have published in prestigious academic journals and have produced policy-facing reports. In addition, the GO Lab hosts an online global knowledge hub and data collaborative, and has an expansive programme of engagement and capacity-building to disseminate insights and allow the wider community to share experiences with one another.

References

ACEVO (Association of Chief Executives of Voluntary Organisations) (2009). Making It Personal: A Social Market Revolution: The Interim Report of the ACEVO Commission on Personalisation, London: ACEVO.

Ansell, Christopher, & Gash, Alison 2013. Stewards, mediators, and catalysts: Toward a model of collaborative leadership. The Innovation Journal, 17 (1), The innovation journal, Vol.17 (1), Article 7.

Ball, Nigel & Gibson, Michael (2022). Partnerships with principles: Putting relationships at the heart of public contracts for better social outcomes. Government Outcomes Lab.

Baron, Samantha, Stanley, Tony, Colomina, Carmen & Pereira, Tricia, 2019. Strengths-based approach: Practice Framework and Practice Handbook. UK Department of Health & Social Care.

Battilana, Julie, & Kimsey, Marissa (2017). Should you agitate, innovate, or orchestrate?. Stanford Social Innovation Review.

Bevan, Gwyn, Evans, Alice and Nuti, Sabina (2019). Reputations count: why benchmarking performance is improving health care across the world. Health economics, policy and law, 14 (2), 141–161.

Broadhurst, Sarah, & Landau, Keri (2017). Learning disability market position statements, are they fit for purpose? Tizard Learning Disability Review, 22(4), 198–205.

Burand, Deborah (2019). Contracting (Incompletely) for Success: Designing Pay for Success Contracts for Social Impact Bonds (SIBS). Cornell Journal of Law & Public Policy, 29 (1), 1-38.

Caldwell, Nigel D., Roehrich, Jens K., & George, Gerard (2017). Social value creation and relational coordination in public‐private collaborations. Journal of Management Studies, 54(6), 906-928.

Carter, Eleanor and Ball, Nigel (in press). Social Outcomes Contracting: seeding a more relational approach to contracts between government and the social economy? In Social Economy Science: Transforming the Economy & Making Society More Resilient. Oxford University Press.

Carter, Eleanor, FitzGerald, Claire, Dixon, Ruth, Economy, Christi, Hameed, Tanyah, & Airoldi, Mara (2018). Building the tools for public services to secure better outcomes: Collaboration, Prevention, Innovation. Government Outcomes Lab, Blavatnik School of Government, University of Oxford.

Carter, Eleanor (2018). Making Markets in Employment Support: Promises and Pitfalls in the Work Programme’s Private Power Market [Phd, University of Sheffield].

Carey, Gemma, Malbon, Eleanor, Green, Celia, Reeders, Daniel, & Marjolin, Axelle. (2020). Quasi-market shaping, stewarding and steering in personalization: The need for practice-orientated empirical evidence. Policy Design and Practice, 3(1), 30–44.

Considine, Mark, O’Sullivan, Siohban, McGann, Michael, and Nguyen, Phuc. 2020. Contracting Personalization by Results: Comparing Marketization Reforms in the UK and Australia. Public Administration  98 (4), 873-90.

Cornforth, Chris & Chambers, Naomi (2010). The role of corporate governance and boards in organisational performance. In Kieran Walsh, Gill Harvey, & Pauline Jas (Eds.), Connecting knowledge and performance in public services: From knowing to doing. Cambridge: Cambridge University Press.

Cutler, Tony, Waine, Barbara, & Brehony, Kevin. (2007). A new epoch of individualization? Problems with the ‘personalization’ of public sector services. Public Administration, 85(3), 847–855.

Cutler, Anotony & Waine, Barbara (1994) Managing the welfare state : the politics of public sector management. Oxford ; Providence (Bloomsbury collections).

Davies, Huw T. O., Nutley, Sandra M. & Smith, Peter (2000). What works? [electronic resource] : Evidence-based policy and practice in public services (University Press Scholarship Online). Bristol, U.K.: Policy Press.

Dickinson, Helen, Carey, Gemma, Malbon, Eleanor, Gilchrist, David, Chand, Satish, Kavanagh, Anne, & Alexander, Damon (2022). Should We Change the Way We Think About Market Performance When It Comes to Quasi‐Markets? A New Framework for Evaluating Public Service Markets. Public Administration Review, 82(5), 897-901.

Disley, Emma, Rubin, Jennifer, Scraggs, Emily, Burrowes, Nina, & Culley, Deirdre May (2011). Evaluation of the Social Impact Bond: Lessons from planning and early implementation at HMP Peterborough. RAND Corporation.

Gash, Tom, Panchamia, Nehal, Sims, Sam, & Hotson, Louisa (2013). Making public service markets work: Professionalising government’s approach to commissioning and market stewardship. Institute for Government.

Ebrahim, Alnoor (2019). Measuring social change: Performance and accountability in a complex world. Stanford University Press.

FitzGerald, Claire, Hameed, Tanyah, Rosenbach, F., Macdonald, James Ruiari, Outes Velarde, Juliana & Dixon, Ruth (2021). An Introduction to Life Chances Fund projects and their early adaptations to Covid-19: Life Chances Fund Evaluation Interim Report. UK Department for Digital, Culture, Media & Sport.

Fitz-Gibbon, Carol Taylor, & Morris, Lynn Lyons (1996). Theory-based evaluation. Evaluation Practice, 17(2), 177–184.

Fox, Chris, & Morris, Stephen (2019). Evaluating outcome-based payment programmes: Challenges for evidence-based policy. Journal of Economic Policy Reform, 0(0), 1–17.

Fraser, Alec, Tan, Stefanie, Lagarde, Mylene, & Mays, Nicholas (2018). Narratives of Promise, Narratives of Caution: A Review of the Literature on Social Impact Bonds. Social Policy & Administration, 52(1), 4–28.

Frydlinger, David, Hart, Oliver, & Vitasek, Kate (2019). A new approach to contracts: how to build better long-term strategic partnerships. Harvard Business Review, 97(5), 116-126.

Fuertes, Vanesa, & Lindsay, Colin (2016). Personalization and street-level practice in activation: The case of the UK’s Work Programme. Public Administration, 94(2), 526–541.

Jantz, Bastian, Klenk, Tanja, Larsen, Flemming & Wiggan, Jay (2018). Marketization and Varieties of Accountability Relationships in Employment Services: Comparing Denmark, Germany, and Great Britain. Administration & Society 50(3), 321-45.

Miles, Matthew B., Huberman, A. M. & Saldaña, Johnny (2020). Qualitative Data Analysis: A Methods Sourcebook. Fourth ed. Los Angeles, California.

Rice, Deborah, Fuertes, Vanesa & Monticelli, Lara (2018). Does individualized employment support deliver what is promised? Findings from three European cities. International social security review, 71(4), pp.91–109.

Rosenbach, F. & Carter, E. (2020). Kirklees Integrated Support Service and Better Outcomes Partnership: The first report from a longitudinal evaluation of a Life Chances Fund impact bond. Government Outcomes Lab, Blavatnik School of Government, University of Oxford.

Ryan, Gery W. & Russell, Bernard, H. (2003). Techniques to Identify Themes. Field methods, 15(1), pp.85–109.

Sainsbury, Roy (2017). Activation in the UK: The frontline and the ‘black box’ of employment service provision. In R. van Berkel, D. Caswell, P. Kupka, & F. Larsen (Eds.), Frontline delivery of welfare-to-work policies in Europe: Activating the unemployed. New York: Routledge.

Tse, Allison E., & Warner, Mildred E. (2020). A policy outcomes comparison: Does SIB market discipline narrow social rights?. Journal of Comparative Policy Analysis: Research and Practice, 22(2), 134-152.

Saldaña, Johnny (2021). The coding manual for qualitative researchers. Los Angeles, CA: Sage.

Savell, Louise, Carter, Eleanor, Airoldi, Mara, FitzGerald, Clare, Tan, Stefanie Outes Velarde, Juliana & Macdonald, J. Ruairi (2022). Understanding outcomes funds: A guide for practitioners, governments and donors. Government Outcomes Lab.

Toerien, Merran, Sainsbury, Roy, Drew, Paul, & Irvine, Annie (2013). Putting personalization into practice: Work-focused interviews in Jobcentre Plus. Journal of Social Policy, 42(2), 309–327.

Van Berkel, Rik, & Knies, Eva (2016). Performance Management, Caseloads and the Frontline Provision of Social Services. Social Policy & Administration, 50(1), 59-78.

Van Puyvelde, Stijn, & Raeymaeckers, Peter (2020). The Governance of Public–Nonprofit Service Networks: Four Propositions. Nonprofit and Voluntary Sector Quarterly, 49(5), 931-950.

Van Slyke, David M., (2006). Agents or Stewards: Using Theory to Understand the Government-Nonprofit Social Service Contracting Relationship. Journal of Public Administration Research and Theory, 17(2), 157-187.

Zahra, Shaker A., & Pearce, John A. (1989). Boards of Directors and Corporate Financial Performance: A Review and Integrative Model. Journal of Management, 15(2), 291-334.

Image credits: Graphic design on tables 4, 5, 6 & 7 - Sam Osborne

Glossary

Cohort: The targeted population of beneficiaries, participants, or service users.

Commissioning: The cyclical process by which entities assess the needs of people in an area, determine priorities, design and contract appropriate services, and monitor and evaluate their performance. This term is used widely in the UK public sector context, but less so elsewhere. It is sometimes used interchangeably with “contracting”.

CDPSoft: The ‘Customer Data Platform Software’ is the central referral and case management system, accessible to all parties involved in the KBOP project. It is administered by Kirklees Council.

DCMS: The Department for Culture, Media and Sport (DCMS) is a department of the United Kingdom government. It hosts the Civil Society and Youth Directorate and Public Sector Commissioning Team (formerly the Centre for Social Impact Bonds), which holds policy responsibility for this policy area within UK central government. In 2016, DCMS launched the Life Chances Fund (LCF), within which it acts as the central government outcome payer.

ETE: Activities aiming to move service participants into Education, Training and Employment.

Fee-for-service contract: In a fee-for-service (also known as fee-for-activity) model, a particular service is specified by the commissioning organisation, and providers are paid to deliver that service. Payment levels may be informed by specific inputs or activities and the accountability focus is usually the activity that service users participate in.

Social investor: An investor seeking social impact in addition to financial return. Social investors can be individuals, institutional investors, and philanthropic foundations, who invest through their endowment. In UK SIBs, these assets are often managed by ‘investment fund managers’ rather than the original investing institutions or individuals who provide the capital.

Investment fund manager: Responsible for providing the project finance and managing the investment strategy on behalf of the social investors.

KBOP partnership: The KBOP partnership constitutes the alliance of service providers and the social prime. The Kirklees Better Outcomes Partnership

Key Performance Indicator: Contractual terms – in this case between the social prime and the individual provider organisation – defining monthly targets (new starts on service, referral numbers, outcome achievements) for providers.

Legacy contract: See Fee-for-Service Contract

Life Chances Fund (LCF): The LCF is a £70m fund launched by the UK government to support the growth and development of outcomes-based commissioning through the use of social impact bonds (SIBs), commissioned by local public sector organisations in England.^{[footnote 2]}

Medium scenario grants: One of three temporary funding options offered to LCF projects during the Covid-19 crisis of 2020. This included activity payments based on projected medium case performance scenarios.

Outcome (outcome metrics/outcome payment triggers): The outcome (or outcome metric) is a result of interest that is typically measured at the level of service users or programme beneficiaries. In evaluation literature, outcomes are understood as not directly under the control of a delivery organisation: they are affected both by the implementation of a service (the activities and outputs it delivers) and by behavioural responses from people participating in that programme. Achieving these outcomes ‘triggers’ outcome payments within an outcomes contract or SIB arrangement.

Outcome-based contract: ‘Outcomes’ can feature in a contractual arrangement in a range of ways. Typically, an outcomes-based contract is understood as a contract where payments are made wholly or partly contingent on the achievement of measured outcomes. Also known as an outcomes contract.

Outcome payer: The organisation that pays for the outcomes in an outcomes contract or impact bond. Outcome payers are often referred to as commissioners.

Outcome payment: Payment by outcome payers for achieving pre-agreed outcomes. Payments may be made to a special purpose vehicle or management entity in an impact bond or to service providers in other forms of outcome-based contracts.

Person-led service provision: Service provision tailored to individual needs and wishes, enhancing user choice.

Payment by Results: A way of delivering services where all or part of the payment is contingent on achieving specified results.

Provider: Also known as service provider, service delivery organisation or delivery partner. Providers are the entity(ies) responsible for delivering the intervention to participants. Depending on the SIB’s contractual structure, providers work with the social prime, fund manager and/or outcome payer(s) to make the impact bond work. A provider can be a private sector organisation, social enterprise, charity, NGO or any other legal form.

Procurement: Acquisition of goods and services from third party suppliers under legally binding contractual terms. In outcome-based contracts where the government is the outcome payer, the procurement processes may play a role shaping the market, in defining the outcome specifications, the terms of the outcomes contract, pricing the outcomes, and selecting the parties.

Rate Card: A schedule of payments for specific, pre-agreed outcome measures that a commissioner (outcome payer) is willing to make for each participant, cohort or specified improvement that verifiably achieves each outcome.

Service users: See Cohort.

Social Impact Bond (SIB): A type of outcome-based contract that incorporates the use of independent, third-party funding from social investors to cover the upfront capital required for a provider to set up and deliver a service. The service is set out to achieve measurable outcomes established by the commissioning authority and the investor is repaid only if these outcomes are achieved. In the literature SIBs are also referred to as Social Outcome Contracts (SOCs) or Social Outcomes Partnerships (SOPs). This report uses the term SIB to refer to the commissioning arrangement; the term Social Outcomes Contract is used to refer to the contract between the council and the social prime.

Social Prime: The KBOP social prime is the investor-owned contract holding and project management entity sitting between the council and the alliance of service provider organisations. It is the contract party to the Social outcomes contract with Kirklees Council and it also holds the bi-lateral contracts with providers.

Strengths-based approach: This is a form of person-led service provision which seeks to increase service users’ ownership of the support process by encouraging each person participating in a service to centre their strengths and ambitions as they journey beyond formal service provision.

The National Lottery Community Fund (The Community Fund): The Community Fund, legally named the Big Lottery Fund, is a non-departmental public body responsible for distributing funds raised by the National Lottery. The Community Fund aims to support projects which help communities and people it considers most in need. The Community Fund manages the Life Chances Fund on behalf of DCMS.

Top-up funding: An outcomes fund may provide a partial contribution to the payment of outcomes where the remainder of outcomes payments are made by another government department, local government, or public sector commissioner. In the LCF the partial contribution from DCMS ‘tops up’ the locally funded payment for outcomes and is intended to support the wider adoption of social impact bonds (SIBs) commissioned locally.

Voluntary, community and social enterprise (VSCE) sector: A ‘catch all’ term that includes any organisation working with social objectives ranging from small community organizations to large, registered charities operating locally, regionally, and nationally.

---

---

1. Analysis of the fee-for-service model was the subject of the first interim evaluation report. ↩

2. The LCF was launched by the Cabinet Office in 2016. Policy responsibility for the LCF now sits in the Civil Society and Youth Directorate, part of the DCMS. The overall fund spend of the LCF was reduced to £70m from £80m as part of the DCMS budget negotiations in September 2020. This does not affect the ability to deliver existing commitments to projects in the Fund. ↩ ↩²

3. See the overarching LCF evaluation strategy. ↩

4. 2 of the 38 interviews constitute clarification interviews where interviewees are ‘re-interviewed’ to enable further depth of insight. ↩

5. Interviews for data clarification were conducted over summer 2022. The interview with the pro-bono legal advisor was also conducted in summer 2022. ↩

6. The numbers in brackets refer to the number of interviews. ↩

7. Further documentary data include for instance COVID-19 project plans, capacity-building slides and screenshots of the central data management system. ↩

8. An exception is the limited use of frontline survey data related to personalised service provision (see section 6.4.2). ↩

9. The description in this report is based on findings from the first interim evaluation report on the pre-SIB fee-for-service arrangement. ↩

10. In the report the term ‘KBOP SIB’/’SIB’ is used to refer to the commissioning arrangement, whereas the term ‘social outcomes contract’ is used to refer specifically to the contract between Kirklees Council and the KBOP social prime. ↩

11. KBOP is part of the Bridges Outcomes Partnership, a non-for-profit social enterprise owned by Bridges Fund Management, a private investment fund management company. ↩

12. The KBOP social prime usually pays delivery partners a month in arrears, guaranteeing payment within 10 days of accurate invoice. However, often – in particular for smaller providers - payments are negotiated in advance to support cashflow. ↩

13. Provider F and H. ↩

14. In the KBOP SIB the personalised, strengths-based service provision was based on the ‘person-led, transitional and strength-based (PTS) response approach’, developed by the Mayday Trust. ↩

15. The KBOP team clarified that this is to ensure confidentiality for the individual and GDPR compliance. ↩

16. Provider C ↩

17. See for definitions of service failure and negative outcome assessment section 4.3.1. ↩

18. The rate card also applies to Community Links, a provider which offers specialist mental health support. ↩

19. Here, providers upload supplementary evidence on the service user history into the central intelligence system. ↩

20. The evidence requirements for sustained employment are stricter, requiring payslips/employer confirmation or confirmatory data from HMRC. ↩

21. The figure combines the outcomes from the generic housing related support rate card and the rate card of the specialist domestic abuse provider (PDAP). ↩

22. A summary of data on the SIBs funded through the Fair Chance Fund can be found on the Government Outcomes Lab’s INDIGO website. ↩

23. The colour coding illustrates how outcome measures have evolved between the first and second iteration of the rate card. ↩

24. A service failure is a breach of service obligations that materially affect service delivery or contract partners. ↩

25. A negative outcome assessment results from a failure to meet base case KPIs for three months in a row or downside case KPIs. ↩

26. Senior operational manager, Provider I ↩

27. Investment fund director ↩

28. A detailed description of the governance structure is provided in Annex C2 ↩

29. From SIB service launch in September 2019 until December 2019 an interim project manager was in post. ↩

30. Director, Provider H ↩

31. Regional heads of operations, Provider C ↩

32. Provider D ↩

33. Undisclosed participant ↩

34. Director, Provider H ↩

35. Provider I ↩

36. See section 6.4.4 for further exploration. ↩

37. Rosenbach, F. and Carter, E. (2020). Kirklees Integrated Support Service and Better Outcomes Partnership: The first report from a longitudinal evaluation of a Life Chances Fund impact bond. Government Outcomes Lab, Blavatnik School of Government, University of Oxford. ↩

38. The central data intelligence system is administered by Kirklees Council and grants full access to the KBOP social prime and limited access to the SIB providers. All participant and outcomes data are saved on CDPSoft. ↩

39. Senior service manager, Provider F ↩

40. Director, Provider H ↩

41. Provider A ↩

42. Senior operational manager, Provider I ↩

43. Senior council contract manager, Kirklees Council ↩

44. KBOP project director ↩

45. Finding based on first-interim evaluation report (Rosenbach & Carter, 2020) ↩

46. This chart summarises the evidence underpinning market stewardship in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim report, which can be accessed here. The KBOP SIB signals an enhanced practice of market stewardship compared to the preceding fee-for-service contract. A final assessment of the degree to which this change is attributable to the SIB as a commissioning tool is the subject of the final process evaluation report. ↩

47. Provider F ↩

48. In the KBOP SIB, providers are rewarded based on a monthly fee, paid in arrears. Hence, different to a traditional payment-by-results contract, they are shielded from the implementation risk. However, the annual KPI re-negotiation accounts for providers’ performance. ↩

49. The service providers can only access their own service data. ↩

50. Only referral numbers that turned into KBOP service placements are included in the figure. ↩

51. Provider C ↩

52. Analysis of the ‘dual phase’ model is based primarily on interview data with KBOP management and documentary evidence sourced from providers (e.g. service reviews). Some interview data from providers added to the analysis. ↩

53. Senior Operational Manager, Provider I ↩

54. Director, Provider H ↩

55. To remove this issue, the KBOP social prime is currently working to secure an automated feed for relevant participant data from HM Revenue and Customs. ↩

56. Provider C ↩

57. The person-led, transitional and strengths-based (PTS) course and qualification was facilitated by the Mayday Trust. Strengths-based service provision is an intervention approach supporting personalised support. Section 6.4 entails a detailed description of the strengths-based approach. ↩

58. Provider F ↩

59. Provider A ↩

60. This chart summarises the evidence underpinning data and performance management in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim report, which can be accessed here. The KBOP SIB is associated with a more intensive practice of performance management compared to the preceding fee-for-service contract. ↩

61. A detailed analysis of the challenges associated with the Floating Support fee-for-service contract in Kirklees can be found in the first interim evaluation report of the Government Outcomes Lab (Rosenbach & Carter, 2021). The challenges of the fee-for-service contract outlined in this second report are based on primary data of the second wave of research and correspond with the findings from the first report. ↩

62. Provider F ↩

63. Regional head of operations, Provider C ↩

64. See section 5.1.4 for a detailed description of the operational governance meetings. ↩

65. This finding relates to interviews conducted in late 2021 and early 2022. Subsequently, KBOP management indicated that the frequency of meetings has been reduced. ↩

66. Provider B ↩

67. Provider C ↩

68. Provider I ↩

69. Regional head of operations, Provider C ↩

70. Provider H ↩

71. Set up by the KBOP social prime, the personalisation fund provides additional funding to support any potential enhancement in service delivery, as determined by frontline staff. ↩

72. Provider I ↩

73. Provider C ↩

74. This chart summarises the evidence underpinning the collaborative practice across service providers in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim report, which can be accessed here. The KBOP SIB signals an enhanced collaborative practice across providers compared to the preceding fee-for-service contract; a final assessment of the degree to which this change is attributable to the SIB as a commissioning tool is to be explored in the final process evaluation report. ↩

75. Provider A ↩

76. The QAF (Quality Assessment Framework) was the central auditing tool in the preceding fee-for-service contract. ↩

77. The KBOP social prime used the ‘person-led, transitional and strengths-based response’, developed by the Mayday Trust. ↩

78. For more details on the strengths-based approach, see for instance the practice framework and handbook published by the UK Department for Health and Social Care. ↩

79. The Homelessness Outcomes Star is licensed by Triangle Consulting. ↩

80. Besides its use in support provision, the Homelessness Outcomes Star serves as an evidence tool for wellbeing outcomes achievements. ↩

81. Data are drawn from a longitudinal survey examining frontline staff delivery experiences under the fee-for-service and SIB commissioning arrangements. The first survey wave was implemented in early 2019 prior to the roll out of the SIB approach (W1, N=48), while the second wave was conducted once the SIB was established in autumn 2021 (W2, N=39). ↩

82. Provider A ↩

83. Senior contracting and procurement manager, Kirklees Council ↩

84. Provider H ↩

85. Provider E ↩

86. It is important to acknowledge that outcomes-based contracts are not automatically linked to high caseloads. ↩

87. Senior Manager Wellbeing and Support, Provider F ↩

88. Service director, Provider A ↩

89. Service manager, Provider F ↩

90. Head of services, Provider H ↩

91. This chart summarises the evidence underpinning flexibility and personalisation of the service in the pre-SIB fee-for-service contracts and under the KBOP social impact bond. Detail on the challenges and implications of the fee-for-service contract is provided in the first interim report, which can be accessed here. The KBOP SIB is associated with enhanced flexibility in service delivery compared to the preceding fee-for-service contract. ↩

92. Kirklees Contract Manager ↩

93. KBOP Project Director ↩

94. Provider H ↩