Guidance

Work Programme National Statistics background information note

Updated 5 October 2020

This guidance was withdrawn on 10 February 2023

The Work Programme statistical series has ended. The final release of statistics was published on 30 September 2020.

Applies to England, Scotland and Wales

The final release of Work Programme statistics was published on 30 September 2020.

1. Purpose of the statistics

This document provides an overview of the performance of the Work Programme with the most up-to-date statistics and breakdowns.

2. Context

What was the Work Programme?

The Work Programme was launched in Great Britain in June 2011 and was one of a number of welfare to work reforms aimed at getting unemployed people into lasting employment. It replaced initiatives including the New Deals programme, Employment Zones and Pathways to Work.

Work Programme was delivered by organisations called Providers who were working under contract to DWP. Providers were sent claimants from Jobcentre Plus at specified points in their claim to help into employment. Generally, participants remained on the Work Programme for a maximum of 2 years.

Providers were paid at different points in the claimant’s work journey and this depended on the complexity of the case. Initially, payments were made at six or three months in work, depending on those requiring more assistance. Further payments were made for those staying in work beyond this, for a maximum of an additional 12 to 24 months for hard to help groups.

Work Programme Provider Guidance: Chapter 1 provides further information.

3. Time period covered

Referrals to the Work Programme started in June 2011 and ended in March 2017. The statistical release provides data from June 2011 to June 2020. The latest publication provides the final release related to the performance of the Work Programme. It updates previously published series and provides the final position on this information.

4. Definitions and terminology

4.1 Joining the Work Programme

Referrals

Claimants had varying entry points and joining criteria (such as age and benefit type) depending on their specific circumstances.

Typically, eligible customers were claiming the following benefits at the point of referral:

• Employment and Support Allowance (ESA)
• Jobseekers Allowance (JSA)
• Income Support (IS) – in England only
• Incapacity benefits – in England only
• Pension Credit
• Universal Credit (from August 2013 onwards)

Statistical information relating to Universal Credit Claimants on the Work Programme was included from the September 2015 release onwards. Payment Groups were defined with the legacy benefit system in mind: people who were claiming Universal Credit at the time of their referral were assigned to the most relevant Payment Group for their circumstances.

For a subset of claimants, the Work Programme was a mandatory requirement in order to receive their benefits while for other claimants the scheme was voluntary. See Work Programme Provider Guidance: Chapter 2 for details.

Participants could receive up to a maximum of 104 weeks of support on the Work Programme, from the date of being referred. They could only join this programme once.

See section 4.3 for more information on completing the Work Programme.

Referrals to the Work Programme ceased in March 2017, although Providers are still able to claim outcome payments in respect of any employment spell that started during a participant’s two-year spell on the programme.

Attachments

Once a claimant was referred to a Provider, the Provider was responsible for discussing the programme and supporting them into work. Subsequently the Provider would register an attachment to the Work Programme.

The statistics on referrals show the number of people referred to the Work Programme by Jobcentre Plus, while attachments show the number of people with whom the Provider had completed the initial engagement activity. Therefore, the number of attachments at any point in time will always be fewer than the number of referrals. This is because an attachment occurs after a referral. In addition, some claimants may never register an attachment, for example, if they moved into work and signed off benefits after being referred to the programme. (In this example, if the individual made a further claim to benefit during the two years following their referral date, then the Provider would then engage with the claimant and register the attachment).

Payment Groups

Work Programme Providers were paid primarily for the results they achieved and they were paid more for supporting harder to reach groups (see Work Programme Provider Guidance: Chapter 9).

Upon referral, claimants were assigned to one of the Payment Groups in the table below, based on their circumstances.

Aggregated Payment Group	Payment Groups Included
JSA 18 to 24	1. JSA claimants aged 18-24
JSA 25 and over	2. JSA claimants aged 25 and over
Other JSA	3. JSA Early Entrants
Other JSA	4. JSA ex-Incapacity Benefit (IB) claimants
Other JSA	9. JSA Prison Leavers
ESA New customer	6a. Excluding 12 month prognosis claimants
Other ESA	6b. New ESA 12 month prognosis claimants only
Other ESA	7. ESA ex-IB claimants
Other ESA	8. IB/IS Volunteers
Other ESA	5. ESA Volunteers

‘JSA Prison Leavers’ (payment group 9) was introduced in February 2012 to provide immediate employment support to claimants of Job Seeker Allowance, who were either about to leave prison and were making an advance claim or those who were applying within 13 weeks of being released from prison.

Referrals to ‘JSA Prison Leavers’ (payment group 9) started on 15 February 2012 due to advanced claims.

Attachments to JSA prison leavers started from the 1 March 2012.

Referrals to ‘New ESA customers with 12-month prognosis’ (payment group 6b) started on 12 November 2012. These referrals were mandatory after the claimant’s Work Capability Assessment outcome.

‘IB/IS Volunteers’ (payment group 8) was funded through a European Social Fund allocation and ceased taking referrals in March 2015. This was reflected in the statistical releases from June 2015 onwards.

The statistics produced reflect the information recorded on the administrative systems, which includes some known minor inaccuracies. This is most notable in the JSA 18 to 24 and JSA 25+ payment groups, which contain some claimants who were not in the specified age groups. Universal Credit claimants were also assigned to the legacy payment groups recorded by the administrative systems.

Contracts and Providers

The Work Programme was delivered by private, public and voluntary sector organisations, working under contract to DWP. These organisations were known as Prime Providers.

For the purposes of Work Programme contracts, DWP divided Great Britain into 18 contract package areas (CPAs). In each contract package area there were at least two Prime Providers and in some contract package areas there were three Prime Providers. This means that there were 40 individual contracts for the delivery of the Work Programme. The same Prime Provider may have held multiple contracts, provided that these were in different CPAs.

In late 2014 the department renegotiated its contracts with Prime Providers. As a result of these changes the department brought in a new, performance measure based on the actual number of referrals, against which Providers’ performance was assessed. Formal performance interventions could then take place on a quarterly basis rather than annually.

The department also made a series of changes to the validation procedures. One example is for outcomes paid from April 2016, the HMRC check examined Real Time Information instead of using P45 data. More details can be found in section 4.2 and Work Programme Provider Guidance: Chapter 9.

Providers working with DWP were expected to achieve the Merlin Standard of behaviour. Guidance for Providers on how to administer the programme and the minimum service delivery standards Providers agreed to were published.

Prime Providers had supply chains of organisations that provided services. Subcontractors in tier one of the supply chains were responsible for delivering the end-to-end process or a specific element of the service, such as job-broking. These partners were usually paid by results. Second tier organisations, worked on an on-call basis, as and when a Prime Provider judged a participant could benefit from the organisation’s help. The voluntary and community sector were represented at all levels of Work Programme delivery.

Administrative data was held against the Prime Providers only and therefore the statistics report figures against the prime contracts.

4.2 Outcomes of the Work Programme

Providers were paid when the claimants completed defined periods in work. Providers were expected to deliver 104 weeks of continuous support regardless of whether a participant changed benefits, or moved into employment.

Participants completed the Work Programme when they had either completed 104 weeks on the scheme or achieved the maximum number of Sustainment Outcome payments permitted, unless a participant completed early.

Work Programme Provider Guidance Chapter 9 provides more detail on payments and Chapter 10 provides more detail on completing the Work Programme.

Job Outcome payments

A Job Outcome payment was made after either three months (for payment groups 3 to 8) or 6 months (for payment groups 1, 2 and 9) of continuous or cumulative spells in employment. Only one Job Outcome payment could be paid per participant.

Payment validation

The outcome validation regime went live for the Work Programme in April 2012. The introduction coincided with an update to the Work Programme payment system on March 26 2012 which supported the process. This regime involved a number of validation procedures which were performed on Job Outcome payments and Sustainment Outcome payments paid to Providers to ensure that only valid outcomes were paid for.

Pre-Payment validation

All Job Outcome payment claims were subject to an ‘off-benefit check’ before payment. This involved an automated check to match participant information with the Customer Information System (CIS) to ensure that participants for whom Job Outcome payments were claimed were not claiming benefit during the reported period of employment.

Initially, Job Outcome payments failing the ‘off-benefit check’ were subject to further validation and only paid when further in-work checks directly with employers and participants were successful. From October 2012, this ‘off-benefit’ process was refined to reduce the number of valid claims failing the initial automated check and requiring further validation before being released for payment.

In September 2012, around 8,000 Job Outcome payments had accumulated from the beginning of the programme, which had incorrectly failed the initial check and were awaiting further validation. These were released for payment in September 2012 and were fully validated through the post-payment process. The automated ‘off-benefit check’ then had an increased window, from the initial 2 days to 15, in which the check was applied, to allow for minor discrepancies between the details of the Provider’s claim and the details on Departmental systems. Job Outcome payment claims that failed this automated check were removed from the system and not paid, unless they could be validated.

Claims which passed the ‘off-benefit check’ were released for payment, and then subject to further post payment in-work checks. This validation was not performed on cases which failed the initial ‘off-benefit check’, as they were already subjected to further in-work checks.

Post-Payment validation

A validation rate was calculated and used to strengthen the controls against fraud and error in the Job Outcome payments claimed by Work Programme Providers. The rate helped extrapolate financial recoveries against all payments made to a contract in the extrapolation period, rather than for the sampled claims alone.

To calculate this rate, a sample of 17 claims per contract was randomly selected every month from the total population of Job Outcome payments that passed the automated ‘off-benefit check’ and were subsequently paid in that month. For outcomes paid from April 2016, the HMRC check examined RTI data to validate employment, rather than P45 data as previously used. This allowed for better coverage and more accurate identification of employment spells. Failed HMRC checks were validated by confirming employment with either the employer or the participant.

Job Outcome payments that were found to be invalid were used to calculate the validation rate which was extrapolated across the total population (the sample had been chosen to be large enough to enable extrapolation). DWP reviewed each extrapolation round to ensure that it operated as intended.

Once the percentage of invalid payments (from October 2014, a proportion of payments which could not be fully validated was classified as invalid) had been calculated from the sample, the validation rate was applied to the total population minus the sample. The ‘fails’ in the sample were removed and money recouped.

Time was allowed for Providers to challenge and for DWP to assess and arbitrate challenge. This process could take up to approximately 3 months, therefore the National Statistics on Job Outcomes for some Providers could have changed slightly in subsequent releases.

The validation rates were also used to derive adjustment factors which were then used to rate the National Statistics. These were derived using the number of the Job Outcomes which failed the post-payment validation process divided by the total number of Job Outcomes sampled. This ratio was applied to Job Outcomes (less the sample and those already validated) to adjust the National Statistics.

The 40 adjustment factors (one for each contract) were derived by the payment validation team every extrapolation period and sent to the National Statistics production team. These were then used to ‘rate’ the Job Outcome data from Provider Referrals and Payment System (PRaP) to produce the National Statistics. This was important for the accuracy of the statistics, consistency between financial recoveries from Providers and counts/rates of Job Outcomes paid.

Prior to April 2013, monthly samples of 100 were brought together every two months. From April 2013, this has moved to monthly samples of 33 brought together every three months. This was in response to the views of users of the statistics and that supported a quarterly statistical publication (within 3 rather than 4 months of the reporting period and on a quarterly basis aligned to the financial year). From October 2014, the extrapolation period became every six months with monthly samples of 17. The results of 6 rounds of validation (one for each month being brought together every six months) align to the financial year (April to September, October to March) to provide validation rates.

DWP had previously carried out analysis using a one off sample of HMRC RTI data to the end of December 2013; examining off-benefit periods covered by employment for Work Programme participants.

Since the RTI system was not fully rolled out until October 2013 this sample data did not include information for all people who have taken part in the Work Programme. However, by identifying all weeks off-benefit containing employment from the period October-December 2013, an estimate for the full duration of the Work Programme for all participants could be made by applying rating factors to weeks off-benefit prior to October 2013.

Using this methodology, it was estimated that in total to the end of December 2013 around 26,000 participants satisfied the conditions for a Job Outcome payment but no Job Outcome payments had been claimed (by the end of March 2014). Whilst alternative methodologies might result in different estimates, we expected at least 20,000 participants to satisfy these conditions.

The December 2014 National Statistics publication covering data to the end of September 2014 used the last of the quarterly samples (extracted from June-September 2014). Due to the operational move to six monthly financial recoveries from October 2014, validation rates were only available every six months, covering the periods October-March and April-September.

In order to continue with a quarterly release schedule, the National Statistics were adjusted using the latest available adjustment factor. This was in line with operational and management information procedures. March and September releases from 2015 used the previous adjustment factors derived from the most recent validation exercises at that time. So, for example, statistics published in March 2015 used validation rates from samples extracted to September 2014. The statistics published in June 2015 covering to the end of March 2015 used samples extracted from October 2014 - March 2015 and reflect the final payments made to Providers. The full historic statistical series was refreshed each time the figures were released.

The end to end post-payment validation process took approximately 8 and a half months to complete. The routine sampling, checks and production of validation rates took just over one month. These were performed monthly, one for each of the previous 6 months Job Outcomes payments.

The Job Outcome adjustment factors, for the relevant 6 months were sent to the National Statistics production team in the form of a dataset just over 3 weeks following the end of the period to be adjusted.

The first post payment validation exercise did not commence until June/July 2012 and was applied to all outcomes from the start of the Work Programme to May 2012. The associated adjustment factors used to rate the statistics relevant to this period were received by production team in August 2012. All Job Outcomes relating to this period were recorded clerically and uploaded to the Work Programme payment system in 2 stages during April and early May 2012.

Sustainment Outcome payments

Following a Job Outcome payment, Sustainment Outcome payments were paid for every subsequent 4-week period the participant spent in continuous employment.

Payment Group	Maximum number of Sustainment Outcome payments achievable
1, 2, 8	13
3, 4, 5, 6, 9	20
7	26

A Job Outcome payment and subsequent Sustainment Outcome payments could be claimed beyond the two-year point (up to the payment group maximum number of weeks) as long as the claimant began the employment during the 104 weeks’ maximum Work Programme period and did not have a break in employment during this period. Following a break in employment (2 calendar days or more) after the 104-week period a Provider would no longer be eligible to receive further payments.

Payments to Providers were subject to validation procedures. More details can be found in Work Programme Provider Guidance: Chapter 9.

Sustainment Outcome payment validation

Sustainment Outcome payments could not be paid until there was a valid Job Outcome payment to support the Sustainment Outcome.

Pre-Payment validation

All Sustainment Outcome payments were subject to a pre-payment automated ‘off-benefit check’. Claims were not paid where a legitimate claim to benefit had been identified in the relevant 4-week period.

Post-Payment validation

Until October 2014 the post-payment process of extrapolation was applied only to Job Outcomes, not to Sustainment Outcome payments. Instead, invalid Sustainment Outcome payments were removed from the PRaP system and therefore not included in the administrative data used for National Statistics. From October 2014 the process for Sustainment Outcome payments changed to the same regime as for Job Outcomes. The first National Statistics release which reflects this change is June 2015.

Unclaimed Job Outcomes

Unclaimed Outcomes were paid to reflect cases where a Provider was unable to formally claim for an outcome achieved i.e. the participant refused to share employment details or the Providers lost contact after assisting the participant into employment.

In September 2015, the first round of identification covering the first 15 referral cohorts (Jun11 to Aug12) was completed and the volumes of Unclaimed Outcomes were reported in the December 2015 release for the first time.

DWP analysis, which was reported in the March 2014 Work Programme Official Statistics publication, looked at participants’ off-benefit periods covered by HMRC RTI employment evidence. The routine identification of these instances had been operationalised and Providers had begun receiving respective Outcome payments at a reduced rate.

Unclaimed Job Outcomes were identified by the Department using HMRC RTI and DWP benefit (National Benefits Database) data. Instances where time in employment and off-benefit meriting a Job Outcome payment which had not been successfully claimed by the Provider 32 months after referral date were identified.

These were combined with a prescribed number of Sustainment Outcome payments expected to have been achieved, where the Provider was able to claim for the Job Outcome and was paid at a reduced rate as an Unclaimed Outcome. This recognised Provider contributions whilst not overpaying for an outcome which did not meet the full requirements. It also retained the incentive for Providers to maintain their current in-work support process in a bid to obtain the outcome at full-price.

4.3 Completing the Work Programme

People could stay on the Work Programme for a maximum of 2 years.

4.4 Performance measures of the Work Programme

Since Providers were paid when participants reached 3 or 6 months in work, an effective way to monitor the Work Programme was to track the progress of those joining in a particular month. This enabled an assessment of progress by comparing monthly cohorts over time with the same duration of support.

Contractual measure

The department calculated expectations of what Providers ought to deliver. A summary is below. Work Programme Provider Guidance: Chapter 12 provides more detail.

All expectations were based on the analysis of historical data. Minimum expected levels for Providers were set. The levels were set at 10% above those that would be expected to occur in the absence of the Work Programme. For certain Payment Groups (1, 2 and 6a), these expectations were contractual minimum performance levels. This meant that if Providers did not achieve them, they may have been subject to a formal performance improvement process.

The manner in which expected performance was measured changed. From March 2015, new contractual minimum performance levels were calculated for rolling measurement periods. The levels were calculated by profiling the number and makeup of persons actually joining in a particular month over the lifetime on the Work Programme. Work Programme Provider Guidance: Chapter 12 provides more detail on these profiles. At the end of each quarter the number of Job Outcome payments achieved in the prior twelve months were measured against minimum performance levels for each contract and payment groups 1, 2 and 6a.

The rolling 12-month measure replaced the former approach where minimum performance levels were calculated for a fixed time interval of a financial year. Performance was assessed on the ratio of the number of claimants who did reach the 3 or 6 months’ work compared to the number of claimants joining the Work Programme. At the end of each financial year, the ratio of Job Outcome payments paid in that year to referrals made in that year (for contract and payment groups 1, 2 and 6a respectively) were compared against the minimum performance levels for these payment groups.

This method had limitations; notably that some of those joining would not have had long enough to reach three/six months in work and also the impact of changes between expected and actual numbers joining. See also the review of Work Programme minimum performance levels.

In 2015, the contract operated by Newcastle College Group in North East Yorkshire and the Humber ceased, with Maximus Employment UK Ltd taking on a new contract, replacing Newcastle College Group from February 2015. For the purposes of the final release document, Newcastle College Group and Maximus Employment UK Ltd in North East Yorkshire and the Humber have been treated as a single contract, although the two are treated separately in the accompanying data tables and Stat-Xplore release.

Transparency measure

The indicator looked at performance by cohort once claimants had completed 12 months on the Work Programme; 12 months was considered a reasonable duration of support. This approach allowed comparison of performance between cohorts. The indicator for June 2011 would be calculated by dividing the number of those who, by the end of June 2012, had at least three/six months in work by the total number joining the Work Programme in June 2011. A higher indicator for later cohorts would demonstrate that the proportion of paid Job Outcomes payments for later cohorts had risen.

The indicator was published on the same release schedule and covered the same time period as the National Statistics. For example, the statistics released in June 2014 covered to the end of March 2014. To reflect any updates to the figures the full historic statistical series was refreshed each time the figures were released.

The indicator was published as part of the National Statistics release in the collection of Work Programme National Statistics.

Minimum expected levels were published alongside the indicator. These were based on the profiles for those joining in a particular month. Work Programme Provider Guidance: Chapter 12 provides more detail on these profiles.

5 Users and uses

User Engagement

Prior to the first publication of Work Programme statistics, a note was placed in the collection of Work Programme National Statistics and internal stakeholders were consulted on their requirements.

To make sure interested users could tell us what they thought about our statistical products and to provide a forum to discuss statistical issues, throughout the continuing development, we made our plans known to users in advance via the statistics web pages and invited user views via email stats-consultation@dwp.gsi.gov.uk, Work Programme statistics questionnaire and the ‘Welfare and benefits’ community. A public consultation on the content and presentation of Work Programme National Statistics after June 2013 was also conducted.

Internal users to DWP were consulted on methodology, classifications, definitions, coverage, timing and so on.

Additionally, views were gathered via Departmental links to wider groups such as the (independently chaired) Work Programme: Building Best Practice group which included prime contractors, subcontractors, voluntary and community sector organisations, the Employment Related Services Association (ERSA), the Association of the Chief Executives of Voluntary Organisations (ACEVO), the Confederation of British Industry (CBI), the Federation of Small Businesses (FSB), the Skills Funding Agency, Business in the Community and the Business Disability Forum.

DWP statisticians engaged routinely with those responsible for answering Parliamentary Questions, Freedom of Information requests and request for wider briefing as well as press office colleagues to explain the statistics and guide their use and interpretation. Media releases were also monitored the once the statistics were released. Internal Users

Through the engagement detailed above, we identified users within DWP as:

• Work Programme policy, strategy, management information, operational support and associated analytical teams
• DWP Private Office and Press Office
• Parliamentary Question, Freedom of Information and Briefing teams

Their use of the statistics included:

• to evaluate, develop and support policy, strategy and operational decisions, initiatives and options and business plans
• to answer Parliamentary Questions and Freedom of Information requests
• to inform Departmental Responses to Commons Select Committees
• to inform press office statements
• to provide briefing lines including for devolved administrations

External Users

Since the first release of Work Programme statistics in February 2012 DWP have identified the following external users:

• Parliament: House of Commons Library, Parliamentary Committees
• other government departments: Cabinet Office, HM Treasury, Scottish and Welsh Government, local authorities and groups of local authorities
• Employment Services organisations: Prime Providers, subcontractors, trade bodies
• Charities and not-for-profit organisations
• Academics
• external commentators – leading news media and blogs
• UK Statistics Authority
• National Audit Office

The statistical products and data are used to:

• provide general information on the Work Programme in Great Britain
• measure Government targets relating to DWP
• develop and evaluate policy within local authorities
• provide briefing
• inform academic research
• feed into analysis papers
• evidence National Audit Office and Parliamentary Committee reports

User experiences

The Statistical Summary outlines the valid uses of the statistics, which is to monitor the type and volume of participants and outcomes achieved as defined in the payment model and contractual agreements.

Feedback through the engagement outlined has provided further information about how the statistics and data is ordinarily accessed and used and to what extent user needs are met. We have also gathered feedback on the information we provide on methodology, quality and via commentary in support of the statistics.

In summary, users:

• welcomed the large amount of statistics, breakdowns and underlying data that is released via Stat-Xplore
• found this information extremely beneficial in particular for local analysis
• felt that the presentation of the statistics in the Statistical Summary is helpful and user friendly
• endorsed the use of cohort statistics
• previously, have requested more frequent and timely release of statistics – we published quarterly with only a 3-month lag, compared to 6 monthly
• were in general, satisfied with the level of information provided on methodology, quality and commentary to support the use of the statistics
• felt that in general their needs are being met
• identified some limitation with the tools
• use the Tabulation Tool, in terms of the ability to download data to excel which was overcome with its replacement (Stat-Xplore)
• use the Visualisation tool in terms of the structure of the underlying data
• felt that we should add more context to the statistics in terms of rates of benefit for example

6 Data quality and sources

6.1 Sources

Data was obtained from the Labour Market System (LMS). This is the system Jobcentre Plus uses to administer customer claims and refer customers to the Work Programme Providers. The data contains information on the claimants’ individual characteristics and claim details.

Data on attachments, Job Outcomes and Sustainment Outcome payments was obtained from PRaP. Providers used PRaP to attach customers and claim Job Outcomes and Sustainment Outcome payments. Data on payments was also used from payment validation procedures – further information on this can be found in section 4.2

Statistics on those completing the Work Programme were derived from a combination of PRaP and the National Benefits Database (NBD). However, this information was not included in the final September 2020 release. Statistics on benefit status were derived by combining data from the Labour Market System (Jobcentre Plus administrative system) with the NBD. However, this information was not included in the final September 2020 release.

The NBD (based on benefit administrative systems) is used to identify spells of Jobseeker’s Allowance (JSA), Employment and Support Allowance (ESA), Incapacity Benefits (IB), Income Support (IS) and Pension Credit (PC) by Work Programme participants.

The NBD is constructed by combining scans of live benefit claims from administrative systems. These scans are snapshot extracts taken at frequent intervals, for most systems this is fortnightly, consisting exclusively of claims live at the scan date. A claim that ends between extracts will not appear on the next one. Claim end dates are not included in the data extracts and are randomly allocated between the last scan in which the claim was live and the first scan in which the claim is no longer included. End dates are received for Jobseeker’s Allowance claims due to the nature of the administrative scans. Therefore, end date estimation primarily affected non-JSA Work Programme referrals. The volume and proportion of non JSA claimants varied from cohort to cohort. Whilst this might have affected individual records, since estimated dates were random between the extract dates, there was minimal impact on reported proportions off benefit at a point in time.

Data from these sources were combined and quality assured (see section 6.2) to create the Work Programme analytical dataset (WPAD). This dataset contained information relating to the published statistics, including dates of referral, Attachment, Job Outcome payment and Sustainment Outcome payments. This enabled derivation of the time taken for referrals to Attach, plus comparison of information with previous.

6.2 Quality Assurance

Details of the initial quality assurance and routine quality assurance conducted ahead of each quarterly release are provided below.

Initial quality assurance conducted during data development

A large number of quality assurance checks were carried out during the early development of the statistics to ascertain:

• reliability, completeness and level of disclosure of individual variables
• levels of duplicate, missing or contradictory information
• consistency across computer systems (LMS and PRaP) and with management information, via cross checking
• monthly build-up of figures in system data
• trends and variation in characteristic, time series and geographical breakdowns
• a methodology to allocate early clerical outcomes to months
• consistent results during time series and cohort development via dual methodology testing and manual checking
• consistent results during Stat-Xplore development through dual methodology testing and manual checking
• trends and differences in post payment adjustment factors via secondary quality assurance

This quality assurance found no issues; it showed the data was robust, consistent with management information and suitable for publishing. Individual variables were complete and consistent with existing sources, comparisons across systems were always within a 0.1% tolerance, including across key breakdowns and time series.

The above processes were subsequently used to develop data cleansing rules, assess retrospection and timeliness as well as form routine quality assurance procedures.

Routine Quality Assurance

A standard set of quality assurance procedures are conducted for each statistical release which consist of checking:

• duplicate, missing or contradictory information
• accordance across computer systems (LMS and PRaP) and with management information
• trends and variation in characteristic, time series and geographical breakdowns
• trends and differences in post payment adjustment factors
• dual methodology testing of time series and cohort data
• checks of Stat-Xplore variables and a sample of cross tabulations against perturbed Work Programme data and accompanying tables

Individual variables remained complete and consistent with existing sources, comparisons across systems remained predominantly within a 0.1% tolerance, including across key breakdowns and time series.

In addition to the quality assurance of the statistics, assurance of the underlying administrative data was also carried out. The National Audit Office published a report in August 2013 on their review of the data systems associated with the Work Programme Business Plan Transparency indicator (and therefore all key Work Programme National statistics). Section 2.3 contains details of the payment validation processes carried out in relation to financial recoveries against payments to Providers.

Work Programme has also been used as a case study by the UK Statistics Authority in relation to Quality Assurance and Audit Arrangements for Administrative Data.

6.3 Limitations

Work Programme National statistics were sourced from data originally collected via systems which administered the Work Programme. As such, published figures reflect the information recorded both manually and automatically on these systems.

The statement of the administrative sources of DWP statistics describes the system assurances associated with DWP data sources.

Ethnicity is self-assessed and recorded on the Jobcentre Plus administrative system. This system records ethnicity using the 2001 harmonised ethnic groups. The latest harmonised standards were published in August 2011 but these had not been adopted in the system at the time of recording. This means that for Work Programme data:

• ‘Chinese’ ethnic group has not moved to the ‘Asian/Asian British’ broad ethnic group and is in the ‘Chinese or other’ broad ethnic group
• No ‘Gypsy or Irish Traveller’ and ‘Arab’ groups were listed
• The broad ethnic group ‘Black or Black British’ was not listed as ‘Black/African/Caribbean/Black British’
• The ‘Mixed’ broad ethnic group was not listed as ‘Mixed/Multiple ethnic groups’

Where manual or automatic processes were known to record information incorrectly, diligence was taken to ensure the published figures represent the Work Programme as accurately as possible. Quality assurance conducted both during development and routine production is covered in section 6.2 Information on known issues is recorded in this document.

Footnotes were applied to known anomalies and where possible data was corrected, although inevitably due to the nature of the data and systems, some inaccuracies will remain.

DWP have a policy for planned revisions describing how DWP will handle revisions and give confidence that all revisions will be handled in a transparent manner. For Work Programme the latest release provides the full historic statistical series including any revisions to past published data.

We allowed some time for additional information to be incorporated into DWP’s data systems. This time period is referred to as retrospection. Information may have been submitted, corrected or resubmitted sometime after the event occurred, for example payment information being received late or subsequently changed or removed. This means data presented in each release is subject to some retrospection and figures may be revised in subsequent releases.

As well as revisions each quarter caused by general retrospection associated with data submission, further revisions result from retrospection associated with Work Programme processes, specifically the validation challenge process which can take approximately 3 months. Quality assurance procedures discussed in section 6.2 identify such changes between releases.

The biggest change between monthly figures occurs in the statistics on those completing the Programme. This is since benefit information was used to inform current Work Programme status, in particular statistics on those completing the allotted time and returning to Jobcentre Plus at 104 weeks. Due to the time taken to receive benefit information after the event, revisions were higher (revisions effectively reclassified those from other groups in the subsequent release) for statistics relating to the two most recent intakes to reach 104 weeks. Benefit status information was not included in the final September 2020 release.

There are minor differences between this publication and DWP’s operational MI caused by a small number of Gross Job Outcomes being recorded in adjacent months, this leads to minor discrepancies in recorded Net Job Outcomes in the two datasets.

Whilst benefit status statistics represent evidence of movement off benefit, they do not provide direct evidence of sustained employment as the analysis makes no assumptions towards the destinations of participants leaving benefit.

6.4 Measuring performance

Referral and Attachment statistics were first published in February 2012 and Job Outcome and Sustainment Outcome payments in November 2012. Outcome statistics were published later due to the time needed for outcomes to be achieved and for PraP to be updated.

On 26 March 2012, PRaP was updated with functionality to record Work Programme Job Outcome payments electronically.

From April 2012, the functionality was used to allow all outcomes recorded clerically from September 2011 to be uploaded to the system. This took place in two stages during April and early May 2012.

The first stage related to outcomes to the end of March 2012 and stage two covered April outcomes. The new system functionality was opened to Providers in mid-May 2012, from which point onwards all Job Outcome payments and Sustainment Outcome payments, including those from early May, were recorded directly on the Work Programme payment system.

A methodology was adopted for the statistics to allocate the clerical outcomes to specific months based on what would have been input had the system been live.

For stage one uploads relating to the end of March 2012, allocation was determined by the qualifying date of the outcome, stage two uploads relating to April were allocated to April and from mid-May onwards outcome statistics use the recorded payment date. When the new system functionality was opened to them in mid-May 2012, Providers began to input outcomes relating to early-May to the system.

The information available to allocate early outcomes to a particular month coupled with failures, errors and delays during the upload and input process impacted on the resultant time series to May 2012. For example, no Sustainment Outcome payments were allocated to March 2012.

The November 2012 release noted that a number of Job Outcomes were incorrectly assigned on the payment system to April 2012 rather than earlier months due to being allotted to stage two rather than stage one, in error, during the upload process. The issue was known to affect two contracts, Merseyside, Halton, Cumbria, Lancs (Ingeus UK Ltd.) and Devon, Cornwall, Dorset, Somerset (Prospect Services Ltd) with very few Job Outcome payments assigned pre April 2012. The cumulative total number of Job Outcome payments was not affected just the month to which they were attributed. It was also noted that other contracts may have been affected but to a lesser extent.

DWP subsequently consulted all other Providers and confirmed that no other contracts were affected. The originally incorrectly assigned payments were allocated to the correct months by amending the original methodology slightly for the affected contracts.

Historical validation factor improvement

Some of the participants sampled had their details matched against HMRC RTI data to check for periods in work. This resulted in small changes to the validation factors applied. This impacted Job Outcomes from April 2014 to March 2015.

The same process where RTI was used to complete the check was opted as the chosen method for Job Outcome and Sustainment Outcome payments from April 2016 onwards. It replaced P45 data as it offered better coverage and more accurate identification of employment spells. The figures were retrospectively revised using this method.

The methodology of validation factors was reviewed in May 2017. Previously, all cases found to have a UC interest were fully validated at the pre-payment stage. It was discovered that UC cases were subtracted from the denominator of the validation factor. This meant that the procedure in which adjustment factors are applied to payments omitted Universal Credit claimants; the new approach amended this issue and figures from April 2015 onwards were corrected. This had a small effect on outcomes within extrapolation period but had a greater influence as the number of UC claimants with the programme increased.

6.5 Changes to the statistics

September 2016 release

Due to a technical issue a small number of Job Outcomes paid in June 2016 were not recorded within performance data.

December 2016 release

A new policy on rounding was applied to the Work Programme publication as follows:

Range of values	Round to the nearest
0 to 1,000	10
1,001 to 10,000	100
10,001 to 100,000	1,000
100,001 to 1,000,000	10,000
1,000,001 to 10,000,000	100,000
10,000,001 to 100,000,000	1,000,000

March 2017 release

Changes from previous releases:

• delayed March 2016 Job Outcomes have been correctly assigned to March 2016
• there were approximately 200 cases with missing Contracts and CPA groups; this should be resolved in future releases
• there were 2 new JCP Districts; Greater Manchester and Cheshire

Data included some demographic information on people that were claiming Universal Credit at the point of referral.

September 2017 release

In February 2018, an error was detected in the number of Job Outcomes in September 2017’s release, covering the period July 2015 to March 2016. This was due to the wrong data file being incorporated into production of the September 2017 analytical dataset. This meant an over count of 0.2% for Job Outcomes at National level. The relevant tables have been amended.

September 2020 release

The Work Programme statistical summary: data to June 2020 publication is the final release of these statistics.

There have been some changes to the final release from previous publications:

• prison leavers have their address listed as their address when they are referred, which may be on the day they left prison or a few weeks later. Previously their addresses could have been the address they had before they were in prison, the prison’s address, or their address after prison
• referrals before October 2012 have their address as at October 2012 in the data and not their address when they were referred
• the methodology for calculating the number of months that have passed between the referral date and Job Outcome Payment date has been improved by basing it on the number of months passed between the two dates, rather than the number of month boundaries
• some Stat-Xplore breakdowns are unavailable (see section 7)
• the relative likelihoods of achieving a Job Outcome payment for different ethnic groups have been included. These were calculated following the standard cross-government methodology advised by the Cabinet Office. Read more details about this.

During the development of the final release (Work Programme statistical summary, data to June 2020), an error was detected in the releases published from April 2015 to March 2018. In those releases, for Sustainment Outcome payments, the wrong validation rates were applied to the contracts. The weighting factors were mismatched to the contracts, which may have resulted in an over count of up to 10% for some specific contracts over some periods. The correct weighting factors have been applied to the final release and the back series has been revised. However, there may be a few small discrepancies when comparing Sustainment Outcome payments by contract in the September 2020 release to releases between April 2015 and March 2018.

6.6 Changes to Contracts and Providers

In March 2014 it was announced that the contract in North East Yorkshire and the Humber run by the Newcastle College Group would be terminated with 12 months’ notice. The replacement, following a procurement competition, Maximus Employment UK Ltd began receiving referrals from 2nd February 2015. An existing Provider G4S received all referrals in North East Yorkshire and the Humber between 5 December 2014 to 30 January 2015.

With regard to the presentation of the statistics for the June 2015 release in North East Yorkshire and the Humber, the referrals already made to Newcastle College Group which had yet to have a Job Outcome were transferred under Maximus. Newcastle College Group kept those who had already had a Job Outcome. From December 2014, the Careers Development Group were known as Shaw Trust; their name is now Shaw Trust CDG. From February 2015, Interserve Working Futures acquired ESG Holding Ltd. These changes to the Prime Providers have been reflected in previous releases.

Changes to the Prime Providers which were not reflected in the June 2015 release but are reflected in the September 2015 release are as follows:

• Interserve Working Futures and ESG were both renamed Interserve Learning & Employment
• EOS-Works Ltd was replaced by Avanta Enterprise Ltd
• A4e was acquired by Staffline Plc which also own Avanta Enterprise Ltd
• A4e and Avanta were rebranded People Plus Inc
• JHP Group Ltd were renamed Learndirect Ltd
• Newcastle College Group was abbreviated to NCG

A further change occurred at the end of 2015: Pertemps became AMP. This change of name is not reflected in the December 2015 publication but is enacted in subsequent releases.

In 2018, some Providers agreed to modify the way they received Sustainment Outcome payments. Instead of receiving a Job Outcome payment when a participant reached 6 months in work (or at least 3 months for groups expected to require more assistance) and also for further sustained work beyond this, they were eligible for a single payment after a participant achieved 6 months in work. This applied to all customer groups, and no further payments were made with respect to each participant. The Providers who opted for this revised payment scheme are shown in the following table.

Contract	Contract Package Areas
People Plus Inc	East Midlands, East London, North East, Merseyside, Halton, Cumbria, Lancs, Manchester, Cheshire, Warrington, Thames Valley, Hamps, Isle of Wight, Surrey, Sussex, Kent, Birmingham, Solihull, Black Country, South Yorkshire
Serco Ltd	Coventry, Warwicks, Staffs, The Marshes, South Yorkshire
Seetec	East of England, East London Manchester, Cheshire, Warrington
Maximus Emp. UK Ltd	West London, NE Yorks, The Humber, Thames Valley, Hamps, Isle of Wight
Reed in Partnership	West London
ShawTrust CDG	East London
Working Links	Scotland, Wales
Rehab jobfit	Devon, Cornwall, Dorset, Somerset, Glouc, Wilts, Swindon, West of England, Wales
Interserve Learning & Employment	Coventry, Warwicks, Staffs, The Marshes, West Yorkshire
Learndirect Ltd	Glouc, Wilts, Swindon, West of England

The September 2020 publication includes all contracts that were on the programme as an overview of the performance of the Work Programme.

7 Status of the statistics

The UK Statistics Authority designated Work Programme statistics as National Statistics, in accordance with the Statistics and Registration Service Act 2007 and the Code of Practice for Official Statistics.

Designation can be broadly interpreted to mean that the statistics:

• meet identified user needs
• are well explained and readily accessible
• are produced according to sound methods and
• are managed impartially and objectively in the public interest

The final release of the statistics (Work Programme statistical summary data to June 2020) has been reassessed and has been reclassified as Official Statistics. This is because the statistics have not been updated since the March 2018 release (data to December 2017) and there have been changes to the payment scheme and to how performance information has been recorded. These issues have resulted in changes to methodology (see section 6.5) in order to publish the final update to the series of Work Programme statistics.

Furthermore, some elements of the statistics have not been included in the final release due to resource limitations. These exclusions are:

• Breakdowns on Stat-Xplore:
  • Lone parent flag
  • current Work Programme status
  • Job Centre plus district
  • first Sustainment Outcome indicator and length of Job Outcome breakdowns
  • CSV files containing off-benefit information for multiple cuts of cohorts

Stat-Xplore

Stat-Xplore is a statistical dissemination tool that allows users to create customised Work Programme tables on the numbers of referrals, attachments, Job Outcomes and Sustainment Outcome payments on a cumulative or monthly basis (cohort statistics are available via the Open Data. See Cohort and Time series data.

The breakdowns available online are:

• Month
• Geography
• Age
• Contract
• Work Programme Status
• Disability Indicator
• Ethnicity
• Gender
• Jobcentre Plus District
• Length of Job Outcome
• Lone Parent Indicator
• Mandatory or Voluntary Indicator
• Payment Group

The final release of the Work Programme statistics does not include Lone Parent Indicator, Work Programme Status, Jobcentre Plus District, Primary Health Condition and Length of Job Outcome breakdowns.

During the quality assurance process, the data underlying the previously included “Primary Health Condition” variable was found not to be of sufficient quality for publication and has not been included as a breakdown on Stat-Xplore.

These variables can be broken down even further; Geography, can be split into countries, regions, counties, parliamentary constituencies and local authorities, for example. Some of these breakdowns are not available for Sustainment Outcome payments.

Cohort and Time series data

The cohort and time series data for the Work Programme Statistics is also published, and is available in CSV format.

Work Programme cohort data includes breakdowns of people who have been referred to the programme by their month of referral:

• Referral counts for each cohort
• For each month following referral for each cohort
• Count and percentage to have achieved a Job Outcome
• Count and percentage to have spent some time off benefit (excluding Universal Credit)
• Count and percentage to have spent 13 weeks off benefit (excluding Universal Credit)
• Count and percentage to have spent 26 weeks off benefit (excluding Universal Credit)
• Count and percentage to be off benefit (excluding Universal Credit) at the end of that month

The Time series data looks at referrals, attachments, Job Outcomes and Sustainment Outcome payments over time.

The cohort and time series data is not available for the final release.

8 Feedback

DWP would like to hear your views on our statistical publications. If you use any of our statistics publications, we would be interested in hearing what you use them for and how well they meet your requirements. If you have any feedback, please contact Kirsten Mackenzie, Data as Statistics, kirsten.mackenzie@dwp.gov.uk.

9 Useful links

The main mechanisms for viewing the National Statistics are the Statistical Summary and Stat-Xplore available on the collection of Work Programme National Statistics.

The statistics can also be accessed on the Employment Programme Support Official Statistics page which provides access to other employment programme Official Statistics, the latest release timetable and other related information.

Find more information relating to arrangements on the Work Programme, available via the Department for Work and Pensions StatXplore tool.