Guidance

Methodology and quality statement: ESA: outcomes of Work Capability Assessments

Updated 14 September 2023

Find the latest release of these statistics in the Employment and Support Allowance: outcomes of Work Capability Assessments collection.

1. Overview

The main purpose of this document is to provide users with information about all the methods and quality assurance steps used to produce the Employment and Support Allowance (ESA): Work Capability Assessments (WCA), Mandatory Reconsiderations (MR) and Appeals statistical release in accordance with practices set out in the Code of Practice for National Statistics.

The Department for Work and Pensions (DWP) publishes the above statistical release every quarter, usually in March, June, September and December of each year.

The release is used to present information on both new ESA claims and Incapacity benefit claims undergoing reassessment (IBR) for ESA within Great Britain. It provides statistics on:

• the outcome of completed ESA health assessments
• ESA claims still in progress
• ESA claims closed before the health assessment process is complete
• statistics on appeals outcomes

Since June 2016, it also includes statistics on MR registrations and outcomes.

From September 2016, median MR clearance times were published as well as ESA WCA Cohort information which allows us to look at the overall ESA claimant journey through ESA WCA, MR and appeal stages.

From September 2017, statistics on ESA WCA end-to-end clearance times for initial claims were published.

The ESA WCA Cohort including MRs and the ESA WCA end-to-end clearance times for initial claims were badged as experimental to reflect the fact that these statistics were new and methodologies and definitions for the statistics could develop over time. In Spring 2023, the Chief Statistician for DWP led an internal review of all experimental official statistics produced by DWP, in line with the Code of Practice for Statistics. The review found that it was appropriate to remove the experimental label from these statistics because they are no longer under development. From September 2023 these statistics are badged as Official Statistics.

From June 2018, more geographical breakdowns covering WCA outcomes were made available on DWP’s online tabulation tool, Stat-Xplore and for initial clearance times in our published tables. Disseminating the statistics on Stat-Xplore allows users greater flexibility of the data by the ability to table more variations of selected breakdowns compared to the fixed Excel tables.

The statistical release provides a high level overview and more detailed figures can be found in the accompanying excel tables.

Note: In previous quarters, to derive the median clearance times, DWP statisticians have used the claim registration date from the Atomic Data Store (ADS) from the DWP data warehouse. From April 2020, the ADS has been switched off and replaced by the Historic Data Store (HDS) within the DWP data warehouse. Therefore, claim registration dates from April 2020 onwards have been sourced from the HDS.

2. Source data

The statistical release uses data from the following 4 main administrative data sources:

1. DWP’s benefit data covering new ESA claims (starting from 27 October 2008). These snapshot datasets are fed into the National Benefits Database (NBD) which holds claim level information on what benefits a person has claimed past and present. The NBD contains benefit information on Incapacity Benefit (IB), Severe Disability Allowance (SDA), and Employment and Support Allowance (ESA) and is used as the base data to merge with health assessment data (MSRS), which contains WCA information from the Assessment provider, and Appeals data from HMCTS.

2. Health Assessment data (MSRS data) from the Health Assessment provider including information on face to face assessment, medical report form data (via form ESA85) and limited capability for work questionnaire data (via form ESA50) – this covers those cases where the assessment phase is completed;

3. Mandatory Reconsideration data from the DWP administrative system named “Decision Maker and Case Recorder” (DMACR) which provides individual level data for MRs. The term ‘Mandatory Reconsideration’ refers to the process where a claimant approaches DWP to dispute a decision on their ESA claim.

4. Appeals data from HM Courts and Tribunals Service (HMCTS) – note that this data only includes information on completed appeals and does not include appeals still in progress. The data from HMCTS also only captures appeals with a ‘‘WCA” issue type recorded. ESA appeals recorded with other issue types such as “SG placement”, where customers dispute the group they have been placed in, are not captured in the HMCTS data.

Simplified data process flow

A simple overview diagram, showing how information from these 4 data sources are combined.

Simplified representation of ESA WCA, MR and appeals dataset production

3. Published data, methodology and definitions

DWP’s statistical team combines the aforementioned data sources and creates 3 new databases to underpin their suite of official statistics.

3.1 ESA WCA Official Statistics cohort dataset

The ESA WCA Cohort dataset is a historic dataset containing ESA benefit data, health assessment data and appeals data. Production is performed using a statistical software package named SAS. The dataset contains data from when ESA was introduced in October 2008 to the latest administrative data held. For example, the September 2016 release contains:

• NBD benefit administrative data up to March 2016
• MSRS health assessment data to June 2016
• HMCTS appeals data to June 2016

This enables us to table ESA benefit, health assessment and appeals information for claims which had a health assessment up to March 2016 (6-month lag).

We also provide tables by ESA claim start date, for claims which started up to the end of December 2015 (9-month lag).

From June 2018, new lower-level geographical breakdowns were included in the cohort dataset. Previously, geographical information in the statistical release was obtained from postcodes using the underlying ESA administrative data and derived using 2001 Census Output Areas (COA). The methodology has now been improved and involves using more reliable Customer Information System (CIS) data to source address information and geographies are now based on 2011 COAs.

Read section 3.5 more information about the methodology.

Timeliness

It is important to note that combined data on WCA outcomes, MRs and appeals is not available until at least 9 months in arrears, and even then this may not be enough time to capture the full effect of appeals and longer assessment periods which can sometimes take more than 3 months. The reasons for this are to:

• enable processing of data sourced from inside the department and outside it from HMCTS and the Assessment Provider
• allow time for all decisions to be made and recorded; the assessment phase is usually 13 weeks, but in some cases it will take longer for a decision to be reached

Headline statistics on MRs are timelier as they aren’t linked to the ESA WCA or Appeals data and therefore include less retrospection and data processing time.

How data is created

The ESA WCA Cohort dataset is created by data merging procedures in SAS to combine ESA benefit, health assessment and appeals information and because we re-create the entire historic back series each quarter it usually takes approximately 5 days of SAS running time to create.

Refreshing the back series entails updating historic numbers with each issue to reflect change based appeals on initial decisions, which can take time to come through, and WCA decisions which are not recorded until after the 13-week assessment phase. It is therefore likely that the statistics underestimate the proportion of claimants who will ultimately be awarded the benefit, by greater amounts for more recent periods.

Retrospection

Figures by claim start date are released at least 9 months in arrears because of the time needed to process data, allow claimants to complete their assessment and align with the Ministry of Justice quarterly publication of Tribunal Statistics release schedule.

Definitions

On-flow – a claim is an on-flow to ESA that is made after 27 October 2008, and not a pre-existing IB claim.

Claimant – a claimant is anyone who has claimed the benefit.

Outcome – an outcome is where the assessment is completed, meaning in most cases that the limited capability for work questionnaire and face to face assessment has been undertaken, and the claimant is found Fit For Work (FFW) or else placed into the Work Related Activity Group (WRAG) or Support Group (SG).

Status – a status is where the assessment phase is not completed. The bulk of these are cases closed before assessment and the rest are those which are still at some point in the ESA WCA process.

For reassessments of IB, a claim is a pre-existing IB, SDA or IS claim going through the reassessment process for ESA after 11 October 2010:

• a claimant is anyone who has claimed incapacity benefits

• an outcome is where the assessment is completed, meaning in most cases that the limited capability for work questionnaire and face to face assessment has been undertaken, and the claimant is found FFW or else placed into the WRAG or SG. Outcomes are the final DWP Decision Maker’s (DM) decision, or the recommendation made by the Assessment Provider, when the DM’s decision is not available

• a status (on table 10a) is where the reassessment process is not completed. This includes cases closed before assessment and those which have not yet completed the ESA WCA process

The publication uses the final DWP DM’s decision, or the recommendation made by the Assessment Provider, when the DM’s decision is not available. This provides a more complete analysis. The DM’s decision may not be available for the following reasons:

• for some cases the DWP decision was not yet available when the data was extracted, in these cases the Assessment Provider recommendation is classed as the outcome. The Assessment Provider recommendation will be replaced by the DWP decision once this is available

• DWP’s benefit administration datasets which underpin this publication do not contain information on all FFW decisions. Information on whether a DM has found a claimant FFW is only available if the claimant appeals this decision. For claimants who have to date not appealed their FFW decision, only information on the Assessment Provider FFW recommendation is available. This will be replaced by the DM’s decision if this becomes available (for claimants appealing) or it is inferred that the DM agreed with the Healthcare provider recommendation if the claim was subsequently closed

• a claim will be identified as having started the reassessment process when it is referred to the Assessment Provider by DWP Operations. Upon receipt of the referral to the Assessment Provider, the claim is recorded on the MSRS live system. This data received from this live system is matched with DWP administrative data to identify outcomes or statuses. Data is matched using the encrypted National Insurance number and the dates relating to the assessment and decision making process. Note that small discrepancies in dates are tolerated in the matching process, to allow for the time taken for claims to be transferred between DWP and ATOS and any time taken between receipt of case data and when records are updated

3.2 Mandatory Reconsiderations data

The MR statistics are derived from data sourced from the Decision Maker and Case Recorder (DMACR) computer system used by DWP Decision Makers and Jobcentre Plus staff to record decisions within the ESA claim process.

How data is created

Data is extracted from the DMACR LIVE system on a monthly basis. Data processing involves merging the current data with historic data and removing duplicates due to cases from the ‘live system’ being removed after 14 months.

Sometimes an earlier error is found or a claimant provides additional evidence following a decision on their ESA claim before an MR is raised. In such cases, the DWP Decision Maker (DM) will consider whether this additional evidence changes the original decision. The DM will then choose to revise or not revise the decision accordingly. This is known as ‘reconsideration’. If the claimant still disagrees with the resulting decision, they can then raise a MR.

These ‘reconsiderations’ mimic MRs, but because they are easier to resolve, they are ‘revised’ at the earliest opportunity by the Benefit Centre instead of being sent to an independent Dispute Resolution team (DRT). Therefore, due to operational practices performed within the DMACR system, some MRs are recorded on the source data as ‘reconsiderations’.

Following investigation and analysis, analysts found it is not possible to differentiate which of these system actions is a true MR or ‘reconsideration’. Therefore, for the purpose of these statistics, ‘reconsiderations’ occurring after the WCA (for reasons typically associated with MRs) are included in this data. This typically increases the total number of MRs by around 10%, as indicated by analysis to determine the mean on figures since MRs were introduced in 2013.

Retrospection

The DMACR data is received monthly and contains little retrospection, which enables us to publish MR registrations, outcomes and clearance times with just over 1-month lag. For example, MR outcomes covering to the end of July 2016 were published on 8th September 2016. The historic figures are refreshed each quarter for each publication therefore the most recent figures may change slightly in the next release.

Definitions

MR decision outcomes - there are multiple outcomes that refer to whether or not the Decision Maker (DM) chooses to revise the decision under dispute:

• if a decision is categorised as ‘Not Revised’ this means that the decision the claimant is questioning has not been changed
• if the decision under dispute is classed as ‘Revised – Allowed’ then the DM has changed the decision in the claimant’s favour. So someone found Fit for Work (FFW) would be awarded ESA or someone put in the Work Related Activity Group (WRAG) claimant would be put in the Support Group (SG)
• if the decision is categorised as ‘Revised – Disallowed’, the DM, having reconsidered the decision, decides that the claimant is not entitled to ESA at all. This would apply where someone put in the WRAG seeks a revision to go into the SG but is then found FFW. These decisions are not at all common – less than 2% to date
• if the decision is categorised as ‘Withdrawn’, the claimant has chosen to remove their MR request

MR Decision categories: There are reasons that result in an ESA claimant raising a MR. For these statistics only those reasons relating to the WCA process have been included.

The main reasons have been categorised as follows:

1. the primary reason for a claimant raising a MR is when they are found FFW following a medical assessment. All MRs relating to this topic are categorised as ‘Customer disputes FFW decision’

2. the second category ‘Customer failed to attend WCA, provide medical evidence or return questionnaire’ incorporates reasons where the claimant has failed to follow the required claim procedures. The MR gives them an opportunity to explain why they didn’t comply

3. the third category is ‘Customer disputes ESA group’ and contains MRs where the claimant is already assigned to an ESA group

Median ESA WCA MR clearance time: The average clearance times is derived using calendar days by calculating the median of the time taken from the date the MR is received in the Benefit Centre (BC) until the date the decision is cleared by the decision maker at the Dispute Resolution Team (DRT). These dates are obtained from the DMACR reporting system.

The date logged on DMACR as the date received is the date when the BC has decided that the MR received is a valid MR, having considered whether they can initially change the decision in the light of any new information.

The cleared date is the date when the decision maker at the DRT has cleared and logged the final decision on the DMACR live system.

The total clearance time therefore includes the time taken to transfer the case to DRT.

3.3 ESA WCA Cohort data

From September 2016, we created an ESA WCA Cohort dataset incorporating MR data from DMACR into the existing ESA-WCA Cohort dataset. This was to enable us to publish information on the different stages of the ESA claim process and better understand the claimant’s end-to-end journey through claiming ESA, WCA, MR and appeal.

This is a particularly complex process due the difficulty matching MRs with the correct referrals. Claimants can have multiple repeat assessment referrals and these referrals can have more than one MR occurring at any point in the customer journey.

How data is created

We create the MR data by running a series of additional SAS programs which merges the latest ESA WCA Cohort dataset with the DMACR data for the same period. Once merged together, to ensure we capture all possible cases in the joined-up dataset, we need to keep a record of the number of ESA WCAs and MRs per individual (using a unique customer identifier).

We then then split the merged individual level data into 6 specific scenarios (based on the results) and handle each scenario differently to help identify and keep the best matched record(s) based on observed data characteristics and minimal time periods (using tailored matching windows) for each specific individual.

The 6 scenarios are:

1. No ESA-WCA, One or more MRs
2. One or more ESA-WCA, No MRs
3. Single ESA -WCA, Single MR
4. Single ESA-WCA, Multiple MR
5. Multiple ESA-WCA, Single MR
6. Multiple ESA-WCA, Multiple MR

The 5 datasets are then concatenated before being sorted by individuals (using a unique customer identifier) and relevant dates.

Matching data across administrative systems is not an exact science and there are risks around proportionate burden (as we are dealing with millions of records) so we’ve devised a methodology to balance production of the most accurate end-to-end statistics on the ESA WCA, MR and Appeals process with efficiency.

To reflect an accurate picture of all MRs, we’ve included ESA WCA MRs that occur both before the work capability assessment (for reasons such as non-return of questionnaire, or failed to attend a medical assessment) and those that occur after the work capability assessment (often raised to dispute an outcome decision on a particular case).

MRs for official statistics purposes contain ‘reconsiderations’ that occur after the WCA, due to them displaying similar characteristics to MRs. This increases the total number of MRs by around 12%.

There are limitations in recording statuses at some points of the end-to-end process, which may mean that outcomes may not be recorded 100% accurately. Where this occurs we have used alternate data variables as proxy’s and labelled our outputs as such. For example:

• the regular cohort data does not capture the initial DWP decision as only the latest DWP decision is kept, therefore we have used the ATOS recommendation as a proxy
• to reflect the final DWP decision where an MR has been overturned (but the outcome doesn’t show this) we have allocated a small number to a ‘higher group’ category: for example, overturned FFW decisions are allocated to WRAG, and overturned WRAG MRs are labelled SG

Due to merging methods there may be occasions (particularly when dealing with multiple MR records) where we get slightly more records than we should. The best matching record(s) are chosen based on the individual’s scenario and minimal time periods – but due to limitations caused by many to many joins in SAS, there may be a small number of duplicates inherent in the data. Analysis shows that these duplicates account for less than 1% of the total number of MR’s.

As above, multiple MRs will cause issues when joining the individual pre-WCA MRs, post-WCA MRs and post-WCA Reconsideration datasets, therefore we need to use only the latest MR record in the cohort for each WCA (the latest outcome is likely to be more reliable, as statuses are often re-written as individuals move through the process). Only 1% of these are multiple MR’s so should not impact greatly.

Not all MRs are included in the cohort as there may not be a match across all systems. Therefore, when joining the individual pre-WCA MRs, post-WCA MRs and post-WCA Reconsiderations together, we work on the assumption that all individuals exist in ESA-WCA Cohort dataset. This means a small number of MRs may be dropped, because we are unable to use them to tell the full end-to-end story). Around 1% of records are dropped at this stage.

Table showing MRs which we would not include.

Individual	ESA WCA record	Pre-MR	Post-MR	Post-Recon
NINO1	Yes	No	Yes	No
NINO2	Yes	No	Yes	Yes
NINO3	Yes	No	No	Yes
NINO4	Yes	Yes	No	No
NINO5	Yes	Yes	Yes	No
NINO6	Yes	Yes	No	Yes
NINO7	No	No	Yes	No
NINO8	No	Yes	Yes	No

Retrospection

The time lags and retrospection for the the ESA WCA Cohort dataset mirror that of the regular official statistics cohort dataset. The only difference is that we restrict the ESA WCA Cohort data to ESA claims with a start date after October 2013, which is when MRs were introduced.

This ESA WCA Cohort dataset will be extended each quarter by including the next 3 months’ worth of data.

3.4 ESA clearance times

From September 2017 we published ESA clearance times statistics. This was following a Public Accounts Committee (PAC) report ‘Contracted out health and disability assessments’, published on 31 March 2016, which recommended the department should publish more data to enable the tax payer, the department and parliament to better understand provider performance. The department accepted this recommendation with the caveat that we would explore what was possible.

The clearance times are published for initial claims by month of clearance in median weeks. They are broken down as follows:

• full end-to-end process times from claim registration to final DWP decision
• time from claim registration to WCA referral
• time spent from WCA referral to Assessment Provider (AP) recommendation
• time from Assessment Provider (AP) recommendation to final DWP decision

From June 2018, regional information for the above breakdowns for initial claims were published as Excel tables, further fulfilling the PAC recommendations to publish national and regional data on provider performance.

How data is created

The clearance times data uses the MSRS data from the healthcare provider as the spine of the dataset. Referral dates are taken from this data to enable derivation of the provider clearance times. The provider clearance times (from WCA referral to Assessment Provider recommendation) include claims that ‘loop’ around the system. These cases can occur, for example, where a claimant fails to attend their assessment. They then loop back around the process and this looping can result in several rows of referral data in the MSRS dataset. In these instances, the first referral start date and last referral end date are recorded for the provider time for that claim.

The MSRS data is then joined to the ESA WCA Cohort dataset to obtain required variables and dates to derive clearance time breakdowns for other stages of the process. Initial claims only are identified and taken forward. Repeat assessment clearance times cannot be derived because their start dates are linked to the initial claim start dates and therefore timings cannot be extracted independently.

After investigations and Quality Assurance against MI developed within the department, we chose not use the claim start date from the ESA WCA dataset as a starting point for the process end-to-end clearance time. This start date shows the date from which the claim has been in payment and can be backdated. A backdated start date is not an accurate representation of when a claim was registered to indicate the start of the claim process.

• Therefore, because the claim registration date was not available in the ESA NBD data we had to obtain it from an alternative data source. As a result the Atomic Data Store (ADS) from the DWP data warehouse has been used to source the claim registration date. This data has been developed within the department from legacy feeds and provides the information needed to obtain the required clearance time accuracy. However, the ADS was switched off on 12 April 2020 and replaced by a tactical data set

• From 12 April 2020 claim registration dates have been sourced from the new tactical feed the Historic Data Store (HDS) from the DWP data warehouse

• The final data is obtained using relational joins within the ADS for claim registration dates up until 12 April 2020, then registration dates from the the HDS from 12 April 2020. This data is then joined to the MSRS and cohort datasets and used alongside the final DWP decision date to derive end-to-end process clearance times

• The ADS-HDS claim registered date is also used in calculating the time from claim registration to referral to the assessment provider

All average clearance times reported in the statistics are based on the median measure. The median time is the best measure to use when estimating how long people have been waiting to receive ESA. The median time is the middle value if you were to order all the times from lowest value to highest value. The median is presented here instead of the mean, as the mean measure can be unduly affected by outlying cases, for example, cases where the person has been hard to reach due to being in prison, hospital or failed to attend their assessment on numerous occasions.

The measure of median weeks is calculated using weekdays and rounded to the nearest number of weeks. The end-to-end median is derived from all clearances across the full process. Median clearance times across different stages of the process will not sum to the end-to-end clearance figure as clearance times at each stage are a median of time taken across all clearances for that stage only and are therefore independent to clearance measures at other stages. Only claims that have completed the ESA process are included.

Around 6% of claims have overlapping dates between the stages resulting in the end to end process time not summing exactly to the sum of the individual clearance components. This refers to the raw clearance times at claim level rather than the monthly median times and has minimal impact on the published figures showing median weeks per month.

Retrospection

The time lags and retrospection for the clearance times data mirror that of the regular National Statistics cohort dataset.

In deriving the last part of the process – assessment provider recommendation to final DWP decision – the DWP decision date is the latest decision held on that claim at the time the statistics were extracted. If a mandatory reconsideration has been recorded at that time, then the date will include those decision dates. Therefore, the clearance times at this stage of the process are subject to slight retrospection once open MRs are cleared. For the same reason, the end-to-end process times are also subject to slight retrospection.

Definitions

Claim registration date – date the claimant has first contact with the department when making a claim to ESA.

Referral start date – date the claimant is referred to the assessment provider for assessment.

Referral end date – date the assessment provider makes a decision recommendation on the assessment.

DWP decision date – date the DWP decision maker makes a final decision on the ESA award following the provider recommendation. This can also include MR and appeal dates where they are completed.

Median weeks – the clearance time measure is derived in working days and converted to the nearest number of weeks for publication.

Cleared claim – these statistics only hold clearance times for those claimants that have completed the end-to-end process. For example, claims that have closed before assessment or been withdrawn are not in scope.

Revisions

From the June 2019 publication, a revision has been made to the figures to the ESA WCA median clearance times, which has resulted in figures being revised from May 2017. This was due to the introduction of additional codes to the source data which were not reflected in our data processing. The previous published figures have therefore been revised and more in line with historic trends.

Please treat with caution clearance time data relating to May 2017 and after in publications before June 2019. The figures have been revised in the June release and will be updated on Stat-Xplore in the September 2019 release.

3.5 Lower-level geographical breakdowns

Due to increasing demand from users, new lower-level geographical breakdowns (Region, Local Authority and Westminster Parliamentary Constituency) were included in the ESA WCA dataset from June 2018 and made available on Stat-Xplore. A further amendment was made from March 2023 providing breakdowns to Ward level and Scottish Parliamentary Constituencies.

Previously, geographical information in the statistical release was obtained from postcodes using the underlying ESA administrative data and derived using 2001 Census Output Areas (COA). An improved methodology involves now using more reliable data to source address information using the Customer Information System (CIS) and geographies are derived using 2011 COAs. The COA 2011 hierarchy is built up from COA to the National Level based on the address of the claimant.

Residency based geographies are derived from address information as recorded on the CIS. CIS is a more reliable source of addresses as it links to all of the DWP benefit systems and contains the most up to date address for each individual.

These addresses are then put through a data cleansing procedure which makes sure postcodes are formatted correctly and the address fields are populated correctly. 2011 COAs are then assigned to claimants using the ONSPD (ONS Postcode Directory), starting with a direct postcode to COA lookup and then working through a logical allocation routine. These COAs are then used to merge on higher level geographies from the National Statistics Postcode Look-Up (NSPL).

This file is then matched to the ESA WCA data, ensuring that the date of decision is between the address start and end of spell dates. If no known address can be matched for a period of time, the closest address for that claimant is used, as this is more accurate than random allocation. There may still be a very small number of records that are still recorded as unknown. Previously published regional information in Table 1a assigns ‘abroad’ cases as ‘unknown’. Using the new methodology, ‘abroad’ cases appear as ‘abroad’ and have been differentiated from ‘unknown/missing’ cases on Stat-Xplore.

Read our DWP statistical geographies note for more information about the methodology changes.

3.6 Medical condition codes: revised ICD code groupings

In March 2018, the Department for Work and Pensions, as part of their quality assurance process, noticed discrepancies regarding the published figures for some disability benefit claimants by medical condition, which impacted the ESA WCA release. Whilst the issues were investigated, all the information relating to medical conditions was removed from our online sources and a note was attached asking users to exercise caution whilst using these statistics.

Information on the scale of the revisions for sanctions.

The issues stemmed from the mapping of medical conditions on the DWP administrative system from IRG (Incapacity Reference Guide) to ICD (International Classification of Diseases), the form in which the statistics are published. In some cases, the IRG framework had been changed without the relevant mapping having been updated to reflect this, whilst in others the mapping was found not to have been suitable from the outset. As such, certain medical conditions were being assigned to IRG codes which were then being mapped to ICD codes that did not correspond to the original conditions.

Having engaged with and received support from DWP doctors, an updated mapping document was created and applied to the data, meaning that the statistics could be republished. Whilst many medical grouping caseload figures were unaffected, others have seen their numbers altered by the new system of mapping used with approximately 18% of the DWP medical codes now mapping to a different statistical medical condition group. At the higher-level at which the statistics are published, with the revised mapping applied to the data as of December 2017, 91% of the high level medical codes caseloads remain the same.

Effect of revised mapping on high level ICD codes for ESA WCA as at December 2017

ICD Chapter	Description	Difference (thousands)	Percentage Difference %
All	Total	0.0	0
1	Certain Infectious and Parasitic Diseases	-0.8	-67
2	Neoplasms	-0.1	-2
3	Diseases of the Blood and Blood forming organs and certain diseases involving the immune mechanism	0.0	0
4	Endocrine, Nutritional and Metabolic Diseases	0.0	0
5	Mental and Behavioural Disorders	1.0	3
6	Diseases of the Nervous System	0.2	7
7	Diseases of the Eye and Adnexa	0.0	0
8	Diseases of the Ear and Mastoid Process	0.0	0
9	Diseases of the Circulatory System	0.0	0
10	Diseases of the Respiratory System	0.0	0
11	Diseases of the Digestive System	0.0	0
12	Diseases of the Skin and Subcutaneous System	0.0	0
13	Diseases of the Musculoskeletal system and Connective Tissue	-0.2	-2
14	Diseases of the Genito-urinary System	-0.1	-20
15	Pregnancy, Childbirth and the Puerperium	0.0	0
16	Certain Conditions Originating in the Perinatal Period	0.0	0
17	Congenital Malformations, Deformations and Chromosomal Abnormalities	0.0	0
18	Symptoms, Signs and Abnormal Clinical and Laboratory findings, not elsewhere classified	-0.0	0
19	Injury, Poisoning and certain other consequences of external causes	-2.4	-42
20	External causes of morbidity and mortality	0.1
21	Factors influencing Health Status and Contact with Health Services	2.2	200
22	Codes for special purposes	0.0
99	Unknown or claimants without diagnosis on the system	0.0

4. Which data should be used for what purpose?

It is important to note that the regular official statistics dataset should be used in the first instance to answer ESA WCA queries, however if the information cannot be obtained from this data, then the ESA WCA Cohort data should be used.

The ESA WCA Cohort data is mainly used to give customers a view of the end-to-end process.

Queries relating to MRs should be initially answered using the MR dataset, as this dataset contains all MRs and the most up to date data. The ESA WCA Cohort data should only be used if the information can’t be obtained from the regular MR official statistics data.

5. Quality assurance

An overview of our quality assurance stages and processes.

Source data collection and validation methods for ESA WCA

Derived ESA-WCA dataset processes and quality assurance

Quality assurance overview

Further details of initial and routine quality assurance conducted ahead of each quarterly release.

Initial quality assurance conducted during data development

A large number of quality assurance checks were carried out during the early development of the statistics to ascertain:

• reliability, completeness and level of disclosure of individual variables
• levels of duplicate, missing or contradictory information
• consistency across computer systems (Benefit data, DMACR, HMCTS, ATOS) and with management information, via cross checking
• monthly build-up of figures in system data
• trends and variation in characteristic, time series and geographical breakdowns
• data linking checks to make sure the benefit claims, ATOS referrals, MRs and Appeals have been linked together accurately
• consistent results during time series and cohort development via dual methodology testing and manual checking
• visits to DWP Benefit Centres to assess procedures and use this to help shape quality assurance checks
• initial meetings and regular contact with Data Providers to inform of planned changes to source data

This quality assurance found no preventative issues; it showed the data was robust, consistent with management information and suitable for publishing.

The above processes have subsequently been used to further develop data cleansing rules, assess retrospection and timeliness as well as form routine quality assurance procedures.

Routine quality assurance

A standard set of quality assurance procedures are conducted for each quarterly statistical release which consist of checking:

• duplicate, missing or contradictory information
• accordance across computer systems (DWP and ATOS) and with management information
• trends and variation in characteristics, time series and geographical breakdowns
• reference file checks
• underlying data distributions and quarterly dataset comparisons
• dual methodology testing of time series and cohort data
• checks against Management Information

These checks are automated wherever possible and manual checks are performed when further investigation is required.

Additional quality assurance for ESA WCA Cohort data

To quality assure our ESA WCA Cohort data we:

1. Ensure underpinning data meets individual quality assurance standards before being used in the overall cohort matching process by defining and channelling data through a set of business rules that valid cases can follow.

The approximate proportion of data records that match with these rules is as follows:

• pre ESA WCA MR individual level QA: 96%
• post ESA WCA MR individual level QA: 98%
• post ESA-WCA Reconsiderations individual level QA: 69%

You can see from the rates shown above that we are able to better minimise the matching window for MRs as we can pinpoint matching dates with more certainty – we know MRs can be accepted and processed up to 13 months after the initial benefit decision is made. So our match rate is much higher.

However, in principle, as reconsiderations can happen at any point within the end-to-end ESA WCA process we need to specify a larger matching window to ensure we capture as many matches as we can. We have set the matching window at 1 year before and 3 years after the initial decision being made. This appears to maximise the matching rate, but on comparison, provides a much lower matching rate than the equivalent rate for genuine MR’s.

The lower percentage does not mean the records should not be included, but indicates that the records don’t necessarily match an expected pattern of behaviour and therefore we may not have matched the exact record – as some elements of case information may still be in progress across the systems. The match will have been made to the correct person at an individual level – but the outcome status may not reflect the precise outcomes. This affects only a small number of cases.

2. We then make aggregate level comparisons against wider data sources and existing published statistics (e.g. appeals rates). MI counts in general are likely to be higher – as official statistics are more refined. We run automated checking programs against the final cohort dataset – to check for unexpected changes in the underlying data distributions for key variables and review high-level time-series charts showing how key statistics change over time.

3. We than table the cohort data and carry out “by eye” checks at summary levels and across lower level time period breakdowns.

Here we look at cumulative ESA WCA, MR and Appeals cohort data broken down by a range of assessment periods including year, quarter and month of completed ESA WCA assessment.

This gives a sense of data quality/accuracy and identifies differences in underlying trends. This lower level cohort information is made available to users within our existing suite of published Excel tables.

6. Statistical publication

This statistical publication brings together all of the relevant information and figures from the detailed Excel tables, Stat-Xplore and where necessary provides links to more information. It can only be completed once all quality assurance stages have been completed on the data. The statistical publication is published on this website and contains headline figures (for example, national totals, top-level figures). It is published quarterly.

DWP pre-announce the release date of the statistical publication at least 1 month in advance, in accordance with release practices set out in the Code of Practice for Official Statistics. Dates of future publications can be found on the statistics release calendar.

As the statistical publication is a National Statistics release there is an internal departmental requirement to accompany this release with a submission. The submission is circulated to the Minister’s office and contains input from policy areas. The submission is sent one day prior to release day and is fully compliant with the UKSA rules surrounding pre-release access of statistics. It contains a blend of the most notable issues within the latest release, key features, and policy guidance. This document is also quality assured in the same manner as the statistical publication, before being sent out.

Once completed, the lead statistician will ensure everything has been checked and passed, and if happy with everything, will sign off the Excel tables, Stat-Xplore data and publication and summary. All actions are recorded on a release timetable, and are checked off, once complete, to ensure that no steps are missed.

The statistical publication was revamped from September 2016 to include MR information and new, ESA WCA Cohort information showing the full end-to-end claimant journey. From September 2017, experimental statistics on ESA WCA end-to-end clearance times for initial claims were included in the publication. From June 2018, selected breakdowns from the cohort dataset, including new lower-level geographical breakdowns (Region, Local Authority and Westminster Parliamentary Constituencies), were made available on DWP’s online tabulation tool, Stat-Xplore. Further breakdowns were added in March 2023 for Ward level and Scottish Parliamentary Constituencies and were made available on Stat-Xplore.

Data confidentiality

The Code of Practice for Statistics (CoP), specifically Principle T6: Data Governance and the National Statistician’s guidance: Confidentiality of Official Statistics (CoOS guidance) set out principles for how we protect data on individuals from being disclosed.

Some figures in this publication are derived from Stat-Xplore and are subject to Introduced Random Error to ensure that no data is released which could risk the identification of individuals. Figures contained within the Statistical First Release (SFR) are also subject to additional rounding unless otherwise stated. The level of rounding applied, which is dependent on the magnitude of the figure being quoted, is shown in the table below.

Since March 2017, the following rounding policy has been applied to the statistical first release. Please note that percentages shown within the release are calculated using figures prior to rounding.

Statistical first release rounding policy

Range	Rounded to the nearest:
0 – 1,000	10
1,001 – 10,000	100
10,001 – 10,000	1,000
100,001 – 1,000,000	10,000
1,000,001 – 10,000,000	100,000
10,000,001 – 100,000,000	1,000,000

7. Release strategy

The Excel tables and statistical publication are released each quarter. Stat-Xplore will also be refreshed with the latest data with each publication.

The publication is aligned with the Ministry of Justice Tribunals Statistics release. The publication is released on the same date and also uses the data which is 3 month lagged at publication date, for example appeals data covering up to June 2016 was used in the September 2016 publication.

The “ESA: outcomes of Work Capability Assessments including mandatory reconsiderations and appeals” quarterly official statistics publication is added to the GOV.UK website at 9:30am on the day of release.