Annex A: methodological and technical details
Published 29 July 2021
© Crown copyright 2021
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/statistics/child-and-working-tax-credits-error-and-fraud-statistics-2019-to-2020/annex-a-methodological-and-technical-details
Annex A
The tax credits Error and Fraud Analytical Programme (EFAP): methodological and technical details
Introduction
The tax credits system is designed to respond to changes in circumstances as they happen. A claimant’s entitlement can therefore change throughout the year which could lead to over or underpayments depending on when the claimant tells HMRC about the change, either in year or at finalisation. Error and fraud can therefore only be found after the claim has been finalised with the actual circumstances of the tax year. The tax year 2019 to 2020 exercise could not start until recipients had provided HMRC with details of their final tax year 2019 to 2020 circumstances, which meant that compliance officers were unable to start work on some cases until after 31 January 2021.
The impact of COVID-19 on HMRC operations meant that publication of the first estimate of tax credits error and fraud was delayed from the usual June publication date to July 2021.
Error and fraud
When Claimant Compliance Officers identified non-compliance, they were required to indicate whether they believed it was due to genuine error or fraud. To be classified as fraud, a caseworker needs to have found evidence that the claimant deliberately set out to misrepresent their circumstances to get money to which they are not entitled (for example claiming for a child that does not exist).
Error covers instances where there is no evidence of the claimant deliberately trying to deceive HMRC. It covers a range of situations, including cases where a claimant inadvertently over-claims because they simply provided HMRC with the wrong information. It could also cover a situation where the correct information has been provided but this information has been incorrectly processed by HMRC.
As part of the working of each case, compliance officers were asked to classify whether or not a case that was found to be incorrect was due to either error or fraud, as well as whether or not the error was due to HMRC.
For cases where error or fraud have been identified the Claimant Compliance Officer also has to identify the causes of the error or fraud, the monetary consequence of this and the adjustment categories which are shown in section 2 of the commentary.
Due to the nature of organised fraud and HMRC compliance procedures the vast majority of organised fraud claims are stopped quickly and awards in payment are terminated. Any cases identified as having an ongoing organised fraud investigation during the EFAP process are not taken up by the EFAP caseworker. Any overpayments as a result of organised fraud are included in the annual National Statistics publication on under and overpayments.
Sampling
The sample for the tax year 2019 to 2020 EFAP is constructed from 2 strata of claimants; these strata, together with the sample sizes, are shown in table A1 below.
From the tax year 2018 to 2019 EFAP onwards, nil awards were removed from the sample, as these have been found to contain negligible amounts of error and fraud. This has increased the number of cases selected in the other strata improving the confidence levels of the outputs.
Table A1: sample strata and sample sizes
| Stratum | Sample Size |
|---|---|
| WTC only | 490 |
| Others | 3,510 |
| Total | 4,000 |
The sample was stratified in this way to ensure that an appropriate number of both Other and WTC only awards were included in the sample. If a purely random sample had been used this would have consisted of a high number of Nil and Flat rate awards which show relatively low rates of non-compliance, thus reducing the accuracy of the results of the EFAP.
The use of a stratified sample allows for the levels of error and fraud in each stratum to be estimated more accurately by ensuring the number of cases in each stratum is representative of the likelihood of error and fraud occurring in that strata of the population.
Cases that fall under special customer records policy (secure and sensitive cases) are excluded from the sample.
An individual award can fall into a number of different strata during the year depending on the circumstances of the household at a given point in time. For example a couple could initially be receiving WTC only and then half way through the year have their first child thus moving them to our other strata.
There are 7 possible categories (which we aggregate into our strata) that a household in award could find themselves in at a given point during the year depending on their circumstances and income. When an award moves between these categories we say that a new entitlement sub-period has been created.
It is important to note that our sample base is awards and not families – these two differ as a family can have a number of awards during a year. Take the following example: initially a lone parent family is in award then a new household is formed when a partner moves in and later in the year the partner moves out (the household breaks down) and they become a lone parent again. In total they have had three separate awards during the year. We follow awards as this is the unit that the tax credits system is based around and hence is most suitable for constructing a representative sample from.
The sample base contains all tax year 2019 to 2020 awards present on the HMRC tax credit system at the end of the first week of August 2019. An award may last for a period of anywhere between one day and the whole year.
The sample for each stratum was selected at random. The minimum sample size for each stratum is 50 to allow for further breakdown of the results internally.
It is possible for awards to migrate to Universal Credit (UC) during the EFAP estimation year. If an award migrated to UC prior to 1 September of the EFAP estimation year (before 1 September 2019 for the tax year 2019 to 2020 estimation), the award is excluded from the sample and a different award selected. This occurs during the sampling process, therefore the total number of cases selected will always be 4,000. If the award migrates to UC after 1 September of the estimation year, it is included in the sample for the period of the year that the award was in payment.
Sampling errors around the estimates
Estimates in the tables are rounded to the nearest £10 million or 10,000 in tables 2, 4, 5, and for all the overall totals in the other tables. The breakdowns in the other tables are rounded to the nearest £5 million or 5,000. The error and fraud rates are rounded to the nearest 0.1% in tables 1 and 3.
The estimates presented are the central estimates derived from the sample taking account of the methodological approach set out below. Since these estimates are based on a sample they are subject to sampling errors. These margins of error have been expressed by calculating a 95 per cent confidence interval around the estimates. These have been calculated and are shown in tables 1 to 4.
Confidence intervals are calculated using the variance of the values in the closed case data. The uncertainty around the open case projections is assumed to be the same as the closed cases.
Methodology
The following section sets out a number of different methodological issues - such as how we process the data, how cases in the sample have been scaled up to represent population estimates, how certain cases have been treated, etc.
Processing
The underlying data are recorded by the compliance officers who carried out the enquiries. It then undergoes a number of steps where it is checked and processed before it is used to calculate the figures in this publication. Compliance officer decisions are checked at the case closure stage by reviewing all supporting evidence used to make the decision, both that supplied by the caseworker and contained in HMRC systems. All calculations are also checked for financial accuracy at the case closure stage.
The final data used are created by cross checking the information held in our compliance management information system against that held in the main tax credit computer system and against information recorded about the case by the compliance officer who worked it. The data is corrected if there is a discrepancy between the systems to assure all of the data is correct before completing the analysis.
Each award has a number of entitlement sub-periods and it is clear that some of these sub-periods cannot be associated with certain types of error or fraud that are recorded, for example if 25 per cent of an award’s time is spent in a WTC only sub-period and 75 per cent of its time in sub-periods relating to CTC then a claimant favour error or fraud relating to a child could only have occurred in the latter 75 per cent of the award. We therefore allocate the error to the sub-periods that it could be associated with, so in the earlier example the child error would be allocated to the 75 per cent of the award spent in sub-periods relating to CTC. Error favouring HMRC has been reallocated between sub-periods based on the proportion of that award spent in that sub-period.
Classification of the 4,000 sample
The EFAP cases can either end with a claimant favour, revenue favour, or no adjustment after the intervention. We will receive information from the claimant through the enquiry in the majority of cases with a number not responding to the investigation. Table A2 sets out how the tax year 2019 to 2020 cases are broken down.
Table A2: breakdown of EFAP cases by response and outcome
| Net Claimant Favour | Net HMRC Favour | Total | |
|---|---|---|---|
| Responded: | |||
| with error and fraud | 531 | 321 | 855 |
| without error and fraud | - | - | 1,836 |
| No Response: | |||
| with error and fraud: | 154 | 8 | 162 |
| without error and fraud | - | - | 717 |
| Not Taken Up | - | - | 138 |
| Open | - | - | 292 |
| Total: | - | - | 4,000 |
Cases can have both claimant favour and HMRC favour error and fraud. Table A2 shows the net position of those cases, where a case with a total claimant favour adjustment is classed as in claimant favour and a case with a total HMRC favour classed as HMRC favour.
Cases that do not have error and fraud, and have not been worked or are still open will not be in either Claimant or HMRC favour and so no breakdown is provided in the table.
Note that it is possible for a case to contain equal values of claimant favour error and fraud and HMRC favour error, meaning that although the case contains error and fraud, the net value of error and fraud is 0, and is neither in net claimant or HMRC favour. This means that the sum of cases with net claimant favour error and fraud and net HMRC favour error may not match the total.
Non-response
Approximately 25 per cent of claimants in the sample that is used to compile this estimate do not respond to HMRC’s investigations. The issue of non-response is monitored in several ways, including ensuring that compliance officers are in a position to make a valid decision without a response, completion of extensive quality checks of compliance officers’ decisions and monitoring of the outcome of non-response cases against those where claimants do respond.
Follow-up analysis has shown that non-response cases are no more or less likely to contain error and fraud favouring the claimant than cases where the claimant does respond. Consequently, we are satisfied that compliance officers are able to make a valid decision on non-response cases by using information held by HMRC. No adjustment is made to the estimate of error and fraud favouring the claimant to account for non-response.
Error favouring HMRC is more likely to be identified in cases where the claimant does respond. It is not possible to determine whether the non-response cases do in fact contain higher levels of error and fraud than we have identified but we hold no evidence to suggest that they do. No adjustment is made to the estimate of error favouring HMRC to account for non-response.
Not taken up cases
In this year’s exercise 138 cases were not taken up for enquiry for reasons including death or other exceptional circumstances. This number is in line with the amount of cases not taken up in previous years. These cases have been excluded from the results, implicitly assuming that if they had been worked they would have the same incidence of error and fraud as the cases that have been successfully completed.
Cases are also not taken up if they fall under special customer records policy. These cases are deemed to require additional protection. Because of this both EFAP caseworkers and analysts do not have the required permissions to access the customer information. These cases are therefore removed from the sample. Types of special customer records can include: Members of the Royal Household, members of UK legislative bodies including Scottish and Welsh Assemblies, VIPs and those in high–risk employment, victims of domestic violence and other high-risk individuals.
Open cases
There were 292 cases which had been opened but not completed when the first estimate was made. A projection was made to cover the estimated additional amount of extra error and fraud these cases would provide. When these 292 cases have been closed, the estimated projection will be replaced with actual values, and any revision will be published alongside the following year’s estimate.
It is assumed in this analysis that these incomplete cases exhibit the same characteristics, on average, to those that had been settled most recently and assumed that the cases left to work to the end will on average exhibit this average level of non-compliance. Where there is only a small number of sample cases for recently settled cases, the average level over a longer time period is used.
Projections for mandatory reconsiderations
Claimants that have been found to be in error and fraud are able to appeal the decision within 30 days of receiving the award notice unless there are exceptional circumstances. These are known as Mandatory Reconsiderations (MRs) and can change the estimated levels of error and fraud by removing amounts of error and fraud from closed cases.
Any MRs that are known before the results are estimated are incorporated into the analysis. To ensure the estimate in this publication is central, a projection is made to take into account MRs that are likely to be received after the publication of the results.
When the value of all MRs is known, the data is included in the final estimate.
Grossing
The sample results of the cases that have been worked to completion plus the projected results from the cases still being worked are grossed to reflect population estimates. Grossing factors have been applied depending on the value of the finalised award and the characteristics of the claimant during the year.
Sample results are grossed to the total of entitlement sub-periods for the population over the whole year rather than to the single entitlement sub-period present at the end of the year.
The sub-periods are grossed up to the position of the award on each of the sample strata which gives increased accuracy over groups with potentially differing rates of error and fraud.