Research and analysis

Comparing methods used in the COVID-19 Infection Survey and NHS Test and Trace, England: October 2020

Updated 6 October 2020

Applies to England

Main points

This is a definitional article outlining the methods used in the coronavirus (COVID-19) Infection Survey (Office for National Statistics (ONS)) study and NHS Test and Trace (Department of Health and Social Care (DHSC)).

Whilst comparisons have been made between these 2 data sources, they measure different aspects of the COVID-19 pandemic.

And whilst these are complementary data sources, drawing direct comparisons between these 2 data sources would result in misrepresenting the statistics, discrediting the conclusions drawn from them and misunderstanding the current infection rate of the pandemic.

Additional analysis undertaken by the ONS to understand the differences and how complementary the data sources are will follow this article.

Reasons the data cannot be directly compared

The methods and approaches from the COVID-19 Infection Survey and NHS Test and Trace are different. These differences are outlined in this section and demonstrate why direct comparisons should not be drawn between the data published from each study.

The way the data is reported is different

The COVID-19 Infection Survey is a longitudinal study which means participants can test positive more than once. For positivity rates, all positive tests are included in the modelling even if one individual has tested positive multiple times. This is because the positivity rate is estimating the number of people testing positive for COVID-19 at a point in time.

The incidence modelling only includes new positive cases. This means someone with multiple positive tests would be included in the incidence modelling the first time they tested positive but not subsequently (unless they tested negative and then positive again). Individuals must have had a previous negative test to be included.

NHS Test and Trace publishes the number of people newly testing positive each week. Figures for people testing positive have been de-duplicated so people who have multiple tests only appear once. Because of this, it is currently not advisable to calculate a positivity rate from the NHS Test and Trace data because the number of people tested in a given week will exclude some people who have been tested in a previous week, so may not be an accurate denominator to use. NHS Test and Trace counts a positive test at the time that the test was taken, regardless of when the person may have been infected.

The populations measured are different

The COVID-19 Infection Survey tests people who are in private households. As such, people living in care homes, other communal establishment and hospitals are not included.

Positive cases from NHS Test and Trace come from private households, care homes and hospitals.

The reasons for people getting tested are different

Individuals participating in the COVID-19 Infection Survey are randomly selected. Participants are tested every week for the first month and then every month from their first visit for a year. The COVID-19 Infection Survey tests everyone in the sample, regardless of whether they report they are experiencing symptoms.

The population being tested by NHS Test and Trace is self-selected and due to the way that people can get tested, those who come forward are more likely to be infected. However, there may be a significant proportion of people who will be asymptomatic or choose not to access testing. Individuals can get a test through NHS Test and Trace if:

• they have COVID-19 symptoms
• an employer has referred them
• they are in an NHS facility such as a hospital as a patient or NHS worker
• they are in a care home as a member of staff or resident (since 6 July, care homes should test staff weekly and residents every 28 days)

This means that the likelihood of an individual testing positive having symptoms between the COVID-19 Infection Survey and NHS Test and Trace are different. The ONS will conduct additional analysis to understand the impact of this difference.

The representativeness of the studies is different

The COVID-19 Infection Survey is based on a nationally representative survey sample. The results are weighted by age, sex and region to be representative of the community population in England.

People are tested through NHS Test and Trace based on whether they are experiencing symptoms or if they are in a higher risk occupation or area. As not all populations have the same risk of infection, the population of those tested will not be nationally representative.

The symptoms of those tested are different

The COVID-19 Infection Survey looks at the symptoms reported by participants who test positive for COVID-19 from a nose and throat swab. This can be on the day of the test or around the time of the test (day of, or preceding or following swab test).

This determines the percentage of people who had symptoms when testing positive for COVID-19. It is important to note that symptoms were self-reported rather than professionally diagnosed, and those without any evidence of symptoms will include instances where all questions about symptoms were not answered. Therefore, the number and percentage of individuals without any evidence of symptoms cannot be used as a proxy for the number and percentage of asymptomatic cases.

Whilst the Office for National Statistics does not report asymptomatic cases, there are people who do not report any evidence of symptoms. This could suggest the COVID-19 Infection Survey is finding a large proportion of the community population have COVID-19 asymptomatically. Because of the underlying principle of testing those at risk, NHS Test and Trace would be unlikely to find these cases. This means the rates of testing positivity would likely be different and should not be directly compared.

NHS Test and Trace predominantly tests people who are reporting symptoms, in particular a high temperature, a new continuous cough or a loss or change to sense of smell or taste. However, some people may have the symptoms other than the three symptoms above, commonly associated with COVID-19. Since April there has been increased testing of asymptomatic people as part of NHS Test and Trace. Testing of individuals who do not report symptoms is targeted in settings like care homes and is therefore not representative.

Conclusions

Given the different methodologies and approaches to collecting data on the number of people testing positive for COVID-19 in England from a nose and throat swab, we would expect to see differences in the reporting of those testing positive.

One of the main differences is the people being tested. In particular, in the COVID-19 Infection Survey, individuals are randomly selected and tested, whether they are showing symptoms or not. Given indications from other data sources about the proportion of asymptomatic cases this could mean the COVID-19 Infection Survey is finding significantly more cases than NHS Test and Trace, where those who are tested are either reporting symptoms or have been advised to have a test (for example, they might work in an occupation where there is a higher risk of transmission such as a hospital). Additionally, as NHS Test and Trace is self-selected, not all of those who have symptoms may request a test.

Because of the methodological differences, the NHS Test and Trace statistics and COVID-19 Infection Survey estimates cannot be directly compared. Drawing direct comparisons between these two data sources would result in misrepresenting the statistics, discrediting the conclusions drawn from them and misunderstanding the current infection rate of the pandemic.

Analysis will be undertaken by the ONS to further understand the differences in the number of people testing positive between the COVID-19 Infection Survey and NHS Test and Trace.

Data sources and quality

Background on the COVID-19 Infection Survey

The COVID-19 Infection Survey is conducted by the Office for National Statistics (ONS) in collaboration with Oxford University, Manchester University, the Wellcome Trust and IQVIA.

This surveillance study is one of several to address pillar 4’s purpose as set out in the government’s testing strategy. The survey has been running since the 26 April 2020.

ONS publishes a weekly bulletin and monthly article detailing the results of the survey. Read more information about the methods used in the COVID-19 Infection Survey.

Background on NHS Test and Trace

NHS Test and Trace was launched in England on 28 May and ensures that anyone who develops symptoms of (COVID-19 can be tested to find out if they have the virus. There is also asymptomatic testing of key workers and in care homes which helps to identify people in high-risk situations who have the virus.

NHS Test and Trace then helps trace recent close contacts of anyone who tests positives for COVID-19 and, if necessary, notifies them that they must self -isolate at home to help stop the spread of the virus. More information about NHS Test and Trace can be found at NHS Test and Trace: how it works.

DHSC publishes weekly statistics on NHS Test and Trace (England). Similar services are in place in Scotland, Wales and Northern Ireland and statistics relating to these services are published separately. The purpose of NHS Test and Trace data is to provide a weekly update on the implementation and performance of NHS Test and Trace in England.

More quality and methodology information on strengths, limitations, appropriate uses, and how the data were created is available in the NHS Test and Trace methodology paper.

Collaboration

This publication was written in collaboration with colleagues at the DHSC.