Accredited official statistics

Adult oral health survey 2021: data quality statement

Published 25 January 2024

Applies to England

Introduction and background

This document is a background quality report for the adult oral health survey 2021. It provides a summary of the key quality related matters that users of the survey need to be aware of. A more detailed discussion of the survey methodology and how it affects quality is in the technical report.

This report is aligned to the UK Statistics Authority’s Code of Practice for Statistics and official guidance on quality statistics in government. It covers the Code of Practice for Statistics assured quality principles of accuracy and reliability, timeliness and punctuality and coherence and comparability. It also covers the European Statistical Service’s dimensions of quality, which form principles 11 to 15 of the European Statistics Code of Practice. These are:

• relevance
• accuracy and reliability
• timeliness and punctuality
• coherence and comparability of the survey over time
• accessibility and clarity

The 2021 adult oral health survey was carried out in February and March 2021 with a representative sample of adults in England aged 16 and over. The survey was commissioned by Public Health England, now known as the Office for Health Improvement and Disparities (OHID) within the Department of Health and Social Care (DHSC).

The survey was carried out by a consortium led by the National Centre for Social Research (NatCen). The consortium includes: the University of Birmingham; King’s College London; the School of Dental Sciences at Newcastle University; the Dental Public Health Group and Department of Epidemiology and Public Health at University College London, and; the Office for National Statistics.

The University of Leeds and School of Clinical Dentistry, University of Sheffield also provided guidance and support to the survey and its design.

The AOHS is the latest in a series of nationally representative surveys of adults’ oral and dental health in England, carried out between 1968 and 2009. These surveys, previously known as adult dental health surveys (ADHSs), have been used to estimate the prevalence of dental and oral health conditions. In addition, the survey has collected data on associated behaviours, including service use; the impact of poor oral and dental health; and barriers to accessing care.

Development work for the 2021 survey was commissioned in early 2020 and included the proposal to rename the survey as the adult oral health survey, to reflect the scope of its topics. At that time, it was expected that the survey would follow closely the design of previous adult dental health surveys, including a face-to-face interview and an oral examination. By late 2020, when data collection was commissioned, the progress of the COVID-19 pandemic meant that this was not feasible at that time. Consequently, the 2021 AOHS was carried out as a web and paper survey with no oral examination, thereby reducing comparability with previous surveys in the series.

The survey findings are published as reports.

Assessment of statistics against quality dimensions and principles

Relevance

This dimension covers the degree to which the statistical product meets user needs in both coverage and content.

The 2021 adult oral health survey was designed as a continuation of the national adult dental health survey series that has been carried out approximately every 10 years since 1968. Like the previous surveys, the adult oral health survey covered the following areas:

• general health
• behaviours related to oral health, including oral hygiene
• oral health (health of teeth and gums)
• use of dental services
• barriers to dental attendance
• impacts of oral health problems

The 2021 adult oral health survey was informed by a consultation carried out with stakeholders. They included representation from Public Health England (central team, consultants, trainees), NHS (NHS England commissioners, consultants, specialists), British Dental Association, Faculty of Dental Surgery, Faculty of General Dental Practice, Health Education England, General Dental Services, Community Dental Services, third sector organisations, academia (in the UK and internationally) and scientific societies. Overall, 89 individuals and organisations contributed their views to the stakeholder consultation, which highlighted the importance of the survey and identified priority areas of interest.

The 2021 adult oral health survey reflected these priorities. As well as the core content listed above, it included questions on access to dental care during the first 12 months of the COVID-19 pandemic. It also included sociodemographic and geographical information to enable analysis of inequalities, including gender, age, household income, ethnicity, socioeconomic classification, English index of multiple deprivation (IMD).

Accuracy and reliability

This dimension covers, with respect to the statistics, their proximity between an estimate and the unknown true value. Accuracy of the statistics: estimating and reporting uncertainty.

Surveys gather information from a sample rather than from the whole population. Results from sample surveys are estimates, not exact numbers.

The precision of survey estimates is dependent on the number of people in the sample on which the survey is based. The larger the sample, the more precise the estimates, and, conversely, estimates based on smaller samples are less precise and more variable. Estimates for small groups are less reliable and tend to be more volatile than for larger aggregated groups.

Survey estimates are subject to sampling error. The 2021 adult oral health survey used a clustered, stratified multi-stage sample design and in addition, weights were applied when obtaining survey estimates. One of the effects of using the complex design and weighting is that standard errors for survey estimates are generally higher than the standard errors that would be derived from an unweighted simple random sample of the same size. The statistical significance of differences between groups mentioned within the survey reports have taken into account the clustering, stratification and weighting of the data. Differences between estimates have not been commented on unless they are significant at the 95% confidence level, thus indicating that there is less than 5% probability that the observed difference could be due to random sampling variation when no difference occurred in the population from which the sample is drawn.

In common with previous adult dental health surveys, the 2021 adult oral health survey was based on a multi-stage stratified probability sample design. The sample was provided by the Office for National Statistics  and was drawn from the AddressBase Premium database. This database combines data from local authorities, Royal Mail and Ordnance Survey, and enables more accurate identification of residential units than the postcode address file (PAF) used in previous ADH surveys.

The sample was designed to be representative of adults aged 16 and over living in private households in England. The sample excludes people living in institutional settings (for example students in halls of residence, adults in care homes, prisoners) and temporary housing (for example hostels and bed and breakfast accommodation), as well as people sleeping rough. It is likely that the oral health of these groups could vary from people in private households, but these groups make up a small proportion of the adult population and their exclusion is unlikely to have a noticeable effect on survey estimates.

Because of restrictions associated with the COVID-19 pandemic, a face-to-face survey, similar to previous adult dental health surveys, could not be carried out. Consequently the 2021 adult oral health survey was carried out using web-based questionnaires, with an option of completing the survey on paper. The response rate was lower than would have been expected from an interviewer-administered survey: a household response rate of 24% was achieved. In total, productive responses were received from 6,321 individuals. Details of the sample design and survey methods and sampling errors and design effects are published in the technical report.

Data and published findings have been quality assured at each stage of production. The checks applied are described in the technical report.

Timeliness and punctuality

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

The survey data were processed and validated in the summer of 2021. Analysis and publication of the findings were not part of the original contract with Public Health England (PHE) and were subsequently commissioned by OHID in April 2022. The first report was published in December 2022.

This publication was released on the planned and pre-announced publication date.

Comparability and coherence 

Comparability is the degree to which data can be compared over time and domain. Coherence is the degree to which data which have been derived from different sources or methods but refer to the same topic are similar.

The 2021 adult oral health survey was designed to be comparable with the 5 previous adult dental health surveys carried out between 1968 and 2009. The populations covered by these surveys have varied over time. All surveys covered England and Wales. Scotland and Northern Ireland were also covered in some but not all of those surveys. The 2021 survey covered England only. Since the 2009 survey, the different nations have published their own adult oral health surveys. Scotland published an Adult Oral Health Survey 2016-2018 in 2019. This included the results of 2 dental surveys undertaken in dental practices. In 2020, the Welsh Dental Survey of 18 to 25 Year Olds was published. This reported on the oral health of young adults attending a range of purposively selected community settings during 2017 to 2019. Ireland published An Overview of Dental Services and Oral Health in Northern Ireland and Ireland but no new oral health data was collected. Due to the different methods used in these surveys, the results are not comparable with the results of adult oral health survey 2021.

The 2021 adult oral health survey sample design was consistent with previous surveys, a multi-stage stratified probability sample of private addresses. Household response rate was lower than in past surveys. As in previous surveys, the data were weighted to make them representative of the adult population of England.

Because of restrictions associated with the COVID-19 pandemic, a face-to-face survey, similar to previous adult dental health surveys, could not be carried out, and data was collected using self-completion questionnaires, online or paper. This meant that the 2021 survey differed from its predecessors in several ways:

• a length of no more than 20 minutes is recommended for web-based questionnaires (and other self-completion modes) because it is easier for respondents to lose interest and fail to complete the survey than when they are interviewed. Consequently, the overall amount of content had to be reduced from what would have been practical in an interviewer-administered mode
• the content and presentation of questions had to be adapted to a self-completion format (web or paper). This included some simple changes (for example, removing interviewer instructions, changing response options to the first person etc.). Crucially, the language, content and presentation of the questions needed to be accessible to enable unsupported completion by as wide a range of people as possible
• in order to maximise response, the questionnaire was also made available in a paper self-completion format. For reasons of space and to avoid confusing respondents with an over-complex structure, the questionnaire needed to be mainly linear, with minimal routing or questions directed at subsamples
• a key element of the original survey design was a dental examination in participants’ homes. This was no longer feasible, so it was necessary to include questions that would collect reliable self-reported information on the health of respondents’ teeth and mouths

Reporting of the 2021 findings make it clear that they are not directly comparable with findings from earlier adult dental health surveys.

Accessibility and clarity

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

The publication is available on gov.uk free of charge. Reports of findings are published in HTML format. They are written in accessible language and illustrated with charts. The tables are published in Excel and ODS formats. Text, charts and tables have been produced in line with government accessibility guidelines. 

Detailed metadata are published in the technical report.

Quality assurance by the Office for Health Improvement and Disparities

In addition to the quality assurance checks described above that were undertaken by the consortium led by the National Centre for Social Research (NatCen), additional quality assurance checks were undertaken at various stages of the survey by the Office for Health Improvement and Disparities (OHID) and the Department of Health and Social Care (DHSC), which published the survey.

Survey materials prior to fieldwork

OHID checked copies of the covering letters and questionnaire used in the survey. This was to ensure they were free from error. Checks include a 2-stage process whereby a consultant and professor independently reviewed all survey materials against the scope of the project.

Online survey

OHID reviewed the online survey including checking for question routing, question text and response codes.

Fieldwork monitoring

During the fieldwork period, OHID monitored weekly the progress of the survey. This included reviewing the response rates both at national and NHS England regional level. This was important because low response rates can limit the representativeness, and therefore usability, of the data. It also included reviewing the approach to ensure this was in line with the survey protocol and that the agreed number of mailings were made to non-responders to the online survey and that the deadline for the final mailing and completion of the online survey were met.

Approval for publication

As a final stage, OHID, DHSC and a professor of dental public health from the University of Leeds reviewed all reports and data tables before statistical clearance and organisational sign off were sought. No outputs were published until all necessary approvals had been received.