Research and analysis

Evaluation of changes in dietary methodology in the National Diet and Nutrition Survey rolling programme from year 12 (2019 to 2020): stage 2

Published 6 November 2023

Acknowledgements

The National Diet and Nutrition Survey rolling programme (NDNS RP) is funded by the Office for Health Improvement and Disparities (OHID), Department of Health and Social Care (DHSC) and the UK Food Standards Agency (FSA), and is currently carried out by a consortium comprising NatCen Social Research (NatCen) and the Medical Research Council (MRC) Epidemiology Unit, University of Cambridge.

This report was prepared by:

• Caireen Roberts, David Collins, Michelle Venables, Albert Koulman and Polly Page (MRC Epidemiology Unit, University of Cambridge)
• Suzanne Hill (NatCen Social Research)
• Gillian Swan and Jo Nicholas (OHID, DHSC)

We would like to thank all of those who gave up their time to be interviewed as part of the NDNS RP. We would also like to acknowledge the professionalism and commitment of the fieldworkers who worked on the survey and who are so important to the survey’s success.

We would like to thank everyone who contributed to the work behind and production of this report, in particular:

• colleagues at the MRC Epidemiology Unit (University of Cambridge): Suzanna Abraham, Birdem Amoutzopoulos, Anila Farooq, Jackie Foreman, Dan Griffiths, Steph Moore, Angela Mulligan, Elise Orford, Toni Steer, Kirsty Trigg
• colleagues at NatCen: Beverley Bates, Robert Ashford and Katy Robertson
• members of the Project Advisory Group: Jean Adams and Nita Forouhi (MRC Epidemiology Unit), Robin Clifford (FSA), Adrienne Cullum, Rachael Wall and Natasha Powell (OHID, DHSC)
• members of the NDNS Project Board: Julie Lovegrove and Susan Fairweather-Tait (Scientific Advisory Committee on Nutrition (SACN)), Paul Niblett (OHID, DHSC), Joseph Shavila (FSA)
• former member of the NDNS Project Board: Professor Hilary Powers (SACN)

This research was also supported by the NIHR Cambridge Biomedical Research Centre (NIHR203312).

Executive summary

This report presents findings from the second stage of evaluation of the changes in dietary assessment methodology in the National Diet and Nutrition Survey rolling programme (NDNS RP) implemented from fieldwork year 12 (2019 to 2020).

The NDNS RP is a continuous, cross-sectional survey, jointly funded by OHID, DHSC and FSA, and carried out by a consortium comprising NatCen and MRC Epi. It is designed to assess the diet, nutrient intake and nutritional status of the general population aged 1.5 years and over living in private households in the UK.

For NDNS fieldwork years 1 to 11 (2008 to 2019), dietary assessment was based on a paper food diary completed by participants over 4 consecutive days with open text entry, review by interviewers, and retrospective coding of foods and portions by trained coders. In 2018, following a review of available automated tools, the decision was taken to move the dietary assessment method to Intake24, a web-based automated self-administered 24-hour dietary recall tool. Data collection using Intake24 began in October 2019 (fieldwork year 12) following a dress rehearsal and developmental updates to the tool.

The original intention had been to evaluate the new dietary assessment method over fieldwork year 12 (October 2019 to June 2020). However, fieldwork was suspended between mid-March and October 2020 as a result of the COVID-19 pandemic and year 12 was not completed. Fieldwork continued to be impacted by the pandemic for the rest of 2020 and 2021 (fieldwork years 13 and 14). Therefore, it was decided to conduct the evaluation in a staged approach over a longer period. 

A first stage evaluation report was published in September 2021 based on data collected from October 2019 to March 2020 (year 12).

While based on a limited amount of data, the findings indicated that the introduction of Intake24 was compatible with NDNS RP requirements. Overall comparability with the previous data collected using the food diary appeared to be good. However, there was an observed fall in the percentage of participants who reported taking supplements, and a drop in the percentage of consumers and amounts consumed for some foods - in particular vegetables, fish and, for younger children, sugar confectionery and buns, cakes and pastries.

This report, the stage 2 evaluation, updates stage 1 and is based on data collected from October 2019 to May 2022. This includes the data from year 12 (October 2019 to March 2020) and new data from years 13 and 14 (October 2020 to May 2022). Data from the NDNS follow-up study during COVID-19 (August to October 2020) (PHE, 2021b) has also been used to fill in the time gap left by the main fieldwork suspension.

This stage 2 evaluation includes:

• findings of the doubly labelled water (DLW) study to assess misreporting of energy intake. The DLW method measures total energy expenditure in free-living individuals over 1 to 2 weeks. This is compared with reported energy intake
• assessment of how the new fieldwork model is working in terms of participation rates for dietary recalls and spread across days of the week
• quality measures for use of Intake24, including completion time and number of foods reported
• analysis of the impact of changes to the dietary assessment method on time trends for foods and nutrients

Findings of the stage 2 evaluation

Misreporting of energy intake

The DLW study was carried out in a sub-sample of NDNS RP participants aged 4 years and above. 279 participants provided valid total energy expenditure (TEE) as measured by DLW and energy intake (EI) data as reported using Intake24.

Overall, mean EI:TEE was 0.70 and ranged from 0.60 for males aged 16 to 49 years to 0.84 for boys aged 4 to 10 years. EI:TEE was significantly lower than 1.0 in all age or sex groups, indicating under-reporting.

There were no significant differences between EI:TEE in this study and the previous DLW study in years 6 and 7 of the NDNS RP where intake data was collected using the food diary.

Dietary recall participation

It has not been possible to evaluate the impact of the new Intake24 fieldwork model on participant response as the evaluation period coincided with COVID-19. This necessitated a remote fieldwork protocol that cannot be directly compared with the face-to-face approach. 

Between October 2019 and May 2022, 955 children and 957 adults completed at least one recall, and 79% of participants went on to complete all 4 recalls. The stage 1 evaluation found a shortfall in the proportion of recalls completed on weekend days. An initial adjustment was made to the recall invitation system to increase the proportion of recalls completed for weekend days.

Following review, a further adjustment has been applied to achieve a better balance across both weekend days and the impact of this is being monitored. 

Intake24 quality measures

A range of indicators were examined to help determine whether participants were using Intake24 as intended.

The median recall completion time was 15 minutes (mean 33 minutes). Some very long recall times were recorded, which reflected the system allowing participants to partially complete a recall and return to it later the same day. 28% of recalls were completed in less than 10 minutes - this proportion increased with each successive recall.

26% of recalls for children aged 11 to 18 years and 21% for adults aged 19 to 64 years contained fewer than 10 items. For children 11 to 18 years, this proportion increased with each successive recall.

The mean number of items per recall was similar to the mean number per paper diary day.

The proportion of recalls with very high or very low energy intakes was small and was also comparable with the food diary.

Evaluating impact on dietary data

Individual-level average daily intake using data collected with the paper food diary was plotted for years 1 to 11 and the data for the evaluation period using Intake24 was added to observe any step changes - that is changes large enough to have implications for continuation of the NDNS time series.

Generally, the spread of energy and nutrient intake data collected using Intake24 appeared similar to data collected with the paper diary with no evidence of step changes. 

However, step changes were observed for some foods. There was a reduction in reported vegetable consumption using Intake24 compared with the diary. There was also a fall in the proportion of consumers of total fish and oily fish and, for children aged 4 to 10 years, a fall in the proportion of consumers of sugar confectionery and buns, cakes and pastries.

A fall in the proportion of consumers of salad vegetables is likely to be due to changes in the coding method for salads and sandwiches.

Step changes in reporting consumption of fat spreads and soft drinks seen in the stage 1 evaluation were no longer seen as they had been addressed by changes to the Intake24 food list.

A modification to Intake24 following the stage 1 findings resulted in an increase in reported dietary supplement use.

Conclusion

This stage 2 evaluation report builds on the findings of the stage 1 report.

The results of the DLW sub-study show that use of Intake24 in NDNS is associated with a similar degree of under-reporting of energy intake to that found by previous DLW studies for the food diary method. Mean EI:TEE was 0.70 in this study and 0.71 in the previous years 6 and 7 study.

Misreporting is an inherent feature of any self-reported dietary assessment instrument. The dietary evaluation shows that the 2 methods perform similarly. Both are subject to error, including under-reporting, although the error profile of each instrument will be different.

Generally, the evaluation has not identified any major concerns around the continuation of the time series data set for monitoring ongoing trends over time. Most of the step changes in reported food consumption identified in the stage 1 evaluation have been resolved by making changes to the food lists in the Intake24 tool.

Longer-term work to address known issues with Intake24 may improve data capture relating to:

• the quantities of vegetables reported
• the proportion of consumers of total fish and oily fish
• for younger children, the proportion of consumers of sugar confectionery and buns, cakes and pastries

Although the evaluation of the NDNS method change coincided with the COVID-19 pandemic, it seems unlikely that this is a major factor in any changes seen as no new step changes were observed between the first stage of the evaluation (based on data collected pre-pandemic) and the second stage (based on data collected during and after the pandemic). Nevertheless, it cannot be ruled out that some of the observed changes are real rather than methodological.

Final conclusions about the impact of the change in methodology will be considered alongside the publication of the next full NDNS RP report in spring 2024.

1. Introduction

1.1 Background

The NDNS RP is a continuous cross-sectional survey designed to assess the diet, nutrient intake and nutritional status of the general population aged 1.5 years and over living in private households in the UK.

Results are used by government to monitor progress toward diet and nutrition objectives of UK health departments, and develop policy interventions.

The NDNS RP is a government-commissioned survey, jointly funded by OHID, DHSC^{[footnote 1]} and FSA. The NDNS RP is currently carried out by a consortium comprising NatCen and MRC Epi^{[footnote 2]}.

In years 1 to 11 (2008 to 2019) of the NDNS RP, dietary data was collected over 4 consecutive days using a paper food diary with estimated portion weights. This required paper-based open-text entry by participants with review by interviewers, and retrospective coding of foods and portions into the dietary assessment system Diet In Nutrients Out (DINO) (Fitt and others, 2014) by trained coders.

In 2018, the decision was taken to move to an automated dietary data collection method to enable increased cost-efficiency, and provide opportunities for improving data quality and the potential to scale the survey in the future.

Following a review of available automated tools and full evaluation of 3 shortlisted tools, Intake24 was selected to replace the paper food diary in the NDNS RP.

The dietary data collection methodology change for NDNS, and associated fieldwork model changes, were implemented in October 2019 (the start of year 12 of the NDNS RP) (PHE , 2021a).

1.2 Overview of evaluation and content of this report

The original intention was to evaluate the new dietary assessment method over the full year 12. However, fieldwork was suspended as a result of the COVID-19 pandemic in March 2020, with dietary data collection about half completed, and was not restarted. Due to the limited data collected in year 12 and the continued impact of COVID-19 on NDNS RP fieldwork, the evaluation has been conducted in a staged approach over a longer period.

A stage 1 evaluation report was published in September 2021 based on data collected during the 6 months of year 12 from October 2019 to March 2020. This aimed to gain an early understanding of the implications of the method change and identify any immediate issues.

This stage 2 evaluation report updates stage 1 and is based on data collected from October 2019 to May 2022. This includes the data from year 12 (October 2019 to March 2020) and new data from years 13 and 14 (October 2020 up to May 2022).

As no data was available from the NDNS RP between March and October 2020 due to the suspension of fieldwork due to COVID-19, data collected from the NDNS: diet, nutrition and physical activity in 2020 - a follow-up study during COVID-19 (PHE, 2021b), which ran fieldwork from August 2020 to October 2020, has been used to fill some of the time gap when looking at continuity of the dietary data.

It is not the intention to compare the results of the stage 2 evaluation with the results of the stage 1 evaluation other than where stage 1 identified issues that have been addressed.

Figure 1 below illustrates the data collection periods included in the evaluation reports and how these relate to survey fieldwork. It shows that stage 1 covers October 2019 to March 2020 (year 12), and stage 2 covers October 2019 to May 2022 (years 12, 13 and 14) and also includes data from the NDNS follow-up study (August to October 2020). To enable the most suitable assessment, the data sets used vary across this report. Details can be found in ‘Appendix A1: content of stage 2 evaluation report - data and nutrient databank used’ below.

Figure 1: Intake24 data collection periods included in first and second stage of evaluation

A DLW sub-study was carried out across years 12 to 14 of the NDNS RP to evaluate the change in dietary assessment methodology. The objectives of this DLW sub-study were to assess the degree of misreporting of energy intake using Intake24 with respect to energy expenditure, and to compare this with the degree of misreporting seen in previous DLW sub-studies with the paper diary in the NDNS RP years 1 to 11.

The following sections of this chapter provide an overview of the NDNS RP dietary data collection methodology change and associated fieldwork model changes along with the objectives of the evaluation.

Chapter ‘2. Misreporting of energy intake’ presents the findings of the DLW study to assess and compare misreporting of energy intake.

Chapter ‘3. Dietary recall participation’ sets out how the new fieldwork model is working in terms of participation rates for dietary recalls and spread across days of the week.

Chapter ‘4. Intake24 quality measures’ presents quality measures for use of Intake24 including completion time and number of foods reported.

Chapter ‘5. Evaluating impact on dietary data’ presents analysis showing the impact of changes to the dietary data collection methodology on time trends for foods and nutrients.

1.3 Overview of methodological changes

Intake24 is a web-based, automated, self-administered 24-hour dietary recall tool (Rowland and others, 2018; Bradley and others, 2016; Foster and others, 2019).

Participants are asked to record everything they ate and drank the previous day. The tool includes an embedded database of foods with linked portion sizes and corresponding nutrient composition data from which dietary intakes are automatically calculated.

A number of developments were made to Intake24 and the underlying data before it was used in the NDNS RP to enable it to meet survey requirements. These included updates to tool functionality and addition of questions to provide supporting information - for example, where food was obtained from.

Work was also undertaken to update the food list from which participants select the foods and drinks they consume, and rationalise and update the linked nutrient composition information drawn from the NDNS Nutrient Databank (NDB)^{[footnote 3]}. This aimed to achieve a comprehensive and up-to-date database of foods to adequately reflect the heterogeneity of foods in the UK while remaining manageable for participants. This included moving to more generic codes for mixed dishes (for example, recipes, salads and sandwiches) that would previously have been manually coded as individual components.

The move to a web-based recall from a paper diary is a significant method change for the NDNS RP as it puts the onus on participants to use the online tool to select foods rather than listing their foods freely and then their written entries being retrospectively coded. This data collection model also allows a much-reduced level of interviewer involvement with fewer interviewer visits to the household.

At the time of changing the dietary assessment method to Intake24, there was also a change in the method for the selection of NDNS RP participants within a household. All these changes gave the opportunity to reduce the cost and improve the efficiency of the survey model.  

Table 1 below gives an overview of the main methodological changes. Following a dress rehearsal in April to June 2019 to test the new survey fieldwork model and processes, NDNS RP fieldwork was launched for year 12 using Intake24^{[footnote 4]} in October 2019.

Table 1: summary of methodological differences in respect of changes introduced from year 12 (October 2019) of the NDNS RP

Component	Years 1 to 11 (2008 to 2019)	From year 12 (October 2019)
Dietary assessment method	Prospective using estimated (un-weighed) paper-based diary	Retrospective using online multiple pass 24-hour recall, Intake24
Dietary data collection	Participant self-completed using open-text with retrospective food and portion coding into the dietary assessment system DINO by trained coders Participants encouraged to report recipes, ingredients and, to aid coding, provide food packaging	Self-completed where participants search for foods and select best match for food names and portion options; auto-linked to food codes and portion amounts; nutrient information calculated within the tool Participants encouraged to match to pre-defined list of foods. Limited reporting of recipes or individual ingredients. Option to report a missing food if match cannot be found
Recording days	Consecutive over 4 days, start day randomly allocated (designed to include at least one weekend day)	Non-consecutive, total 4 days selected randomly (designed to include at least one weekend day)
Portion size assessment	Portion sizes mainly reported as household measures, often estimated as small, medium or large servings. Limited number of portion size photos available	A large proportion of food codes in Intake24 are linked to a range of portion size photos from which the participant can select the best match to their portion size. Household measures also available for some foods
Food coding	Individual coding of food items. Very few generic codes for mixed dishes - for example, all reported recipes and sandwiches entered as individual ingredients or components	Pre-set food list and embedded coding. More generic codes to allow a single code to represent a range of similar foods - for example, ‘cheese sandwich’. Participants can report individual foods or recipes if they cannot find a match in the food list
Reporting dietary supplement use	Dietary supplement use reported by free text description in the same way as foods	Questions on dietary supplement use with pre-set list in the same way as foods, linked to generic supplement codes. Option to report a missing supplement if match cannot be found
Dietary data set for analysis	Data set for analysis only includes participants providing 3 or 4 diary days	Data set for analysis includes all participants with at least one recall
Interviewer visit	2 to 3 visits or contacts to each household to complete all interviewer-stage components. Additional visits for participants requiring assistance	Single visit to household to complete all interviewer-stage components. Additional visits for participants requiring assistance (including with no internet)
Interviewer involvement or support	Diary introduced at first visit and then reviewed at a mid-diary visit or telephone call. Reviewed and information clarified on third visit	First recall completed by participant during interviewer visit. Subsequent recalls completed independently with no interviewer involvement, except for households where participants required assistance with completing the recalls
Participant selection	One adult and, where present, one child selected in around one-third of addresses, and one child (no adult) in around two-thirds of addresses	Two adults and, where present, one child selected in around one-third of addresses, and up to 2 children (no adult) from the remaining two-thirds of addresses

Further details of the changes in methodology introduced from year 12 of the NDNS RP and the dress rehearsal can be found in chapter ‘2. Changes in methodology’ of the stage 1 evaluation report along with details of the updates and modifications made to Intake24, including preparation of the embedded food list and rationalised NDNS NDB.

1.4 Objectives of the evaluation

Given the purpose of the NDNS RP to provide trend information on food and nutrient intakes for the UK population, the implications of survey methodological changes for data interpretation are important to understand.

Alongside evaluating the ability of the new tool to effectively capture dietary intake in the national survey setting, an assessment of the implications of the methodological changes on the quality, coverage (for example of days of the week) and detail of nutrient data collected, and the comparability of results with previous years, is also needed.

Therefore, the primary objectives of the evaluation of the changes in dietary methodology in the NDNS RP setting were as follows:

1. Describe how the new dietary method is performing in the NDNS RP.
2. Identify aspects of data discontinuity and assess the feasibility of continuation of the time series data set for monitoring trends over time.
3. Assess the degree of misreporting of EI by comparing EI from the online 24-hour recall tool Intake24 in the NDNS RP with TEE measured by the objective biomarker DLW.
4. Compare differences in EI:TEE measured by the DLW sub-study using Intake24 and previous DLW sub-studies conducted in the NDNS RP using the paper diary method.

The stage 1 evaluation covered objectives 1 and 2, the analysis for which has been repeated in this second stage report using a larger data set. Analysis for objectives 3 and 4 is presented for the first time.

1.5 Summary findings of evaluation stage 1

The stage 1 evaluation (PHE, 2021a), published in September 2021, was based on all available data from NDNS RP October 2019 to March 2020 (year 12) and reported on: 

• overall response to the survey, participation rates for completing dietary recalls, and representation of weekdays and weekend days in the data set
• measures around use of Intake24, including completion time and number of foods reported
• impact of rationalisation and updating of the NDB on resulting dietary data
• impact on continuity of the NDNS RP trend data series

While based on a limited amount of data due to the impact of COVID-19 on NDNS RP fieldwork, the findings provided an early indication that, overall, the introduction of Intake24 as a new dietary assessment tool was compatible with NDNS RP requirements. Overall comparability with the previous data collected using the food diary appeared to be good - however, a number of specific issues were identified for action and these are addressed in this stage 2 report (see chapter ‘5. Evaluating impact on dietary data’ below).

2. Misreporting of energy intake

This chapter presents the results from the years 12 to 14 DLW sub-study to estimate the degree of misreporting of EI using Intake24.

The sub-study was carried out during the NDNS RP fieldwork periods October 2019 to March 2020 (year 12) and October 2020 to May 2022 (years 13 and 14).

The following commentary is supported by ‘Appendix B: doubly labelled water sub-study’ below.

2.1 Background

If a healthy adult participant is in energy balance, their habitual EI equals their habitual TEE and their ratio of EI:TEE is 1.0. Determination of adequacy of dietary reporting for a group of individuals is based on the ratio of reported EI and measured TEE.

Because of the variability of energy intake and energy expenditure, an individual may not be in perfect energy balance at any given time and EI:TEE will not equal 1.0. For some individuals, their ratio at that time will be less than 1.0 and for some it will be greater than 1.0 - but, for a group, the expectation is that the mean ratio will be 1.0.

Where the mean ratio for a particular group is lower than 1.0, this indicates a discrepancy between reported EI and measured TEE, potentially due to under-reporting of intake or undereating during the dietary intake assessment.

The misreporting of EI is known to be an issue for all dietary surveys and studies where the assessment of usual diet relies on self-reporting instruments and studies have demonstrated that individuals can over or under-report their dietary intake, with an overall bias towards under-reporting (Black and others, 1993; Livingstone and Black, 2003).

The DLW method is an established method, widely agreed to be the most accurate way of measuring TEE in free-living individuals over 1 to 2 weeks (Barrie and Coward, 1985; Bluck, 2008), and hence providing the ability to assess misreporting of EI.

Even though growing children, and adults losing or gaining weight intentionally or unintentionally, are by definition not in energy balance, the DLW method can still be used to assess TEE in such individuals.^{[footnote 5]} Although corrections can be made to TEE in adults not in energy balance, this would require further measures of body composition or, at the very least, weight to be made at the end of the measurement period. For that reason, the NDNS DLW protocol excludes all adults declaring that they are intentionally losing or gaining weight (Prentice, 1990).

A DLW sub-study has been an integral part of the UK NDNS since its inception. The NDNS RP is one of the few national dietary surveys to include this method. It provides a vital reference point to estimate the level of misreporting of energy intake in adults and children aged 4 years and above within the NDNS dietary data set. It also provides information to help establish the TEE of the UK population (Brage and others, 2020).

A DLW sub-study has taken place once every 5-year contract phase, with previous sub-studies in years 1 and 3 (2008 to 2009 and 2010 to 2011) and years 6 and 7 (2013 to 2014 and 2014 to 2015).

2.2 Overview of the DLW sub-study in years 12 to 14

2.2.1 DLW method

For the DLW sub-study in years 12 to 14, interviewers invited eligible^{[footnote 6]} participants to take part until a quota for each age or sex group was filled (see section ‘2.2.2 DLW sample’ below). For those participants taking part, the aim was to complete the DLW protocol as close to the interviewer visit as possible and ideally to overlap with some of the dietary recall time period. 

Each participant was asked to provide a baseline urine sample before receiving a weighed oral dose of water enriched in 2 naturally occurring stable isotopes, hydrogen (²H, deuterium) and oxygen (¹⁸O) (day 0). Participants were asked to provide further single samples of their urine every day for a total of 10 consecutive days following the day of dosing. The date and time of sample collection was noted by the participant in a log sheet.

Urine samples were labelled and stored in 7ml glass bijou vials in the participants’ fridge until the end of the 10-day collection. They were then collected by the interviewer and posted back to MRC Epi where they were frozen at −20°C and later analysed. Isotopic enrichments of the dose provided and of the urine samples were analysed using continuous flow isotope-ratio mass spectrometry (IRMS) (see ‘Appendix B1: background to method’).

TEE was calculated using the Schoeller equation (Schoeller and others, 1986), as described in the SACN Dietary Reference Values for Energy report (SACN, 2011), from slopes and intercepts of the isotope disappearance curves based on urine samples collected on days 1 to 10.

Basal metabolic rate (BMR) for each individual was estimated using the Schofield equations (Schofield, 1985). Physical activity level was expressed as TEE divided by BMR. This ratio removes virtually all the differences between individuals due to sex, age and body size.

2.2.2 DLW sample

The DLW study was carried out in a sub-sample of NDNS RP participants aged 4 years and above. Due to the high participant burden and high financial cost of the DLW protocol, the participant number targets for years 12 to 14 were reduced compared with previous years to enable the sub-study to run within the available funding.^{[footnote 7]}

A power calculation was undertaken to assess:

• the level of concordance (EI:TEE) that could be detected as statistically significant
• the degree of change in concordance possible to detect as statistically significant between the years 12 to 14 DLW sub-study and the NDNS RP DLW sub-studies in earlier years

This informed the sample size for achieved valid DLW samples for years 12 to 14 that were feasible within budget constraints. A sample size of 280 was selected as this enabled the DLW sub-study to detect a EI:TEE of 0.83 and a change in EI:TEE between Intake24 and the previous paper diary method of 0.14 within each age or sex group as statistically significant.

Recruitment targets to achieve the required number of valid DLW samples were set as follows with equal numbers within group for each sex:

• 4 to 10 years old (n=40)
• 11 to 15 years old (n=60)
• 16 to 49 years old (n=80)
• 50 to 64 years old (n=60)
• 65 years old and over (n=40)

In total, 344 eligible participants were recruited to the DLW sub-study. Of these, 279 (82%) participants provided valid data for both TEE, measured by DLW, and EI estimated using data collected via Intake24 (see ‘Appendix B2: participant flow diagram’).^{[footnote 8]}

The recruiting target was generally met in all groups except in:

• males aged 50 to 64 years, which under-recruited by 6 against a target of 30
• males and females aged 11 to 15 years, which under-recruited by 1 and 3 respectively against a target of 30
• females aged 65 years and over, which under-recruited by 1 against a target of 20

Of the 279 participants, 109 (39%) completed recalls and DLW concurrently^{[footnote 9]}, while the remaining 170 participants had a mean lag of 10 days (min=0, max=51) between recall 4 and DLW dosing day. Of the 170 non-concurrent participants, 24 completed recall 4 on the same day as DLW dosing took place.

Analyses have been carried out to assess the representativeness of the DLW sub-sample in relation to the core survey sample in years 12 to 14 for:

• total EI (MJ/day)
• free sugars intake (% of total energy)
• saturated fatty acids intake (% of total energy)
• total fruit and vegetables consumption (g/day)

The results indicate that the DLW sample is representative of the main NDNS RP sample with respect to these measures.

Further details and analyses will be included in the final stage of the evaluation to be published with the NDNS RP years 12 to 15 results report.

2.3 Comparison of reported energy intake and measured energy expenditure

Mean values for reported EI estimated from recalls completed using Intake24 and TEE measured by DLW, along with the difference between TEE and EI and the ratio of EI:TEE, are shown in Table 2 below.

Overall, in combined age or sex groups, mean EI:TEE was 0.70, indicating under-reporting of EI. Mean EI:TEE was 0.63 for male adults and 0.68 for female adults. Mean EI:TEE ranged from 0.60 for males aged 16 to 49 years at the lowest to 0.84 for boys aged 4 to 10 years at the highest. For all age or sex groups, mean EI:TEE was significantly different to 1.0 (p<0.05) (see Figure 2 below).

Table 2: mean values of reported EI and measured TEE in the NDNS RP DLW sub-study years 12 to 14

Age group	Sex	EI (kcal)	TEE (kcal)	TEE-EI (kcal)	EI:TEE
4 to 10 years	Males	1,518	1,908	390	0.84
4 to 10 years	Females	1,340	1,682	341	0.82
4 to 10 years	Sex-combined	1,427	1,792	365	0.83
11 to 15 years	Males	1831	2,677	845	0.69
11 to 15 years	Females	1655	2,243	589	0.77
11 to 15 years	Sex-combined	1,746	2,468	722	0.73
16 to 49 years	Males	1995	3,369	1,373	0.60
16 to 49 years	Females	1710	2,535	826	0.69
16 to 49 years	Sex-combined	1,851	2,947	1,096	0.64
50 to 64 years	Males	2,213	3,312	1,099	0.67
50 to 64 years	Females	1,646	2,547	901	0.65
50 to 64 years	Sex-combined	1,898	2,887	989	0.66
65 years and over	Males	1,787	2,749	962	0.66
65 years and over	Females	1,425	2,062	637	0.70
65 years and over	Sex-combined	1,619	2,430	811	0.67

2.4 Comparison with previous NDNS RP DLW results

Figure 2 below presents the EI:TEE ratio across all age and sex groups for NDNS RP years 1 and 3, years 6 and 7, and years 12 to 14.

The EI:TEE ratio did not differ significantly between years 12 to 14 (0.70) and years 6 and 7 (0.71), or when years 1 and 3 and 6 and 7 were combined (0.73).

For years 1 and 3 and years 6 and 7, EI was estimated from paper food diaries. There was, however, a significant difference (p<0.05) in males aged 11 to 15 years between years 1 and 3 and years 6 and 7 (0.78 vs 0.68).^{[footnote 10]}  

The data underlying this figure is in Table B2 of ‘Appendix B3: Tables’.

Figure 2: mean values with 95% CI of reported EI:TEE in all NDNS RP DLW sub-studies

Figure 3 below presents the reported EI across all age or sex groups for years 1 and 3, years 6 and 7 and years 12 to 14.

The only significant differences (p<0.05) between the sub-studies were between years 1 and 3 and years 6 and 7 in males aged 11 to 15 years (2,058kcal compared with 1,775kcal) and 16 to 49 years (2,262kcal compared with 2,052kcal).

The data underlying this figure is in Table B3 of ‘Appendix B3: Tables’.

Figure 3: mean values with 95% CI of reported EI in all NDNS RP DLW sub-studies

Figure 4 below presents the measured TEE across all age or sex groups for years 1 and 3, years 6 and 7, and years 12 to 14.

The only significant differences (p<0.05) between the sub-studies were between years 1 and 3 and years 6 and 7 in males aged 16 to 49 years (3,462kcal compared with 3,231kcal) and between years 6 and 7 and years 12 to 14 in males aged 50 to 64 years (3,074kcal compared with 3,312kcal).

The data underlying this figure is in Table B4 of ‘Appendix B3: Tables’.

Figure 4: mean values and 95% CI of measured TEE in all NDNS RP DLW sub-studies

3. Dietary recall participation

This chapter looks at the number of dietary recalls achieved and the spread across days of the week to assess how the Intake24 fieldwork model is working. Findings from the stage 1 evaluation report have been updated using combined data from October 2019 to March 2020 (year 12) and October 2020 to May 2022 (years 13 and 14) (referred to as October 2019 to May 2022 throughout the chapter).

The following commentary is supported by ‘Appendix A: background, fieldwork and data processing’ below.

3.1 Overview of fieldwork changes as a result of the COVID-19 pandemic

Year 12 fieldwork began in October 2019 with face-to-face interviews and was due to run until August 2020. However, as a result of the COVID-19 pandemic, all year 12 interviewer and nurse fieldwork was suspended on 18 March 2020 and did not resume.

NDNS fieldwork subsequently re-started in October 2020 with year 13. Interviews were conducted using a remote protocol, enabling the continuation of data collection throughout the ongoing COVID-19 pandemic.

Year 14 fieldwork started in April 2021 initially under the same remote interviewing approach as year 13.

For the remote protocol, after participant selection at the doorstep, all interviews were conducted via telephone. This protocol applied between October 2020 and September 2021. From September 2021, face-to-face interviewing was re-introduced for any households who were happy with in-home visits.

Full details of changes to fieldwork as a result of COVID-19 can be found in ‘Appendix A2: fieldwork changes due to the COVID-19 pandemic’ below.

Due to the impact of COVID-19, it has not been possible to evaluate the impact of the new Intake24 fieldwork model on participant response. Therefore, direct comparisons to response rates achieved in past NDNS RP years have not been made here.

3.2 Dietary data collection

In the NDNS RP (for face-to-face and remote interviews), interviewers introduced Intake24 and provided each participant with a unique URL that would be used to access Intake24 online for completion of all their recalls. Participants were told that, once they accessed the link, they would be invited to watch a short tutorial video about Intake24.

For face-to-face interviews, the first recall was completed with the interviewer present. Interviewers then checked that participants had been able to complete and submit their recall.

For remote interviews, interviewers asked participants to complete their recall following the telephone interview. A follow-up phone call was then scheduled on the same day or the next day for interviewers to check that participants had submitted their recall with no issues.

Subsequent recalls were completed independently by the participant and participants were notified when their next recall was due by text and/or email. Where participants were unable to complete recalls independently - for example, due to internet access issues or lack of confidence with technology - assistance was available.

Further details on the support procedures can be found in ‘Appendix A3: Intake24 support’ below.

3.3 Participation rates for dietary recalls

Between October 2019 and May 2022, a total of 1,912 individuals completed at least one recall (and were therefore defined as productive participants): 955 children and 957 adults (see Table A2 in ‘Appendix A4: number of completed recalls’ below).

The majority of productive participants went on to complete all 4 dietary recalls (1,519, 79%), 61 (3%) completed 3 recalls only, 126 (7%) completed 2 recalls only and 206 (11%) completed one recall only.

The proportions of participants completing each of the recalls are similar to those reported in the stage 1 evaluation (PHE, 2021a). 

3.4 Recalls by day of the week

The NDNS RP study design aims to provide an even representation of all days of the week in the overall dietary data set.

In years 1 to 11 (2008 to 2019), the food diary could start on any day of the week and would run for 4 consecutive days. At the first interviewer visit, the diary start day was randomly assigned for each participant.

With Intake24 from year 12, the first recall was completed at the main interviewer visit (which could take place on any day of the week but was less likely to take place at the weekend) and dates for subsequent recalls were randomly allocated within the next 2 to 6 days by the recall invitation system. If a participant did not complete their recall on the requested day, the recall invitation system sent up to 4 reminders (firstly in the evening of the initial requested day and then at intervals over the next 9 days), always requesting completion of the recall for the preceding day.

Figure 5 shows the percentages of recalls obtained for each day of the week overall and by recall number for October 2019 to May 2022. Data is shown for the day of the week the recall represents, rather than the day upon which it was completed (as each recall represents the diet of the previous day).

The pattern of interviewer fieldwork has resulted in fewer first recalls being completed for Fridays and Saturdays (as less interviewing takes place on weekends). It should also be noted that some participants completed their recalls on a date that was different from the one they were initially allocated.

An adjustment was made to the recall invitation system in October 2020 to attempt to balance out the overall proportions of recalls completed for respective days of the week. This ensured that third and fourth recall invitations were sent on Sunday or Monday if the participant was yet to complete a recall for a weekend day. However, the adjustment did not evenly sample weekend days, and this has resulted in an oversample of recalls completed for a Saturday (with an increase from 12% for the period October 2019 to March 2020 to 16% for the period October 2019 to May 2022) and a reduced proportion of recalls completed for a Sunday (from 15% to 11%).

A further adjustment has been made for NDNS RP fieldwork going forward to redress the balance across the weekend days and this is being monitored. 

Figure 5: percentage of recalls by days of the week (October 2019 to May 2022)

4. Intake24 quality measures

This chapter describes how the new dietary method is performing in the NDNS RP by looking at a range of indicators to help determine whether participants are using Intake24 as intended.

Figures from the stage 1 evaluation report have been updated using combined data from October 2019 to March 2020 (year 12) and October 2020 to May 2022 (years 13 and 14) (referred to as October 2019 to May 2022 throughout this chapter) to maximise the number of participant and recall numbers across the age groups.

The following commentary is supported by ‘Appendix A: background, fieldwork and data processing’ below and ‘Appendix C: Intake24 quality measures’.

4.1 Missing foods

Intake24 automatically assigns food codes and gram weights to the foods and portion sizes selected by participants, allowing the tool to auto-generate nutrient data.

Participants are asked to select, wherever possible, foods from the food list provided in Intake24.

If they cannot find an exact match for the food consumed, the tool prompts them to choose the closest matching item. If a participant still cannot find a suitable match, they can report their food as a missing food (by food name and/or providing ingredients of a recipe). Foods reported as missing are later manually assigned to an appropriate food code and portion size, using the missing food details and original free text search term provided by the participant. The raw Intake24 output is imported into a bespoke database to facilitate coding of missing foods and further dietary data checks (see ‘Appendix A5: quality checks’ below).

The October 2019 to May 2022 dietary data set included a total of 6,715 recalls.^{[footnote 11]} Across all age groups (as shown in Tables 1.1 and 1.2 of ‘Appendix C: Intake24 quality measures’), 11% of recalls had at least one food reported as missing by participants. Of these:

• 15% of first recalls had at least one missing food
• 11% had at least one missing food for recall 2
• 10% had at least one missing food for recall 3
• 9% had at least one missing food for recall 4

Just over 1 in 100 (1.1%) of all food items recorded were reported as missing foods by participants. Around half (53%, n=569) of these foods could be matched exactly to an existing food code in Intake24 by the research team.^{[footnote 12]} Similar percentages were seen when split by age group, except for older adults aged 65 years and over where 63% of foods reported as missing could be matched to existing food codes and children aged 1.5 to 3 years where 43% could be matched.^{[footnote 13]}

For those missing foods that could not be matched to existing codes (47%, n=504), the food was allocated a ‘closest match’ code available in Intake24. Monitoring these ‘closest match’ codes is an ongoing task so that a decision can be taken as to whether a new food needs to be added in Intake24 at a later stage based on the reported amount and frequency of consumption. This helps to ensure that the food list in Intake24 reflects general consumption patterns within the population.

4.2 Recall completeness

Measures such as recall completion times and number of food items reported were reviewed as an indication of recall completeness. A number of thresholds were adopted pragmatically to identify recalls that may be less likely to be complete – those with:

• fewer than 10 food items
• 3 or fewer eating or drinking occasions
• completion time of under 10 minutes

Thresholds were also set to identify recalls where the calorie intake represented obvious misreporting in all age groups: less than 400kcal or more than 4,000kcal after data quality checks. The purpose of these analyses was observational rather than statistically assessing any associations and to compare, where possible, the same measures from the paper diary.

For October 2019 to May 2022, the median recall completion time was 15 minutes (the mean was 33 minutes). A previous study testing Intake24 in a national survey setting reported an average (mean) completion time of 14 minutes (Rowland and others, 2016). Online recall dietary assessment tools MyFood24 and ASA24 have mean completion times of 16 and 24 minutes respectively (Albar and others, 2015; National Cancer Institute, 2023).

For Intake24, completion times are taken from the time stamps within the Intake24 system, which are recorded when a participant first logs in and when they finally log out or submit their recall.

For NDNS, to provide maximum flexibility for participants, Intake24 is configured such that a participant can complete their recall in one go or in stages over the day (including using different devices), providing it is submitted before midnight on the day of the recall request. So, if a participant takes a break before submitting their recall and leaves their device logged on to Intake24, this will be reflected in their completion time. Some long completion times were seen (10% of recalls took between 30 to 59 minutes and 8% of recalls took more than 60 minutes).

Recall times have been included in the analysis without adjustment, but they may not necessarily reflect the time actively spent on recall completion, particularly for longer recall times.

Thirty-nine percent of recalls were completed within 10 to 19 minutes, and 28% in less than 10 minutes (5% for adults aged 65 years and over, and 28 to 37% for children and adults aged 19 to 64 years). Only 35 recalls (0.5%) were completed in under 3 minutes. Overall, the proportion of participants completing a recall in less than 10 minutes increased for each successive recall with the biggest jump between the first and second recall - 11% of participants took less than 10 minutes for the first recall while 27% took less than 10 minutes for the second recall, 37% for the third recall and 41% for the fourth recall.

The reduced completion time with successive recalls has been observed in similar tools (Subar and others, 2020) and could be related to a learning effect (see Tables 2.1 and 2.2 of ‘Appendix C: Intake24 quality measures’).

Twenty-six percent of recalls for children aged 11 to 18 years and 21% of recalls for adults aged 19 to 64 contained fewer than 10 items. This is compared with 5 to 11% of recalls in the other age groups. For children aged 11 to 18 years, this proportion increased with each successive recall (22% for the first recall up to 29% for the fourth recall) (see Tables 2.1 and 2.3 of ‘Appendix C: Intake24 quality measures’).

Figure 6 shows completion times and number of food items reported plotted against each other for all recalls.

As noted above, longer completion times may not necessarily fully reflect the time actively spent on completing the recall. The vertical line indicates the 10-minute threshold and the horizontal line indicates the 10 food items threshold. The plot shows that, while 36% of the recalls that were completed in less than 10 minutes had fewer than 10 food items, up to 25 items were being recorded in under 10 minutes.

For children aged 11 to 18 years, the spread of data was similar to that seen overall in Figure 6, even though this age group had the lowest mean number of food items (13 per recall) and highest percentage of recalls with fewer than 10 items. For this age group, 43% of recalls completed in less than 10 minutes had fewer than 10 food items.

Figure 6: recall completion times and number of food items - all recalls (n=6715) (October 2019 to May 2022)

Consideration was also given as to whether participants reported a similar number of food items using Intake24 compared with the paper diary used in years 1 to 11 (2008 to 2019).

For the last 3 years in which the food diary was in use (2016 to 2019), it was possible to count items coded in the DINO dietary assessment system in a way that was similar to counting items recorded through Intake24, although some differences are to be expected given the revised coding approach with Intake24 and the rationalisation of food codes. For example, in the paper diary, a cheese sandwich would be coded as at least 2 food items (bread and filling) whereas in Intake24, a generic cheese sandwich code would most likely be selected with the sandwich coded as 1 item.

Figure 7 shows that the distribution of number of food items per 24-hour recall in years 12 to 14 (combined) appears to be similar to the number of food items reported per diary day in years 9 to 11 (combined).

The mean number of food items per recall was 17 while the mean number per diary day was 20 items. There was a higher proportion of Intake24 recalls at the lower end of reported number of food items, with 11% of 24-hour recall days having fewer than 10 food items reported, compared with 5% of diary days.

Figure 7: percentage of diary days (n=14140) (years 9 to 11 combined) and percentage of 24-hour recalls (n=6715) (years 12 to 14 combined) by number of food items

It was considered that one strategy participants might use to speed up recall completion would be to aggregate several different eating occasions together - for instance, recording all food items under breakfast and evening meal rather than reporting them at different times across the day. If this was the case then the data might show fewer eating occasions for 24-hour recalls completed in less than 10 minutes, perhaps along with an increase in the number of food items reported per eating occasion.

For recalls completed in less than 10 minutes, the mean number of eating occasions was 4.3, and the mean number of food items recorded per eating occasion was 2.7, compared with means of 5.2 (eating occasions) and 3.2 (items recorded) for recalls completed in 10 minutes or more. While there are differences, these figures suggest that this strategy was not being widely used.   

Only 6% (n=374) of all recalls were less than 10 minutes, had 3 or fewer eating occasions and had fewer than 10 food items.

Overall, the proportion of 24-hour recalls with very high or very low energy intakes was small, and was comparable with the proportion of diary days from years 1 to 11 (combined).

In the years 12 to 14 (combined) data, 56 recalls (0.8%) had energy intakes less than 400kcal/day, in 24 of which the participant reported eating less than usual. The percentage of paper diary days below 400kcal in years 1 to 11 (combined) was similar (0.6%). After winsorization of pizza portions^{[footnote 14]}, 46 recalls (0.7%) had energy intakes more than 4000kcal/day, in 15 of which the participant reported eating more than usual. This was also similar to the percentage of diary days above this cut-off (0.5%). For both recalls and diary days, the majority of records with energy intakes more than 4,000kcal/day were in the age groups 11 to 18 years and 19 to 64 years.

Figure 8 shows recall completion times and energy intake plotted against each other for all recalls.

The vertical line indicates the 10-minute threshold. It shows a wide spread of energy intakes for recalls completed in under 10 minutes with 0.5% of these recalls exceeding 4,000kcal/day and 1.9% being below 400kcal/day. This pattern was similar across all age groups.

Figure 8: recall completion times and energy intake - all recalls (n=6715) (October 2019 to May 2022)

5. Evaluating impact on dietary data

The changes in survey methodology implemented from year 12 (October 2019) of the NDNS RP are summarised in section ‘1.3 Overview of methodological changes’ above. To what extent any differences seen in the subsequent dietary data can be ascribed to these changes is difficult to quantify, but this evaluation can look at the overall impact and consider possible reasons behind changes observed.

The objective of this evaluation is to understand any differences between the methods and, therefore, any potential implications for continuing the NDNS RP trend data series over the method change. It is not to align the data collected using the 2 methods - measurement error applies across all self-report dietary assessment methods, but it is likely that different methods have different error profiles.

This chapter considers aspects of data discontinuity and assesses the feasibility of continuation of the time series data set for monitoring ongoing trends over time.

5.1 Updates to the nutrient databank

The development of Intake24 for the start of year 12 required a major review and rationalisation of the linked NDB.

In order to measure the impact of this on monitoring trends over time, dietary data from NDNS RP year 10 (2017 to 2018) was recalculated after matching paper diary entries to the foods available in the year 12 rationalised NDB. This exercise, reported in the first stage of the evaluation (PHE, 2021a), aimed to test if any changes observed in the NDNS data collected pre and post the move to the new dietary assessment method could be the result of changes in the food codes used.

Mean daily intake of selected foods and nutrients using the original year 10 data was compared with the daily intake based on recoded year 10 data calculated with the rationalised NDB, for all ages combined.

While, overall, no major differences were observed following the year 10 code replacement, a few relatively small differences were observed as a result of the change from individual recipe coding in the paper diary to using more generic recipe codes in Intake24. These were:

• an increase in mean total energy intake (47kcal/day)
• an increase in mean total fat intake (2.8g/day or 0.6% of total energy)
• a decrease in mean fruit and vegetable intake (13g/day or 0.2 portions/day)

These comparisons were not statistically tested in the stage 1 report, but subsequent analysis showed no statistical differences (Amoutzopoulos and others, 2022).

Following the stage 1 evaluation report, a further review was carried out on the recoded year 10 data to look at individual days with the largest differences seen in key foods and nutrients.

The findings suggested some improvements were needed to the rationalised NDB and identified some issues with the recoding method where better recipe matches could have been made. Differences were also identified in the disaggregation values for smoothies in DINO and Intake24 used in calculating total fruit and vegetables, meaning that smoothies contributed more to fruit and vegetable intake with the diary method than with Intake24. After further updates and re-matching, recalculation of nutrients showed a slight narrowing of the differences seen between the original year 10 and the recoded data.

Changes have included improving the detail of recipes and alignment with standard recipes (FSA, 2017), and restoring some foods that were removed in the original rationalisation in order to address issues identified from the stage 1 evaluation, alongside routine NDB updates.^{[footnote 15]} The effect of these refinements was assessed to understand the impact on population nutrient intakes (see ‘Appendix A6’ below).

While the changes implemented since the stage 1 evaluation report do not represent such a major overhaul as the initial NDB rationalisation, it is useful to understand and monitor their impact on reported population nutrient intakes to evaluate changes and ongoing refinements to the NDB.

It is important to note that small changes have been made to the NDB throughout the rolling programme, but their impact from one version to the next has not been quantified.

5.2 Impact of dietary assessment method change on monitoring trends over time

For this part of the evaluation, a number of key foods and nutrients were selected for inspection on the basis of the following considerations:

• importance for policy and monitoring over time
• as indicators that have been relatively constant over time and where change would not necessarily be expected
• items commonly omitted
• items which may be misclassified by participants when using Intake24

These are listed in ‘Appendix D: impact of dietary assessment method change on monitoring trends over time’.

The first stage evaluation report presented this analysis using data from October 2019 to March 2020 (year 12) and showed that, generally, the spread of energy and nutrient intake data collected using Intake24 was similar to that for years 1 to 11, which used the paper diary. However, step changes were observed for some foods, which were likely to be methodological rather than other factors, such as the impact of the COVID-19 pandemic, given the timing of data collection.

The analysis has been repeated for this second stage evaluation using a larger data set. NDNS RP data from October 2020 to May 2022 (years 13 and 14) and from the NDNS follow-up study August 2020 to October 2020 (PHE, 2021b) has been added to the time trend analysis (see section ‘1.2 Overview of evaluation and content of this report’ above) alongside data from the stage 1 evaluation (year 12, October 2019 to March 2020) to allow inspection of differences.

The timing of the evaluation data collection spans periods of change in the availability of foods and patterns of consumption in the UK, and it is therefore possible that changes in reported intakes may reflect actual changes in eating habits. However, any step changes that have persisted since the first stage evaluation are unlikely to be as a result of the COVID-19 pandemic.

For ease of reference, October 2019 to May 2022 is used in this chapter to indicate the inclusion of all data above unless stated otherwise. The commentary is supported by ‘Appendix D: impact of dietary assessment method change on monitoring trends over time’.

For selected foods and nutrients, for each NDNS age group (1.5 to 3 years, 4 to 10 years, 11 to 18 years, 19 to 64 years and 65 years and over), individual-level average daily intake (obtained using the paper food diary) was plotted per quarter of a year for years 1 to 11 (2008 to 2019) and a weighted linear regression line was presented along with combined year weighted means.^{[footnote 16]} Individual-level average daily intakes for October 2019 to May 2022 (obtained using Intake24) were added to the plots, but the regression line was not extended as the additional data from October 2019 onwards was not weighted.^{[footnote 17]}

While it was not possible to judge any shift in the centre of the distribution from the plots, they showed the range of intakes so that any obvious changes in variation between the years using Intake24 and previous years using the paper diary could be visually identified. No statistical testing was performed between the 2 periods because survey design information (such as weightings and stratification) was not available for the more recent data.

In addition, for foods, percentage of consumers per quarter of a year (unweighted) was plotted alongside population intakes for years 1 to 11 for comparison with October 2019 to May 2022 to identify any changes.^{[footnote 18]} For foods where there were a large number of non-consumers, percentage of consumers and intakes for consumers only were presented instead of population intakes.

The potential impact on the NDNS RP dietary data of including data from all participants who completed at least 1 recall as opposed to only including data from those with 3 or 4 recalls (which would be comparable with the food diary method in years 1 to 11 that only included data from participants completing 3 or 4 diary days) was examined.

The time trend plots in ‘Appendix D: impact of dietary assessment method change on monitoring trends over time’ distinguish between those participants in October 2019 to May 2022 who only completed 1 or 2 recalls (n= 445) from those who completed 3 or 4 recalls (n=2397).

5.2.1 Observations 

Energy and nutrients

The plots in ‘Appendix D: impact of dietary assessment method change on monitoring trends over time’ show that, generally, the spread of energy and nutrient intake data collected using Intake24 appeared similar to data collected with the paper diary with no evidence of step changes.

For vitamin D and folate, intakes for all years are presented including the contribution from supplements. Functionality to allow reporting of dietary supplements using the ‘Add a meal’ function had been added to Intake24 prior to the start of year 12.

Results from the stage 1 evaluation raised concerns that the percentage of participants reporting taking supplements containing vitamin D or folate in October 2019 to March 2020 appeared lower than in years 1 to 11 in most age groups. This led to a modification from mid-July 2021 so that all participants were asked a question specifically about supplements, which then linked to a direct search using a more appropriate food list and prompts.

Figure 9 shows that, in the last year of the paper diary (2018 to 2019), 22% of diary days had at least one supplement reported.

Before the introduction of the new question in Intake24, only 12% of recalls had at least one supplement reported, increasing to 33% of recalls with at least one supplement reported after the modification. This may reflect a general increase in the uptake of supplements over recent years, partly in response to the COVID-19 pandemic, that was not adequately captured by Intake24 prior to modification.

Figure 9: percentage of days reporting at least one dietary supplement using the diary (n=4358) and using Intake24 before (n=4788) and after (n=1927) the introduction of new supplement question in July 2021

Data from October 2021 to March 2022 showed a noticeable increase in the percentage reporting taking supplements for vitamin D across all age groups - higher than in years 1 to 11.

The percentage reporting taking supplements for folate also showed an increase in the same period, although less marked than that seen with vitamin D.

Foods

The following observations are shown in the plots in ‘Appendix D: impact of dietary assessment method change on monitoring trends over time’.

Foods with dietary recommendations

Fruit and vegetables, red and processed meat, and fish (total fish and oily fish) are foods with specific dietary recommendations. Therefore, they are key measures when monitoring changes over time. The stage 1 report had observed step changes in consumption of fruit and vegetables, and in total fish and oily fish using Intake24 compared with the paper diary. 

For October 2019 to May 2022 using Intake24, there was an observed decrease in the proportion of adults aged 19 to 64 years achieving 5 A Day fruit and vegetable portions compared with years 1 to 11 using the paper diary. There was no evidence of change in the percentage of consumers of fruit or vegetables or in the amount of fruit consumed in this age group. However, reported vegetable intakes (including the contribution from composite dishes) were lower with noticeably fewer high intakes of vegetables reported compared with years 1 to 11.

Since the stage 1 evaluation, a number of factors that could be contributing to this difference have been identified, and a number of improvements to capturing intake of fruit and vegetables in Intake24 have been implemented (see Table 3 below). It is also possible that the large amounts of vegetables reported in some diaries were overestimated and that Intake24 provides more accurate estimates.

The impact of these changes on fruit and vegetable intakes will continue to be monitored and reported in the final stage of the evaluation to be included with the NDNS RP years 12 to 15 results report.

Table 3: reporting of consumption of fruit and vegetables - possible causes for observed changes and action taken

Possible cause	Investigation	Action taken
Generic recipe (mixed dishes) codes in Intake24 do not adequately represent the vegetable content of homemade recipes Burdensome to report more than one vegetable in Intake24 Vegetables entered in Intake24 as one of multiple foods in initial search term and ultimately not entered (for example, search term ‘fish chips peas’ will return results only for one food listed)	Reviewed diaries of consumers of large amounts of vegetables, and compared diaries and Intake24 for participants who took part in NDNS follow-up study. Considered patterns of reporting in diary and ease or barriers to recording equivalent frequency or amounts in Intake24 Compared portion sizes and number of discrete vegetables portions recorded in diary with Intake24 Reviewed search terms used by participants in Intake24 that contain vegetables including multiple food items in one search entry	Added more vegetable-based recipes and other non-meat substitute dishes - for example lentil dishes - to food list Added more associated food prompts for fruit or vegetables commonly eaten with other foods

There was no evidence of method-related changes in reported intakes for total meat and red and processed meat in any age group.

The percentage of consumers of total fish and oily fish appeared to fall in most age groups in the October 2019 to May 2022 data using Intake24, but there was no evidence of change in amounts consumed. This is likely to be methodological and is being investigated.

Commonly omitted foods

Snacks, foods eaten as additions (for example, salad vegetables and condiments) and beverages are commonly under-reported foods in dietary assessment (Gemming and Mhurchu, 2016). Our evaluation therefore looked at percentage of consumers to understand whether these foods were similarly reported in the 2 dietary assessment methods.

At stage 1 of the evaluation, a drop was seen in percentage of consumers aged 4 to 10 years for sugar and chocolate confectionery and buns, cakes and pastries in October 2019 to March 2020 (year 12) compared with years 1 to 11. In August 2020 to May 2022, the drop in sugar confectionery and buns, cakes and pastries compared with years 1 to 11 was still observed, but not for chocolate confectionery. No changes were seen in the percentage of consumers of biscuits or crisps and snacks in any age group.

A drop was seen in the percentage of consumers of salad vegetables across all age groups in October 2019 to May 2022 compared with years 1 to 11, but not in the amount consumed by consumers. This is possibly due to a coding difference, which has impacted on the ability to identify salad consumers in the Intake24 data. In the paper diary, if a participant recorded salad as part of a composite dish (for example, sandwiches or burgers) then the salad would have been coded separately, and the participant would be identified and counted as a consumer of salad vegetables. In Intake24, salad in sandwiches and burgers is included in the new single generic food codes (for example, ‘cheese and tomato sandwich’ code) and, while the salad component will be disaggregated for contribution to vegetable intake, participants are not identified and counted as salad consumers in the same way as with the diary.

For cooked vegetables, there was a drop in percentage of consumers aged 11 to 18 years, but not in amounts consumed by consumers.

For sauces and condiments, there was a drop in the percentage of consumers of mayonnaise in October 2019 to May 2022 compared with years 1 to 11 across all age groups. This may be because, like salad above, mayonnaise is included as a constituent ingredient in single food codes in Intake24 (for example, ‘tuna mayonnaise sandwich’), whereas it would have been coded separately in the diary data. This could explain why a similar change was not seen for tomato ketchup, because it is less likely to be part of a composite food code. There was a drop in the amount of gravy consumed by consumers but not in the percentage of consumers. This may be due to differences in the 2 methods in estimating quantities of gravy with the possibility that amounts were overestimated in the diary.

Misclassifications

In the stage 1 evaluation, there appeared to be some step changes for fat spreads and soft drinks between the Intake24 data collected in October 2019 to March 2020 (year 12) compared with years 1 to 11 diary data.

Fat spreads, soft drinks and fruit juice were reviewed as these categories were considered at higher risk of misclassification due to the range of products on the market and the common use of colloquial descriptors, which may not reflect the product specification. For example, the term ‘juice’ is sometimes used in the UK to describe drinks that are not 100% fruit juice or ‘butter’ may be used to describe any fat spread. These step changes observed at stage 1 were considered to be related to the tool rather than genuine changes in consumption and action was taken to improve classification (see Table 4 below).

In August 2020 to May 2022, these differences with years 1 to 11 were no longer observed apart from the decrease in the percentage of consumers of reduced-fat spread, which was still evident. This may be as a result of an over-representation of this type of fat spread in the diary data (see Table 4 below).

There was no change observed for fruit juice, which indicated that this was being reported and recorded consistently in Intake24 and the diary.

Table 4: reporting of consumption of fat spreads and soft drinks - possible causes for observed changes and action taken

Food group	Observation at stage 1	Possible cause	Action taken
Fat spreads	A rise in the percentage of consumers of butter but a fall in the percentage of consumers of reduced-fat spread in Intake24 recalls compared with the diary A fall in amount of low-fat spreads consumed in Intake24 compared with the diary	Intake24 recall: – inconsistencies in food descriptions – use of inaccurate colloquial descriptors such as ‘butter’ used as a generic search term for any fat spread – fat spreads in sandwiches now part of generic food codes so not identifying consumers in same way as diary Paper diary: – default fat was reduced-fat spread so may have been over-represented – fat spread coded separately for sandwiches	Amended food names to improve how foods appear in the Intake24 food list Rearranged how food categories appear in Intake24 food list
Soft drinks	A rise in the percentage of consumers of sugar-sweetened soft drinks (children only) and a drop in the percentage of consumers of low-calorie or no added sugar soft drinks (children and adults) in Intake24 compared with the diary	Intake24 recall: – inconsistency in naming for sugar-sweetened soft drinks and low-calorie or no added sugar drinks – squash and cordial linked to incorrect codes - for example ‘ready to drink’	Amended food names and how they appear in the Intake24 food list Amended food codes linked to soft drinks

Consistency in portion size estimation

Breakfast cereals, rice and pizzas were chosen for review as these are commonly consumed foods and have different options for portion estimation methods available in Intake24 compared with the diary. There appeared to be no step changes for amounts consumed of these foods (pizza data was analysed after winsorization - see section ‘4.2 Recall completeness’ above).

Comparing intakes from 1 or 2 recalls vs 3 or 4 recalls

For average daily intake of nutrients and foods consumed in October 2019 to May 2022, some relatively extreme high and low individual values were observed for those with 1 or 2 recalls compared with those with 3 or 4 recalls. However, not all participants with 1 or 2 recalls had extreme values so the overall impact of this on a group mean level will be small.

The extreme low values (zero consumption) for some foods (for instance, total fish) will have an impact on the percentage of consumers (because a participant with 1 or 2 recalls is less likely to be a consumer than someone with 3 or 4 recalls). However, the plots in ‘Appendix D: impact of dietary assessment method change on monitoring trends over time’ showed that including those with only 1 or 2 recalls in the NDNS RP data set did not have a large effect on the proportion of consumers.

The intention is to continue including participants with only 1 or 2 recalls to maximize use of the data obtained and this will be monitored.

6. Conclusions and next steps

This stage 2 evaluation report builds on the findings of the stage 1 report published in September 2021 (PHE, 2021a) and provides a new assessment of misreporting based on a DLW sub-study. Issues and data discrepancies identified in the stage 1 report have been further considered here and progress made to address these is reported.

The aim of the evaluation was to:

• understand the implications of the dietary assessment method change in the NDNS RP on the quality, coverage and detail of the data collected
• identify aspects of data discontinuity
• assess the feasibility of continuing the time series data set for monitoring trends

While the evaluation has attempted to compare the methods, it is important to note that the paper diary and Intake24 are different in a number of respects and would not be expected to produce identical results. Key differences include that Intake24 relies on the participant identifying and selecting appropriate foods from a pre-coded food list, whereas the paper diary was free text entry and retrospectively coded.

While both methods were introduced via an interviewer in the home, there was intentionally less involvement from the interviewer in the administration of the online recall method compared with the paper diary. It is not possible to quantify the impact of all aspects of the method change on data quality.

The results of the DLW sub-study show that use of Intake24 in NDNS is associated with a similar degree of misreporting of energy intake to that found by the previous NDNS RP DLW studies for the food diary method.

The findings do not suggest any major differences between the 2 methods in the age or sex groups who misreport to the greatest extent, but this analysis is limited by small sample sizes.

Furthermore, the dietary evaluation shows that results from the 2 methods are largely comparable, so we can conclude that the new NDNS dietary assessment method performs similarly to the old method.

Misreporting (usually under-reporting) of energy intake is a well recognised issue across dietary surveys and studies worldwide and, as expected, under-reporting remains an issue in the NDNS RP. It does not appear that the method change from a paper diary to an online dietary recall has mitigated under-reporting, suggesting that while there may be small differential factors, misreporting overall is an inherent feature of any self-reported dietary assessment instrument rather than a factor of the specific method used. The DLW method only describes misreporting of energy intake and cannot characterise how this translates into nutrients or foods, or where the sources of misreporting lie. Further work, beyond the scope of this evaluation, is being considered to improve our understanding of misreporting.

The quality measures part of the evaluation shows that Intake24 appears to be acceptable to NDNS participants as assessed by the participation rates for recalls. The majority of participants who complete one recall go on to complete all 4, indicating high protocol compliance. The average recall time is comparable with other online recall tools and indicators for recall completeness, such as the number of eating occasions, food items and range of foods reported, are broadly comparable with the previous paper diary method.

The evaluation has not identified any major concerns around the continuation of the time series data set for monitoring ongoing trends over time. Some discrepancies in reported food consumption were identified in the stage 1 evaluation, but these have mainly been resolved by making changes to the food lists in the Intake24 tool.

However, there is some evidence that vegetables are not being reported by adults using Intake24 to the extent reported using the paper diary method. It is likely this is related to method factors and longer-term work to address known issues with Intake24, such as improved search functionality to identify and capture multiple foods reported in one search entry, should result in better capture of consumption.

Also, consumption of total and oily fish and, for younger children, sugar confectionery and buns, cakes and pastries are not being reported as often as they were in the diary. There is no other data available on consumption of these foods but household purchase data does not suggest a reduction in purchases (OHID, 2022). As with vegetables, improving identification of multiple foods in one search entry should assist with capturing these foods. Qualitative work may also help understand the challenges with recording these particular foods in Intake24.

It is important to note that future work will focus on improving the quality of the data collected with Intake24, rather than attempting to align it with the diary data. The observed differences seen with these foods may persist and it may not be possible to definitively establish the reasons for those differences in every case. It also should not be assumed that the food diary provided a better estimate of consumption than does Intake24. All self-reported dietary data is subject to error - different methods and tools will have different error profiles.

The evaluation of the NDNS method change has coincided with the COVID-19 pandemic - a period of unprecedented disruption to the food supply chain, restrictions on the eating-out-of-home sector and consequent changes to eating behaviour for many people. However, as the stage 1 evaluation was based on data collected before the start of the pandemic and this subsequent stage 2 evaluation did not identify any new step changes in foods or nutrients, it seems unlikely that the impact of COVID-19 on eating habits is a major factor in any changes seen. Nevertheless, it cannot be ruled out that some of the observed changes are real rather than methodological.

Furthermore, following the move to Intake24, the statistical analysis methods for reporting dietary intake results have been reviewed, and the decision taken to apply a method to estimate the population habitual or usual intake distribution. This method is appropriate for use with multiple 24-hour recalls and is commonly used by researchers globally. It enables better estimation of the tails of the intake distribution, including proportions above or below a threshold, than does the day average method previously used in NDNS with the paper diary. Results for NDNS years 12 to 15 will therefore be reported using the usual intake method. Implications of this change for data continuity will be explained in the report.

Final conclusions about the impact of the change in methodology, including an update on outstanding issues noted in this report, will be published alongside the next full NDNS results report in 2024. This will include an updated analysis based on data collected up to May 2023. It will also include an assessment of the impact of selecting more individuals from the same household (see Table 1 above).

This update will complete the planned evaluation of the method change - however, the impact of further improvements to data collection will be assessed on an ongoing basis. 

References

Albar SA, Carter MC, Alwan NA, Evans CEL and Cade JE on behalf of the myfood24 consortium group. ‘Formative evaluation of the usability and acceptability of myfood24 among adolescents: a UK online dietary assessments tool.’ BMC Nutrition 2015: volume 1, article number 29.

Amoutzopoulos B, Steer T, Roberts C, Collins D, Trigg K, Barratt R and others. ‘Rationalisation of the UK Nutrient Databank for Incorporation in a Web-Based Dietary Recall for Implementation in the UK National Diet and Nutrition Survey Rolling Programme.’ Nutrients 2022: volume 14, issue 21, page 4,551.

Barrie A and Coward WA. ‘A rapid analytical technique for the determination of energy expenditure by the doubly labelled water method.’ Biomedical Mass Spectrometry 1985: volume 12, issue 9, pages 535-541.

Black AE, Prentice AM, Goldberg SA, Jebb SA, Bingham SA, Livingstone MBE and others. ‘Measurements of total energy expenditure provide insights into the validity of dietary measurements of energy intake.’ Journal of the Academy of Nutrition and Dietetics 1993: volume 93, issue 5, pages 572-579.

Bluck LJC. ‘Doubly labelled water for the measurement of total energy expenditure in man – progress and applications in the last decade.’ Nutrition Bulletin 2008: volume 33, issue 2, pages 80-90.

Bradley J, Simpson E, Poliakov I, Matthews JNS, Olivier P, Adamson AJ and Foster E. ‘Comparison of INTAKE24 (an Online 24-h Dietary Recall Tool) with Interviewer-Led 24-h Recall in 11–24 Year-Old.’ Nutrients 2016: volume 8, issue 6, page 358.

Brage S, Lindsay T, Venables M, Wijndaele K, Westgate K, Collins D and others. ‘Descriptive epidemiology of energy expenditure in the UK: findings from the National Diet and Nutrition Survey 2008–15.’ International Journal of Epidemiology 2020: volume 49, issue 3, pages 1,007-1,021.

Fitt E, Cole D, Ziauddeen N, Pell D, Stickley E, Harvey A and Stephen AM. ‘DINO (Diet In Nutrients Out) - an integrated dietary assessment system.’ Public Health Nutrition 2014: volume 18, issue 2, pages 234-241.

FSA. Standard Recipes Database, 1992-2012. UK Data Service (viewed on 11 July 2023).

Foster E, Lee C, Imamura F, Hollidge SE, Westgate KL, Venables MC and others. ‘Validity and reliability of an online self-report 24-h dietary recall method (Intake24): a doubly labelled water study and repeated-measures analysis.’ Journal of Nutrition Science 2019: volume 8, eLocator 29.

Livingstone MBE and Black AE. ‘Markers of the Validity of Reported Energy Intake.’ The Journal of Nutrition 2003: volume 133, issue 3, pages 895S-920S.

National Cancer Institute. ASA24® Respondent Website Features (viewed on 11 July 2023).

OHID. Sugar reduction programme: industry progress 2015 to 2020. 2022 (viewed on 11 July 2023).

PHE. Evaluation of change in dietary methodology in NDNS rolling programme: stage 1. 2021a (viewed on 11 July 2023).

PHE. NDNS: Diet and physical activity – a follow-up study during COVID-19. 2021b (viewed on 11 July 2023).

Prentice A. ‘The Doubly-labelled Water Method for Measuring Energy Expenditure. Technical recommendations for use in humans. A consensus Report.’ (PDF, 5.74MB) International Dietary Energy Consultative Working Group 1990 (viewed on 11 July 2023).

Rowland M, Poliakov I, Christie S, Simpson E and Foster E. ‘Field testing of the use of INTAKE24 in a sample of young people and adults living in Scotland.’ Food Standards Scotland 2016 (viewed on 11 July 2023).

Rowland MK, Adamson AJ, Poliakov I, Bradley J, Simpson E, Olivier P and Foster E. ‘Field Testing of the Use of Intake24—An Online 24-Hour Dietary Recall System.’ Nutrients 2018: volume 10, issue 11, page 1,690.

SACN. SACN Dietary Reference Values for Energy., PHE 2011 (viewed on 11 July 2023).

Schoeller DA, Ravussin E, Schutz Y, Acheson KJ, Baertschi P and Jequier E. ‘Energy expenditure by doubly labeled water: validation in humans and proposed calculation.’ American Journal of Physiology in Regulatory, Integrative and Comparative Physiology 1986: volume 250, issue 5, pages R823-R830.

Schofield W. ‘Predicting basal metabolic rate, new standards and review of previous work.’ Human Nutrition Clinical Nutrition 1985: volume 39, supplements 5-41.

Subar AF, Potischman N, Dodd KW, Thompson FE, Baer DJ, Schoeller DA and others. ‘Performance and Feasibility of Recalls Completed Using the Automated Self-Administered 24-Hour Dietary Assessment Tool in Relation to Other Self-Report Tools and Biomarkers in the Interactive Diet and Activity Tracking in AARP (IDATA) Study.’ Journal of the Academy of Nutrition and Dietetics 2020: volume 120, issue 11, pages 1,805-1,820.

Appendix A: background, fieldwork and data processing

Appendix A1: content of stage 2 evaluation report - data and nutrient databank used

This stage 2 evaluation uses data collected over different periods of the NDNS RP fieldwork.

Table A1 below shows the data sets used for the different parts of analysis in this report.

Data from the NDNS follow-up study was used to evaluate changes in food consumption and nutrient intakes over the full-time period, but not for evaluating measures around the use of Intake24 (as some follow-up study participants had previous experience of using Intake24).

Table A1: summary of data sets used for analysis in stage 2 evaluation report

Report content	Data used	Nutrient databank[footnote 19] used
Overall response to the survey, participation rates for completing recalls and representativeness of days of the week	NDNS RP October 2019 to March 2020 and October 2020 to May 2022	Not applicable
Measures around use of Intake24 - for example, completion time or number of foods reported	NDNS RP October 2019 to March 2020 and October 2020 to May 2022	Not applicable
Trends in food and nutrient intake over time	NDNS RP October 2019 to March 2020 and NDNS follow-up study August 2020 to October 2020)	UK_NDB_pre1
Trends in food and nutrient intake over time	NDNS RP October 2020 to May 2022	UK_NDB_1
Misreporting of energy intake	NDNS RP November 2019 to November 2021 (DLW participants only)	UK_NDB_1

Appendix A2: fieldwork changes due to the COVID-19 pandemic

In year 12 of the NDNS RP (October 2019 to March 2020), interviewers conducted a face-to-face interview with all selected participants within a household, during the same visit if possible. This visit included:

• a computer-assisted personal interview
• dietary recall
• a physical activity questionnaire
• height and weight measurements
• a spot urine sample
• seeking agreement for a nurse to visit (and take a blood sample and other physical measurements)

In response to the COVID-19 pandemic, there was a full suspension of fieldwork activity for around 6 months and it was decided not to resume year 12. This meant that just over half of the planned year 12 interviewer fieldwork, and a quarter of nurse fieldwork, was completed.

Subsequently, changes to the fieldwork model were introduced to enable further data collection during periods of government restrictions. When year 13 started in October 2020, restrictions meant that in-home interviewing was not possible and almost all interviewing was conducted remotely over the telephone, after a household visit and participant selection at the doorstep. Some adaptations had to be made for remote data collection - for example, spot urine sample collection was moved to the nurse visit (for participants who agreed), and self-reported rather than interviewer-measured height and weight were collected. Changes to the dietary recall process are discussed in ‘Appendix A3: Intake24 support’ below.

Year 14 fieldwork started in April 2021, initially under the same remote interviewing approach as year 13. Face-to-face interviewing was re-introduced in September 2021 for households who were happy with an in-home visit, and the telephone option remaining available for those who were not.

From the start of year 15 fieldwork in April 2022, face-to-face interviewing was the preferred mode with interviewers offering this option first.

In summary, the COVID-19 pandemic impacted fieldwork across the 3 survey years:

• year 12 fieldwork was truncated (resulting in a lower number of issued addresses)
• year 13 interviewer fieldwork was conducted via telephone only in Great Britain, with some face-to-face visits in Northern Ireland which, for practical reasons, were conducted at a later date
• year 14 fieldwork was conducted using a mixture of telephone and face-to-face modes^{[footnote 20]}

Appendix A3: Intake24 support

During the initial interview (face to face or telephone), participants were asked to complete their first recall independently using Intake24. Interviewers would then check with participants that they did not have any issues completing their recall (for face-to-face visits this was a verbal check^{[footnote 21]} while for telephone interviews a follow-up phone call was scheduled). In the majority of cases, participants would then go on to complete subsequent recalls independently with no further involvement from interviewers.

For participants who were unable to complete their first recall independently (for example, due to internet access issues or lack of confidence with technology), face-to-face recall assistance was offered by interviewers during in-home interviews.

If a participant indicated that they were unable to complete the second recall independently, a follow-up visit would be arranged whereby the interviewer would schedule a date and time to return to the household and complete the recall with the participant.

At the end of each visit, the participant would indicate whether they could complete the following recall independently (such that some participants may have had assistance with the second recall but not the third and fourth).

For remote interviews, or where internet access was poor in the area, telephone recall assistance was scheduled with research staff at MRC Epi. Participants were sent a hard-copy food photograph atlas prior to the scheduled phone call to aid the estimation of portion sizes during completion of the recall. The researcher read out the instructions and prompts in Intake24 with the participant providing the information on their food and drink consumption, which was then entered into Intake24 by the researcher. At the end of the recall, the researcher arranged an appointment for the next dietary recall if assistance was still required.

Appendix A4: number of completed recalls

Table A2 below shows the number of completed recalls for all participants in the NDNS RP for the period October 2019 to May 2022.

Table A2: number of completed recalls for all participants in the NDNS RP for the period October 2019 to May 2022

Completed recalls	Adults (N)	Adults (%)	Children (N)	Children (%)	Total (N)	Total (%)
Completed 1 recall only	81	8%	125	13%	206	11%
Completed 2 recalls only	55	6%	71	7%	126	7%
Completed 3 recalls only	29	3%	32	3%	61	3%
Completed 4 recalls	792	83%	727	76%	1,519	79%
Productive participants[footnote 22]	957	Not applicable	955	Not applicable	1,912	Not applicable

Appendix A5: quality checks

Recall data was examined according to a series of quality checks informed by the research team’s experience with processing and checking dietary data, and published studies using Intake24 and other similar dietary assessment methodology.

This included monitoring the number of recalls:

• with fewer than 10 food items
• with 3 or fewer eating or drinking occasions (occasions when a participant reported consuming only a drink without food were included in the count)
• with completion time of under 3 minutes
• where the calorie intake was very low (less than 400kcal) or very high (more than 4,000kcal)

These counts were compared across survey years and against other UK surveys using Intake24 to check whether percentages were within a similar range.

The recall data includes the original search term used by the participant.

In addition to the counts, more detailed checks were undertaken on a sub-set of 10% of recalls in certain time periods:

• multiple food items in the participant’s search term (for example, ‘toast cereal yoghurt’) and only one food item was coded
• inconsistencies between the search term and the food code selected (for example, searched for ‘chicken stir-fry’ but selected ‘prawn stir-fry’)
• ‘orphan’ foods - a reported food that appeared to have been eaten on its own (for example, beef steak) when it would commonly be eaten with other foods (such as chips or potatoes, or salad)

Due to the manual nature of these detailed checks, it is not feasible to carry them out comprehensively throughout routine fieldwork. However, they are useful at the start of a study, where the tool is used in a new population or setting, and at times of methodological change. In the NDNS RP, therefore, they were undertaken to monitor performance of Intake24 on introduction of the new method to the survey and where there was a change of fieldwork protocol due to COVID-19.

Specifically, these checks were carried out on the year 12 data (October 2019 to March 2020) and on the first 4 months of data collected in year 13 (October 2020 to February 2021) following the move to remote interviews (see sections ‘3.1 Overview of fieldwork changes as a result of the COVID-19 pandemic’ and ‘3.2 Dietary data collection’ above).

Undertaking and reporting on the above checks was part of the monitoring of Intake24 to identify potential issues and improvements for tool functionality and usability. The objective was to identify the frequency of known issues in the NDNS data set and to consider improvements to Intake24, for example:

• additional food prompt questions
• improved portion estimation
• clarification around the naming of foods

Such changes were implemented at intervals during NDNS RP data collection. No adjustments were made to the dietary data itself as a result of the above checks as it was not possible to apply the checks and adjustments systematically to the overall NDNS data set, hence bias may have been introduced if selected adjustments had been made.

In addition to the above, when all missing foods had been coded, individual recalls were reviewed where the total energy intake was less than 400kcal and where the participant had not stated that they consumed ‘less than usual’ or that they were on a weight loss diet. Two recalls were considered ‘incomplete’ and excluded from this evaluation.

Portion size boxplots were generated by NDNS standard sex and age groups (1.5 to 3 years, 4 to 10 years, 11 to 18 years, and 19 years and over) to identify any extreme outliers within each food group. Extreme outliers were identified from the boxplots as individual data points separate from the box and whiskers since they were more than 3 times the inter-quartile range (75th percentile to 25th percentile) from the nearest quartile for that intake (either the 25th or 75th percentile). These were examined on a case-by-case basis and reviewed in the context of the participant’s overall reported consumption. Portion sizes that were considered to be implausible, and likely to be the result of errors in portion size selection, were adjusted. Adjustments were carried out in the bespoke dietary database by changing the portion code at the individual recall level. 

Finally, boxplots were generated by age group to identify any infeasible or extreme energy and nutrient values. Extreme outliers were identified as described above for portion sizes and looked at on a case-by-case basis. Extreme intakes that were considered to be the result of errors in portion size estimation or food composition in the NDB were adjusted, otherwise values were left in the data set as they were assumed to reflect consumption by participants.

As a result of the extreme outlier checks for portion size, energy and nutrients, 0.4% of all food and drink entries in the final data set were adjusted.

Appendix A6: changes to the nutrient databank

Following the rationalisation process, UK_NDB_pre1 was the version of the NDB incorporated into Intake24 for the NDNS RP in October 2019. A programme of updates continued in parallel with NDNS fieldwork, and an updated version of the NDB (UK_NDB_1) was incorporated into Intake24 in June 2022 with the facility to back-apply to the data already collected since October 2019.

In order to assess the effect of these changes on population nutrient intakes, the year 12 data (October 2019 to March 2020) that had used NDB version UK_NDB_pre1 was recalculated using UK_NDB_1 data.

Table A3 below shows differences for selected nutrients and foods (all ages combined).

Table A3: impact of NDB updates on intakes for selected nutrients and foods in NDNS RP year 12 (October 2019 to March 2020)

Food or nutrient	Year 12 using UK_NDB_pre1	Year 12 using UK_NDB_1	Difference	Difference %
Energy (kcal/day)	1,621	1,596	−25	−1.5
Total fat (g/day)	62.8	60.9	−1.9	−3.0
Total fat (%TE)	34.5	34.0	−0.5	−1.4
Free sugars (g/day)	50.3	48.7	−1.6	−3.2
Free sugars (%TE)	11.3	11.1	−0.2	−1.8
Fruit and vegetables (g/day)	225	228	3.0	1.3
Fruit and vegetables - 5 A Day portions (portions/day)	3.3	3.4	0.1	3.0
Fruit and vegetables - 5 A Day portions (% achieving 5 portions/day)	19	20	1	Not applicable
Total meat (g/day)	79.1	80.6	1.5	1.9
Sodium (mg/day)	1,758	1,678	−80	−4.5

Comparison of year 12 data did not reveal any major changes. Those food codes contributing most to the differences seen were reviewed to confirm the updates between versions of the NDB were correct.

Appendix B: doubly labelled water sub-study                                                      

Appendix B1: background to method

The DLW method is an established method, widely agreed to be the most accurate way of measuring TEE in free-living individuals over 1 to 2 weeks, and hence providing the ability to assess misreporting of EI.

The method uses an oral dose of DLW - that is, water enriched in 2 naturally occurring stable isotopes, hydrogen (²H, deuterium) and oxygen (¹⁸O). In brief, the method works as follows.

The ingested DLW equilibrates with the total body pool of water, from which the rate of disappearance (r) of ²H from the body represents water (²H₂O) lost - for example, in urine, breath, sweat and breast milk.

The rate of disappearance of oxygen-18 (¹⁸O) represents the sum of both water (H₂¹⁸O) loss and carbon dioxide (C¹⁸O₂) loss in breath.

Rapid exchange and equilibrium of ¹⁸O between water, and carbon dioxide in body fluids, occurs via the action of the enzyme carbonic anhydrase in red blood cells and the lungs.

The difference between these rates therefore equates to CO₂ production: rH₂O + rCO₂ – rH₂O = rCO₂.

Energy expenditure can be calculated from CO₂ production using standard respiratory equations because there is a known amount of heat (energy) associated with each litre of CO₂ produced during metabolism.

The exact amount of CO₂ produced depends on the composition of the diet - that is, the mixture of carbohydrate, fat, protein and alcohol consumed.

It should be noted that the DLW method gives an integrated estimate of energy expenditure for the period of measurement and not data for individual days.

By following the excretion of these isotopes from the body, through analyses of samples of body water (typically urine) over the subsequent 7 to 14 days, a mean daily rate of CO₂ production is obtained for the participant. From this average, a daily TEE can be calculated, which comprises the energy expended on basal metabolism, digestion and metabolism of food, and on physical activity.

An overview of the DLW sub-study in years 12 to 14 of the NDNS is provided previously in chapter ‘2. Misreporting of energy intake’.

Isotopic analyses

Measurements of deuterium^{[footnote 23]} content of the samples were made using a Sercon ABCA-Hydra 20-22 IRMS (Sercon Ltd, Crewe, Cheshire, UK).

This was done by equilibration of a 400 μL aliquot of urine with approximately 3 bar/mL hydrogen gas over a platinum catalyst. A 500 μL aliquot of the sample and equilibration with 5% CO₂ in N₂ balance (4) was used to determine the oxygen isotopic composition of the urine samples. Analysis was completed using a Sercon ABCA IRMS (Sercon Ltd, Crewe, Cheshire, UK).

In all cases, analytical standards prepared in house and traceable to the international standards Vienna Standard Mean Ocean Water (V-SMOW) and Standard Light Arctic Precipitation (SLAP) were included in each batch of samples analysed.

Appendix B2: participant flow diagram

Figure B1: flow diagram of the years 12 to 14 DLW sub-study showing drop-out rates at the different stages

Text alternative to figure B1

Of the 344 participants recruited, 308 returned kits (sample sets) to the laboratory. Of the remaining 36 participants, 2 dropped out before the kit was issued from the laboratory, 20 dropped out after and 2 could not be contacted after dosing. A further 11 participants did not continue due to suspension of fieldwork during COVID-19.

Of the 308 kits returned to the laboratory, 11 were excluded - 2 because of a possible dose mix up, 7 because the post-dose sample volumes were insufficient and 2 because no pre-dose sample was returned.

Of the 297 sample sets analysed for 18-oxygen, 12 were excluded - 2 due to insufficient dietary data, one because the pre-dose sample was too enriched, 2 because all samples were at natural enrichment and 7 because there were insufficient unique post-dose samples.

Of the 285 sample sets analysed for deuterium, 6 were excluded - 2 because the pre-dose sample was too enriched and 4 because the ratio between the volume of deuterium and the volume of oxygen (known as the space ratio) was outside the physiological range. This may be due to small analytical errors and subject variation.

This resulted in 279 participants in the complete DLW data set.

Appendix B3: tables

Table B1: number of participants in all NDNS RP DLW sub-studies

Age group	Sex	Years 1 and 3 target (N)	Years 1 and 3 achieved (N)	Years 6 and 7 target (N)	Years 6 and 7 achieved (N)	Years 12 to 14 achieved (N)	Years 12 to 14 achieved (N)
4 to 10 years	Males	40	41	30	33	20	20
4 to 10 years	Females	40	41	30	32	20	21
4 to 10 years	Sex-combined	80	82	60	65	40	41
11 to 15 years	Males	40	34	40	42	30	29
11 to 15 years	Females	40	38	40	42	30	27
11 to 15 years	Sex-combined	80	72	80	84	60	56
16 to 49 years	Males	40	38	50	51	40	43
16 to 49 years	Females	40	40	50	51	40	44
16 to 49 years	Sex-combined	80	78	100	102	80	87
50 to 64 years	Males	40	41	40	42	30	24
50 to 64 years	Females	40	37	40	42	30	30
50 to 64 years	Sex-combined	80	79	80	84	60	54
65 years and over	Males	40	29	30	32	20	22
65 years and over	Females	40	32	30	32	20	19
65 years and over	Sex-combined	80	61	60	64	40	41

Table B2: mean values of the ratio of reported EI and measured TEE in all NDNS RP DLW sub-studies

Age group	Sex	Years 1 and 3 (N)	Years 1 and 3 (EI:TEE)	Years 6 and 7 (N)	Years 6 and 7 (EI:TEE)	Years 12 to14 (N)	Years 12 to14 (EI:TEE)
4 to 10 years	Males	41	0.90	33	0.85	20	0.84
4 to 10 years	Females	41	0.90	32	0.88	21	0.82
11 to 15 years	Males	34	0.78	42	0.68[footnote 24]	29	0.69
11 to 15 years	Females	38	0.74	42	0.70	27	0.77
16 to 49 years	Males	38	0.67	51	0.65	43	0.60
16 to 49 years	Females	40	0.64	51	0.68	44	0.69
50 to 64 years	Males	41	0.70	42	0.69	24	0.67
50 to 64 years	Females	37	0.66	42	0.64	30	0.65
65 years and over	Males	29	0.72	32	0.73	22	0.66
65 years and over	Females	32	0.73	32	0.71	19	0.70

Table B3: mean values of reported EI in all NDNS RP DLW sub-studies

Age group	Sex	Years 1 and 3 (N)	Years 1 and 3 EI (kcal)	Years 6 and 7 (N)	Years 6 and 7 EI (kcal)	Years 12 to 14 (N)	Years 12 to 14 EI (kcal)
4 to 10 years	Males	41	1,610	33	1,565	20	1,518
4 to 10 years	Females	41	1,552	32	1,426	21	1,340
11 to 15 years	Males	34	2,058	42	1,775[footnote 24]	29	1,831
11 to15 years	Females	38	1,712	42	1,575	27	1,655
16 to 49 years	Males	38	2,262	51	2,052[footnote 24]	43	1,995
16 to 49 years	Females	40	1,609	51	1,709	44	1,710
50 to 64 years	Males	41	2,160	42	2,065	24	2,213
50 to 64 years	Females	37	1,588	42	1,577	30	1,646
65 years and over	Males	29	1,900	32	2,000	22	1,787
65 years and over	Females	32	1,548	32	1,541	19	1,425

Table B4: mean values of measured TEE in all NDNS RP DLW sub-studies

Age group	Sex	Years 1 and 3 (N)	Years 1 and 3 TEE (kcal)	Years 6 and 7 (N)	Years 6 and 7 TEE (kcal)	Years 12 to 14 (N)	Years 12 to 14 TEE (kcal)
4 to 10 years	Males	41	1,843	33	1,862	20	1,908
4 to 10 years	Females	41	1,763	32	1,655	21	1,682
11 to15 years	Males	34	2,714	42	2,705	29	2,677
11 to 15 years	Females	38	2,379	42	2,307	27	2,243
16 to 49 years	Males	38	3,462	51	3,231[footnote 24]	43	3,369
16 to 49 years	Females	40	2,530	51	2,606	44	2,535
50 to 64 years	Males	41	3,148	42	3,074	24	3,312[footnote 25]
50 to 64 years	Females	37	2,432	42	2,474	30	2,547
65 years and over	Males	29	2,661	32	2,763	22	2,749
65 years and over	Females	32	2,163	32	2,212	19	2,062

1. From October 2021, responsibility for the NDNS contract transferred from DHSC’s executive agency Public Health England (PHE) to OHID.  ↩

2. NatCen has led the consortium since the beginning of the RP. The MRC Epidemiology Unit at the University of Cambridge joined the consortium in November 2017 when responsibilities transferred from the former MRC Elsie Widdowson Laboratory. ↩

3. The NDB is a bespoke database of nutrient composition information maintained on an ongoing basis for the NDNS RP by the survey consortium in conjunction with government. Data is compiled using data from the UK food composition tables McCance and Widdowson’s the Composition of Foods and other sources to provide best available estimates of nutrient composition values for foods consumed in the NDNS RP. ↩

4. Intake24.org (UK locale, system version 3, 2019, University of Cambridge): an open source dietary assessment research tool, maintained and developed by the Nutrition Measurement Platform, MRC Epidemiology Unit, University of Cambridge, in collaboration with Open Lab, Newcastle University. ↩

5. When growth rates are not extremely rapid, such as in older children, correcting for weight change during DLW measurement has been found to make only a very small difference to calculated CO₂ production rate (and therefore TEE). ↩

6. DLW sub-study participants were deemed eligible if height and weight had been measured at the visit or, when interviewing was not face-to-face due to COVID protocols, self-reported (measured by the participant or elsewhere in the previous month); they were not actively trying to lose or gain weight; and the first recall had been completed. ↩

7. In previous NDNS RP DLW sub-studies (years 1 and 3, and years 6 and 7), DLW was administered to an approximate 10% subgroup of total survey participants aged 4 years and over, following dietary data collection. In years 1 and 3, the aim was to recruit 40 participants in each of the 10 age or sex groups - 4 to 10 years, 11 to 15 years, 16 to 49 years, 50 to 64 years and 65 years and over - for both males and females. A recruitment of 371 participants was achieved. However, as the observed level of variation in years 1 and 3 was greatest in the 16 to 49 years age group, the recruitment design was amended for years 6 and 7 to as follows: 4 to 10 years (n=60), 11 to 15 years (n=80), 16 to 49 years (n=100), 50 to 64 years (n=80) and 65 years and over (n=60), with equal numbers within group for each sex. 399 participants were recruited. ↩

8. Of the 279 participants with valid data, 274 completed all 4 dietary recalls using Intake24, 3 participants completed 3 recalls, and 2 participants completed 2 recalls. ↩

9. With the design of the study, no participants were concurrent with all 4 dietary recalls, and the DLW dosing and sample collection period. Four participants had some overlap with recalls 2 to 4, 51 had overlap with recalls 3 to 4, and 54 completed the fourth recall only within the DLW sample collection period. ↩

10. It should be noted that some or all of the 5 significant differences listed in section ‘2.4. Comparison with previous NDNS RP DLW results’ could have arisen at random. These 5 significant differences from all 90 comparisons performed equate to 5.6% of the comparisons, which is very close to the statistical error rate of 5% that would be expected to be significant if nothing had changed. This should be taken into account when interpreting the results. ↩

11. Two recalls were excluded during quality checks and so were not included in the final dietary data set. ↩

12. Numbers for missing foods do not include nutrient supplements, duplicate entries or items that did not require coding - for example, non-nutrient supplements or cold and flu remedies. ↩

13. Caregivers would be responsible for completing the recall on behalf of younger children. ↩

14. During the early stages of year 12 data collection, extreme portions of pizza were identified. A review of the portion size pathway in Intake24 indicated an issue with the selection of number of ‘slices’ versus ‘whole’ pizza. The pathway was immediately amended in the tool to address this and subsequently fewer of these implausibly large portions have appeared in the data. To reduce the effect of these large pizza portions within the data analysis for this evaluation, winsorization was applied, which involves recoding extreme values to the nearest ‘reasonable’ value. In this case, a 1,000g cut-off was applied based on the pizza portion data from years 1 to 11. ↩

15. A programme of updates and revisions (generally annually) are made to the NDB so that the databank is up to date and, as far as possible, reflects the nutrient composition of the food supply for each survey year reported. Updating of the NDB may include the addition of new foods as well as revision of nutrient composition of existing foods, either at food group level following a programme of re-analysis, or to take account of reformulation reported by manufacturers and changes in fortification practices. ↩

16. Details on weighting the NDNS RP data can be found in Appendix B of the NDNS: results from years 9 to 11 (2016 to 2017 and 2018 to 2019) report. ↩

17. Weights will be created later in 2023 for the NDNS RP years 12 to 15 results report. ↩

18. Due to the small number of participants in some quarters of a year in the period October 2020 to May 2022, data for this period has been combined and plotted for half of a year instead of quarters. ↩

19. See section ‘5.1 Updates to the nutrient databank’ and ‘Appendix A6: changes to the nutrient databank’ for more details. ↩

20. In year 14 April 2021 to May 2022, 40% of households had opted for a face-to-face interview. By September 2022, this had increased to 66% of households. Year 15 started in April 2022 and, by September, 79% of interviews had been face to face. ↩

21. In year 12, interviewers were asked to review the first recall before submission if possible. In year 14, this review moved to a check that participants did not have any issues completing and submitting their recall. ↩

22. ‘Productive participants’ are those who completed at least one recall. ↩

23. Deuterium analysis of year 12 urine samples was made at Iso-Analytical Ltd, Crewe, using a Europa Scientific ANCA-GSL elemental analyser and 20-20 isotope ratio mass spectrometer. Analysis of years 13 and 14 samples was made at the Nutritional Biochemistry Laboratory, MRC Epidemiology Unit, University of Cambridge, Cambridge. ↩

24. Significantly different to years 1 and 3. ↩ ↩² ↩³ ↩⁴

25. Significantly different to years 6 and 7. ↩