Research and analysis

Methodology

Updated 12 April 2022

Applies to England

This methodology statement provides information on how the ‘COVID-19: mental health and wellbeing surveillance report’ (the report) is produced. The information will help users to:

• understand the strengths and limitations of the report

• understand the methods used to create the report

• decide suitable uses for the report

Method for data collection and analysis

Adults

Geography

Covers whole of UK or England (publications in English).

Eligibility

16 years to adult for all groups.

Group 1: pre-existing ongoing longitudinal surveys with mental health and wellbeing variables, data from pre- and post-March 2020 – which capture changes between pre and during COVID-19 pandemic (good to moderate quality). Survey uses validated diagnostic tool(s).

Group 2: longitudinal surveys established in March, April and May 2020 with mental health and wellbeing variables, with at least one follow-up point. Captures changes through the COVID-19 pandemic (good to moderate quality) (good to moderate quality). Surveys use validated diagnostic tool(s).

Group 3: additional longitudinal or cross-sectional surveys that concentrate on specific populations or topic areas and mental health and wellbeing variables. Captures data post March 2020 (good, moderate, low quality).

Group 4: weekly public opinion survey, nationally representative sample with mental health and wellbeing variables, which captures data from March 2020, and is ongoing.

Group 5: national mental health and wellbeing remote service provider with weekly service use data available for publication.

Search strategy

One main search conducted at beginning of report production.

Group 1: eligible cohorts selected from Catalogue of Mental Health Measures

Group 2: eligible cohorts selected from COVID-Minds.

Group 3: eligible cohorts identified through snowball method (for example academic and professional network), ongoing search throughout reporting period.

Group 4: eligible surveys identified through snowball method (for example academic and professional network).

Group 5: snowball method - contacted Organisation for the Review of Care and Health Acts (Orcha) accredited services and other services recommended by experts.

Extraction

Reviewers carry out a routine check for updates (new analyses, results or submitted data).

Group 1: via contact with principal investigators or checking latest results online.

Group 2: via contact with principal investigators or checking latest results online.

Group 3: as and when new cohorts are identified via principal investigators.

Group 4: via contact with principal investigators or statistical lead, or checking latest results online.

Group 5: via contact with service provider or data sent to team regularly. Each reviewer enters data into a standard extraction template.

Synthesis

Results are synthesised regularly from the extraction template. Findings relating to more recent time periods are prioritised. Those relating to older time periods are included if they offer a new perspective to that which has already been covered. Findings from specific groups are allocated to appropriate sections of the report:

Important findings - Groups 1, 2 and 3

Weekly data and survey triangulation - Groups 1, 2, 3 and 4

Weekly data tracking of the general population - Group 4

Spotlight - Groups 1, 2 and 3

Updates from surveys - Groups 1, 2 and 3

Data presented in WICH tool - Groups 4 and 5

Children and Young People

Geographical

Covers whole of UK or England (Publications in English).

Group 1:

• CYP <18 years including evidence pertaining to this age group, for example 16 to 24s. Parents or carers of CYP <18 years

• published from March 2020 to present, with any data collection from December 2019 to present

• mental health and wellbeing outcomes (for example, excluding educational outcomes unless linked to wellbeing)

• published documents including academic literature (including pre-print and working papers), grey literature, government and public sector analysis, public, academic and third sector press releases

• all qualitative and quantitative methods including opportunity sampled and randomly sampled surveys, qualitative research, administrative statistics, secondary analyses, reviews and secondary reviews

Group 2:

• CYP 4 to 19 years and parents of CYP

• weekly public opinion survey

• nationally representative sample with mental health and wellbeing variables

• captures data from March 2020 and is ongoing

Group 3:

• CYP 4 to 19 years

• national mental health and wellbeing remote services provider with weekly service use data available for publication

Search Strategy

Group 1: this is an extensive and complex search strategy along with key words - these are available on request.

Group 2: one main search conducted at beginning of report production. Eligible surveys identified through snowball method (for example academic and professional network).

Group 3: one main search conducted at beginning of report production. Snowball method - contacted Orcha (Organisation for the Review of Care and Health Apps) accredited services and other services recommended by experts.

Extraction

Group 1: each reviewer enters data into a standard extraction template. Research team reviews publications retrieved through search, summarises findings related directly to research questions.

1. Reviewers select studies or sources, decides on relevance for inclusion in tracker. If study not selected by any reviewer, lead reviewer assigns to a reviewer, notifies them via email.

2. Reviewers read study or source, provides summary of key findings in thematic extraction document, including key limitations and notes of interpretation.

3. 20% or three studies (whichever is higher number) peer reviewed by another member of review team. Areas for improvement and rationale for final decisions are logged on DH Exchange.

Group 2: Via contact with principal investigators or statistical lead, or checking latest results online.

Group 3: Via contact with service provider or data sent to team each week.

Synthesis

Group 1: extracted evidence reviewed for relevance to CYP mental health and wellbeing in COVID-19, including experiences relating to COVID-19. Include if it:

1. Supports statements already evidenced in the synthesis unless large number of evidence sources corroborate statement and new source is lower quality or earlier fieldwork.

2. Contradicts statements already evidenced in synthesis unless large number of evidence sources already cited contradict same statement and new source is lower quality or earlier fieldwork.

3. Supports new statement and new evidence is high standard, for example:

• representative sample OR

• new evidence covers specific group of CYP which more rigorous sources have not engaged OR

• evidence is at least moderate standard, for example large non-representative sample and authoritative source and new evidence is about a new topic which more rigorous samples have not engaged OR

• at least one other piece of evidence supports new statement

Group 2: results are synthesised regularly from the extraction template. Findings from specific groups are allocated to appropriate sections of the report.

Group 3: results are synthesised regularly from the extraction template. Findings from specific groups are allocated to appropriate sections of the report.