Official Statistics

2. Methodology and Interpretation - Community Life COVID-19 Re-contact Survey 2020

Published 8 December 2020

Applies to England

2.1 Sampling and recruitment

The sample for wave 2 was drawn from respondents who participated in wave 1 from July 2018 through to February 2020 inclusive. Wave 1 data was mainly collected online, although some respondents (for example those without internet access) completed the survey using a paper self-completion questionnaire.

Wave 2 used a mixed-mode approach to maximise coverage of the re-contact population. Data for wave 2 was collected through online self-completion questionnaires and telephone interviews. Given the time-limited nature of the re-contact agreement for respondents (two years), this window ensured the maximum re-contact sample size from the period before the COVID-19 pandemic.

The sample was divided into four groups, with different recruitment approaches. The make-up of the four groups was determined by the contact details available, the mode of completion of the original wave 1 interview, and online proficiency levels.

The four groups were:

Group 1: respondents who completed wave 1 online, and were invited to take part in wave 2 online

Group 2: respondents who completed wave 1 online, were invited to take part in wave 2 online, with a telephone follow-up for non-responders

Group 3: respondents who completed wave 1 on paper, and were invited to take part in wave 2 by telephone

Group 4: respondents who completed wave 1 on paper, and were invited to take part in wave 2 online

Further information on who was included in each group can be found in the Technical Note.

2.2 Questionnaire

The wave 2 questionnaire largely replicated the wave 1 questionnaire and included the following eight sections:^{[footnote 1]}

1. Opening demographics

2. Social contact and neighbourhood

3. Formal volunteering

4. Informal volunteering

5. Charitable giving (web survey only)

6. Wellbeing and loneliness

7. Closing demographics

8. End of interview admin

Where necessary, questions were modified to capture respondents’ behaviours or economic situation specifically in relation to COVID-19.

For example, the questions on formal and informal volunteering were adapted so they asked about the time period when restrictions due to the COVID-19 pandemic were put in place. The original volunteering questions in wave 1 had a recall period of 12 months, while wave 2 had a recall period of four months. This should be taken into consideration when interpreting findings from this report. Further details of these considerations can be found in section 1.7 and in Appendix B.

Beyond these specific changes, there were also other changes in survey coverage between wave 1 and wave 2. In order to keep the average questionnaire length at wave 2 to 15 minutes, some questions from wave 1 were not included in wave 2. In addition, a small number of new questions were added to capture behaviours specifically related to COVID-19, for example whether people volunteered to help people affected by the pandemic. Full copies of the wave 2 questionnaire can be found in the Technical Note.

2.3 Fieldwork

Fieldwork for wave 2 ran from 7th July through to 26th July 2020. In total, 6,519 cases were issued for wave 2 and 2,812 questionnaires were completed, representing a 43% conversion rate.^{[footnote 2]} The majority (n=2,469, 88%) completed the survey online, and n=343 (12%) by telephone interview.

The cumulative response rate amongst those who completed the wave 1 questionnaire in the eligible timeframe was 16%.

There were three types of attrition (totalling 84%) between wave 1 and wave 2:

1. Refusal to be re-contacted about additional research when completing the original survey (57%).

2. Non-provision of either an email address or a full name (5%).

3. Survey non-response (22%).

The weighted wave 1 response rate from the period covering July 2018 through to February 2020 inclusive was 20%. Therefore, the total wave 2 response rate was approximately 3%, 16% of the original 20% wave 1 response rate.

Although this is low, the availability of highly relevant wave 1 data for both the wave 2 respondents and non-respondents allows for a sophisticated weighting protocol. This substantially reduces the risk of additional bias, despite the sharp reduction in response rate from 20% (wave 1) to 3% (wave 2).^{[footnote 3]}

Further information can be found in the Technical Note.

2.4 Weighting, analysis and reporting conventions

In total, 2,812 questionnaires were completed and passed a quality control test.^{[footnote 4]} The majority 2,469 (88%) completed the survey online, and 343 (12%) by telephone interview. The data were weighted to compensate for net attrition between wave 1 and wave 2. The weights ensured that the wave 2 respondent profile matched that of wave 1 with respect to the full range of characteristics recorded in wave 1, both demographic and topic-specific (see Technical Note). A separate weight was also computed for online respondents because some items within the wave 2 questionnaire were not included in the telephone version.

This report provides net and gross change analysis for survey measures covering volunteering, charitable giving, social cohesion, wellbeing and loneliness. The report highlights statistically significant differences at the 95% confidence interval (see Glossary for questions asked in both wave 1 and wave 2, and for cross-sectional estimates between subgroups in wave 2.

The tables refer to the percentage for any given measure. Confidence intervals are included and are referred to as the ‘upper estimate’ and ‘lower estimate’. The upper and lower estimates have been calculated using a 95% confidence interval. This means that if the survey was conducted 100 times, a finding of the same nature would be found in at least 95 cases. Confidence intervals are also included in charts as error bars where this is feasible. The base descriptions in the tables and charts refer to the respondents who were asked the question(s). Any exclusions are made clear in these descriptions.

Percentages included in charts and tables in this report may not add up to 100% due to rounding, the exclusion of some categories (e.g. ‘Don’t know and ‘Refused’) and the option for more than one category to be selected at some questions.

Further information can be found in the Technical Note.

2.5 Limitations

A number of limitations should be considered when interpreting findings from this report. Where these limitations apply to specific findings, this has been detailed in the text.

• Unable to infer direct causation. Although this report details the level and dynamics of change between the pre COVID-19 pandemic period and the period during the pandemic, we cannot infer that the pandemic itself caused the change.

• Mode variation between waves. It is important to note that respondents were invited to complete wave 2 online or via telephone, while at wave 1 respondents were invited to complete online or via paper questionnaire. This may affect some gross change analyses if measurement effects vary between modes.

• Question wording variation between waves. There were some minor changes in question wording between wave 1 and wave 2 which affect the ability to make direct comparisons between waves. Some questions were also adapted to capture respondents’ behaviours in relation to COVID-19 and are therefore not strictly comparable.

• Question coverage varied by mode. For pragmatic reasons, some questions were not asked across all modes. Some questions were too complicated to be asked via paper questionnaire at wave 1 and some questions were too lengthy to be administered by telephone interview at wave 2. Therefore, these measures have smaller base sizes compared to the overall wave 2 sample size. Any exclusions are made clear in the base description in the tables and charts. Weights have been calculated to account for these exclusions. All reported data are weighted to ensure that they are representative of adults aged 16+ in England.

• Recall period varied by wave. The recall period for the volunteering measures was amended from the original 12-month recall period (at wave 1) to four months (wave 2).^{[footnote 5]} This means wave 2 respondents were asked to recall their volunteering activities and behaviours over a shorter period than in wave 1. Consequently, some volunteering measures covered in this report are not directly comparable with their wave 1 counterparts. This is covered in more detail in Chapter 3 (formal volunteering) and Chapter 4 (informal volunteering), and in the Technical Note.

• Unobserved differences. Finally, although the weighting protocol makes the wave 2 sample approximately equivalent in observable respects to the wave 1 sample, there may be unobserved (and therefore uncorrected) differences between the two that affect comparability. This risk exists across all surveys. The risk here is small but not negligible.

• Comparability with previous annual survey years. Data from the re-contact survey should not be compared with previous years of Community Life Survey data because it is a different cohort. The cohort is made up of those who completed the 2018-19 or 2019-20 Community Life Survey and agreed to be re-contacted for follow-up research. While weights and statistical techniques have been applied to correct for this difference, a reasonable margin of error still exists. This report should therefore be used as a standalone product and further analysis should be run separately from the Community Life Survey.

Further information can be found in Appendix B.

2.6 Government guidance and context during fieldwork

To contextualise these findings, it is important to understand the restrictions in place during the fieldwork period. Fieldwork for wave 2 ran from 7th July through to 26th July 2020 during the COVID-19 pandemic, just as restrictions were starting to ease. Restrictions which were in place during end of May to June were relaxed under the Health Protection Regulations 2020 on 4th July, allowing almost all non-essential public facing businesses to re-open over the course of the month (by 24th July). Travel was no longer restricted, although social distancing rules remained in effect. Mask use was required in increasingly more settings over the course of the month, becoming mandatory in shops on the 24th July.

The questions on formal and informal volunteering included in wave 2 asked respondents to think about the last four months; therefore respondents who completed the survey on the first day of fieldwork were asked to think back to the 7th March 2020. The charitable giving questions asked respondents to think about the last four weeks; therefore respondents who completed the survey on the first day of fieldwork were asked to think back to the 7th June. The government in England advised social distancing over the course of March and restricted all activity outside of the home on the 23rd March which persisted for six weeks. The period from mid-May to the end of June included a gradual lifting of national restrictions to allow for increased movement, interaction, and commercial activity, combined with local restrictions to address new outbreaks. Some settings remained closed and the public were not allowed to attend large business events, nightclubs, and sports events in stadiums.

1. A copy of the annual Community Life Survey’s online questionnaire can be found in Appendix A of the Community Life 2018-19 and 2019-20 technical reports. ↩

2. Conversion rate is calculated as: number of completed interviews/number of cases issued. ↩

3. Additional weights were also computed to allow longitudinal analysis of variables included in the online but not paper version of the original wave 1 questionnaire. ↩

4. 131 cases were removed following a quality control test checking for consistency between waves on sex and age; one case was removed for completing both online and via telephone. ↩

5. The possible recall period for wave 2 covered 7th March to the 26th July. ↩