Guidance

UKSPF: Local Survey Tool

Updated 16 April 2025

About this document

With a host of local level policies and projects focused on improving residents’ perceptions of their local community, local areas might wish to measure perceptions of community in a tested and robust way. This document aims to provide local authorities with practical guidance for measuring community perceptions at a local level. This could be selected micro-geographies within a local authority or the local authority as a whole. It draws on questions from the long running Community Life Survey (CLS) to help local authorities understand more about their communities. This enables benchmarking at local levels and helps local authorities with evaluating the impacts of projects.

The CLS is an annual survey of people aged 16+ in England that tracks trends and developments across areas that are key to encouraging social action and empowering communities. The Ministry of Housing, Communities and Local Government (MHCLG) have joined up with the Department for Culture, Media and Sport (DCMS) to produce local authority level results for the first time in 2023/24.

This guidance is based on CLS 2023 to 2024 methodology. Ensuring consistent research methods will allow for the data collected by local authorities to be more easily and accurately compared to larger samples, such as the CLS. It can also ensure that the data collected is valid and reliable, meaning it is accurately measuring what it intends to. 

This document covers:

• why you should use standard questions
• key survey questions
• survey methods – considerations to be made when using these questions
• taking a qualitative approach – possible qualitative methods that could be used to capture more insights
• how to analyse – suggested quantitative analysis that can be done with survey results
• appendices
  • A – key survey questions
  • B – demographic questions
  • C – methodological steps to consider
  • D – suggested qualitative methods

This document has been developed by MHCLG in partnership with the Institute for Community Studies and Verian. If you have questions related to this document, email: UKSPFsurveys@communities.gov.uk.

Why use standard questions?

The questions highlighted below have a considerable history in local growth and communities research. Many of these questions have been used nationally since push-to-web data collection commenced on the CLS in 2013/14, and some go back to the Citizenship Survey in the early 2000s. As a result, many of these questions have a considerable time series of data, and they have been comprehensively tested.

As such, using these questions enables local authorities to:

1. Compare themselves to:

• national level data – to see how they compare to the national picture, particularly looking at national trends for these questions over time

• regional data – to see how they compare with the other local authorities in the region

• similar local authorities – to see how they compare with like local authorities

• within local authority – to see how micro-geographies within a local authority compare to each other and the local authority as a whole.

2. Evaluate the impacts of projects, using time series data for local authorities or benchmarking against similar local authorities to identify the impacts of projects.

Key survey questions

The 15 key survey questions MHCLG suggest prioritising to measure community perceptions are set out in Appendix A. They cover the following key themes:

• the community and neighbourhood: people pulling together to improve the neighbourhood
• wider local area (15-20 mins walk):
  • satisfaction as a place to live
  • access to facilities
  • pride in living here
  • green and natural spaces
  • culture, sport, and heritage
  • opportunities for children
  • safety and security
• civic activities people have been involved in
• social action people have undertaken
• life chances:
  • personal wellbeing
  • satisfaction with work
  • life skills
  • income
  • life satisfaction

To enable comparisons with CLS data, it is also important to include a number of key demographic questions. This will enable local authorities to understand the differences between key groups within their communities and to understand whether projects have impacted differently on different groups of people. Appendix B suggests 9 key demographics which will allow direct comparisons with the CLS.

Keeping respondent data anonymous is imperative to good survey design and data protection legislation. See guidance on sampling for more information.

Survey methods

While many local authorities have considerable experience in running their own surveys, others may have less experience. Local authorities might choose to incorporate these questions into their existing surveys or create bespoke surveys. Appendix C sets out the key steps to take when planning a survey.

Research questions – identify the research question(s) you most want to answer with the survey and reflect on why the work is being undertaken and what the most useful outcomes are. A Theory of Change can be a good first step in achieving this.

Timings – when planning a survey, it’s important to consider the timing of its distribution, ideally conducting an initial survey before a project starts to establish a baseline, followed by subsequent surveys after the project’s implementation to assess changes in attitudes and behaviours. More details of considerations to be made here can be found in Appendix C.

Survey mode – the questions set out in the appendices will work with most survey modes, from face-to-face interviewing to web-based design or postal surveys. If you’re intending to compare your data to the CLS, a self-completion mode would be preferable to ensure consistency. Details on different survey modes are set out in Appendix C. Whichever survey design is used; it is imperative that the sample is designed in a robust way to ensure it is as representative as possible. The choice of survey mode may affect how representative the achieved sample is, and weighting may be required to compensate for imbalances.

Sampling – an optimal approach would be drawing a random sample of addressees within a local authority or catchment area based on a standard ONS geography (for example, Census Output Areas), to maximise robustness and generalisability of data. Using a standard geography will mean that population statistics can be sourced from the ONS to assess whether the achieved sample is a good representation of the general population, and if imbalances are observed, to develop a suitable weighting scheme.

Sample sizes – the size of the desired sample will depend on the research objectives, resources, and budget. Generally, it is best to obtain as large a sample as possible. It is recommended to target a sample of more than 300 completed surveys per area. Where possible, a target of 500 would be preferable, with this being extended to over 1,000 respondents if there is an intention to conduct demographic sub-group analysis.

Geography – selecting the correct geography is essential in ensuring that the correct population are being surveyed. If you’re interested in studying the impact of a project site, then people within a set distance from the project are the target (perhaps a few postal sectors). However, if broader comparisons are to be made, then it is essential to ensure that a representative group of people in a local authority are surveyed.

Survey questions – the questions detailed in this guidance cover a wide range of community perception topics. Local authorities may wish to add these questions to existing surveys or add additional questions to the 15 selected.

Weighting – the profile of the achieved sample may not be fully representative of the general population (living in the sampled area) as different types of people tend to respond to survey requests at different rates.

Once your survey data has been collected, we would recommend that you compare your achieved sample profile to ONS benchmarks. We would recommend considering variables such as age, gender, education level, ethnicity, household size, and housing tenure. Suitable ONS benchmarks can be found online:

• local authority level data is available from different sources:
  • mid-year population estimates with age and sex profiles for each local authority in England and Wales are published annually
  • The ONS NOMIS website offers other variables at the local authority level,
    such as the latest Annual Population Survey
• if your catchment area has been defined at a more granular level, you can source data from the 2021 England and Wales Census

To enable comparisons with the CLS, we suggest aligning with CLS methodology as much as possible. See the most recent CLS technical report for a more detailed outline of the methodology.

Taking a qualitative approach

Supplementing survey questions with qualitative data often adds richness and understanding to the traditional survey approach.

Open-text questions are often included in survey questionnaires allowing respondents to elaborate more on their answers and ask questions that are more complex than a multiple-choice answer.

Appendix D sets out other qualitative approaches which are often used, including:

Semi-structured interviews – with relatively small samples, often tightly focused on specific themes or working through solutions. Topic guides are used to drive the interview, comprising set questions to ask all interviewees, with follow-up questions and deeper probing on specific themes.

Stakeholder workshops (or focus group) – bringing together a range of people from different backgrounds to share ideas/opinions on the research questions. These allow for shared ideas and participants to build-off each other’s contributions. This approach is particularly useful for developing solutions and testing ideas.

Site-based observations and interviews – conducted by attending the particular site/project of interest, researchers may observe visitor numbers and behaviours firsthand to build a sense of who uses the site. Interviews could be conducted with active users to understand their uses of the site.

How to analyse

The following section sets out basic quantitative analysis that can be done with survey results. These approaches will generate helpful insights that can inform projects, policy, and follow up research.

As outlined in the previous section, it is essential to weight data before conducting any analysis to make sure that the survey sample is representative of the population of interest.

Approach

Descriptive statistics for the sample area

Description

These are summary statistics that describe the data. Typically, this will be done by calculating the percentages of different responses to questions. This approach forms the foundation of most quantitative analysis.

Most survey packages will create spreadsheets of the raw data that you can analyse in software packages like Microsoft Excel. However, it’s important to check and clean all data for any anomalies or errors.

We recommend including confidence intervals on visuals, where possible, and reporting on the base (or “n”) of the sample, to display the precision of your data and enable an informed interpretation of your results.

Sample outputs and graphics

“40% of respondents agreed (tend to or strongly) that they are proud to live in their local area”

Figure 1: Sample stacked bar chart

A bar chart showing survey responses to three statements:

1. I am proud to live in my local area

2. In 5 years time I would like to still be living in my local area

3. I would recommend my local area to others as a good place to live

Each statement has a horizontal stacked bar representing the percentage of respondents who selected each response on a scale of disagreement to agreement with each level colour-coded, with percentages marked along the top from 0% to 100%.

Approach

Comparisons by demographic characteristic

Description

This approach looks in detail at particular questions and compares responses by demographic characteristics.

What can be done here will depend on the sample size that your study achieves. The credibility of the analysis can be reduced when you are comparing sub-populations with small numbers of responses. Comparison is possible with any number above 30, but estimates are less precise and so less likely to detect differences. We recommend a sample of 100+.

Note that where sample sizes are small there is risk of identifying individuals. In these cases, it’s advised to suppress results for that population. Guidance for a minimal number of respondents varies, based on sensitivity of the issues and demographics, but as a rule of thumb we’d suppress cells with less than 30 observations.

We recommend significance testing when doing any comparisons to assess whether any differences found are statistically significant.

Sample outputs and graphics

“71% of those aged 75 and over agreed that that are proud to live in their local area, compared to 47% of 16- to 24-year-olds.”

Figure 2: Sample bar chart

A bar chart showing the percentage of people who agree with the statement “I am proud to live in my local area” across different age groups. The x-axis represents different age brackets. The y-axis represents the percentage of agreement. Each bar has an error bar.

Approach

Comparisons over time

Description

If you have committed to sharing the survey at multiple timepoints, you can analyse the same questions over time to observe trends. This can be particularly effective when planned before and after particular projects.

Sample outputs and graphics

“There was a decrease in people who agree that people in their neighbourhood pull together to improve it, decreasing from 62% in 2021/22 to 56% in 2023/24.”

Figure 3: Sample bar chart

This image shows a bar chart comparing survey responses from 2021 to 2022 and 2023 to 2024. The responses are categorised into three statements:

1. People in this neighbourhood pull together to improve the neighbourhood 2. I can personally influence decisions in the local area 3. People from different backgrounds get on well together in my local area

Each category has two bars representing the percentage of people who agree with each statement for the respective time periods. Error bars are present on each bar.

Approach

Comparisons with national, regional, and local authority data

Description

Due to the way the local tool is anchored to the Community Life Survey, it could be possible to compare local responses to national, regional, and local authority level responses. However, as stated above, if methods deviate significantly from the Community Life Survey, including the timings of fieldwork, then any patterns in the results could be due to methodological differences rather than genuine differences on the ground.

It’s also important to identify similar local authorities when looking for comparisons to reduce the number of variables that may contribute to differences in responses.^{[footnote 1]}

The first local authority level statistical release was published in December 2024.

If other local authorities opt to use the tool there will also be scope for collaboration and cooperation across the country.

Sample outputs and graphics

“40% of respondents in location X agreed they were proud to live in their local area, compared to the national average of 30%.”

Figure 4: Sample stacked bar chart

This image shows a bar chart with the title ”I am proud to live in my local area.” The chart compares responses from ”National data” and ”Location X” on a scale from disagreement to agreement.

Approach

Geographic Information Systems (GIS) analysis

Description

Depending on the number of responses, it is possible to map the responses to particular questions. As long as postcode data has been collected, it is possible to allocate those to different geographical scales commonly used by the Office for National Statistics and UK Government: OA (Output Areas), LSOA (Lower layer Super Output Areas), MSOA (Middle layer Super Output Areas), and LAD (Local Authority Districts) scales.

This approach requires access to GIS software, but there are online versions available, including basic tutorials.

GIS can illustrate how sentiment varies across a geographical area and highlight whether there are any particular places within a locality with especially strong sentiments.

This method would require an absolute minimum of 30 respondents for each place (OA, LSOA, MSOA, etc.) that results are generated for, but ideally 100+, so it may be possible to pool multiple years of data if necessary.

Sample outputs and graphics

“There are strong positive sentiments in the East of [location], while more negative sentiments in the South of [location]”

Figure 5: Sample geographic heat map

This image shows a heat map with various regions shaded using light colours for low values and dark colours for high values. There is no text or numerical data visible on the map itself.

Appendix A: 15 Questions identified for measuring community perception

This section contains the 15 questions MHCLG has used to measure community perceptions and life chances. The question list is not exhaustive, and local authorities may wish to add to the questions provided.

Using the Community Life Survey, 15 questions have been selected that cover a range of topics in the communities space. The benefit of using the questions selected is in the ability to collect consistent data and compare results, where appropriate, with the CLS.

1. Your community (neighbourhood - area within a few minutes’ walk from your home)

To what extent would you agree or disagree that people in your neighbourhood pull together to improve the neighbourhood?

Definitely agree

Tend to agree

Tend to disagree

Definitely disagree

Nothing needs improving

2. Wider local area (area within 15 to 20 minutes’ walk from your home)

Overall, how satisfied or dissatisfied are you with your local area as a place to live?

Very satisfied

Fairly satisfied

Neither satisfied nor dissatisfied

Fairly dissatisfied

Very dissatisfied

3. Local area facilities (15 to 20 minutes and further away)

For each of the following, indicate whether there is at least one within a 15 to 20 minute walk from your home, further away but still in your local area, or there is not one in your local area at all. Select the first two answers if they both apply.

	Yes, within a 15 to 20 minute walk from my home	Yes, further away but still in my local area	No, not in my local area at all	Don’t know / not sure	Not applicable
General/grocery shop
Pub/bar
Park
Library
Restaurant/café
Community centre/hall
Sports facilities
Health Centre/GP
Chemist or pharmacy
Post Office
Place of worship for my faith or religion (such as church, mosque, temple)
Public transport links

4. Civic participation

In the last 12 months, that is since [date one year ago], have you… (select all that apply)?

Contacted a local official such as a local councillor, MP, government official, mayor, or public official working for the local council
Attended a public meeting, rally, or taken part in a public demonstration or protest
Signed a paper petition or an online/e-petition
None of these

5. Social action

Have you been involved in any of the following activities in the last 12 months, that is since [date one year ago]? (select all that apply)

Trying to set up a new service/amenity for local residents
Trying to stop the closure of a service/amenity
Trying to stop something else happening in my local area
Running local services on a voluntary basis
Organising a community event
Another local issue
None of these

6. Community pride

How much do you agree or disagree with the following statements?

	Definitely agree	Tend to agree	Neither agree nor disagree	Tend to disagree	Definitely disagree	Don’t know
I am proud to live in my local area
In five years’ time I would like to still be living in my local area
I would recommend my local area to others as a good place to live

7. Satisfaction with green and natural spaces

How satisfied or dissatisfied are you with the green and natural spaces in your local area?

Very satisfied

Fairly satisfied

Neither satisfied nor dissatisfied

Fairly dissatisfied

Very dissatisfied

There are no green or natural spaces in my local area

Don’t know

8. Culture, sport, and heritage

For each of the following, indicate whether there are opportunities to take part in the activities, groups, and events in your local area?

	Yes, lots	Yes, some	No, none	Don’t know
Arts and cultural activities – such as: - facilities such as cinemas, theatres, museums, art galleries, studios, music venues - groups or classes such as book clubs, theatre or dance groups, choir or music groups, art or craft classes, film clubs, video games clubs - events such as street festivals, musical festivals, carnivals
Heritage sites and places of historic interest – such as: - ancient monuments or archaeological sites with historic significance (such as a burial site or castle) - a place that you visited specifically for its historic nature (such as a local historic town, coastline or countryside) - a building or place with historic or artistic features (such as a public garden, country house, place of worship attended as a visitor, historic civic buildings) - a place connected with industrial or other local history (such as an old factory, mine or railway, shipwrecks, historic sports venue visited for its heritage)
Sports teams, clubs or classes or exercise at sports facilities

9. Parental attitudes

Thinking about your local area, how much do you agree or disagree with the following statements?

	Definitely agree	Tend to agree	Neither agree nor disagree	Tend to disagree	Definitely disagree	Don’t know
My area is a good place to bring up children
There are activities for children in my local area
Children that live in my local area have a good quality of life
Children that live in my local area have opportunities available to them
It would be better to bring up children in a different area
It is easy for children to get involved in crime or join gangs in my local area

10. Safety and security

To what extent, if at all, is anti-social behaviour a problem in your local area? Anti-social behaviour is any behaviour that causes people nuisance, annoyance, alarm or distress. This can include behaviours that are aggressive or intimidating or that impact upon a person’s quality of life.

A very big problem

A fairly big problem

Not a very big problem

Not a problem at all

11. Wellbeing

For each of the following, indicate how you feel about different aspects of your life. There are no right or wrong answers. For each aspect, give an answer on a scale of 0 to ten, where 0 is ‘not at all’ and 10 is ‘completely’.

	0 Not at all	1	2	3	4	5	6	7	8	9	10 Completely	Prefer not to say
Overall, how satisfied are you with your life nowadays?
Overall, to what extent do you feel the things you do in your life are worthwhile?
Overall, how happy did you feel yesterday?
Overall, how anxious did you feel yesterday?

12. Job, work, location and attitudes

How much of the time do you feel each of the following statements applies to your job?

	All of the time	Most of the time	Some of the time	Hardly ever	Never	Don’t know	Prefer not to say	Does not apply
I find my job fulfilling
I enjoy the type of job I do
I feel productive when I am working
I find purpose in my job

13. Life Skills

How confident would you feel in each of the following situations?

	Very confident	Fairly confident	Neither confident nor unconfident	Not very confident	Not at all confident	Don’t know
Meeting new people
Working with other people in a team
Being the leader of a team
Explaining my ideas clearly
Speaking in public
Seeing things from other people’s viewpoints
Expressing myself to others
Speaking up in a group of people I don’t know well

14. Income

Which of these statements best describes your current financial situation? (select one)

I am saving a lot
I am saving a little
I am just managing to make ends meet
I am having to draw on savings
I am running into debt
Don’t know
Prefer not to say

15. Life satisfaction

Overall, how satisfied or dissatisfied are you with…?

	Very satisfied	Fairly satisfied	Neither satisfied nor dissatisfied	Fairly dissatisfied	Very dissatisfied	Don’t know	Prefer not to say
Your income
Your job
Your health

Appendix B:  Demographic questions

Local authorities may wish to understand the differences between demographics when analysing community perceptions. The following 9 questions can be used to collect this data, which are also collected by the CLS and Your Community Your Say survey to compare group differences.

It is important to keep respondent data anonymous, therefore, some data may not be appropriate to collect if the sample size is too small and respondents could be identified. See guidance on sampling for more information.

1. Postcode^{[footnote 2]}

What is your full post code? (e.g. AB12 3CD)

2. Age

What is your age?
16 . . . 99
Prefer not to say

3. Sex^{[footnote 3]}

What is your sex?
Female
Male
Prefer not to say

4. Ethnic identity

What is your ethnic group?
White	English / Welsh / Scottish / Northern Irish / British Irish Gypsy or Irish Traveller Any other White background
Mixed	White and Black Caribbean White and Black African White and Asian Any other mixed / multiple ethnic background
Asian or Asian British	Indian Pakistani Bangladeshi Chinese Any other Asian background
Black or Black British	African Caribbean Any other Black / African / Caribbean background
Other ethnic group	Arab Any other ethnic group
Don’t know
Prefer not to say

5. Tenure type

Do you or your household own or rent this accommodation?
Owns outright
Owns with a mortgage or loan
Part-owns and part rents (shared ownership)
Rents it (with or without housing benefit)
Lives here rent-free

6. Children

How many children aged under 16 currently live in your household?

0

1

2

3

4

5

6

7

8

9

10

7. Working Status

What is your current working status?
Working full time (30+ hours a week)
Working part time (less than 30 hours a week)
On government supported training
Unemployed and available for work
On maternity or paternity leave
Retired and not working at all
Full-time education at school, college or University
Looking after home or family
Permanently sick or disabled
Doing something else
Don’t know
Prefer not to say

8. Income

Select the income group which represents your household’s total annual income before any deductions such as income tax or National Insurance?
Up to £2,599
£2,600 up to £5,199
£5,200 up to £10,399
£10,400 up to £15,599
£15,600 up to £20,799
£20,800 up to £25,999
£26,000 up to £31,199
£31,200 up to £36,399
£36,400 up to £41,599
£41,600 up to £46,799
£46,800 up to £51,999
£52,000 up to £74,999
£75,000 up to £99,999
£100,000 up to £149,999
£150,000 or more
Don’t know
Prefer not to say

9. Education

Have you achieved any of the follow qualifications? (select all that apply)

Degree level or above	PhD Bachelors or Masters degree HND or HNC NVQ level 4 and above, teaching or nursing
GCSEs or equivalent	5 or more GCSEs (A* to C, 1 to 9), O levels (passes) or CSEs (grade 1) Any other GCSEs, O levels or CSEs (any grades) or Basic Skills course Standard grade, National 4 or 5 (Scotland)
AS, A level or equivalent	2 or more A levels, 4 or more AS levels 1 A level, 2-3 AS levels 1 AS level Highers (Scotland)
NVQ or equivalent	NVQ level 3, BTEC National, OND or ONC, City and Guilds Advance Craft NVQ level 2, BTEC General, City and Guilds Craft NVQ level 1
Other or no qualifications	Any other qualification, equivalent unknown No qualifications
Don’t know
Prefer not to say

Appendix C: Methodological steps to consider

1. Research questions

Before starting the project, it is important to ask, as a researcher, what research questions are you trying to answer? And therefore, is conducting a survey the most appropriate way of answering these questions.

The approach taken in the research – from sampling, through analysis, to visualisation and accompanying narrative – should all work towards answering that research question. A good research question will be specific, relevant, and answerable with the resources available.

Identifying a research question can also help clarify thinking as it encourages reflection on key issues: why is this work being undertaken; how will it inform plans and practice going forward; what is our theory of change?

Examples of research questions for this programme of work might include:

• How has a new leisure facility affected community sentiment in an area?

• Is the community’s perception of an area changing over time?

2. Survey questions

The questions in this local survey tool have been carefully selected and are anchored to the Community Life Survey. These should not be changed as it will limit what analysis you can do in terms of benchmarking and generalisability. Collecting data on these specific questions will allow you to compare local results to national results in England and understand how your geographical area of concern is doing compared to other areas across the UK also using this method. Even though the Community Life Survey is focused on England, this does not mean there is no value in using these questions for local authorities in Scotland, Wales, and Northern Ireland – the questions are still relevant and important to monitor, but additional care will need to be taken when comparing with the similar questions asked in the Scottish Household Survey and National Survey for Wales. 

The questions selected are not an exhaustive list of the questions a local authority may want to ask their local communities. Adding further questions on particular topics of interest is okay, however we do not recommend amending the pre-existing questions provided. Any questions added cannot be used in the same comparative way outlined here, unless they are also contained within another available local authority level survey. Similarly, if local authorities would like to add additional answer options to the questions included, then comparison with the Community Life Survey will not be possible.

If a local authority would like to add additional questions, we would recommend following the principles of good research practice discussed below to ensure the additional questions meet the same standard of validity and reliability as the questions in the Community Life Survey.

It is possible to use the 15 recommended questions in surveys that have already been developed. Many local authorities already conduct regular annual surveys, and the questions outlined here would be a great addition to these pre-existing surveys. Although, if the survey methodology and mode differ substantially from the Community Life Survey then it won’t be possible to make robust comparisons.

3. Geography

It is important to identify a sensible geographical range for the research. This geographical area selected will impact how you interpret results – as findings will only be generalisable to the population living in this specific area.

The geographical area that is most appropriate for your study will depend upon the research question you are hoping to answer.

• If you want to look at the impact which a new facility - or improvements to an existing facility - have had, it will be important to consider what a reasonable “catchment area” around the facility is likely to be. For instance, this could be a distance in miles, or the length of time it takes to walk to the facility.

• If you want to measure the impact more generally across a geographical area - then it might be appropriate to select an entire local authority, or a combination of wards within a local authority, or some other definition. It is possible to obtain local authority level estimates of questions from the Community Life Survey – this would be a good starting point for geographical comparisons.

When determining the geographical area for a study, it is also important to consider the size of the population that lives in the area and whether it is sufficiently large to support a quantitative survey. The population size required will depend on the target sample size and the expected response rate to the study.

When defining the geographical area covered by the study, it is essential to use one of the standard geographical classifications used in the UK. The ONS has valuable resources describing the different levels of classification. Using a standard geographical classification to define the geographical area for your study will make it easier to:

• draw a random sample of addresses to include in the survey
• assess sample representativity once fieldwork has been conducted and apply weighting to compensate for observable imbalances
• map the data using Geographic Information Systems (GIS) software 

4. Timing

As well as considering where the survey is sent, it is necessary to consider when the survey will be sent. It is good practice to circulate the survey at multiple timepoints – ideally, an initial survey prior to a project starting to attain a local baseline, followed up with another survey once a project has been implemented and had time to bed-in to see how attitudes and behaviours have changed.

In the absence of a formal control, changes between the baseline and follow-up survey will not be evidence of causality. However, this design would identify trends that could suggest impact. For example, identifying an increase in positive sentiments after a project can be used as evidence to highlight a positive impact of the project.

If a project is expected to have long-term effects, it could also be valuable to consider circulating the survey at regular intervals (every 12 months, for example). This would then enable you to track trends over time and monitor sentiments over an extended period.

Frequency and length of surveying will need to be considered to avoid over burdening respondents leading to survey fatigue.

5. Survey mode

Surveys can be conducted in different ways, for example face-to-face, by telephone, self-completion online, or using a paper questionnaire.

For consistency with the Community Life Survey, we would recommend inviting people to participate online, but with the option to complete via paper if requested. By using a similar approach this means that you can more confidently make comparisons between your survey and Community Life Survey findings.

Should you use a different survey mode, then differences between findings from your survey and the Community Life Survey could be caused by differences in approach rather than reflecting real differences.

6. Sampling

There are various ways in which a sample can be generated for a survey. The most appropriate method to use will depend on factors such as research objectives, survey mode, and budget.
The optimal approach to use for sampling would be to draw a random sample of addresses. Royal Mail licence a database of UK addresses that is commonly used by researchers for this purpose. Standard geographical classifications can then be appended using the ONS National Statistics Postcode Lookup (NSPL) file to limit the Royal Mail file to the specific geographical areas in scope for your study. A list of addresses should then be selected at random for the survey. For example, one of the approaches most commonly used is to select every nth address in the database. If your geographical area of interest is very small you may choose to select all addresses in scope – a census.

This random sampling approach is the gold standard for social research and will help to ensure that your results are robust and generalisable. This is also the approach used by the Community Life Survey, and using a consistent approach will ensure comparability.
Alternative approaches could be used for sampling where budgets do not allow for a random sampling approach to be used. For instance, this could include advertising the study using posters displayed in community centres or posting a link to the survey in social media groups. However, it is important to be aware that these approaches are less robust than using a random sample. For instance, some risks include:

• not all individuals in the population are likely to see the survey advertised – so those that respond may not be representative of the entire population

• people outside of the geographical area of interest may respond

• some individuals may respond multiple times

Constructing a good sample can be a complicated challenge, however a fundamental principle for any survey is to achieve enough responses for the findings to be robust (i.e., to be confident that the findings from the survey are genuine rather than happening by pure chance).

The size of the achieved sample that you require will depend on your research objectives and on your budget. In general, it is best to obtain as large a sample as possible. We would typically recommend targeting no less than 300 completed surveys per area. Where possible it would be preferable to target 500 responses or more. In addition, if you intend to conduct demographic sub-group analysis within a survey, then a much larger sample is likely to be required (1,000+ responses).

The volume of sample needed to generate your target number of interviews will depend on the response rate that is achieved for your study. Response rates can vary substantially from survey to survey depending on a range of factors such as: the population profile of the local area covered by the survey, the incentivisation strategy, reminder strategy, survey mode(s) offered, survey branding and sponsor, etc.
Typically when issuing surveys (sent via the post, a couple of reminders, paper questionnaires in final reminders, and a £10 conditional incentive for completion) then you can expect to receive a completed survey for approximately every  3-3.5 addresses that mailings are sent out to.^{[footnote 4]} However, in areas with very low deprivation, it can be closer to a completed survey for every 2.5 that are sent out.

If you are deviating from this approach, then this is likely to affect the response rate you will achieve. We would recommend that you conduct a small pilot in advance of launching the main study to determine what the response rate is likely to be.

7. Representation

It is critical to ensure the achieved sample is representative of the target population – adults living in the geographical area in scope for the study. If the sample is not representative, there is a risk that the survey estimates may be inaccurate.
It is a good idea to do preliminary research on the general background demographic characteristics of the geographical area you are researching. Demographic characteristics that you may want to consider including but are not restricted to: sex or gender identity, ethnic background, age profile, housing tenure, and education level.

By defining your geographical area of interest using standard classifications you will be able to source population statistics that are specific to your area of interest. Sources you may want to consider using:

• Local authority profiles obtained via NOMIS

• census data published at different geographical levels:

  • for England and Wales 2021, a range of statistics are available using the custom dataset tool

  • for Scotland 2022

  • for Northern Ireland 2021

It can be sensible to develop a sampling strategy of particular demographic groups in advance to ensure they are represented in the results. e.g., offering translated interviews if there is a high proportion of individuals with English as a second language.

Once your data has been collected you can compare the profile of your survey sample to the population profile. Observable imbalances can then be corrected for by weighting the survey data you have collected. Weighting is a statistical technique used to adjust a survey sample by assigning numeric values to different individuals in the data to increase/decrease their contribution to the survey estimates if they are under-/over-represented in the collected data. Most statistical software will include a range of methods for weighting data. The most common approach is to use raking to ensure the sample margins match the population margins for a range of variables.

8. Data storage and GDPR

It is essential to have good data management practices in place. This means that:

• data is stored securely on password protected devices
• no extraneous personal information is collected
• participants are able to offer informed consent – they know what data is collected, why it is being collected, and how it will be used
• there are privacy policies in place, explaining participants’ rights and providing information they are entitled to under UK data protection legislation. This covers how data will be handled and processed, when it will be deleted, and how participants are able to withdraw consent

This list is not exhaustive, consult your data protection processes to ensure adherence to GDPR (General Data Protection Regulation). Guidelines are also available online on how to ensure that research is compliant with GDPR. UK Research and Innovation has a short FAQs document for researchers.

Appendix D: Suggested qualitative methods

1. Free text responses

A quick and easy adjustment to the local survey tool would be to include a few free text response questions. These are text boxes with an open question that allows respondents to share extended thoughts and reflections. For this survey we would recommend up to 5 free text response questions, with a strict word limit (somewhere between 240 characters and 240 words is usually more than enough space). These questions might ask a respondent to explain more in relation to a given question, ask about any recommendations they have for the area, or ask if anything hasn’t been covered that they feel is relevant.

This method can be an efficient way to collect a lot of qualitative data. Including it within the survey also makes it easier to place the qualitative question in direct relation to the quantitative question – allowing you to ask for context, further detail, or explanation. It is also an accessible way for people to share their thoughts without having to take part in an interview or workshop.

Free text responses can very easily collect too much qualitative material that can overwhelm analysts and become unwieldy. Nor is there a guarantee that respondents will use the free text response in the way the survey designer anticipated or intended.

2. Semi-structured interviews

These are perhaps one of the most common methods used by researchers to understand the qualitative dimensions of a research question. They are a flexible tool that allows the researcher to explore a range of material and understand the interviewees’ thoughts, perspectives, and attitudes to a situation. The topic guide for a semi-structured interview can be based around the same structure as the survey – asking a key question for each of the areas covered in the survey, for example. To compliment this local survey tool, you might consider sampling a cross section of local residents who use a new facility that has been opened to understand the difference that it has made to them.

This method can be an excellent way to explore the relationship between personal biography and outcomes – what is it about a person’s individual situation that has led to a particular outcome. The method is also designed to be adaptive, so that follow up questions can be asked about particular areas, or new directions taken in the interview as new material emerges.

Semi-structured interviews can be quite laborious. They are time consuming to arrange, undertake, and analyse. They can also be demanding for the researcher to build rapport in a short space of time with a new interviewee and create an environment that is conducive to the sharing of information.

3. Stakeholder workshops

Hosting a workshop with a group of concerned stakeholders can be a creative way to gather a range of perspectives in a short space of time. There is no single format for a workshop – but it typically involves inviting a range of people from different backgrounds to a shared space and running them through a range of activities that each relate to a different research question. Activities might involve deconstructing a problem statement, undertaking community asset mapping, or developing solutions to problems together. A workshop design that might compliment this local survey tool would be to run a session that is focused on the results of the survey – asking local residents to deconstruct the results, develop potential solutions, or share information they feel is missing. Alternatively, a workshop might be focused on understanding the precise details of how effective a particular project has been – inviting key stakeholders and users to attend.

This method can work particularly well when there is a good mix of people in the room – such as users, local politicians, service managers etc. - this can enable people with different life experience to share their perspectives on an equal footing. In comparison to semi-structured interviews, workshops can be an effective way to understand how communities are thinking rather than individual people.

Despite their strengths, workshops can be difficult to undertake. Facilitating a workshop requires a precise mix of skills and arranging a group of people to be available at the same date, time, and location can be challenging. As with the other qualitative methods it can also generate a huge amount of material that can be difficult to analyse unless sufficient thought has been given to how data will be collected in the room.

4. Site-based observations and interviews

This approach is about attending to a particular site in a systematic way. Site-based observations are about attempting to document how a space is being used – observing the number of visitors, what people are doing, and who is there. If a new facility has been opened in an area, this might involve observing it at particular times of the day and week over a period of time to understand how the space is being used. While observing, on-the-spot interviews might be undertaken with members of the public – where you can ask a few key questions about the situation.

The strength of this approach is that it is highly place-based. The fieldwork will be happening in a particular location, and it is possible to develop a well-rounded sense of why a particular environment might be important.

This method also demands a lot of the researcher, requiring them to pay close attention over extended periods of time. Having a researcher in a space can also affect how it is being used. To do this well the researcher will need to reflexively engage with how they are affecting use of the space and acknowledge the limitations of what their direct observations will be able to capture.

1. Due to compositional differences between local authorities that cannot be controlled for, it is recommended that local authorities are only compared to ones in similar regions or of similar compositions. ↩

2. For spatially specific projects, you might choose to ask respondents for postcodes. However, caution should be taken as it risks identifying individuals. ↩

3. The CLS also asks a follow-up ONS harmonised question on gender identity, however this question is currently under review for methodological reasons.  See the Government Statistical Service  Harmonisation Team’s workplan for the latest guidance on this. We will update this report with the latest gender identity question once established. ↩

4. Nelson and Dufour (2023) Community Life Survey Hyperlocal Booster Report. November 2023. Power to Change. ↩