Analysis of comparability of People and Nature Surveys across UK nations
Published 19 December 2023
An assessment of the comparability of People and Nature Surveys across the UK nations, providing guidance on the comparability of statistics produced across the 4 surveys and recommendations for future harmonisation.
Executive Summary
Each of the UK nations (England, Northern Ireland, Scotland, and Wales) delivers a population level social survey to gather evidence about how the public enjoy, access, and understand the natural environment. This report provides an assessment of the comparability of these surveys, as they currently stand across the UK nations:
- The People and Nature Surveys for England (PaNS)
- Scottish People and Nature Survey (SPaNS)
- People and Nature Survey Wales (PaNSW)
- People in the Outdoors Monitor for Northern Ireland (POMNI).
The surveys were mapped against 6 themes identified prior to the review, with 17 additional subthemes identified during the mapping process. Users can currently draw some conclusions about the 6 main themes, but there are differences between surveys which mean subthemes are not directly comparable. The key themes are:
- engagement with nature
- access to greenspace
- benefits of time spent outdoors in nature
- nature connection
- perceptions of the quality of greenspace near to where people live or visit
- visit expenditure
PaNS and PaNSW are largely identical and readily harmonise across all themes. SPaNS is a slightly shorter survey covering all themes except ‘visit expenditure’ and POMNI is a significantly shorter survey which restricts comparability. The subthemes are tabulated below under the main themes (Table 1), with information on whether cross-study comparability is possible across each subtheme. The comparability classifications are:
-
Likely: subtheme comparison is relatively straightforward because each survey is seeking comparable information and many questions in this subtheme have similar phrasing across surveys
-
Possible: subtheme comparison is possible but phrasing and question context may vary somewhat or there are only a small number of responses
-
Unlikely: subtheme comparison is difficult due to vastly different question contexts and any composite statistics may be poor quality
-
None: subtheme not comparable because at least one survey has no or too diverse responses
In future, the 4 nations may consider introducing a core subset of identical questions which would allow high quality composite UK statistics to be reliably produced across key themes and constructs.
Table 1. Overview of comparison potential across subthemes, including the number of response statementsY in each subtheme.
England PaNS |
Scotland SPaNS |
Wales PaNSW |
N Ireland POMNI |
Comparison potential |
||
---|---|---|---|---|---|---|
Engagement (Engagement with nature) | Frequency and duration | 10 | 7 | 10 | 6 | likely |
Recreational activity | 6 | 14 | 6 | 6 | likely | |
Visit destination | 4 | 7 | 4 | 1 | likely | |
Social engagement | 6 | 4 | 6 | 5 | likely | |
Access (Access to greenspace) | Barriers to access | 4 | 5 | 4 | 1 | likely |
Transport and travel distance | 7 | 12 | 7 | 4 | possible | |
Environment type | 1 | 2 | 1 | - | none | |
Benefits (Benefits of time spent outdoors in nature) | Health | 19 | 16 | 19 | 7 | unlikely |
Social connection | 3 | 2 | 3 | 3 | unlikely | |
Personal growth | 4 | 2 | 4 | 3 | possible | |
Connection (Nature connection) | Personal connection | 18 | 8 | 18 | 3 | unlikely |
Environmental stewardship | 18 | 4 | 18 | 1 | none | |
Environmental action | 4 | - | 4 | - | none | |
Perceptions (Perceptions of the quality of greenspace near to where people live or visit) | Environmental quality | 10 | 6 | 10 | 2 | possible |
Facilities quality | 1 | - | 1 | 1 | none | |
Safety | 1 | 1 | 1 | 1 | possible | |
Expenditure (Visit expenditure) | Expenditure | 2 | - | 2 | 1 | none |
Y Questions with multiple choice answers which span multiple themes are categorised by individual answer statement.
1. Background
Each of the 4 UK nations (England, Northern Ireland, Scotland, and Wales) delivers a social survey gathering evidence about how the public enjoy, access, and understand the natural environment. This includes assessing health and other benefits derived from engagement with nature, environmental attitudes, and any actions taken to look after the natural environment. Currently, each UK nation operates individual surveys for their respective populations, which presents a challenge for cross national comparison as each survey is shaped by distinct policy, funding, and region-specific factors.
This report aims to provide guidance for users by assessing the comparability of People and Nature Surveys across the UK nations: The People and Nature Survey for England (PaNS), Scottish People and Nature Survey (SPaNS), People and Nature Survey Wales (PaNSW1), and People in the Outdoors Monitor for Northern Ireland (POMNI). Overall similarities and differences between surveys are evaluated across the following themes:
- engagement with nature
- access to greenspace
- benefits of time spent outdoors in nature (including physical and mental health benefits)
- nature connection
- perceptions of the quality of greenspace near to where people live
- visit expenditure
The report also considers the broader methodological details of the surveys (for example delivery mode, recruitment, and sample size). The primary purpose of this report is to provide users with information about the comparability of statistics produced across the 4 surveys. The potential, if any, for future composite statistics to be produced is detailed throughout and high-level recommendations for improving future harmonisation between surveys are provided.
2. Overview of Surveys
The 4 surveys have broadly similar aims: to investigate public engagement with, behaviours in, and attitudes towards the natural environment. Since 2020, each survey has transitioned from in-person data collection (typically door-to-door) to an online format using panel surveys. Participants, matched using the Office for National Statistics (ONS) demographic quotas for each nation, are engaged throughout a given survey year to allow analyses of seasonal trends. Comparable demographic data, such as sex, age, and ethnicity, is also captured across all surveys, either as a dedicated survey module (PaNS and PaNSW) or as part of the wider omnibus survey (SPaNS and POMNI). Table 2 summarises an overview of each nation’s survey delivery including frequency, modality, sample size, inclusion criteria, length, survey structure, geographic resolution, and previous surveys.
Table 2. An overview of each nations survey implementation and design.
England PaNS |
Scotland SPaNS |
Wales PaNSW |
N Ireland POMNI |
|
---|---|---|---|---|
Frequency | Annual | Triennial | Biennial | Funding dependent1 |
Design | 6 modules | 7 modules | 6 modules | 1 module |
Modality | Online, Kantar panel survey | Online, Prodege panel survey | Online, Kantar panel survey | Online, Cognisense panel survey |
Participants2 | 25,000 | 12,000 | 6,000 | 6,000 |
Age | 16+ years | 16+ years | 16+ years | 18+ years |
Length | 50+ questions | 48 questions | 50+ questions | 17 questions |
Geographic resolution |
Mandatory modules: local authority Other modules: nationally aggregated |
Mandatory modules: nationally aggregated Other modules: nationally aggregated |
Mandatory modules: local authority3 Other modules: nationally aggregated |
Mandatory modules: nationally aggregated Other modules: N/A |
Start year | 2020/21 | 2013/14 | 2021/22 | 2020/21 |
Predecessor | Monitor of Engagement with the Natural Environment (MENE) | Scottish Recreation Survey (ScRS) | National Survey for Wales and Wales Outdoor Recreation Survey | General national survey |
Other notes | Children’s survey delivered separately |
1 With the aim to implement triennially.
2 Maximum number of participants - some modules have a smaller sample size.
3 The raw data can be disaggregated at the local authority level; however, due to the smaller sample size compared to England (6,000 respondents compared to 25,000 respondents, respectively), users are recommended to disaggregate the data by the 6 regions of Wales.
PaNS, delivered by Natural England, is an extensive survey with 6 modules covering engagement with nature and gardens in the previous 7- and 14-day periods, as well as wider environmental attitudes and behaviours. Each month approximately 2,000 adults complete the survey. Mandatory modules, including those asking about recent engagement, are completed in line with demographic targets. The modular design minimises survey fatigue as some modules are only asked to a randomly assigned subset of participants. As a result, there is a smaller sample size for these randomly assigned modules which are aggregated nationally, whereas mandatory modules have high data resolution available at local geographic scales. PaNS notably differs from the other surveys by also delivering a children’s survey and (unlike all surveys except PaNS Wales) allowing users to disaggregate data by the month in which it was collected.
SPaNS is delivered by Nature Scot and utilises a modular design similar to PaNS, albeit on a slightly smaller scale. A distinctive feature of SPaNS is that 6 modules are deployed in specific months of the year (for example bi-monthly, quarterly), each meeting demographic population quotas. Implemented in every third year, the survey length of Nature Scot has fluctuated since it was first deployed in 2013/14, including 19 questions as of 2023/24.
PaNSW, delivered by Natural Resources Wales, was developed with direct support from Natural England. In the first instance, the survey was launched as a direct copy of the English survey. Since its launch, there have been slight changes to survey questions, but survey structure and questions in PaNSW remain almost identical to PaNS. The raw data is disaggregated at the local authority level; however, due to the smaller sample size compared to England (6,000 respondents compared to 25,000 respondents, respectively), users are recommended to disaggregate the data by the 6 regions of Wales. First implemented in 2021/22, PaNSW will be deployed every other year and survey results will be released from 2023/24.
POMNI is delivered by Outdoor Recreation Northern Ireland (ORNI). This survey has 17 questions - all mandatory - and has the goal of being implemented across 6 non-consecutive months every third year, dependent on funding. In 2020/21, during the first year of its operation, the survey collected data in November, December, March, April, August, and October, maintaining good seasonal coverage.
3. Survey Content
Survey questions were compared across 6 themes that were identified as being of most interest to users, reflecting policy priorities across the 4 nations:
- engagement with nature
- access to greenspace
- benefits of time spent outdoors in nature (including physical and mental health benefits)
- nature connection
- perceptions of the quality of greenspace near to where people live
- visit expenditure
Mapping of each individual survey across the 6 themes and 17 subthemes is provided in Appendix 1. The analysis shows that there are comparable questions in most thematic areas (see summary in Table 1). However, there are clear differences in question structure and phrasing across all themes, which may limit comparison of cross-national survey data. A detailed description of potential comparability of each theme and subtheme is provided in Appendix 2. While acknowledging structure and phrasing differences, findings in this report suggest potential to compare cross-national survey data for 5 of 6 themes and 12 of 17 subthemes:
- engagement with nature
- frequency and duration
- recreational activity
- visit destination
- social engagement
- access to greenspace
- barriers to access
- transport and travel distance
- benefits of time spent outdoors in nature
- health
- social connection
- personal growth
- nature connection
- personal connection
- personal connection
- perceptions of the quality of greenspace near to where people live or visit
- environment quality
- safety
4. Summary and Recommendations
There are core similarities across the 4 surveys. Their aims are broadly aligned, and each survey uses online panel surveys and similar sampling approaches. Given collaboration in the development of PaNSW, questions from PaNS and PaNSW are nearly identical and thus data from these two surveys is readily comparable across all themes. Conversely, SPaNS and POMNI are shorter and have more varied phrasing, therefore comparability between all 4 surveys depends on the theme or subtheme of interest. Notably, differences in question phrasing between surveys means that composite metrics are largely unlikely. However, by using response statements (as outlined in this report), it is possible for some cross-national comparisons to be made. A significant caveat is that taking response statements out of context may impact data quality. This should be considered on a case-by-case basis when extracting data to consolidate national variables and the corresponding uncertainty around data quality duly reported.
In future iterations of surveys, the 4 nations may consider changes to improve cross-study harmonisation and potential UK-wide statistics. This could involve:
- ongoing collaboration between survey teams
- creating a small set of core questions which are identical across all 4 surveys, including key themes such as visit frequency, access, and perceptions of greenspace
- delivering the surveys in the same years, for example biennially
5. Glossary of Abbreviations
NCI refers to the Nature Connection Index, which is a method for measuring how people connect to nature.
ONS refers to the Office for National Statistics, the UK’s largest independent producer of official statistics and the recognised national statistical institute of the UK.
PaNS is the People and Nature Survey, delivered by Natural England.
PaNSW is the People and Nature Survey for Wales, delivered by Natural Resources Wales.
POMNI refers to the People in the Outdoor Monitor for Northern Ireland, delivered by Outdoor recreation Northern Ireland.
SPaNS is the Scottish People and Nature Survey, delivered by Nature Scot.
Appendix 1. Mapping of Individual Questions and Responses For Each Survey
Please refer to the accompanying Excel sheet.
Appendix 2. Detailed Comparison of Survey Content
Survey questions were compared across 6 themes that were identified as being of most interest to users, reflecting policy priorities across the 4 nations:
- engagement with nature
- access to greenspace
- benefits of time spent outdoors in nature
- nature connection
- perceptions of the quality of greenspace near to where people live or visit
- visit expenditure
Subthemes were determined during mapping using inductive coding, a bottom-up approach in which codes are developed as the dataset is analysed. The most recent version of all 4 surveys was used, specifically PaNS from the survey year 2022/23, SPaNS from 2023/24, PaNSW from 2022/23, and POMNI from 2020/21. Questions related to COVID-19 restrictions or lockdowns and questions about children’s interaction with the outdoors were excluded. Questions with multiple choice responses which span multiple themes were categorised as distinct individual response statements. An example of this is:
“Please read the following list of statements and select any which applied to your last visit to the outdoors:
- the activities I did made me feel out of breath or sweaty
- I learned something new about the natural world
- I felt closer to nature
- I enjoyed spending time with family”
In this case, each item falls under distinct subthemes of health, personal growth, personal connection and social connection, respectively. Conversely, where all responses fall under the same subtheme, the question was classified as one unit with one response statement. An example of this is:
“Did you go on [your most recent] visit:
- with children
- with friends
- with an organised group
- by yourself”
Here, all responses fall under the same subtheme of social engagement and therefore are considered as a single response statement.
Detailed descriptions of prospective subtheme comparison are described in subsequent sections; however, it should be noted that question-specific comparability recommendations are beyond the scope of this report.
3.1. Engagement with nature
Engagement with nature has the largest number of questions across the surveys and constitutes many mandatory questions in all surveys. Four subthemes were identified:
- frequency and duration
- recreational activity
- visit destination
- social engagement
Frequency and duration
For the first subtheme, all 4 surveys ask about frequency and duration of time spent in the outdoors. Overall, the frequency and duration sub-theme is likely to be comparable between the surveys. Key similarities and differences are:
- all 4 surveys ask about the frequency of visits in the last 12 months
- PaNS and PaNSW ask about visits in the previous 7- and 14-day periods whereas SPaNS and POMNI asks about visits in the previous 4 weeks
- all 4 surveys ask about duration of total visit and time spent doing an activity
Recreational activity
Comparison between surveys in the subtheme recreational activity is also likely, as all surveys ask about the reasons for the last visit and the type of activities undertaken. However, the response options (broad range of activities covered) provided to participants differ slightly. See Table 3 for a comparison of question phrasing in this subtheme.
Visit destination
Similarly, questions about the environment of the visit destination are also likely to be comparable. All surveys ask about the type of place participants had visited, such as beach, farmland, park.
Social engagement
Finally, social engagement concerns social interactions in nature, asking about who accompanied the participant on the visit and whether participants spent time spent outdoors with the aim of socialising. All 4 surveys have several questions in this subtheme; therefore, it is likely this subtheme is comparable cross-nationally.
3.2. Access to greenspace
Three subthemes were identified related to access to greenspace:
- barriers to access
- transport and travel distance
- environment type
Barriers to access
All surveys ask about barriers to access of outdoor spaces and this subtheme is likely to be comparable cross-nationally. This subtheme includes topics around:
- mobility
- mental health
- busy scheduling (i.e., “too busy at home”)
- lack of facilities (including for people with disabilities)
- feelings around safety
- unaffordability
- general lack of interest
- inclement weather
A full comparison of question phrasing can be found in Table 3.
While all surveys ask classification questions about participants’ health and wellness, PaNS and PaNSW specifically ask a question with notably more detail about health-related barriers to access:
“How important were the following health related reasons in stopping you from visiting green and natural spaces in the last 14 days:
- my mobility
- concerns that I will become ill during this visit
- lack of disabled facilities
- unsuitable / poorly maintained sites
- no one to go with me / help me
- tiredness / fatigue”
Transport and travel distance
All surveys also ask about the distance travelled and mode of transport used for participants’ last visit to the outdoors. It is possible this subtheme is comparable between surveys given considerations about differences in question phrasing. Furthermore, PaNS and PaNSW ask how easy participants think it is to visit the outdoors using public transport or without a car and SPaNS includes questions about the National Cycle Network.
Environment type
Finally, PaNS, PaNSW, and SPaNS ask whether participants have access to private or communal outdoors spaces such as gardens or allotments. POMNI doesn’t ask any questions about these topics, therefore, this subtheme is not comparable cross-nationally.
3.3. Benefits of time spent outdoors in nature
Each of the surveys asks about the benefits participants gain from visiting the outdoors, which have been classified under the subthemes:
- health
- social connection
- personal growth
Health
The subtheme health is unlikely to be comparable across surveys due to large differences in question context and response phrasing. Key aspects in this subtheme were:
- all 4 surveys ask about both mental and physical health benefits of accessing the outdoors
- PaNS and PaNSW phrasing is similar, for example local greenspace is “good for mental health” or are places which “encourage physical health”
- SPaNS and POMNI phrasing is similar, for example going outdoors “to relax and unwind” or to find “peace and quiet”
- PaNS and PaNSW have additional nuanced response options about why participants took their most recent visit, for example “to get a break from TV”, “to get fresh air”, or “because I was advised to spend more time spent outdoors by my GP”
Social connection
All surveys ask about whether participants gained social benefits from their last visit to the outdoors. However, it’s unlikely this subtheme is comparable cross-nationally due to highly variable phrasing:
- “[the visit was] to look after children / other family members” (PaNS and PaNSW)
- “[the visit] allowed to me to connect/ interact with others socially” (SPaNS)
- “I enjoyed spending time with other people [on my last visit]” (POMNI)
Personal growth
The subtheme personal growth has similar phrasing across only few responses, therefore it is possible to compare this subtheme between surveys. All surveys have response statements referring to “learn[ing] something new” and PaNS, PaNSW, and SpaNS also have options referring to the opportunity to “challenge myself”, “be creative”, or “achieve something”.
3.4. Nature connection
Participants’ connection with nature is explored within three identified subthemes:
- personal connection
- environmental stewardship
- environmental action
Personal connection
The subtheme personal connection directly explores participants’ connectedness with the outdoors. PaNS, PaNSW, and SpaNS each ask the 6-item Nature Connection Index (NCI) question:
“How much do you agree or disagree with the following:
- I always find beauty in nature
- I always treat nature with respect
- being in nature makes me very happy
- spending time in nature is very important to me
- I find being in nature really amazing
- I feel part of nature”
Conversely, POMNI asks about the 6th response only, which has been reported to be the key item in the NCI. Therefore, it is possible to compare on this single item only. Overall, comparison of this subtheme between surveys is unlikely because the response statements are from a diverse range of questions and losing question context will considerably affect composite data quality.
Environmental stewardship
The subtheme environmental stewardship includes general attitudes or awareness of environmental issues. This ranges from broad feelings to specific actions taken to protect the environment, for example recycling or walking/cycling instead of using a car.
PaNS and PaNSW have significantly more questions and response statements in this subtheme (n=18) than SPaNS (n=4), while POMNI only has a single response statement in this subtheme on the reason for their most recent outdoor visit: “to help out/volunteer.”
Therefore, it is not possible to compare between surveys.
Environmental action
The subtheme environmental action refers to positive environmental actions specifically carried out during participants last visit to the outdoors. For example, following tenets of the countryside code, or taking care not to disturb wildlife. Only PaNS and PaNSW ask questions in this subtheme, therefore, it is not possible to make comparisons across surveys.
3.5. Perceptions of the quality of green space near to where people live
In this theme about participants’ perceptions of the quality of green space near to where people live, three subthemes were identified:
- environment quality
- facilities quality
- safety
PaNS, PaNSW and SPaNS had more response statements about environment quality (n=6-10) than facilities quality (n=0 or 1).
Environment quality
The subtheme environment quality captures information on wildlife, habitat, or paths and trails, and participants’ views on whether the quality of local greenspaces have changed in the past 5 years. It is possible to compare this subtheme between surveys given that differences in phrasing are overcome.
Facilities quality
Facilities quality refers to infrastructure such as car parks, cafes, playgrounds, or toilets. This subtheme is not comparable across surveys because SPaNS doesn’t ask about this topic.
Safety
All surveys have one response statement under the subtheme of safety. It’s possible this subtheme is comparable cross-nationally, however, each survey has only one response statement each of which have variable phrasing:
- “thinking about the place you visited: the place felt welcoming / safe” (PaNS and PaNSW)
- “how much do you agree or disagree with the following statements relating to your local greenspace(s)? My local greenspace… is a safe place to visit” (SPaNS)
- “how much do you agree or disagree with the following statements relating to your nearest greenspace areas and paths and trails? I feel safe visiting my local greenspaces” (POMNI)
3.6. Visit expenditure
It is not possible to compare this subtheme between surveys because SPaNS does not include questions on expenditure. PaNS and PaNSW ask for a detailed breakdown of itemised expenditure for participants’ most recent visit outdoors, whereas POMNI asks a single question:
“In total how much, if anything, did you personally spend during [your most recent] visit to the outdoors” (a comparison of question phrasing can be found in Table 3).
Whilst it is possible to sum individual item responses from PaNS and PaNSW to compare total expenditure to POMNI, this may cause a discrepancy in data quality when asking participants to recall overall trip expenditure compared to individual items.
Table 3. Comparison of phrasing of several example questions across different themes from each survey.
Theme | Subtheme | England PaNS |
Scotland SPaNS |
Wales PaNSW |
N Ireland POMNI |
---|---|---|---|---|---|
Engagement | Frequency and duration | How many times, if at all, did you make this type of visit to green and natural spaces in the last 14 days? | How many visits to the outdoors for leisure and recreation in Scotland did you make during [preceding month]? | How many times, if at all, did you make this type of visit to green and natural spaces in the last 14 days? | How many visits to the outdoors for leisure and recreation in Northern Ireland have you made in the last 4 weeks? |
Engagement | Recreational activity | Which of these activities, if any, did you do on this specific visit?:
|
Thinking about the most recent visit to the outdoors for leisure and recreation that you took during [preceding month], which of the following did your visit include?
|
Which of these activities, if any, did you do on this specific visit?:
|
Thinking about your last visit to the outdoors for leisure and recreation, which of the activities listed below was the main activity that you did?
|
Access | Barriers to access | How many times, if at all, did you make this type of visit to green and natural spaces in the last 14 days?
|
For which of the reasons listed below, if any, did you not make more visits to the outdoors for the purposes of leisure and recreation in the last 12 months?
|
What was the main reason for not spending free time spent outdoors in the last 14 days?:
|
Which of the following reasons, if any, describe why you have not spent [more/any] of your time out of doors?
|
Expenditure | Expenditure | How much did you spend on:
|
How much did you spend on:
|
In total how much, if anything, did you personally spend during this visit to the outdoors? |