Official Statistics

Indicators of species abundance in England: Response to feedback

Updated 24 April 2025

Applies to England

Last updated: 2025

When we first published the ‘Indicators of species abundance in England’ statistical release in May 2024 we asked for feedback. The production team have reviewed all the feedback received before October of that year, and either addressed it directly in the 2025 release, or made a plan to address it in the coming years. Here we detail the technical feedback we’ve received, and how we have responded to that feedback. We will publish a separate, full response to all the feedback we have received in the following weeks. The indicator production team are always keen to hear feedback from users, please do continue to get in touch with suggestions or comments: biodiversity@defra.gov.uk.

Smoothing timescale

In responses to our question about which smoothing interval would be most appropriate (10 year vs 3 year) we noted no consistent preference on which version of the smoothing is best. For example, some respondents preferred shorter or longer smoothing intervals and some also suggest species-specific smoothing may be useful.

In the current publication, we have retained the two versions of smoothing. A future development goal is to determine which version of smoothing should be used for assessing change in the indicator and carried forward (see Development plan).

Communication of smoothed trends

In the feedback there were concerns about the interpretation and communication of smoothed trends, particularly about missing important interannual variation.

These indicators are a summary of distinct species-specific times-series. Essentially, they are an average time-series across a set of species and inherently lose the granular detail (species-time-series) in favour of a broader picture of patterns of change. Similarly, statistical smoothing processes necessarily will result in a loss of granularity, meaning that strong fluctuations in inter-annual species- or group-specific abundance values, will be down weighted in favour of a smoothed trend. Smoothing is used here to ensure the indicators reflect long-term patterns of change rather than short-term variation that may be driven by weather, unaccounted for variation in recording behaviour, or other short-term drivers of change. Two levels of smoothing are used in the current version of the indicator, a strong and a weak smoother, this allows readers to assess the impact of the amount of smoothing on the resulting patterns of change. The future development plan contains a commitment to an investigation of the appropriate level of smoothing to apply to the these indicators. This is particularly important given the relevance of these indicators to the Environment Act.

Concerns about the interpretation of smoothed trends are valid and it is important that the fundamental caveats and assumptions associated with smoothing are clearly communicated. We addressed this in the ‘Smoothing to reveal long-term trends’ section of ‘Caveats and limitations’ of the England Abundance indicator report (see ‘Caveats and limitations)’.

Compatibility with targets

Once developed, the statistic of relative abundance for all-species in England will be used to track the government’s progress towards meeting the statutory target of halting the decline in species abundance by 2030, and then reversing these declines by 2042.

Although a smoothed index is desirable for assessing change over time, as it smooths out some of the impact of between-year fluctuations (e.g. weather patterns), it does mean that it will be difficult to detect notable change between two adjacent years. The statutory target to halt the decline in species abundance will involve comparing index values from 2029 to 2030, the methodology for making this assessment will be important to explore. Understanding the relationship between the Environmental Act targets, the indicator and the smoothing options is a priority area for further development work.

Pre-smoothing

In the indicator, following advice from an independent expert review of the methodology by three academics , we applied an additional step of smoothing (pre-smoothing) to the species level data before creating the composite index with the Freeman method (Freeman et al., 2020). The decision to implement pre-smoothing was also partly driven by the cross-validation testing which showed an increase in within-dataset predictive accuracy of the Freeman method when applied to a pre-smoothed dataset compared to an unsmoothed dataset. However, there are some limitations to this approach and we have expanded the ‘Pre-smoothing’ section of the Caveats and limitations of the main indicator report to better reflect these. Other departures from the original implementation of the Freeman method, and consequences thereof, are discussed here and in the indicator report (see section ‘Model specifics’ section of the Technical Annex).

The Freeman method was specifically designed to accommodate unsmoothed input data and any associated short-term variation. The Freeman method is also structured to account for the fact that input data provide an imperfect representation of the true underlying state (abundance per year), meaning that individual species-year index values are supplied alongside an estimate of uncertainty (available here for most species from the given input datasets). This uncertainty is then propagated to the final indicator produced by the Freeman method. The current indicator production method departs from both of these original strengths of the Freeman method, each are discussed separately below, and in the ‘Pre-smoothing’ section of the ‘Caveats and limitations’ and the ‘Model specifics’ section of the indicator report, respectively.

By design the Freeman method contains two forms of ‘smoothing’. Firstly, growth rates (that is species year to year change) are assumed to follow a log normal distribution parameterised by the data, where the impact of outliers is reduced as they are pulled towards the mean growth rate across species. Secondly, the Freeman method applies spline-based smoothing at the community (composite indicator) level, which is specifically designed to smooth short-term fluctuations, meaning the indicator better represents the long-term direction of change. These inbuilt smoothing approaches mean that any additional smoothing of the input data, such as the pre-smoothing step we perform, should be superfluous. Some of the feedback received expressed concern that using additional smoothing steps risks removing biologically meaningful signals, which would reduce the indicator’s accuracy and interpretability.

The caveats of using cross-validation based performance metrics as a reason to use pre-smoothing have now been explained in more detail in the publication text. Cross-validation assessments are done by fitting the model to a subset of the whole dataset, such that the data which it hasn’t been fitted to can be used to test the models predictive performance. As the data used in the indicator are pre-smoothed (thereby reducing interannual variation) then it is not surprising that the cross-validation performance is better when using pre-smoothed data compared to the more variable unsmoothed input data. Essentially, the pre-smoothed data are easier for the model to fit, leading inevitably to better performance in cross-validation comparisons.

Given the points above, further development work is required to examine the validity and impact of the methodological departures from the original implementation of the Freeman method. Notably, the addition of pre-smoothing and the lack of species-specific uncertainty propagation. This is now highlighted as an area for future work in the Development plan of the indicator report.

Confidence around the start and end of the time-series

Smoothing an index like the all-species abundance indicator generally produces a trend where the start and the end of the time series have the lowest confidence associated with them and this can impact how well we are able to assess meaningful change over the long and short term. In the all-species abundance indicator, the confidence intervals around the index grows over time, as the indicator is baselined to the first year of the time series (1970) and index values refer to the amount of change since that baseline year. This means we can make reliable comparisons between the index at the start of the time series and how this overlaps with the credible intervals in other years. It is less statistically robust to compare index values between other years of the index. We are currently discussing methods for a more robust approach to assessing change over the short- and medium-term.

Standardising to baseline values in the first year is common practice in biodiversity indicators, and results in the confidence interval around the index being smallest in the early years, however this gives the impression that the first (early) years are estimated with more certainty. This is not ideal as data quality and quantity tend to increase over time meaning there is often more certainty around changes in abundance of many species in the more recent years. Uncertainty around the indicator also tends to increase over time when using the Freeman method. This is in part due to the way that randomness is incorporated into the estimates of growth for each species’ population, which causes compounding of uncertainty year after year. Essentially, each year’s estimate depends on the previous year, so uncertainty builds over time. It is possible that as data availability increases in later years, uncertainty overall can decrease as the benefit (reduced uncertainty) of the additional data outweighs the compounding of growth rate uncertainty.

Understanding which areas of uncertainty are captured by the CIs surrounding the indicator method, and critically assessing the areas of uncertainty that are lost (for example uncertainty around the species-specific index values) is an essential development goal for the indicator. These areas for future development are now highlighted in the future developments section of the indicator report.

Representativeness of the indicator

Some of the feedback received raised issues around the representativeness of the indicator, how that changes over time and how that interacts with the assumptions of the Freeman model. This included the suggestion that we cover other ways of describing representativeness, other than species coverage.

The representativeness of the indicator is reported in multiple sections of the indicator report, but to date has mainly focussed on the species representativeness. For example, the ‘Representativeness of the indicator’ paragraph in the ‘Caveats and limitations’ section, exclusively discusses the taxonomic coverage of the indicator, how that coverage changes over time and how that relates to the overall goals of the indicator, i.e. average trends across ‘all’ and ‘priority’ species. Figure 9 of the report provides a clear visualisation of the temporal change in taxonomic coverage of the indicator.

We have now extended the ‘representativeness of the indicator’ section, discussing other important areas of representativeness, for example the spatial coverage of the data. We make a key distinction around two forms of representativeness. Firstly, how representative the data are in terms of the target populations of interest, in this case, to what extend do the data reflect the change in status of ‘all’ and ‘priority’ species across their respective ranges within England. Second, within the data, how consistent is that representativeness over time, for example the large changes in the number of species contributing to the indicator over time. These are both discussed in the context of the Freeman method, where we highlight assumptions and areas of concern that require further understanding and investigation.

Understanding and communicating the representativeness of the sample data to the desired population of inference is essential for readers to assess the validity of the conclusions drawn from these indicators. An area for future work is to complete standardised assessments of representativeness across multiple scales (taxonomic, spatial and temporal), where any limitations of the data and models are clearly described. The assumptions of the Freeman method as applied in these indicators should be tested, revealing the likelihood of those assumptions holding for the given indicator. For example, the assumption that “trends among missing species follow the same overall distribution as those with data” can be tested. In reality this assumption that species are missing at random with respect to their trend is known to not be true, however, simple diagnostics can give some insight into how wrong such an assumption is.

Lack of measurement error

The Freeman model (Freeman et al., 2020) does not treat the input data for the different species as perfect, but recognises they arise from a sampling process that is subject to measurement error. This measurement error is currently estimated from the data, but the method does include the ability to provide species-specific estimates for measurement error, if available.

The ability of the Freeman method to incorporate uncertainty around the input indices of abundance is a key ‘strength’, but is not currently implemented in the all-species and priority abundance indicators. Instead a constant observation variance is estimated globally by the model. This means that uncertainty around the species-specific index estimates is not propagated through to the final indicator plots and assessments.

The use of pre-smoothing inhibits the ability to easily supply uncertainty values as inputs to the Freeman method. A future development goal is to examine the credibility of using pre-smoothing, including an investigation into options for extracting uncertainty values from the pre-smoothed indices. Future iterations of this indicator will include a comparison of the indicator with and without uncertainty values propagated from the input abundance indices. This is crucial to understand the impact of propagating the species-specific uncertainty through to the final indicator.

Options for weighting the indicator

Several responses suggested that we could consider using weightings within the indicator. Currently species-specific trends are amalgamated without weighting, which treats each species equally. When creating a species indicator weighting may be used to try to address biases in a dataset, for example, if one taxonomic group is represented by far more species than another, the latter could be given a higher weight so that both taxonomic groups contribute equally to the overall indicator. Complicated weighting can, however, make the meaning and communication of the indicator less transparent. The main bias on the data is that some taxonomic groups are not represented at all, which cannot be addressed by weighting. We discuss this further in the Technical annex (see ‘Exploring options for a weighted index’).

We could consider weighting at several different levels, including taxonomic group, functional group or habitat associated grouping. Species-specific weighting has been described in the literature but has often been noted to incorporate a degree of subjectivity that can be difficult to reach a consensus on. Furthermore, it is possible to weight by the level of confidence in the species-specific trends, in other words we can down-weight species where the trends are more vulnerable to bias. However, as with any weighting approach, it is crucial to consider the indicator’s inferential target. Down-weighting species based on bias may result in an indicator that is less representative of the intended population.

Recorder effort

Concerns were raised around the underlying species data used in the indicator and whether we account for bias in the species models due to changing recorder effort and/or change in the spatial pattern of recording over time.

Many of the underlying group-specific species models are designed to handle some of the common forms of bias found in these types of structured surveys datasets. We are aware that even the UK Butterfly Monitoring Scheme, arguably one of the most well-recorded groups has underlying issues of bias: see Boyd et al., 2025, for more detail).

The data sources and the modelling approach used to produce the species specific indices are listed in Table 5 and Table 6, respectively. Understanding the extent to which these methods handle biased data in space and time is challenging, and an evolving field of academic research. A key step towards improving this understanding is clear documentation and communication of the spatial and environmental representation of the sampled data and how these changes over time. As part of the Development plan we intend to produce a clear assessment of the spatial, taxonomic and temporal coverage of data in the next iteration of these indicators.

Transparency

In the updated publication, we have addressed several areas of text where we received feedback about a lack of clarity. This includes the Background and methodology section, as well as the Technical annex. Our hope is that the text is now clearer and easier to interpret and understand, enabling further feedback on the methods used to produce these indicators. We will aim to make the code for all stages of the modelling pipeline publicly available.

How will we communicate the impact of changing the methodology?

As Official Statistics in Development, the methodology for the indicators is likely to undergo changes before it gains Official Statistic status. As with all our statistics, we will indicate anticipated developments in the Development plan for this indicator and explain in the relevant sections where the methodology might have changed for the current publication. Following the feedback we received, we are also publishing this response where we highlight the major changes that have been made to this publication this year, along with a Key changes section in the publication itself. Any major changes made to the underlying methodology will also be explored in the Technical annex with a charted comparison to the previous methodology and an explanation of any differences that may arise. This year we have added a ‘Changes since the last publication’ section to the Technical annex, which explores any key differences from the previous year.

Is it our intention to publish all of the underlying data?

Defra do not own the underlying data used to calculate this indicator. We have added links and references to survey data, species indices and peer-reviewed methodology for each scheme, where available (See Table 6, Technical annex). This also addresses concerns raised around transparency of the provenance of the data and the peer-reviewed science that underpins each scheme.

Fundamental issue with indicator lines: certain species can be declining while the indicator line is increasing

Indicators are a summary of distinct species-specific times-series. Essentially, they are an average time-series across a set of species and inherently lose the granular detail (species time-series) in favour of a broader picture of patterns of change. The all-species abundance indicator was developed with the aim of summarising trends in abundance for the broadest possible set of species in England. Species in the indicator are equally weighted and the trend is calculated based on geometric mean, which means that a doubling in abundance of one species would be exactly cancelled out by a halving in another species (regardless of whether they are rare or widespread). While it is possible that large increases in some species could mask declines in others, it is not feasible to entirely avoid this risk whilst ensuring that the indicator is as representative as possible of English biodiversity.

We have published additional figures, alongside the overall trend, that allow more detailed understanding of the underlying changes. For example, we produce a chart showing the number of species that have increased, decreased, or shown little change over the short- and long-term (Figure 2). We also produce a breakdown by taxonomic group (Figure 3; Table 2), which would highlight if different trends were occurring across different groups.

Peer review

There were several suggestions that peer review of the complete methodology used would be desirable and will give more confidence in the approach, including the cross-validation work underpinning the indicator development. The indicator methodology was reviewed by an independent panel of three academic experts and we will look to do further peer review of the whole methodology once further developed. The majority of the current indicator methodology and data analysis do use peer-reviewed methods (see References). Any deviations from these methods (for example, the pre-smoothing) are clearly highlighted in the publication, along with justification for their differences. We also highlight areas for future work where the justification needs further clarification. We intend to publish this year reports relating to the recent development of the indicator, including suggestions and reviews from the expert panel, which should aid with transparency around the decisions taken for this indicator. This page and the main publication will be updated with these links once they become available.

Further suggested developments

In addition to the feedback highlighted here, there were also other suggested developments for the indicator that are too numerous to note here. These suggestions will be kept under review and where possible incorporated into the Development plan.

There were several requests for a species distribution indicator to complement the abundance indicators. As part of our Development plan, we have committed to developing an all-species distribution indicator. This is due to be published in the next update of the England biodiversity indicators, provisionally scheduled for November 2025.

The indicator production team are always keen to hear feedback from users, please do continue to get in touch with suggestions or comments: biodiversity@defra.gov.uk.