Official Statistics

Quality and methodology information: surveillance of bloodstream infections in critical care units, England

Published 17 October 2024

Applies to England

About this report

This report outlines the quality and methodology information (QMI) relevant to the ‘Surveillance of bloodstream infections in critical care units, England: May 2016 to March 2024’ official statistics in development, published by the UK Health Security Agency (UKHSA).

This QMI report helps users understand the strengths and limitations of these statistics, ensuring UKHSA is compliant with the quality standards stated in the Code of Practice for Statistics. The report explains:

• the strengths and limitations of the data used to produce the statistics
• the methods used to produce the statistics
• the quality of the statistical outputs

About the statistics

These statistics show how often bloodstream infections occur in critical care units (CCU), the microbes that cause them, linked factors, and how often people die following an infection.

Geographical coverage: participating critical care units in England.

Publication frequency: yearly.

Contact

You can contact the team that produced these statistics by emailing ICCQIP.surveillance@ukhsa.gov.uk

Suitable data sources

Statistics should be based on the most appropriate data to meet intended uses.

This section describes the data used to produce the statistics.

Data sources

The data presented in this report has been primarily sourced from the Infection in Critical Care Quality Improvement Programme (ICCQIP), a non-mandatory surveillance programme which collects data on bloodstream infections in English CCUs. Staff in participating CCUs enter data from their unit through an online portal, the Data Capture System (DCS), designed by the UKHSA. The submitted data includes both episode data and unit activity data.

An infection episode is defined as a blood culture positive for one or more organisms, regardless of the type of organism (skin commensal or recognised pathogen). The infection episode length is seven days, where day one is the date of the first specimen. In order to minimise potential bias, CCUs submitting data are not asked to determine whether a positive blood culture (PBC) conforms to a specific metric (such as bloodstream infection, BSI). Instead, data is captured that allows the reported case to be categorised according to standard case definitions and corresponding rates of infection are reported. Episode data includes information such as the date and time the specimen was taken, the organisms found in each PBC, the results of any relevant repeat blood cultures, details on central vascular catheter (CVC) use, and clinical symptoms associated with the PBC.

Unit activity data is submitted by units as monthly aggregates, or as daily values. They are used to estimate incidence of various outcomes (see section ‘Calculation of incidence rates’) and other metrics. They comprise:

• the total number of occupied bed-days for each unit per month (described here as bed-days)
• the total number of occupied bed-days for each unit per month, restricted to patients who have spent more than 2 nights in the CCU (where CCU admission is day 1) (described as bed-days over 2 nights)
• of the patients in the unit for over 2 nights , the number with one or more CVC in place (described as CVC bed-days over 2 nights)
• the total number of blood culture sets taken by each unit

Mortality data is acquired from the Personal Demographics Service (PDS) via NHS England’s Demographics Batch Service (DBS).

Further information is available in the surveillance protocol, PDF, 2.279MB.

Data quality

The data that we use to produce statistics must be fit for purpose. Poor quality data can cause errors and can hinder effective decision making.

We have assessed the quality of the source data against the data quality dimensions in the Government Data Quality Framework.

This assessment covers the quality of the data that was used to produce the statistics, not the quality of the final statistical outputs. The quality summary section below explains the quality of the final statistical outputs.

Strengths and limitations of the data

The strengths of the data include:

• the data collection system is specifically designed for surveillance of bloodstream infections
• data includes clinical as well as laboratory information
• all CCUs in England are invited to participate in data submission
• mortality data is sourced from the national master database of all NHS patients in England
• data undergoes checks for uniqueness, consistency and validity

The limitations of the data include:

• the data derives from a non-mandatory surveillance programme, with particular underrepresentation of paediatric and neonatal units
• the participation of units can fluctuate across the years and reduced during the early phase of the COVID-19 pandemic
• data from the most recent calendar quarter might be underreported

Accuracy

Accuracy is about the degree to which the data reflects the real world. This can refer to correct names, addresses or represent factual and up-to-date data.

Episode and unit activity data is specifically collected for this surveillance programme by critical care or microbiology staff. Units have access to automated reports to verify the accuracy of their submissions. Units also have the option to have a separate colleague verify and sign off the data. Once data has been signed off, it cannot be edited or deleted without the unit being granted a data unlock.

It is possible that there has been underreporting of ICU activity during the first year of the COVID-19 pandemic, therefore the incidence rates during this period might be overestimated.

Completeness

Completeness describes the degree to which records are present.

For a data set to be complete, all records are included, and the most important data is present in those records. This means that the data set contains all the records that it should and all essential values in a record are populated.

Completeness is not the same as accuracy as a full data set may still have incorrect values.

All NHS CCUs across England are eligible to participate in the CCU surveillance programme.

Participation in the ICCQIP programme is voluntary and suffers from non-participation and fluctuations in reporting. Overall, 27.1% of CCUs in England submitted data in FY 2023 to 2024 (supplementary Table S1). This relatively low engagement leads to underascertained absolute number of infections and unit activity. However, incidence rates and percentages take this into account and are not expected to be underascertained because of this. Non-participation also has the potential for participation bias, where certain unit factors, including unit size or patient characteristics, may differ between units that report and do not report to the programme. This may in turn influence the ability to apply the findings in this report to all CCUs across England.

Paediatric and neonatal units have much lower participation rates than adult units, which presents issues regarding the generalisability and effect of random error in these unit types. As of April 2024, the total number of CCUs across England as identified through CCU audits in England (ICNARC, PICANet, NNAP) was 420, of which 244 were adult, 23 paediatric and 153 neonatal. The number of CCUs registered with ICCQIP surveillance in March 2024 was 283, of which 237 were adult CCUs (97.1% of all adult units identified in the audit), 20 paediatric (87.0% of all paediatric units), and 26 neonatal (17.0% of all neonatal units). Of the 283 CCUs registered on the ICCQIP CCU Data Capture System (DCS), 189 (66.8%) have submitted at least one quarter of data to ICCQIP surveillance (supplementary Table S1).

The number of units that participate in the programme fluctuates each year. A total of 89 adult CCUs submitting data in the first year of surveillance; this increased to a peak of 109 in FY 2019 to 2020. The number has since reduced to 104 in FY 2023 to 2024. The number of paediatric units submitting data to ICCQIP has declined from 7 in the first year of surveillance to 4 in FY 2023 to 2024. For neonatal units, this number increased from 5 in the first year of surveillance to 6 by FY 2023 to 2024, with a peak of 10 in FY 2021 to 2022.

The COVID-19 pandemic impacted the surveillance of CCU infections, as many units paused reporting. Currently, we do not know whether this reduction in reporting happened independently of characteristics such as unit size, speciality unit type, or demographics of the population served. If certain types of units were more likely than others to cease reporting, then this may confound the presented CCU infection rate estimates during the COVID-19 pandemic.

Uniqueness

Uniqueness describes the degree to which there is no duplication in records. This means that the data contains only one record for each entity it represents, and each value is stored once.

Some fields should be unique. Some data is less likely to be unique, for example geographical data such as town of birth.

The DCS automatically detects duplicated episode records and does not allow users to enter exact duplicates. Further de-duplication is performed during the analysis stage, as described in the methods section below.

Consistency

Consistency describes the degree to which values in a data set do not contradict other values representing the same entity.

Data are consistent if it does not contradict data in another data set.

A research piece is being planned to compare results of our voluntary surveillance system with data obtained by linking other data sources, such as NHS Hospital Episode Statistics on admitted patient care and the Second Generation Surveillance System for laboratory-based infection data.

Timeliness

Timeliness describes the degree to which the data is an accurate reflection of the period that it represents, and that the data and its values are up to date.

Some data, such as date of birth, may stay the same whereas some, such as income, may not.

Data is timely if the time lag between collection and availability is appropriate for the intended use.

Records are included in this report if the initial positive blood culture came from a blood specimen taken no earlier than 1 May 2016 and no later than 31 March 2024. Data were included if reported by participants on or before 27 May 2024.

Data is not always reported by the required deadline. This means that there will be some cases not included because they were not reported in time for the data cut-off. This could lead to an underestimate of counts and denominators for data from the latest quarter (January to March 2024).

Validity

Validity describes the degree to which the data is in the range and format expected. For example, date of birth does not exceed the present day and is within a reasonable range.

Valid data is stored in a data set in the appropriate format for that type of data. For example, a date of birth is stored in a date format rather than in plain text.

The DCS automatically conducts internal validity checks. All data types are in the correct format. Further validity checks are conducted at the analysis stage; for example, to check that the number of total blood cultures reported by a unit does not exceed its number of positive blood cultures.

Sound methods

Statistical outputs should be produced using appropriate methods and recognised standards.

This section describes how the statistics were produced and quality assured.

Dataset production

Participating units submit data as described in the Data sources section above.

Calculation of participation numbers

Participation data, as presented in supplementary Table S1, were calculated based on information from the ICU DCS and external audits conducted by the Intensive Care National Audit and Research Centre (ICNARC), PICANet, and NNAP.

The number of units in external audits was taken from publications external to UKHSA. The number of adult units was taken from the ICNARC Case Mix Programme participation list, published in April 2024. The number of paediatric units was taken from the England section of the PICANet Participating Organisations list. The number of neonatal units was taken from the NNAP report 2022 data. Units listed by ICNARC and NNAP were excluded from supplementary Table S1 if they were from the independent sector (non-NHS) or not in England.

The number of units registered on the ICU DCS is taken directly from the UKHSA DCS website user list. This list was then cross compared with submitted data to produce participation results by year and overall. Units were counted as having ever entered any data if they had entered PBC case data and/or denominator data since the beginning of the surveillance period in May 2016. Units were counted as having participated in any given year if they had submitted (one of the below):

• any complete monthly denominator data, with or without case data
• case data, and had estimable denominator data based on previous submissions

Processing of unit activity (denominator) data

If only monthly denominators have been provided by a unit, then this data is used.

If only daily denominator data have been submitted, the ICU DCS calculates a monthly value as:

Monthly value = (sum of daily values of metric / total days denominator data has been entered) × number of days in the month

If a unit entered both daily data and total monthly data then the entered monthly data is used in rate calculations, unless a unit had informed us in writing that the daily data aggregated to monthly totals should be used instead of the entered monthly data.

Denominator data is considered missing when units submit PBC data but no denominator data for the same time-period. They are also quality-checked against the following validation rules:

• number of occupied patient bed-days in the unit cannot be fewer than the number of occupied patient bed-days in the unit when restricted to only include patients in the unit for more than 2 nights (that is CCU patient bed-days)
• number of occupied patient bed-days in the unit cannot be fewer than the number of occupied patient bed-days in the unit when restricted to only include patients with at least one CVC in place (that is CVC-days)
• number of occupied patient-days in the unit cannot be less than the number of occupied patient-days in the unit when restricted to patients in the unit for more than 2 nights with at least one CVC in place (that is CCU-CVC-days)
• number of occupied patient bed-days in the unit restricted to patients with at least one CVC in place (that is CVC-days) cannot be less than the number of occupied patient bed-days in the unit when restricted to only include patients in the unit for more than 2 nights with at least one CVC in place (that is CCU-CVC-days)
• number of occupied patient bed-days in the unit restricted to patients in the unit for more than 2 nights (that is CCU patient bed-days) cannot be less than the number of occupied patient bed-days in the unit when restricted to only include patients in the unit for more than 2 nights with at least one CVC in place (that is CCU-CVC-days)

Denominator data that breaches validation rules (for example, a greater number of CCU bed-days than patient bed-days) or is missing (even though data on positive blood cultures is submitted for the same period) are verified with the units, who may update the records.

Missing denominator data and denominator data that continues to breach validation rules or erroneous data (notified by the unit) is deleted and is imputed from other data entered by the same unit. Each denominator metric, that is patient bed-days, CCU bed-days, CVC-days, CCU-CVC-days and blood cultures per month, is imputed individually.

Data for a given missing denominator metric is imputed following a 3-step hierarchical process whereby if data is still missing after the current step, the next step is started. Missing data is imputed with:

• data provided by the unit for the same month that is missing but for the previous year
• data provided by the unit for the most recent previous month
• data provided by the unit for the most recent future month (this is possible because units provide data in quarters)

After imputation, data for a given denominator metric in any particular month where there is a more than 100% increase or 50% decrease compared with the previous month is flagged, along with the previous month. The flagged values are checked with the unit, then assessed in the context of all provided data for that metric by the unit and the values dropped if it does not fit in with the rest of the time series. The data is then reimported, rechecked to confirm no unusual values. If the first imputation used historic data as per the hierarchical selection process, this may not reflect capacity or activity change within a unit, then one of the later imputation selection steps may be used to select data for imputation.

Units who have not supplied any denominator data for a particular denominator metric cannot have this metric imputed for them. As such, rates which require this denominator metric cannot be calculated. Furthermore, these units have their numerator data excluded when calculating unit type (adult, paediatric and neonatal) values (counts and rates) to not overestimate this data.

As some units may provide some but not all denominator metrics, there could be different numbers of units that contribute to the unit-type overall totals and rates.

The denominator imputation process carries certain quality risks. If the missing denominators are not missing at random, imputing them can lead to biased denominators and rates. For example, if units were less likely to submit denominator data during periods of higher patient activity, the imputed denominators would be underestimated, and the rates would be overestimated. Imputation of missing denominators continued throughout the COVID-19 pandemic.

De-duplication of infection episode data

Episodes are defined as a 7-day period. If an episode with the same organisms within a single patient is entered within 6 days before or after an existing episode for the same patient, then the second episode is considered a duplicate. Patients are identified using NHS number, date of birth and intensive care unit. Episodes are then created based on patient and all organisms entered within an individual record. If more than one record is found within an episode, the following steps are followed to identify which record to retain:

• if specimen dates for the multiple patient-episode records were the same, the record with the fullest information is retained – if there was the same level of data entered for all records, the record entered earliest (lowest ICU DCS ID number) will have been retained
• for multiple within-patient-episode records where the specimen date is not the same (but is within 6 days before or after the earliest specimen date), the record with the earliest specimen date was retained

Classification of infection metrics

Algorithms determine which PBCs qualify as:

• bloodstream infections (BSI)
• CCU-associated BSI (CCU-BSI)
• CCU-associated central vascular catheter BSI (CCU-CVC-BSI)
• CCU-associated Catheter-related BSI (CCU-CRBSI)
• CCU-associated Catheter-associated BSI (CCU-CABSI)

These algorithms are based on the information provided by participating units. The process is outlined in the surveillance protocol.

In adult and paediatric units, for a skin commensal organism to meet the case definition for BSI, it must be identified in two separate PBCs taken within a 48-hour period, and the patient must have additional signs and symptoms. The data entry fields for repeat blood cultures and clinical symptoms are not mandatory for case submission; this likely leads to an underestimation of BSIs caused by skin commensals in this report.

The full list of bacteria classified as skin commensals is available in the ICCQIP protocol, Appendix 2.

Classification of organisms

Entered organism data is aggregated into the following groups:

• Coagulase-negative staphylococci
• S. aureus
• Streptococcus spp.
  • non-viridans
  • viridans
• Enterococcus spp.
• E. coli
• Klebsiella spp.
• P. aeruginosa
• other gram-negative
  • Acinetobacter spp.
  • other Enterobacteriaceae
  • Serratia spp.
  • other gram-negative bacteria
• Candida spp.
  • C. albicans
  • Candida spp., other
• other
  • other Gram-positive bacteria
  • other bacteria
  • other fungi
  • parasites

Percentages out of total isolates were calculated by each metric (PBC, BSI, CCU-BSI, CCU-CVC-BSI) and unit type (adult, paediatric, neonatal) as follows:

Percentage of organism x in financial year y = (number of organism x isolates in financial year y) / (number of total isolates in financial year y)

Reporting of staphylococci

The DCS uses drop-down menus for users to select which organisms have been identified in PBCs and this list has changed over time. One change that affected data quality concerned coagulase-negative and coagulase-positive Staphylococci (CoNS, CoPS). Until November 2022, one of the options in the organism list in the ICU DCS was “Staphylococcus sp, other”. This was intended to capture CoPS other than S. aureus, as a separate option for CoNS. However, it was discovered that “Staphylococcus sp, other” was predominantly used for CoNS. To remove ambiguity, on 11 November 2022, “Staphylococcus sp, other” was renamed to “Staphylococcus sp., other (coagulase-positive)” on the DCS. As counts of CoPS BSI, CCU-BSI and CCU-CVC-BSI are not comparable before and after this data change, they are currently not reported. This change in the organism reporting led to an underestimation of BSIs. If all the excluded submissions were CoPS, then the rate of BSIs in adults could be underestimated by as much as 8%. However, it is believed that the vast majority of these cases were CoNS, and that the underestimation is likely to be less pronounced.

Analysis of polymicrobial infections

If a patient had multiple organisms isolated from the same blood culture set (or from multiple sets taken on the same day), then this counts as polymicrobial infection.

Calculation of blood culture positivity

The blood culture positivity is calculated as:

(the number of PBCs) / (the number of total blood cultures) × 100

PBCs from a unit for which the number of total blood cultures was not submitted or imputable for a given month were excluded for the purposes of positivity rate calculation, to avoid overestimating this metric.

Calculation of incidence rates

Counts of the various metrics and denominator data were aggregated to totals per financial year for each unit. In addition, values were aggregated by unit type (adult, paediatric and neonatal) and NHS reporting region for each financial year.

Infection counts from a unit for which a denominator was not submitted or imputable for a given month were excluded for the purposes of incidence rate calculations, to avoid overestimating the rates.

Incidence rates were then calculated as:

• rate per 1,000 bed-days = (number of new PBCs or BSIs / bed-days) × 1,000
• rate per 1,000 bed-days over 2 nights = (number of new CCU-BSIs / the total number of occupied bed-days, restricted to patients who have spent more than 2 nights in the CCU) × 1,000
• rate per 1,000 CVC-days over 2 nights = (number of new CCU-CVC-BSIs / the total number of CVC-days, restricted to include patients in the unit for more than 2 nights with at least 1 CVC in place) × 1,000

Calculation of CVC utilisation

CVC utilisation is calculated as the total number of days on which patients who are admitted to the CCU for more than 2 nights have a CVC in situ divided by the total number of days on which patients have been admitted to the CCU for more than 2 nights.

Mortality analysis

Mortality data is linked into the infection data set by patient NHS number and date of birth.

Where multiple records had the same NHS number and date of birth within the 30-day fatality window, only the record with the specimen date closest to the date of death was used in the mortality analyses. This was done to avoid overestimating the numbers of deaths. This deduplication algorithm was applied to both the 30-day fatality, traced and total number of reports to prevent an inflated count of deaths and reports.

The case fatality rate (CFR) is the number of deaths from any cause reported within 30 days of specimen date as a percentage of all reported cases. As this is a measure of all-cause mortality, it includes deaths that may not be directly attributable to the infections.

CFR = (number of deaths from any cause reported within 30 days of specimen date) divided by (number of cases) × 100

Quality assurance

Quality assurance of results relies on two epidemiology scientists conducting all analyses independently using separate code and confirming when results match. All code used for data processing and analysis is version controlled using a local Git repository.

Results are sense checked with previously reported trends and available microbiological knowledge. Tables and graphs are automatically generated from the quality assured data tables by using R and Quarto Markdown. An information officer quality assures all figures in the main text against the data tables.

Senior members of the team and UKHSA Statistics Production Division staff review the statistics and report before publication.

Confidentiality and disclosure control

Personal and confidential data is collected, processed, and used in accordance with the UKHSA Privacy Notice. All UKHSA staff with access to personal or confidential information must complete mandatory information governance training, which must be refreshed every year. Information is stored on computer systems that are kept up-to-date and regularly tested to make sure they are secure and protected from viruses and hacking. UKHSA staff do not store data on their own laptops or computers. Instead, data is stored centrally on UKHSA servers.

Data is only published at aggregated level, so it is anonymous. The smallest population at risk is all patients admitted to adult critical care units in a given NHS England region, which is not small enough to carry a great risk of disclosure. Bloodstream infections are not associated with specific stigma.

Quality summary

Quality means that statistics fit their intended uses, are based on appropriate data and methods, and are not materially misleading.

Quality requires skilled professional judgement about collecting, preparing, analysing, and publishing statistics and data in ways that meet the needs of people who want to use the statistics.

This section assesses the statistics against the European Statistical System dimensions of quality.

Relevance

Relevance is the degree to which the statistics meet user needs in both coverage and content.

This report is relevant for clinicians and public health personnel involved in patient safety in critical care in England. It covers key aspects of bloodstream infections in this setting, including incidence rates, organism distributions, central vascular catheter (CVC) use, regional comparisons, and mortality outcomes. It was developed as part of ICCQIP, which is led by representatives of key stakeholder organisations.

Accuracy and reliability

Accuracy is the proximity between an estimate and the unknown true value. Reliability is the closeness of early estimates to subsequent estimated values.

The total number of infections is underestimated because not all units participate in this surveillance programme; however, the use of unit activity denominators allows us to estimate more accurate rates.

We plan to produce this report on an annual basis. If further data is entered or existing data is amended for the previously reported period, we will include updated estimates.

Timeliness and punctuality

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

The report was published seven months after the end of the reference period. It was punctual.

These are official statistics in development and are pre-announced at least 28 days in advance. Provisional publication dates for the year ahead are pre-announced online in December and can be found on the UKHSA release calendar.

Accessibility and clarity

Accessibility is the ease with which users can access the data, also reflecting the format in which the data is available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

The report and tables have been produced in accessible HTML and ODS format and are freely available from GOV.UK. All figures have been produced within accessibility guidelines.

Coherence and comparability

Coherence is the degree to which data that is derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain.

Definitions and methods are applied consistently throughout the reference period. Data can be compared from financial year (FY) 2017 to 2018 and FY 2023 to 2024. Of note, data collection started in May 2016; this means that data for FY 2016 to 2017 only covers 11 months, and caution must be used when comparing with later years.

Uses and users

Users of statistics and data should be at the centre of statistical production, and statistics should meet user needs.

This section explains how the statistics are used, and how we understand user needs.

Appropriate use of the statistics     

These statistics can be used:

• to inform stakeholders on the national annual trends of rate of bloodstream infections, associated mortality, and CVC use in critical care units in England
• to compare CCU infection trends between NHS reporting regions in England
• to compare with other countries that use similar case definitions

These statistics cannot be used:

• to estimate the total number of bloodstream infections in English critical care units
• to provide demographic insights into patients with bloodstream infections other than the age-specific unit type they were registered with

Users

It is expected that this first report will be used by:

• clinical and infection prevention staff in critical care units in England
• NHS England regions
• ICCQIP board

User engagement

The team will collect user feedback from this first annual report and use it to improve future iterations. The team also holds training sessions for DCS users, covering data collection, data submission, and interpretation of results. Changes to the surveillance system are communicated to users by email and protocol updates. The team responds to user and stakeholder queries via a dedicated email inbox (ICCQIP.surveillance@ukhsa.gov.uk).

Related statistics

Related statistics are listed in the main report.