Guidance

Statistics Statement of Principles and Code of Practice on changes to data systems

Updated 12 February 2020

Part 1: Statement of Principles and Procedures

About the Statement

1.1 The Digital Economy Act 2017 (‘the Act’) amends the Statistics and Registration Service Act 2007^{[footnote 1]} (‘the 2007 Act’) to provide the UK Statistics Authority and its executive office, the Office for National Statistics (ONS; hereafter collectively ‘the Authority’, and references to ‘we’ and ‘our’ within this document are references to the Authority)^{[footnote 2]}, with greater and easier access to a range of data sources held within the public and private sectors. These provisions bring the legal framework governing the production and publication of official statistics and statistical research in the UK into line with that of the UK’s international partners, supporting ongoing improvements in the quality, relevance and timeliness of official statistics in a changing world.

1.2 To provide clarity and transparency about how powers enabling this access will operate the 2007 Act also sets out the requirement, in section 45E of the 2007 Act,^{[footnote 3]} that the Authority prepares, consults on and publishes a Statement of Principles and Procedures (hereafter ‘the Statement’) governing the way data will be accessed under the new powers. The Act further requires the Authority to consult before issuing or reissuing the Statement, and to lay the Statement before the UK Parliament and the devolved legislatures.

1.3 This document represents the Statement required under section 45E of the 2007 Act. In preparing this Statement we, the Authority, have had regard to, inter alia, the:

• Information Commissioner’s Anonymisation: Managing Data Protection Risk Code of Practice (2012)
• Information Commissioner’s Data Sharing Code of Practice (2011)^{[footnote 4]}
• Information Commissioner’s Conducting Privacy Impact Assessments Code of Practice (2014)
• Information Commissioner’s Privacy Notices, Transparency and Control Code of Practice (2016)
• Statistics and Registration Service Act 2007
• Code of Practice for Official Statistics
• The data protection legislation^{[footnote 5]}
• ONS Respondent Charter for Business Surveys
• Government Security Policy Framework
• The Ethical Principles of the National Statistician’s Data Ethics Advisory Committee
• The Concordat on Statistics between the UK Government (including the UK Statistics Authority and Office for National Statistics) and the Devolved Administrations

1.4 This Statement complements and is consistent with the principles and expectations set out in those documents. The principles and procedures outlined in this document apply to all the ways we access and share data, including onward disclosure to the statistical offices of the devolved administrations.

Understanding the power

2.1 Statistics are a vital public good for the information age – the quality and range of official statistics provide key decision-makers in Government, business and beyond with crucial insights into the UK’s society and economy. Official statistics also play a key role in supporting a healthy democracy by enabling individuals to hold their elected representatives to account. Producing high quality official statistics requires sophisticated, robust methodology and an appropriately skilled statistical workforce. But it also requires a legal framework that empowers statistical producers to collect and process the data on which official statistics and statistical research are based. Such a framework needs to recognise:

• that the way data are produced will continue to change in the future;
• that data useful for the production and publication of official statistics and for statistical research is held by an increasing number of public and private bodies in ever larger quantities, and that the proliferation of useful data sources will continue in the future;
• the need for statistical producers to be able to understand the data they are using so they can take advantage of the growing potential uses of these data;
• that the proliferation of data and data holders means greater variation in the quality of data, and that statistical producers therefore need tools that help them to understand the quality of that data and to determine the feasibility of their use in the production and publication of official statistics and statistical research; and,
• that ensuring the increased availability of data to researchers does not impact negatively on citizens’ privacy means obliging those who collect and handle data to implement robust privacy-enhancing measures.

2.2 The 2007 Act as amended, creates a legal framework providing the Authority with access to data held by Crown bodies, other public authorities and undertakings (including charities) to support the Authority’s statistical functions. As amended, the 2007 Act requires these data suppliers to consult the Authority before changes to data systems are made in order to protect the continuity of data supply, as well as the accuracy and reliability of statistics and statistical research derived from these data sources. To support the production and publication of devolved official statistics and statistical research, the 2007 Act permits controlled disclosures of data to the statistical offices of the devolved administrations in Scotland, Wales and Northern Ireland, with the consent of the data supplier. In carrying out its functions under the 2007 Act, the Authority observes the very highest standards of data security, confidentiality and transparency. The Act also sets out strict penalties for those who misuse data collected for statistical purposes and requires the Authority to obtain consent from data providers to use information received under these powers for its functions in relation to statistical services (i.e. under section 22 of the 2007 Act) or before disclosing information to an Approved Researcher under section 39 of the 2007 Act.

Principles and Procedures

3.1 Section 45E(5) of the 2007 Act requires the Authority to prepare and publish a Statement of Principles to which it will have regard in exercising its functions under sections 45B, 45C, and 45D of the Act, and the procedures it will adopt in exercising those functions. We, the Authority, have established principles and procedures, published in this Statement to ensure that we:

• exercise our statutory responsibilities in a fair, proportionate and accountable way, with due regard for principles of privacy and appropriate degrees of internal and external scrutiny;
• work in a collaborative, transparent and fair manner with data suppliers, civil society and the general public, responding to any concerns or opportunities as they arise; and,
• reinforce our full accountability to the UK Parliament and the devolved legislatures in exercising our statutory responsibilities.

3.2 We will only seek access to data for the purposes of fulfilling one or more of our statutory functions, as set out in the Statistics and Registration Service Act 2007, including to produce official statistics and undertake statistical research that meets identifiable user needs for the public good.^{[footnote 6]} In doing so, we will ensure compliance with the data protection legislation. In exercising our statutory functions we will adhere to the following six principles, collectively intended to ensure that the highest ethical and legal standards apply across the full statistical life-cycle, and to provide public assurance and maintain confidence in the trustworthiness, and quality and value of our statistics and statistical research. These are outlined below.

Principle 1: Confidentiality

4.1 We are committed to protecting the confidentiality of data and the importance of adhering to data protection legislation. In order to ensure compliance with those legal obligations we will maintain appropriate security controls to ensure personal information remains protected and secure at all times. We will regularly assess our security infrastructure and procedures to maintain the integrity of, and confidence in, these safeguards.

4.2 This is consistent with the Code of Practice for Official Statistics, which requires that private information about individuals and organisations compiled in the production and publication of official statistics and statistical research be treated confidentially and used for statistical and statistical research purposes only.

Principle 2: Transparency

5.1 As part of our statutory reporting obligations we are committed to publishing information on how we exercise our statutory responsibilities and obligations in respect of access to sources of data for statistical and statistical research purposes. Building on the information we currently provide through our Statement of Administrative Sources, we undertake to publish regular reports setting out details of the data we are receiving under these powers, the duration of data supply, and the uses to which the data is put. These reports will include links to published statistical and research outputs.

5.2 We want to maintain the high levels of trust and confidence among the general public in the integrity and value of our statistics and how we exercise our statutory functions, to ensure that users of our statistics understand the public good of our work and the decisions they can make using our statistics and analysis. To this end we will publish regular reports and maintain user engagement to communicate readily accessible information about the sources of data we are accessing, and intend to access, for our statistical and statistical research functions.^{[footnote 7]} As part of our reporting function, and to help data suppliers and data subjects understand the ways in which their data are used and the privacy and security safeguards around the use of these data, we commit to make details of data access requests and responses publicly available by default.

5.3 In some exceptional cases this degree of transparency may be inappropriate.^{[footnote 8]} Where we are aware that reasons not to disclose this information may exist we undertake to obtain and take into account the data supplier’s advice before deciding whether to (or the extent to which we will) publish details of data access requests and responses. This commitment applies equally to any details we publish in documents we are required or choose to lay before Parliament or the devolved legislatures.

Principle 3: Ethics and the law

6.1 Data access arrangements will meet all legal obligations arising from the data protection legislation and other legislation, as appropriate. We will further ensure that data access arrangements observe the relevant ethical standards, for example the Ethical Principles of the National Statistician’s Data Ethics Advisory Committee, making sure at all times that these arrangements support us in the exercise of our statutory functions, including the delivery of official statistics and statistical research that serve a clear public interest. We will ensure that data access arrangements adhere to recognised standards of methodological integrity and quality, address issues of privacy and transparency, suitably consider the risks and limitations of new technologies and data collection methods, and are subject to appropriate scrutiny, oversight and monitoring. Quality for statistical and statistical research purposes will also be monitored and assured, including in accordance with internationally agreed practices.

6.2 As part of this process, we will have regard to best practice on privacy impact assessments and privacy notices in the establishment of data access arrangements, as covered by the Information Commissioner’s Office’s (ICO) Conducting Privacy Impact Assessments Code of Practice and Privacy Notices Code of Practice (which provides guidance on the contents of these notices, as well as where and when to make them publicly available). The data protection legislation requires ‘data protection impact assessments’ to be conducted prior to the processing when the processing is likely to result in a high risk to the rights and freedoms of individuals. The data protection legislation also requires privacy notices to contain more detailed and specific information than under the Data Protection Act 1998.

6.3 This is consistent with the Code of Practice for Official Statistics, which requires statistical methods to be fully documented and consistent with sound methods and best practices.

Principle 4: Public interest

7.1 We will exercise our statutory functions and responsibilities in ways that are free of the influence of organisational, political or personal interests, ensuring that we take decisions on the data sources we seek to access only on the basis of a sound statistical rationale and a clear public interest. The Authority’s statutory functions are set out in the 2007 Act, including to promote and safeguard the production and publication of official statistics that serve the public good. The public good includes informing the public about social and economic matters and assisting in the development and evaluation of public policy. In order to achieve those public interests in the most effective and efficient manner, we may exercise these new powers to access data. We will only seek access to data where we are reasonably satisfied that the data is of sufficient quality and coverage to support us in the exercise of our statutory functions, including the production and publication of high-quality statistical and research outputs. Before securing access to data held exclusively by public authorities or undertakings (including charities) under the statutory powers conferred upon us, we will also ensure that we have assessed known viable alternatives, particularly where publicly-available equivalent sources would serve as suitable substitutes.

7.2 This is consistent with the Code of Practice for Official Statistics, which requires that statistics serve the public good, and that statistical producers ensure the public interest prevails over organisational, political or personal interests at all stages in the production, management, and dissemination of official statistics and statistical research.

Principle 5: Proportionality

8.1 We are committed to minimising the burdens associated with providing us with access to data to support us in exercising our statutory functions (for example, when producing and publishing official statistics and statistical research). We will seek at all times to ensure that the costs of providing us with access to data are proportionate to the benefits accruing from the use of the statistics produced from these data. This commitment will also inform all decisions concerning the volume or type of data we seek, or the frequency with which data are requested. We will work with data suppliers to establish data access arrangements that minimise the cost burden and potential for disproportionate impacts on taxpayers and data suppliers alike. We undertake to seek data from national or consolidated sources before placing burdens on local service providers and, where the same data can be obtained from multiple sources, to ensure that our decisions on which source(s) to access are informed by considerations of associated costs and burdens to the data suppliers.

8.2 This is consistent with the Code of Practice for Official Statistics, which requires statistical producers to assess the burden on data suppliers relative to the benefits arising from the use of the statistics, and to ensure that these burdens are not excessive.

Principle 6: Collaboration

9.1 We will consult with, and consider the views of, data suppliers before issuing a notice or requesting access to data. We commit to exploring collaborative solutions and negotiated data arrangements in preference to issuing requests or notices to enable this access. This will allow us to tailor data access arrangements to the specific needs, resources, interests and cultures of data suppliers, as well as the particular sensitivities and risks associated with different types and sources of data. A collaborative approach will also enable us to understand the way the data are constructed and therefore any caveats concerning their quality, interpretation and use. We will invoke our statutory powers of compliance and compulsion as a last resort only once all other reasonable means of accessing the data have been exhausted (including senior level discussions between the Authority and the data supplier).

Governance

10.1 We will monitor the ways in which we are using the new powers provided in the 2007 Act (as amended) to access data by establishing an advisory data access oversight function as part of the governance structure of the Authority, and drawing on the experience and independent expertise of individuals from a wide range of sectors and organisations. This function will work in a transparent fashion to assess data access proposals against the principles above, including taking into account the position of data suppliers, and will provide impartial and independent advice to the National Statistician to support him/her on the exercise of his/her functions. We will publish reports and other papers detailing this advice in accordance with the Authority’s general commitment to transparency.

10.2 We will maintain protocols concerning how the Authority will work effectively with the statistical offices of the devolved administrations under the new legislation. This will be overseen by the most appropriate senior-level inter-administrative forum which will manage the interaction between the Authority and the statistical offices of the Devolved Administrations, and inform the Authority and the Devolved Administrations of respective data needs and requirements.

10.3 We will publish, consult on and maintain up-to-date supplementary guidance and best practice documents concerning the way we exercise our new powers under the 2007 Act. These documents, which we may re-issue from time to time to reflect changing practices and the development of professional expertise, will provide additional clarity to help data suppliers understand their obligations. They will provide information for statistical users and the general public seeking to understand the procedural arrangements governing the way third-party data is collected, processed and safeguarded to support us in the exercise of our statutory functions, including:

• Data requests and notices: guidance concerning the format in which we will request data or issue notices under this legislation;
• Minimising burden: the means by which we will assess the burden on data suppliers and the ways we will minimise these burdens.
• Data supply and transmission: technical guidance about the format(s) for the transmission of data and safeguards to ensure data is transmitted securely;
• Security and confidentiality: summary information about the ways in which we ensure the highest levels of data security and the confidential storage, processing, use and dissemination of data held in our systems; and
• Representation and dispute resolution: arrangements where data suppliers can query or challenge requests for data made by us, including representations to our data access oversight function.

Part 2: Code of Practice on Changes to Data Systems

About the Code

11.1 Under section 45G of the Statistics and Registration Service Act 2007 (‘the 2007 Act’), the UK Statistics Authority and its executive office, the Office for National Statistics (collectively referred to hereafter as the ‘Authority’) must prepare, consult on and publish a code of practice on changes to data systems. This document represents the Code required under section 45G.

11.2. This code contains guidance on the matters to be taken into account by public authorities in making changes to their processes for collecting, organising, storing or retrieving information, or their processes for supplying information to the Authority. This code is relevant where public authorities supply data to the Authority to enable it to carry out one or more of its functions, including the production and publication of official statistics and statistical research.

11.3. The 2007 Act requires public authorities to have regard to this code when considering or making changes to their data systems. The code is therefore principally intended to provide guidance for public authorities that are supplying data to the Authority, but will have practical relevance for all organisations who supply data for statistics or statistical research (or those who have been advised they may be required to provide data in the future), regardless of the status of the organisation, or the frequency and means by which data is supplied. In such cases organisations should be encouraged to see this code as an example of good practice, and to consider its recommendations accordingly.

Understanding the power

12.1 The Digital Economy Act 2017 (‘the Act’) makes a number of changes to the law governing the way data is used in the production and publication of official statistics in the UK. The Act amends the 2007 Act by, amongst other things, granting the Authority access to data held by public authorities and undertakings and expanding the range of data sources the Authority will be able to draw on in the production and publication of official statistics and statistical research. This may include a range of types of data, from individual records or variables within a larger dataset to entire datasets and metadata.

12.2 The precise nature of, means by which and the frequency with which data should be made available to the Authority will be set out as part of a data access agreement. In some cases the 2007 Act also permits the Authority to set out obligations as part of a formal agreement to provide access to data requiring data suppliers to consult the Authority before making changes to the data they collect, the way they collect or process these data, and / or arrangements for providing the Authority with access to these data and for monitoring compliance. In such cases, the access agreement will set out details of this obligation including the changes that are sufficient to trigger the obligation and the period of notice the data supplier must provide when making such a change.

12.3 Where the Authority has shared data with any of the devolved administrations for statistical purposes, under the power provided in section 53A of the 2007 Act, the Authority will share the relevant information about changes to data systems with statisticians in the relevant devolved administration(s).

12.4 The purpose of requiring data suppliers to consult the Authority about changes to data systems is to support the continuity of data supply, or to help manage discontinuities of supply, thereby maintaining the integrity, accuracy and reliability of statistics and statistical research derived from these data. This is an essential safeguard if the UK statistical system is to remain robust whilst reducing its reliance on traditional survey-based sources in favour of directly accessing other sources of data that will support better and more relevant analysis.

12.5 Where such an obligation is not set out in the data access agreement, the Authority nonetheless recommends that data suppliers consider the impact that changes they are considering making to data systems may have on any aspect of the data they are supplying (or may supply) to the Authority. The guidance below will help data suppliers identify some of the possible impacts these changes may have, and to help the Authority to minimise the potential such changes may have to disrupt the production and publication or undermine the integrity of official statistics and statistical research.

Importance of consultation

13.1 There are a number of reasons why a data supplier may seek to change the way it collects and processes data – including to meet strategic, financial or information security challenges, or as a consequence of evolving operational or transactional needs or of organisational transformations and changes in staffing structures. Any guidance provided here or statutory obligation set out in a notice issued by the Authority^{[footnote 9]} does not change the right of a data supplier to make such changes. It is instead intended to help make sure that the Authority is made aware of these changes to the extent to which they may impact on the nature of the data, or of the provision of that data, that the Authority receives and relies upon to enable the Authority to carry out one or more of its functions, including the production and publication of official statistics and statistical research.

13.2 There are a number of changes that may impact on the supply of data, including changes to:

• the type of data collected, or the way the data are collected: changes to the nature of the data collected which are or may be passed to the Authority could have impacts on statistics that rely on these data. Where a data supplying organisation decides to stop collecting data it provides for the production and publication of official statistics and statistical research, for instance, the Authority will need to be consulted as early as possible to investigate and secure other sources to avoid disruptions to important statistical outputs that rely on these data. Similarly, changes to the way data are collected may have possible implications for the quality, reliability or usability of the data and therefore the integrity of the statistical outputs that make use of these data;
• the way data are organised, stored and retrieved: in most cases such changes will have no impact on the provision of data for the production and publication of official statistics or statistical research. There are some instances, however, where changes to data infrastructures may impact on some aspect of this provision. Changes to the way data are stored and retrieved, for example, could change the frequency with which data can be provided, potentially interrupting the supply of data, while changes to archival procedures may have an impact on the ability to easily access historical data;
• the way data are supplied: changes to the means by which data are supplied to the Authority may have implications for the security and reliability of these data. The Authority publishes technical guidance concerning the formats it requires for transmitted data and will work with data suppliers to establish data access and / or transmission arrangements that minimise the costs and burdens data suppliers might accrue.

13.3 Being aware of such changes will enable the Authority to adjust its practices and to engage with data suppliers to ensure it continues to access the data held by data suppliers in as safe, reliable, efficient and least burdensome way as possible.

13.4 Not every change to the way data is collected and processed will affect data that is being provided to the Authority for one or more of its statutory functions. However, in order to identify whether this is the case or not data suppliers will need to be aware of all data that is being supplied or shared, of the duration of the agreement, and the way in which that data is being shared or transmitted. As discussed above, the Authority will ensure that all these details are set out within the framework of a data access arrangement, and that any changes to the obligations of a data supplier or arrangements for accessing data are recorded in an amendment. Data suppliers should ensure that staff responsible for considering and implementing changes to data systems are appropriately familiarised with these agreements.

13.5 Similarly, not every change will have a direct impact on the quality and continuity of data upon that is relied upon for the production and publication of official statistics and statistical research. During initial data access negotiations the Authority will engage with data suppliers to ensure they understand the role supplied data will have in the production and publication of statistics, and to therefore help the data supplier to anticipate the impact of changes it is considering to the way it collects or processes this or related data.

13.6 Where a data supplier believes proposed changes may have an impact on any aspect of the statistics that make use of the data, or on the agreed arrangements for supply, it should advise the Authority in a timely fashion. There will be some occasions when changes to data systems happen much more quickly, such as where a security vulnerability has been discovered or a business opportunity has arisen. In all such cases data suppliers should ensure they contact the Authority as soon as possible once they become aware of the need to effect a change. Where a data supplier is uncertain about the impact on statistics or statistical research of changes to their data systems the Authority recommends the data supplier contact the Authority for discussion. The Authority will work with the data supplier to understand the nature of the changes and any impacts, to identify steps to maintain continuity of supply, and to ensure that supply is maintained in a way that keeps burdens to a minimum.

13.7 In the vast majority of cases, the data supplier will itself initiate changes to the way data is collected or processed. There are, however, occasions, where a data supplier may become aware of a change that was unintended but might impact on the Authority’s capacity to use these data in the production and publication of official statistics or statistical research, or may do so in future. Such changes include a change in the nature of the data being supplied, in the way it is processed, or to the infrastructures within which it is processed. Evidence of data breaches, security vulnerabilities or the misuse of data, for instance, will have implications for the security procedures related to the transmission, storage and use of data that have been set out in the data access arrangements. Similarly, evidence of errors, inaccuracies or omissions in data, or fallibilities in the way the data is collected, will have implications for the quality of the data and therefore the methodological commentary accompanying statistical outputs. Large inaccuracies or methodological flaws might even call into question the extent to which the data can be relied upon for the production and publication of official statistics and statistical research. Where such issues cause the data supplier to change the ways it collects or processes data, the data supplier should advise the Authority about these issues at the earliest opportunity as part of its consultation about the changes being made.

1. References to the Statistics and Registration Service Act 2007 are to the Act as amended by the Digital Economy Act 2017 and in force. ↩

2. The legal powers and duties established in the Statistics and Registration Service Act 2007 are vested in the Board of the UK Statistics Authority (‘the Statistics Board’). The Board directs and oversees the work of the UK Statistics Authority and its executive office, the Office for National Statistics, the UK’s largest statistical producer and the internationally recognised national statistical institute of the UK. Within this document ‘the Authority’ is used to reflect this arrangement. ↩

3. As inserted by section 80 of the Act. ↩

4. As altered or replaced from time to time. ↩

5. “The data protection legislation” means the full, applicable data protection framework as set out in the Data Protection Act 2018. This encompasses general processing (including the General Data Protection Regulation and the applied GDPR), law enforcement processing, and intelligence services processing. References to “the Data Protection Act 1998” in the Digital Economy Act 2017 are amended to “the data protection legislation” by the Data Protection Act 2018. ↩

6. See also UK Statistics Authority Better Statistics, Better Decisions: Strategy for UK Statistics 2015-2020 at https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-betterstatisticsbetterdecisionsstrategyfor2015to202_tcm97-44175-5.pdf ↩

7. Further, information sharing practices should be fair and transparent – indeed, this is necessary to comply with Article 5(2) of the GDPR sets out the ‘accountability’ principle, making data controllers responsible for demonstrating that the ‘lawfulness, fairness and transparency’ principle, together with the other principles in Article 5(1), have been complied with. ↩

8. For example, where the publication of such details may be prejudicial to wider public or commercial interests, such as national security, the prevention or detection of crime or where publishing the information would significantly damage the market position of a data supplier. ↩

9. This means a notice issued by the Authority to a data supplier in accordance with section 45C(2) and (6) or section 45D(1) and (5) of the Statistics and Registration Service Act 2007. ↩