Guidance

Essential Shared Data Assets (ESDAs) Policy (HTML)

Updated 26 April 2024

1. Executive summary

The UK government wants to use data to its full potential to enable better data-driven decision-making. This will ultimately help government to:

• improve public service delivery
• understand the effects of policies and programmes
• ensure the effective use of resources

The Central Digital and Data Office (CDDO) roadmap 2022 to 2025 sets out 4 deliverables to support “better data to power decision making”, of which 2 specifically relate to critical data assets.

However, the government does not have a consistent way of defining and identifying critical data assets from a cross-government perspective.

The resource defines the critical data assets from a cross-government perspective as Essential Shared Data Assets (ESDAs). It has been subject to an Alpha trial and is now being rolled out to Beta phase, after which it will be reviewed. It sets out the processes and purposes where organisations need data for:

• public service delivery
• legislative, legal or contractual requirements
• Defence, national security and resilience
• policy development and evaluation
• official statistics

The resource provides a consistent and standardised approach to defining and identifying critical data assets from a cross-government perspective to avoid confusion with organisation-level critical data assets. More detailed guidance is also available to support implementation of this policy.

This definition of ESDAs is essential for delivering government’s commitments defined in the CDDO’s roadmap.

2. Realising our data ambitions

Strategic context

Across government, we aim to unlock the potential in our data and enable better data-driven decision-making. This will help the government improve public service delivery, understand the impact of policies and programmes, and ensure the effective use of resources.

The National Data Strategy set out the government’s ambition for data across the digital economy and the steps required to deliver the transformation. The National Data Strategy identified the priority changes needed to realise the vision for data in government across the technical, technological and cultural landscape, which included:

• improved data quality
• setting and promoting the adoption of standards for data
• taking a ‘data-sharing by default’ approach where legally and ethically appropriate to do so across government to deal with the culture of risk aversion around data use and sharing
• opening government up to greater scrutiny and increasing accountability, ensuring that this improves productivity, policy and services for people while also providing data security
• efficient data-sharing between organisations

Building on the Strategy’s commitments to government’s use of data, the Central Digital and Data Office (CDDO) within the Cabinet Office, published the Transforming for a Digital Future 2022 to 2025 roadmap for digital and data in 2022. The roadmap set out 4 deliverables to support ‘better data to power decision making’ of which 2 specifically relate to critical data assets.

All departments will:

• work to make critical data assets available and in use across government through trusted APIs and platforms such as GDX and IDS
• access to a Data Marketplace, including a Data Catalogue, standards and governance models
• co-develop and adopt a single data ownership model for critical data assets
• ensure that 50% of high-priority data quality issues are resolved within the period defined by a cross-government framework

The role of ‘critical data assets’ in achieving these aims

To achieve the aim of ‘better data for decision making’, the data that powers decisions needs to be:

• trusted
• findable
• accessible
• interoperable
• reusable
• fit for its intended purpose.

Government collects, processes and stores a large amount of data to fulfil its many functions. In this document the term ‘data asset‘ refers, in a generalised way, to ‘discrete values that convey information’. A data asset is a container that holds one or more datasets. Datasets are individual, structured files containing data that are organised within data assets of related materials. For the purposes of this document ‘data asset’ does not include related services, applications or database systems. The importance of the data assets will vary from government to organisation level. However, given the limited resources available to departments, it is crucial to ensure that action to improve the management, access and use of data is prioritised to those data assets most important to the strategic priorities of the organisation and government.
In data management practices, key data assets at an organisation level are called ‘critical data assets’. The specific aim of this initiative is to define and identify those data assets that are critical from a cross-government perspective. We will adopt the term ‘essential shared data asset’ (ESDA) to avoid confusion with organisation-level critical data assets. 

The identification of ESDAs is the first step towards delivering Mission 3 of the roadmap:

• understanding the data necessary for data-driven decision making
• ensuring the data that enables decision-making are available through assured sharing mechanisms and platforms where legal, improving access, efficiency, security and trust
• data identified as an ESDA provides an exemplar for agreeing on a set of fundamental data responsibilities and accountabilities across government via the data ownership model
• the ownership model applied to ESDAs supports consistency and clarity around how data is managed and straightforward ways to resolve quality issues with data
• ESDAs provide the initial implementation targets for the metadata standards related to creating a data marketplace by 2025
• metadata about ESDAs will be included within the data marketplace
• ESDAs are the starting point for addressing high-priority data quality issues

3. Defining ESDA’s for this purpose

An ESDA is:

• data provided by a government department or agency to another government department or agency to support an essential process or purpose which is the responsibility of the receiving organisation; or
• data acquired by a government department or agency from another government department or agency to support an essential process or purpose, which is the responsibility of the acquiring organisation.

Categories and criteria for essential processes or purposes

There is already a need for departments to identify critical assets they hold for the requirements of their organisation, considering factors such as:
- whether there are material risks to the organisation in its ability to fulfil its main functions if that data is no longer available - whether the asset includes personal data - the sensitivity of the data - the data process levels (from raw data for specialist usage up to sophisticated data services for public consumption) - whether the data is an authoritative source essential to the operations of the organisation - whether the data is necessary to improve the meaning and usefulness of other data within the organisation

A subset of the data assets critical at an organisation level will be determined critical at a cross-government level. Alignment of definitions will ensure consistency and reduce burdens. ESDAs will be identified against the following 5 categories and criteria, which may overlap:

Category	1. Delivery of Public Service
Definition	The shared data is provided by a government department or one of its agencies to another government department or agency for the delivery of a public service for which the receiving organisation is responsible; OR The shared data is acquired by a government department or one of its agencies to another government department or agency for the delivery of a public service for which the receiving organisation is responsible.
Criteria	- The service is a direct responsibility of a central government department and/or one of its agencies; and - The service is a Top 75 service or other key service related to the delivery or administration of: - education, training and skills; - taxation; - working age, disability or health welfare benefits; - state pension; - justice; - environment; - business and industry; - transport, or - ownership of land and property; - The service is public facing and the beneficiary or subject of the service is an individual, household or business - Both the data provider and acquirer/receiver are a government department or one of its agencies - The service forms part of big spend decisions or is a strategic priority for Government

Category	2. Legislative, legal or contractual requirement
Definition	The shared data is required or provided to fulfil the condition/s of specified legislation; The shared data is required or provided to fulfil the condition/s of specified contract;
Criteria	The shared data is provided by a government department or one its agencies to another government department or agency to support the receiving organisation meet a regulatory or other legally or contractually defined obligation; OR Data is acquired from another government department or agency to support the receiving organisation to meet its legally or contractually defined obligations.

Category	3. Defence, national security, and resilience
Definition	The shared data is required by a government department or public body for the delivery of a security, defence or intelligence purpose or process
Criteria	The shared data is provided by a government department or one its agencies to another government department or agency for defence, national security or resilience purposes; OR Data is acquired from another government department or one its agencies for defence, national security or resilience purposes which is the responsibility of the acquirer who is a government department or one its agencies

Category	4. Policy development and evaluation
Definition	The data is required for policy formulation or evaluation and/or research for government’s strategic priorities and manifesto commitments (e.g. easing the cost of living, economic growth)
Criteria	Data required to support the development or evaluation of government policy commitments at various levels of maturity from exploratory stages to the delivery of green or white papers that the government intends to publish

Category	5. Official statistics
Definition	The shared data is required for the production of national statistics, the shared data is essential to the delivery of a specified official statistical product
Criteria	The shared data is required to deliver products on the current list of National Statistics; the data is provided by a central government department or one of its agencies

4. Direct outcomes of applying the designation

The Central Digital and Data Office Roadmap 2022 to 2025 requires that all departments:

• work to ensure that ESDAs are available and in use across government through trusted APIs and platforms such as GDX and IDS
• apply a single data ownership model to essential shared data assets

All departments must ensure that access and exchange mechanisms for ESDAs are modern, assured and trusted. Legacy systems and other constraints mean that this may require time to achieve. Where departments or agencies cannot ensure these mechanisms, they should have a clearly defined plan with deadlines for meeting this requirement.

Government departments and agencies have agreed on a single data ownership model. Departments will apply this model to ESDAs for which they are the provider. Where departments have existing and established ownership models, they should map the responsibilities and accountabilities outlined in the single model and resolve any gaps identified between their own and the data ownership model to be applied to ESDAs.

The designation does not:

• force an organisation to share data beyond current provisions
• remove any legal, data protection or regulatory obligations and safeguards related to data sharing and access
• require the data provider to give any guarantees about ongoing availability and supply or quality of the specified data outside of what was previously agreed

Departments must provide a return of their ESDAs and coordinate and assure returns for their arm’s length bodies. Returns must meet a set of core metadata requirements defined by CDDO as per guidance which will be available on GOV.UK. Metadata about data assets will be discoverable in the Government Data Catalogue accessible within government. More detailed guidance for departments setting out the process and how to provide returns is published alongside this paper.   

5. Data in scope

Only data assets classified at Official level are in scope of these proposals. Sensitive data assets at SECRET and TOP SECRET are out of scope given the sensitivities and risks associated with such data assets.

The designation applies to the specific data shared to meet the essential purpose. The provider organisation may apply this as relevant to the data context in their data management systems, applications or databases. For example, it may be faster to apply the responsibilities and requirements from the designation to a dataset rather than a single element.

Where the data being shared is the result of processing by the provider or includes data from another government organisation or third party, the designation only applies to the shared data. It does not apply to data beyond the scope of the provider’s data ownership accountabilities and responsibilities.

‘6. Organisational and administrative scope

The definition and associated responsibilities of determining ESDAs apply to UK government departments (not including councils) and arm’s length bodies. These include:

• executive agencies
• non-departmental public bodies
• non-ministerial departments

Central government departments are responsible for coordinating returns from their arm’s length bodies. Departments should work in partnership with their arm’s length bodies to monitor the completion of and assure returns of ESDAs to CDDO. Departments and their arm’s length bodies are aligned in departmental plans, delivery of services and shared objectives and as such it is important that they are coordinated and consistent in the identification of ESDAs.    

Due to devolution, The Northern Ireland Executive, Scottish Government and Welsh Government have their own approaches to data-sharing governance. However, the local government is not required to implement these principles and actions. There are many benefits to aligning data-sharing governance across the UK, and administrations of the UK must continue to share good practice and learn from each other particularly as the delivery of key public services is dependent on the underlying process of sharing data.

Data is required for essential purposes between the devolved administrations and the UK government. Data provided by central government departments and their agencies to government organisations outside of scope but where other criteria are met should have the designation applied. For example, where data is provided by a central government department to a devolved administration that aligns with a category set out above and meets the criteria of that category is an ESDA.

7. Further sharing/access for ESDAs

Government is getting better value from its data and improving data for policy development and evaluation and decision-making through:

• trusted research environments
• secure and assured government platforms
• data trusts

Many of these initiatives are in the early stages of development and require data from across the government to achieve their aims. In addition, departments and agencies are encouraged to support the growth and value they provide by providing access to their ESDAs as appropriate.

The managers of such government-trusted research environments, platforms and trusts should support the provision of access to ESDAs they have identified and prioritised through the supply and/or resourcing of assured access mechanisms.

8. Legal and copyright notices

© Crown copyright, 2023

Licensed under the Open Government Licence v3.0 except where otherwise stated.

Provenance:

Essential Shared Data Assets

Licence:

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

The licence allows:

Sharing, copying and redistributing the content.

Adapting - remixing, transforming, and building upon the material.

NB. For the purposes of use in government, please contact CDDO before making changes to the guidance.

Under the following terms:

• Attribution — You must give appropriate credit, provide a link to the licence, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
• NonCommercial — You may not use the material with the primary purpose of commercial advantage or monetary compensation.
• ShareAlike —  If you remix, transform, or build upon the material, you must distribute your contributions under the same licence as the original.
• No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the licence permits.