Policy paper

G8 Open Data Charter and Technical Annex

Published 18 June 2013

This was published under the 2010 to 2015 Conservative and Liberal Democrat coalition government

Preamble

1) The world is witnessing the growth of a global movement facilitated by technology and social media and fuelled by information – one that contains enormous potential to create more accountable, efficient, responsive, and effective governments and businesses, and to spur economic growth.

Open data sit at the heart of this global movement.

2) Access to data allows individuals and organisations to develop new insights and innovations that can improve the lives of others and help to improve the flow of information within and between countries. While governments and businesses collect a wide range of data, they do not always share these data in ways that are easily discoverable, useable, or understandable by the public.

This is a missed opportunity.

3) Today, people expect to be able to access information and services electronically when and how they want. Increasingly, this is true of government data as well. We have arrived at a tipping point, heralding a new era in which people can use open data to generate insights, ideas, and services to create a better world for all.

4) Open data can increase transparency about what government and business are doing. Open data also increase awareness about how countries’ natural resources are used, how extractives revenues are spent, and how land is transacted and managed. All of which promotes accountability and good governance, enhances public debate, and helps to combat corruption. Transparent data on G8 development assistance are also essential for accountability.

5). Providing access to government data can empower individuals, the media, civil society, and business to fuel better outcomes in public services such as health, education, public safety, environmental protection, and governance. Open data can do this by:

• showing how and where public money is spent, providing strong incentives for that money to be used most effectively;
• enabling people to make better informed choices about the services they receive and the standards they should expect.

6) Freely-available government data can be used in innovative ways to create useful tools and products that help people navigate modern life more easily. Used in this way, open data are a catalyst for innovation in the private sector, supporting the creation of new markets, businesses, and jobs. Beyond government, these benefits can multiply as more businesses adopt open data practices modelled by government and share their own data with the public.

7) We, the G8, agree that open data are an untapped resource with huge potential to encourage the building of stronger, more interconnected societies that better meet the needs of our citizens and allow innovation and prosperity to flourish.

8) We therefore agree to follow a set of principles that will be the foundation for access to, and the release and re-use of, data made available by G8 governments. They are:

• Open Data by Default
• Quality and Quantity
• Useable by All
• Releasing Data for Improved Governance
• Releasing Data for Innovation

9) While working within our national political and legal frameworks, we will implement these principles in accordance with the technical best practises and timeframes set out in our national action plans. G8 members will, by the end of this year, develop action plans, with a view to implementation of the Charter and technical annex by the end of 2015 at the latest. We will review progress at our next meeting in 2014.

10) We also recognise the benefits of open data can and should be enjoyed by citizens of all nations. In the spirit of openness we offer this Open Data Charter for consideration by other countries, multinational organisations and initiatives.

1. Principle 1: Open Data by Default

11) We recognise that free access to, and subsequent re-use of, open data are of significant value to society and the economy.

12) We agree to orient our governments towards open data by default.

13) We recognise that the term government data is meant in the widest sense possible. This could apply to data owned by national, federal, local, or international government bodies, or by the wider public sector.

14) We recognise that there is national and international legislation, in particular pertaining to intellectual property, personally-identifiable and sensitive information, which must be observed.

15) We will:

• establish an expectation that all government data be published openly by default, as outlined in this Charter, while recognising that there are legitimate reasons why some data cannot be released.

2. Principle 2: Quality and Quantity

16) We recognise that governments and the public sector hold vast amounts of information that may be of interest to citizens.

17) We also recognise that it may take time to prepare high-quality data, and the importance of consulting with each other and with national, and wider, open data users to identify which data to prioritise for release or improvement.

18) We will:

• release high-quality open data that are timely, comprehensive, and accurate. To the extent possible, data will be in their original, unmodified form and at the finest level of granularity available;
• ensure that information in the data is written in plain, clear language, so that it can be understood by all, though this Charter does not require translation into other languages;
• make sure that data are fully described, so that consumers have sufficient information to understand their strengths, weaknesses, analytical limitations, and security requirements, as well as how to process the data; and
• release data as early as possible, allow users to provide feedback, and then continue to make revisions to ensure the highest standards of open data quality are met.

3. Principle 3: Usable by All

19) We agree to release data in a way that helps all people to obtain and re-use it.

20) We recognise that open data should be available free of charge in order to encourage their most widespread use.

21) We agree that when open data are released, it should be done without bureaucratic or administrative barriers, such as registration requirements, which can deter people from accessing the data.

22) We will:

• release data in open formats wherever possible, ensuring that the data are available to the widest range of users for the widest range of purposes; and
• release as much data as possible, and where it is not possible to offer free access at present, promote the benefits and encourage the allowance of free access to data. In many cases this will include providing data in multiple formats, so that they can be processed by computers and understood by people.

4. Principle 4: Releasing Data for Improved Governance

23) We recognise that the release of open data strengthens our democratic institutions and encourages better policy-making to meets the needs of our citizens. This is true not only in our own countries but across the world.

24) We also recognise that interest in open data is growing in other multilateral organisations and initiatives.

25) We will:

• share technical expertise and experience with each other and with other countries across the world so that everyone can reap the benefits of open data; and
• be transparent about our own data collection, standards, and publishing processes, by documenting all of these related processes online.

5. Principle 5: Releasing Data for Innovation

26) Recognising the importance of diversity in stimulating creativity and innovation, we agree that the more people and organisations that use our data, the greater the social and economic benefits that will be generated. This is true for both commercial and non-commercial uses.

27) We will:

• work to increase open data literacy and encourage people, such as developers of applications and civil society organisations that work in the field of open data promotion, to unlock the value of open data;
• empower a future generation of data innovators by providing data in machine-readable formats.

6. Technical annex

1) We, the G8, have consulted with technical experts to identify some best practices (part one) and collective actions (part two) that we will use to meet the principles set out in the G8 Open Data Charter.

2) While working within our national political and legal frameworks, we agree to implement these practices as quickly as possible and aim to complete our activities by 2015 at the latest. This will be done in accordance with the timeframes in our national action plans.

3) The Annex constitutes a ‘living’ set of guidelines that may be subject to amendments after consideration of emerging technology solutions or practical experience gained during the course of implementation of the G8 Open Data Charter.

6.1 Part One - Best Practices

Principle 1: Open Data by Default

4) We recognise the importance of open data and we will establish an expectation that all government data be published openly by default.

5) We will:

• define our open data position in a public statement of intent, such as an announcement, strategy or policy, so that our plans for progressing the open data agenda in our jurisdictions are clear;
• publish a national action plan to provide more specific details on our plans to release data according to the principles in the G8 Open Data Charter; and
• publish data on a national portal so that all government data that has been released can be found easily in one place. A portal may be a central website from which data can be downloaded, or a website which lists all open government data stored at a different location. Each portal will include a registry file that lists all the data and metadata used on the portal, as well as providing APIs for developers. Where it is yet not possible to publish all data on a portal, the location of data will be communicated clearly and not moved without notice.

Principle 2: Quality and Quantity

6) We commit to releasing data that are both high in quality as well as high in quantity. When releasing data, we aim to do so in a way that helps people to use and understand them. This will help to increase the interoperability of data from different policy areas, businesses or countries.

7) We will:

• use robust and consistent metadata (i.e. the fields or elements that describe the actual data);
• publish and maintain an up-to-date mapping of the core descriptive metadata fields across G8 members to enable easier use and comprehension by people from around the world. This will allow countries, in the G8 and beyond, who do not currently have a data portal to consider adopting the metadata fields included in this mapping;
• ensure data are fully described, as appropriate, to help users to fully understand the data. This may include:
  • Documentation that provides explanations about the data fields used;
  • Data dictionaries to link different data; and
  • A user’s guide that describes the purpose of the collection, the target audience, the characteristics of the sample, and the method of data collection.
• listen to feedback from data users to improve the breadth, quality and accessibility of data we offer. This could be in the form of a public consultation on the national data strategy or policy, discussions with civil society, creation of a feedback mechanism on the data portal, or through other appropriate mechanisms.

Principle 3: Usable by All

8) We agree to release data in a way that helps all people find and re-use them.

9) We will:

• make data available in convenient open formats to ensure files can be easily retrieved, downloaded, indexed, and searched by all commonly used Web search applications. Open formats, for example non-proprietary CSV files, are ones where the specification for the format is available to anyone for free, thereby allowing the data contained in a file to be opened by different software programmes.

Principle 4: Releasing Data for Improved Governance

10) We recognise that data are a powerful tool to help drive government effectiveness, efficiency and responsiveness to citizen needs while fuelling further demand for open data.

11) We will:

• develop links with civil society organisations and individuals to allow the public to provide feedback on the most important data they would like released;
• be open about our own data standards, so that we take into account:
  • Data that are released by other national and international organisations
  • The standards emerging from other international transparency initiatives; and
• document our own experiences of working with open data by, for example, publishing technical information about our open data policies, practices, and portals so that the benefits of open data can be enjoyed in other countries.

Principle 5: Releasing Data for Innovation

12) We agree that our citizens can use our data to fuel innovation in our own countries and around the world. We recognise that free access to, and reuse of, open government data are an essential part of this.

13) We will:

• support the release of data using open licences or other relevant instruments - while respecting intellectual property rights - so that no restrictions or charges are placed on the re-use of the information for non-commercial or commercial purposes, save for exceptional circumstances;
• ensure data are machine readable in bulk by providing data that are well structured to allow automated processing and access with the minimum number of file downloads;
• release data using application programming interfaces (APIs), where appropriate, to ensure easy access to the most regularly updated and accessed data; and
• encourage innovative uses of our data through the organisation of challenges, prizes or mentoring for data users in our individual jurisdictions.

6.2 Part Two - Collective Actions

Action 1: G8 National Action Plans

• We will publish individual action plans detailing how we will implement the Open Data Charter according to our national frameworks (October 2013)
• We will report progress on an annual basis (via the G8 Accountability Working Group) (2014 and 2015)

Action 2: Release of high value data

• We recognise the following as areas of high value, both for improving our democracies and encouraging innovative re-use of data.

Data Category* (alphabetical order)	Example datasets
Companies	Company/business register
Crime and Justice	Crime statistics, safety
Earth observation	Meteorological/weather, agriculture, forestry, fishing, and hunting
Education	List of schools; performance of schools, digital skills
Energy and Environment	Pollution levels, energy consumption
Finance and contracts	Transaction spend, contracts let, call for tender, future tenders, local budget, national budget (planned and spent)
Geospatial	Topography, postcodes, national maps, local maps
Global Development	Aid, food security, extractives, land
Government Accountability and Democracy	Government contact points, election results, legislation and statutes, salaries (pay scales), hospitality/gifts
Health	Prescription data, performance data
Science and Research	Genome data, research and educational activity, experiment results
Statistics	National Statistics, Census, infrastructure, wealth, skills
Social mobility and welfare	Housing, health insurance and unemployment benefits
Transport and Infrastructure	Public transport timetables, access points broadband penetration

• In accordance with the principles of “open by default” and “quality and quantity” we will work towards the progressive publication of these data.
• As a first step, we will collectively make key datasets on National Statistics, National Maps, National Elections and National Budgets available and discoverable (from June 2013), and we will work towards improving their granularity and accessibility (by December 2013)
• We recognise that collective action by all G8 members has the potential to unlock barriers and foster innovative solutions to some of the challenges we are facing. We therefore agree on a mutual effort to increase the supply of open government data available on key functions of our States, such as democracy and environment. We will work on identifying datasets in these areas by December 2013, with an aim to release them by December 2014.
• We will set out in our national action plans how and when we will release data under the remaining categories according to our national frameworks (October 2013).

Action 3: Metadata mapping

• We have contributed to and commit to maintaining the G8 metadata mapping exercise (June 2013)
• This mapping can be viewed on Github and comprises a collective mapping ‘index’ across G8 member’s metadata, and a detailed page on each G8 member use of metadata within their national portal.

*categories and datasets to be finalised by December 2015