2. Discovery: understanding user needs

This section helps you to understand users and their needs and begin to diagnose the policy problems and challenges that you need to fix.

An introduction to discovery

Definition

The discovery stage should be used to till research and knowledge gaps with insight and evidence from user research and data science. Following a user’s journeys through a policy or service, experiencing their lives and using data to understand how user needs and challenges at a deeper level will help provide insight. This insight should give you a full picture of what a user needs from a policy or service that you can develop into solutions and policies.

What you should achieve

1. User needs

You should finish discovery with a full picture of how people experience a policy and the service, and the needs that any solutions you think up need to answer.

2. An understanding of the policy context

Understanding the world in which people experience a policy is fundamentally important to the future success of a policy project. There are many push and pull factors that might affect how people experience a policy or service that might need to also be included in a policy challenge.

3. Final project challenge

You should use any data insights or user research to finalise the aim and challenge of your project. This will be key in the next stage of the project (development).

.

Tools and Techniques

Data science

Contents

  1. Introduction
  2. When to use data science
  3. How to start using data science
  4. Tips for data science
  5. Ethics of data science

Data science an introduction

Data science uses advanced software, computer power and artificial intelligence to analyse and visualise big and complex data to provide useful insight that can improve an understanding of a problem and design better policy.

Data science analyses any form of data, from large quantitative (numerical) data sets, to unstructured qualitative (descriptive) data that doesn’t fit in a standard database. This can take the form of free form text, interviews, consultations, images or phone calls, to name a few. It can even analyse real time data that is constantly changing and point out trends, views, themes, sentiments, characteristics or any kind of finding you can think of.

Examples from around government

  1. The London Fire Brigade created a prototype data visualisation to help them see where they needed to improve reaction times. This used data available from various fire brigades and visualised it using a heat map and google maps to easily show where the service needed to focus it’s money.

  2. The Foreign Office allowed policy makers to visualise their international connections on twitter accounts. This used visualisation tools to showcase where any communications they wanted to spread could be best targeted and enabled quicker sharing of information around the globe.

  3. The Government Digital Service have been using feedback to a service’s webpage can be used to spot problems with a service before it becomes a serious problem.

When to use data science

The wide variety of techniques within data science can be tailored to answer a policy question. If policy makers are aware of the types of things data science can do, it can provide new evidence, ideas and insights at different stages of policy development.

  • discovery: data science can help shape and frame your understanding of what the actual problem is, who it affects, and how
  • design: your data science analysis can provide evidence to plan your policy design
  • consultation: data science can help analyse responses and get a more rounded view of public ideas by using social media data
  • delivery: automated tools and processes can help at an operational level to understand how a service is working and how it can be adapted and improved; to increase efficiencies and to reduce human error or bias
  • evaluation: data science lets you take advantage of new data sources to evaluate your policy, like digital and social media data, and can help you present findings in a more accessible and transparent way

How to start using data science

Data science is a specialist skill and requires policy makers to work together with data analysts or scientists. A policy maker needs to understand what is possible so they can commission data science and identify new and alternative data sources for the data scientist to use. Using techniques like data science tool cards can help to inspire policy makers about what data to use and why.

If you are considering a data science project or would like to explore the policy questions you have and the data you hold, contact the analysts in your department. Policy lab can also help organise data science as part of a policy project with them.

You can contact policy lab at policylab@cabinetoffice.gov.uk for more advice on a project that might include data science.

Tips for data science

There are a number of issues non-data scientists can consider and discuss with analysts to understand the potential of data for their policy area.

Read the glossary of common terms in data science.

Try and make the best use of existing data

Do you have data of particular value that you would like to understand better? Can you explore your data assets to see if there is untapped potential?

Complex free text data (e.g. from free text fields in consultations or on social media) can now be analysed in bulk for findings and trends. Some older data therefore can now become more useful.

To take complaints data as an example, the data science approach would look at more than just summary statistics of how many complaints have been received or dealt with. It would analyse the text, examining trends to understand demand fluctuations on a service, or looking at the language people use through sentiment analysis to see how they are interacting with a service.

This data may have previously been available but unused but data science allows you to explore that data and make use of it.

Combine multiple data sources

Your data might be more valuable when combined with data from other sources. The Department of Energy and Climate Change (DECC) has a target of ensuring vulnerable customers get access to schemes such as the Green Deal to install energy efficiency measures (e.g. cavity wall, loft insulation). DECC already held data on physical property characteristics like energy consumption, but when this was combined with Department for Work and Pensions welfare data this gave a richer insight into households that need energy efficiency measures.

Present data better

Datasets sometimes come in spreadsheet form with thousands of rows of complex figures. For a policy this is often incomprehensible. Visualisation data lets you to see the data and findings, share them with interested parties and ministers and make better policy. The Foreign and Commonwealth Office built a prototype of visa demand to show a story with the data.

Ethics of data science

The Cabinet Office has created Data Science ethics to help policy makers and data specialists to work together. The ethics is an ethical framework that brings together the relevant parts of the law and ethical considerations into a simple document that helps Government officials decide what it can do and what it should do.

  1. Start with a clear user need and public benefit: this will help you justify the level of data sensitivity and method you use.
  2. Use the minimum level of data necessary to fulfill the public benefit: there are many techniques for doing so, such as de-identification, aggregation or querying against data.
  3. Build robust data science models: the model is only as good as the data it contains and while machines are less biased than humans they can get it wrong. It’s critical to be clear about the confidence of the model and think through unintended consequences and biases contained within the data.
  4. Be alert to public perceptions: put simply, what would a normal person on the street think about the project?
  5. Be as open and accountable as possible: Transparency is the antiseptic for unethical behavior. Aim to be as open as possible (with explanations in plain English), although in certain public protection cases the ability to be transparent will be constrained.
  6. Keep data safe and secure: this is not restricted to data science projects but we know that the public are most concerned about losing control of their data.

Read the full ethical framework

Data visualisation

Contents

  1. introduction
  2. When to use
  3. Examples of data visualisation
  4. How to visualise data

Data visualisation: an introduction

Data visualisation can be used with data science, open data and data analysis. It is used to take complex and often difficult to interpret data and visualise it in an appropriate manner. This can include maps, graphs, diagrams, journeys maps, personas or performance pages.

It is a powerful tool that can not only improve your understanding but the engagement of other parties and the trust in a policy or institution.

When to use data visualisation

Data visualisation can be used at any point in policy development. It can be used to help you understand a problem better and communicate this to both the public and ministers or senior civil servants. It can also be used to measure success and prove that a policy is working after a policy has been delivered.

Examples of data visualisation

How to visualise data

There are many methods and tools that you can use to visualise data. A good place to start at beginner level is Open Data Tools or RAW. These tools are relatively easy to master and can be experimented with to create compelling data presentations.

There is an easy beginners tutorial on data.gov.uk that explains how to prepare and show open data in Google Maps. The Guardian also has an introduction to data visualisation and a more in depth course.

To further develop your skills, you can looks into courses in data visualisation, data science and coding, there are many available online, and many of these are free to do. Codecademy and Coursera are some examples.

Deliberative dialogue

Contents

  1. Introduction
  2. When to use a dialogue
  3. Examples from around government
  4. How to start a dialogue

Deliberative dialogue: an introduction

Deliberative dialogue uses an an impartial and facilitated environment to bring members of the public, experts and policymakers together to discuss policy issues and discover possible solutions.

A deliberative dialogue is not a public meeting nor discussion. It is a carefully structured conversation that helps people to not only talk together, but think together. Facilitators look for common ground and discover possible solutions by working with users in a safe and open space.

It uses a variety of creative techniques to help people develop and explain their views on an issue.

When to use a deliberative dialogue

You should start a dialogue in the early stages of policy discovery, so that the views of users and the public can be used throughout the design and delivery of policy.

Each dialogue process is different and can use any variety different deliberative techniques, objectives, people, and resources available.

A deliberative dialogue is a complex process that needs time and thought.

You should use experts to create and hold a deliberative dialogue. Using a facilitator is recommended so that a dialogue is independent and impartial. This will help the public share their true feelings and views.

Deliberative dialogue involves long term research and analysis so it can take a long time to do.

Examples from government

Deliberative dialogue has been used across government to understand the public’s views on the ethical, social and regulatory issues surrounding mitochondrial replacement and synthetic biology.

It has also been used to explore how the UK should reduce greenhouse gas emissions and what wellbeing data tells us about policy making on issues like loneliness, the labour market and being involved in your community.

How to start a dialogue

As a policy maker you should be involved in the dialogue but not lead it.

Contact Sciencewise to organise a public dialogue for your policy area.

Ethnography

Ethnographic research usually involves observing target users in their natural, real-world setting, rather than in the artificial environment of a lab or focus group. It aims to gather insight into how people live; what they do; how they use things; or what they need in their everyday or professional lives.

Read more about ethnography and how to do it in the government service design manual

Interviews

Contents

  1. Introduction
  2. Examples from around government
  3. When to use
  4. Styles of interviewing
  5. How to organise and run interviews

Interviews: an introduction

Interviews are used to understand how a person’s lifestyle and deeper emotions impact their experience and needs of policy. Interviews should be used to generate insight into people’s lives, from their points of view and in their own language, rather than the government’s.

Interviews are used when running ethnographic research and user research.

Interviews can be used on anyone with an experience of a policy area. This can include front line staff, volunteers and public sector workers as well as policy users and members of the public.

Examples from around government

The Department of Health and Department of Work and Pensions used interviews to find out how to help people manage their health conditions in work. They conducted ethnographically informed interviews with people with health conditions, GPs, employers and Job Centre Plus staff to understand how they experienced the process of transitioning (or supporting people transitioning) in and out of work.

Policy Lab used the interviews and ideas to focus the policy problem on the needs of those interviewed.

When to use interviews

Interviews are a useful way of understanding your users, what they need and how they might react to your policy ideas. They can also give you first hand access to the people who have knowledge of the policy area.

Use interviews to inform Personas. Personas are fictional characters that help you understand the varied needs of many users.

You should use them when your understanding of a problem and the needs of people is still unclear. They can help you frame the policy problem and understand what you need to fix.

If you are looking for statistically robust evidence rather than emotive qualitative evidence, interviews will not work.

If you have a budget for using specialist ethnographic researchers, do not use interviews as you will be duplicating work.

If you run interviews yourself they can be cheap, but you may gain more insight by using professionals.

Styles of interviewing

Framework interviews

A framework interview uses a question sheet to guide the interview. It’s structured and can help you focus on one particular area of research. Framework interviews are very basic and don’t always allow interviewees to express more than is asked of them. This will give you less insight, but can allow you to use digital tools to conduct these interviews quicker.

Semi-structured interviews

A semi-structured interview is still structured with prepared questions, but is more open to where the conversation flows. These interviews offer much more depth and enable interviewees to share their personal views more easily.

Observation

Observation interviewing involves spending time accompanying the person in their day to day life to get a more deeper understanding of their life. Ideally you need at least several hours or a day for this.

Digital interviewing

Use digital interviewing techniques like online forms or video chatting to ask people questions. You can also ask people to contribute by sending in photos and videos that highlight their views and reactions.

How to organise and run interviews

Before you start interviewing people you should be aware of 3 important things:

  • Your own bias: it is inevitable that this will shape your interpretations but there are methods to minimise this.
  • Ethics: you need to get people’s consent to being interviewed, agree about how you will refer to them or anonymise them, and minimise any harm to them from interviewing them.
  • The work and expertise involved in interpreting the data: the real value of interviewing is to identify patterns and themes that emerge as you immerse yourself in the person’s world. You should capture and discover what’s underneath the people’s stories through the data you gather

Before you begin: decide on your approach to ethics

  1. get or create an informed consent form and clarify if the person is happy for their name and other details to be used when the research is shared. You should give them the opportunity to remain anonymous or change their name

Doing the interview: gather data

  1. introduce yourself and explain the research – build up trust with the person you are interviewing and confirm how long you expect to take

  2. depending on your research questions and the kind of interview you are doing, you might take up to an hour or 90 minutes

  3. right after the interview, make some notes about your impressions and the main themes in the discussion you had, also noting down anything that surprised you

Afterwards: analyse and interpret the data

  1. assemble the materials you gathered from all your interviews – notes, photos, audio clips, drawings – and think back to the location and situation of the interview, reflecting on anything that seemed distinct to that part of that world

  2. write down words or phrases that capture the sense of what you are hearing or seeing – this is called coding the data and might take several hours

  3. write down quotations from what the person said that exemplify a key point

  4. summarise the themes or patterns emerging across the codes into phrases or short sentences

  5. present your findings to a broader group of people to triangulate them

  6. ensure that the notes you’ll share anonymise the person you interviewed, if that’s what you agreed with them

Other resources

Social media engagement

Contents

  1. Introduction
  2. Examples
  3. Ways of using social media
  4. How to use social media

Social media engagement: an introduction

Social media can help you to understand people’s views and needs, and work with them to design policy.

Working with people through social media can help you to connect with their opinions and existing experiences. You can use it to crowdsource ideas and quickly test how ideas will be received and used in the real world.

Social media can be very useful for a quick link to the people your policy will impact. Running ideas, questionnaires or consultations through social media can significantly increase the number of people you reach.

Remember that by choosing to work with people on social media you are choosing to work with a small selection of the audience.

Social media analysis is helpful for analysing your engagement or finding existing data from social media platform.

Examples from around government

  • the NHS Constitution consultation used social media to spread the word about changes to the NHS constitution
  • the Public Sector Efficiency Challenge used social media and crowdsourcing to share ideas about where money could be saved across the civil service
  • UKTI’s Export Jam used social media to ask people what their experience of exporting was and for their ideas to improve it
  • the Government Digital Services and HM Revenue & Customs uses Twitter to help people when they have a problem with a service or policy

How to use social media

The Government Digital Service Social Media Playbook is the government guide for using social media. It outlines the available platforms and the ways that government should speak and use them.

Choose the most appropriate channel

Combine a range of different channels depending on who you need to reach.

You may wish to use a traditional open social media site like Facebook or Twitter, or something more closed like Hackpad or LinkedIn groups. You should find one that works for you and your users.

Remember this is a dialogue not a monologue

You should not simply broadcast what government wants people to hear. Any social media campaign should create a dialogue where government responds to individual people.

Be prepared for negative comments

Social media is a democratic forum and people may want to express their displeasure with the government or a particular policy.

Influencers

Find the influential people who are already talking about your area on Twitter. Follow them and talk to them to see if they will help you increase debate around your crowdsourcing project.

Provide relevant information

Tweet useful information to get the conversation flowing. Infographics, pictures and links to relevant articles will make people more likely to read your tweets and more likely to engage with your question because they have some background data to work with. Ask them to share their information as well.

Closed/open

Most social media is open about the ideas and conversations taking place.

For some specialised or sensitive discussions you might want to use a more closed environment. You can use a members-only groups on LinkedIn, Slack and Hackpad to let people to discuss things which might be commercially sensitive or that they might not want to share openly.

On very sensitive topics such as sexual assault, you may need to let people to comment anonymously, ask them if you can use their contribution anonymously and remove any identifying features from any information they allow you to publish.

User research

This guidance provides a broad overview of the methods and techniques available to conduct user research. Research can incorporate both qualitative and quantitative techniques.

Qualitative techniques are intensive and often small scale. These include focus groups and one-to-one interviews, and are typically used to explore and analyse unstructured data.

Quantitative techniques involve higher-volume research, and include online surveys, face-to-face interviews, and involve a structured approach to data collection and analysis.

Read more about user research on the Digital Service Manual

Guerrilla Testing

Guerrilla user testing is a low cost method of user testing. The term ‘guerrilla’ refers to its ‘out in the wild’ style, in the fact that it can be conducted anywhere eg cafe, library, train station etc, essentially anywhere where there is significant footfall.

Read the full advice and information for guerilla testing from the Government Digital Service

Idea days and policy jams

Contents

  1. Introduction
  2. Examples
  3. Principles of an idea or jam
  4. How to run an idea jam

Idea jams: an introduction

An idea or policy jam gives policy users and experts an opportunity to work with policy makers and designers to understand a policy problem and create solutions together. The ideas created do not always need to be realistic or sensible. Instead the ideas can highlight the needs of users and help policymakers understand what kind of solution they need to create.

Ideas days and policy jams are a way to help policymakers understand how people experience their policies in the real world and work together to create better policy.

The jam itself is a day, or more, in a location outside of government where interested parties, policy makers and users talk through their problems and create solutions together. They often involve interacting in new ways eg, sketching, hope and fear card exercises, journey mapping and acting.

Policy jams and idea days are a examples of ‘design thinking’ that builds, service, policies and government around user’s needs and experiences. Read an introduction to design thinking.

Examples from across government

UK Trade and Investment (UKTI) Ideas Lab held nine Export Jams around the country in 2015 to help them understand what was holding back small to medium sized companies in the UK investing overseas. They worked with users to understand their current experiences and brainstorm future ideas.

Principles of an idea jam

1. Understand what you want to achieve

It’s very easy to hold an ideas day (all you need is a good location, pens and paper), but it’s more difficult to gather useful information and data from it. You will need to work out what you want to achieve before you begin - and make sure the aim is sensible and achievable. Most idea jams either choose to look for solutions with users and citizens, or to work out what their current experience is and to identify the problem.

2. Listen, don’t tell

The point of an idea day is to listen to the problems and ideas people have at them and work with them in an open and productive manner. Rather than shutting ideas down and telling people why things won’t work, look outside the box and think of unrealistic ideas that can then be adapted. You should facilitate and guide discussions and tasks so that they are fixing the problem the idea jam was set up to deal with.

3. Empower people to think differently

An ideas day is a space for both users policy makers to think outside the box and engage in meaningful debate and ideas. What would happen if there was no budget? what would be the perfect answer? What would a major company do?

How to run an idea jam

1. Before the jam

You must plan and structure an idea jam to make sure you gather the best data and ideas from the event. Bringing in expert help and facilitators can help you do to this. The Cabinet Office’s Policy Lab have run several idea jams in the past and will be able to advise or run one for you.

2. Running the jam

You should use a facilitator when running a ideas day. Facilitators let the discussion flow more freely for those at the day, but can also control it to make sure it runs in the direction policy makers need. Their neutrality can enable a more open and level playing field of discussion.

Email policy lab at policylab@cabinetoffice.gov.uk if you want some help and advice.

Open Data

Contents

  1. Introduction
  2. Examples
  3. Why you should use open data
  4. How to use open data
  5. Ethical implications

Open data: an introduction

More data is being generated than ever before, from apps and devices people use everyday, to the massive amounts of administrative data that government departments keep on their services and customers. Open data can benefit policy in numerous ways:

  • It brings in external thinking through hack days to open eyes of politicians and officials and design better solutions. This has been used at places like MoveMaker and SkillsRoute
  • It can help policy makers work with voices of citizens and create choice for users. Ofsted’s school dashboard has done this with schools to help parents choose

Not all data is fully open. Like Open Policy Making there are scales of openness. The ODI spectrum helps you to see how open you can be.

Read more information about Open Data from Open Knowledge Foundation’s School of Data and Open Data Institute

Examples of open data

  • Citymapper is perhaps one of the best known examples of open data. Citymapper takes data from transit authorities and the public sector around the globe to make cities easier to use and ‘reinvent the transport app for the world’s most complicated cities’

  • There are also prototypes available on GOV.UK that highlight how open data can be used to report on the success of a policy. These prototypes are not working yet but they do show how policy can work with digital and open data to improve understanding of the policy. View the prototype on housing and free schoold

Why you should use data to understand users and needs

Data science techniques, such as clustering and predictive analytics, can help to identify an issue quickly, and allow interventions to take place earlier to avoid a major problem occurring.

For instance, by analysing comments and feedback data from GOV.UK, it’s possible to see where the highest frequency of complaints are occurring, what users these complaints are concerning, and how to mitigate against the issues reported in them.

One example of this is passports. Of complaints are increasing in frequency, and often contain words relating to the online service failing, it is possible to flag this at an early stage, and undertake maintenance to ensure a complete service failure does not occur.

How to use open data

1. Get the data

Open data is often good to use to further augment existing data to improve your knowledge and cross reference information from across the globe. Open data is free and available from UK Data Archive, Data.gov.uk and the Office for National Statistics.

Often the best approach is just to explore, there are many places that hold openly available data that can be used for analytical purposes, policymakers should be encouraged to approach issues with the freedom to seek data and not be overly prescriptive about traditional methods of understanding policy issues.

2. Use data to identify policy issues

Data visualisation techniques can make your data more compelling and provide the ability to see where policies are failing to reach their intended targets.

An example would be the visualisation of the LSYPE (Longitudinal Study of Young People in England) survey data to show transitions between different aged cohorts. By viewing data in this way it’s possible to spot trends, such as the sizable amount of people moving from family care into unemployment between the ages of 20 - 21. This highlights an area to look at further and and begin to create policy interventions around.

Data visualisation, analysis and open data are often specialist tools and working with experts is often necessary.

Ethical implications and privacy concerns in using data

There are a huge amount of ethical and privacy considerations when using data. Data that is heavily aggregated will be safe to use, but will not always make for compelling or useful analysis. Personal-level data is great for bespoke tailoring of services and interventions, but creates potential issues around privacy and protecting the anonymity of citizens.

It is important for policymakers to think about what they are trying to achieve and how it could impact on the privacy of citizens, and ensure that they are working within the knowledge and information management guidelines set out by their departments, and the terms of the Data Protection Act.

Social media and data analysis

Contents

  1. Introduction
  2. Why you should use social media analysis
  3. When to use social media analysis
  4. Tools you can use

Social media analysis; an introduction

Social media and data analysis is the process of taking impenetrable data (like consultations or data sets), or social media (like twitter) and transforming it into statistical evidence that can help you to understand users and their needs.

Social media and data can be analysed to look for themes, trends and statistical evidence that can help you to understand users and better inform a policy. The tools available range from free to thousands of pounds and often need specialist people brought into the understand and interpret the data.

These tools can work across social media, crowdsourcing, consultations and sometimes large unorganised data sets.

This section talks about off the shelf tools tools. For more powerful data science tools and techniques read our guide to data science.

Why you should use data and social media analysis

Social media analysis can bring insight to large scale data sets that would have been impossible to see without the use of a computer.

Analysis tools analyse the data in a variety of ways. Some will looking for influencers, hashtags, key terms, sentiment, age and gender.

These tools also use natural language processing, a technique where the software mimics the human brain’s ability to group certain words together and spot correlations between them.

For example, some tools can understand the different uses of the word ‘sick’. Sometimes the word is used to refer to someone who is unwell e.g. ‘my grandma is sick’, whilst others may use it to describe someone’s positive impression of something. e.g. ‘Sick concert’.

Some of the tools use machine learning to allow you to train the tool to create more accurate results for you – you do this by checking whether it has categorised a particular tweet correctly and then correcting the tool’s assumptions. The tools should then re categorise all other tweets using the new rule you have given it.

When to use data and social media analysis

Social media analysis should be used throughout policy development. It can be particularly useful at the beginning of policy development to help you understand the exact problem users are having and what their needs are.

It can also be used alongside engagement and consultations during the the creative and prototyping stages of policy design. Crowdsourcing online can create a large amount of responses and using a social media tool to analyse them is important to ensure you hear every idea.

Online testing and polls can also be analysed through social media. In many cases it is best to talk directly to analysis providers to make sure that they can meet your needs.

Tools you can use for data and social media analysis