Guidance

Events Research Programme Phase III: Service Evaluation - Testing of the NHS App in real-world settings

Updated 8 September 2021

© Crown copyright 2021

This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.

Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.

This publication is available at https://www.gov.uk/government/publications/events-research-programme-science/events-research-programme-phase-iii-service-evaluation-testing-of-the-nhs-app-in-real-world-settings

Events Research Programme Phase III: Service Evaluation Work Statement

Study name

Testing of the NHS App in real-world settings.

Event locations for study

All events during phase 3 will use the NHS App.

Principal investigator

Phillipa-Rose Hodgson, Director of Product, NHSX

Lauren Harkins, Assistant Director of Programmes, NHSX

Key research question(s)

What can we learn from real-world testing to support product development?

Sub questions:

i) What can we learn from citizens during real-world testing to support product development

ii) What can we learn from venues during real-world testing to support product development

iii) What can we learn about the non-digital journey during real-world testing to support product development

NHSX are testing technology. NHSX are not undertaking transmission studies nor undertaking certification policy related enquiries.

NHSX may support DHSC-led research related to exemptions which is not included within this protocol.

Not applicable

Study Design

User journeys

There are two ways in which citizens can digitally surface their NHS COVID Pass based upon vaccination and testing data:

  • Through the NHS App: displays vaccination and test data (pillar 2).
  • Through NHS.UK: displays the same data as the NHS App but the jump-off point into the service is via NHS.UK rather than the NHS App.

Citizens can also request an NHS COVID Pass letter from 119 or via NHS.UK

The current build allows for:

  • vaccination data
  • pillar 2 testing data - LFT and PCR (both at-home and assisted)
  • natural immunity post-positive PCR test

User needs and service design

Aligned to Government Digital Service (GDS) standards, the technology has been built and tested in a user-centric way, considering differing needs of individuals accessing the user journeys.

Testing

Eligible participants are considered to:

  • be registered with a GP in England
  • be over the age of 16 (or 13 with consent from GP)
  • have been fully vaccinated, or undertaken testing within the Cabinet Office specific testing window prior to an event, or have natural immunity

Participants would be able to use any of the three service offerings (NHS App, NHS.UK or an NHS COVID Pass letter) to enter the venue.

The research will be centered around user-oriented testing and user experience.

User-oriented testing and user experience

The NHSX user research team will undertake on-site observations, interviews with both users and venue staff who have used the technology to identify opportunities to enhance the product offering, and associated communications.

The NHSX user research team will undertake user experience (UX) testing to identify opportunities to improve the look and feel of the product and service. This will include:

  • access to the NHS App via P5 and P9 (differing authentication levels)
  • experience using the NHS COVID Pass digitally and non-digitally
  • citizen support required
  • response to comms (part of service design)
  • staff training and support
  • consideration of any technical issues
  • experience in the differing settings

This will enable NHSX to identify items for the product backlog and opportunities to improve the service.

The research will also reflect changes in communications and product development work that take place during the programme.

Key outcome measures

Understanding of the experience of citizens and venues, including thematic analysis.

Items identified for product backlog.

NPIs being changed

Not applicable.

Engagement with participants and communications

Engagement will be directly on-site with those attending the events and event organisers.

Peer review

This work is not subject to peer review.

Ethics approval

This work is not subject to ethics approval.

Back to top