NHS Digital Staff Passport
This is the report from the cross-government alpha assessment of the NHS Digital Staff Passport service, on 5th July 2022
Service Standard assessment report
NHS Digital Staff Passport
| From: | Central Digital & Data Office (CDDO) |
| Assessment date: | 05/07/2022 |
| Stage: | Alpha |
| Result: | Not Met |
| Service provider: | NHS England and NHS Improvement (NHSE&I) |
Service description
Digital Staff Passports will enable postgraduate doctors in training, and staff who move temporarily, to hold a verified portfolio of digital credentials for employment checks and core skills training competencies. So they can move between different NHS organisations more easily and quickly, without the need for repeat form filling, checks and duplicate training.
Service users
- NHS staff – postgraduate doctors in training are the initial focus for MVP. The long term aim is to roll out to all staff.
- NHS employers and for each organisation their Corporate staff (HR and recruitment teams, occupational health and education), and Medical staffing (training and education teams)
- In addition, NHS England IT staff will be required to administer the service.
1. Understand users and their needs
Decision
The service did not meet point 1 of the Standard.
What the team has done well
The panel was impressed that:
- the team has conducted research with both end users (Postgraduate Doctors and Temporary Movers) and admin users (such as HR colleagues, Occupational Health, ‘Verifiers’)
- a number of user needs have been identified and categorised into key themes, such as saving time, avoiding duplication and workload management. Personas have been created to illustrate which users each need applies to
- research has been conducted with a number of admin users and, despite difficulty with recruitment, the team has managed to speak to some Postgraduate Doctors and Temporary Movers during the alpha phase
- the team explained their riskiest assumptions and were able to show that they had tested some of their assumptions in alpha
What the team needs to explore
Before the next assessment, the team needs to:
- conduct research with a wider range of end users. Although the team has researched with a good number of administrative users and have drawn on previous research with doctors, the number of end users they have tested the staff passport with is lower than we’d expect during the alpha phase. The panel would suggest that the team incorporates a wider group of users across a range of demographics into their research (in particular other NHS staff roles - this will enable the team to truly test their assumptions and design a scalable solution). To meet this recommendation, the team should consider using a recruitment agency to support the recruitment of NHS staff for research
- revisit their user needs to explain what drives users to engage with their service. Some needs, such as ‘as a postgraduate doctor, I want to be able to manage my statutory and mandatory training outside of my core skills through the digital staff passport, so that I do not have to repeat mandatory training as often’, are solution specific and don’t explain the behavioural driver for the user. In the assessment, the team explained that their users need to do as little admin as possible so that they can get on with helping patients - it would be great to see the team showcase needs like this at their next assessment to explain the bigger picture surrounding their service
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the team has identified a problem for users and are actively trying to solve that problem from both the end user and internal/admin user perspectives
- significant efforts are being made to ensure that users don’t have to repeatedly provide the same onboarding information multiple times. Where data is already held, the team is planning to surface this data for users to confirm rather than asking for re-entry
- the team was explicit in identifying and defining what they saw as their riskiest assumptions
- they are taking into consideration the dependencies between this service and related services, for example the Inter-Authority Transfer (IAT) process and Oriel to try to solve a whole problem
- for the users identified in the alpha stage, the service appears to solve a whole problem: that of making the transition between different NHS organisations much quicker and smoother
What the team needs to explore
Before the next assessment, the team needs to:
- demonstrate a greater understanding of the full range of potential users so they can be sure they have scoped the service correctly and be sure that the service solves a problem for all it’s potential users, not just a small subset of them
- demonstrate more clearly that this service is the right shape to meet the user needs they’ve identified without reference to the COVID-19 passport. Consider whether if the latter hadn’t existed, this approach to the NHS Digital Staff Passport would still have been identified as the right one
- show that they have further tested their riskiest assumptions. For example, it wasn’t clear whether questions around assumption 5, ‘The new process won’t cause issues with workload and capacity for HR’ were fully resolved
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the current, manual process will continue to be available to staff who are unable or unwilling to use the app
- staff users will be prompted by email to start the process of creating a wallet or sharing credentials
What the team needs to explore
Before the next assessment, the team needs to:
- think about how the offline journey might be improved based on insights from developing the digital service, in case there is a significant minority of people who cannot or will not use the digital passport
- prepare a demonstration of the user journey for someone who has lost their phone. Who do they contact? What do they need to do to get set up again? How long will it take?
- prepare a demonstration of how end users are directed to download and use the app in the first place
4. Make the service simple to use
Decision
The service did not meet point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the service’s content designer is well-integrated in the team (not always the case)
- content design has been led by listening to the language heard in UR sessions
- content has been clearly and consistently documented
- the user centred design trio have been working well together and running regular design reviews and consulting with a range of other NHS design and service teams
- the service is being designed using patterns and components from the NHS Digital design system
- the team recognised that further refinement of the interaction/interface design will be required as they move into private beta
- the team demonstrated iterative changes, based on user feedback, from the initial designs that were adapted from the COVID-19 Digital Staff Passport service
What the team needs to explore
Before the next assessment, the team needs to:
- consider how intuitive the service is for end users. End users have previously expressed confusion over the service (such as not understanding the concept of an ‘access credential’) and the team has endeavoured to reduce confusion via guidance. For a truly simple to use, intuitive service, the team should iterate their content and designs in line with user understanding and expectations
- make the concept of access credentials and downloading a wallet intuitive within the service, rather than address user confusion with guidance and comms. Expecting users to either memorise guidance in advance or break up their journey to go and read guidance could be a poor experience
- validate the name of the service. The GOV.UK guidance says to name your service using a verb so that users can find it when they search online. As this is an app rather than a web-based service, users do not have to search online to find it. Breaking with GOV.UK guidance may therefore be justified in this case, but that needs to be properly established. Although it’s true that there are a number of verbs that could apply to the tasks within the service, it may be possible to come up with a service name that covers them all and aligns with GOV.UK guidance (https://www.gov.uk/service-manual/design/naming-your-service)
- validate that all users understand what ‘ESR’ is – it’s not spelled out in the service
5. Make sure everyone can use the service
Decision
The service did not meet point 5 of the Standard.
What the team has done well
The panel was impressed that:
- the team has conducted research with some users with access needs and have considered the need for accessibility testing in the private beta phase
- the team is planning to test accessibility in the beta phase
- the team has so far tested with a number of different user groups: postgrad doctors, HR and recruitment etc
What the team needs to explore
Before the next assessment, the team needs to:
- assess the digital inclusion of their user base. The panel was surprised to hear that the team had not been assessing users’ digital competency and confidence and, as a result, the team was not able to meet this standard. It is crucial that the team assess their users’ digital skills throughout their research. Some of the team’s riskiest assumptions, such as the assumption that staff members are able and willing to use mobile phones and apps, cannot be validated or invalidated until the team has assessed digital inclusion. The team’s key beta aims, such as the aim to ‘understand barriers for widespread implementation and adoption’ cannot be met until this research is done. The service manual page on digital inclusion will help the team to formulate an approach to identifying the needs and drivers of users with low digital skills
- conduct research with users with access needs. The team highlighted that they had conducted research with three users with access needs in the alpha phase so far, however it’d be advantageous to have a larger sample of users with access needs to support the design and development of an accessible service. The team should consider seeking support from a recruitment agency to help unblock some of the issues they’ve faced in recruiting NHS users with access needs
- testing has been done around content resizing for people with vision disabilities and how well the content structure works for dyslexic users, but more needs to be done – for example with users of screen-reading software
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- the team has filled all the expected roles for a multi-disciplinary team as per the GOV.UK Service Manual guidance
What the team needs to explore
Before their next assessment, the team needs to:
- establish what its long term plans and strategy are for the support, maintenance and continuous improvement of the service are, and ensure it has the right team mix in place between Civil Servants and suppliers in order to deliver that in a value for money way
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- the team has an empowered Service Manager and Product Owner who are both able to make evidence based decisions for the team in a timely fashion without having to refer or rely on a Steering Committee unnecessarily
- the team is following a classic Scrum framework approach with all the relevant ceremonies at the right cadence
- the team will be able to deploy changes quickly and frequently in private beta
What the team needs to explore
Before the next assessment, the team needs to:
- explore whether Scrum remains the best agile framework to follow for a team that will inevitably have support and service level requests to resolve as unplanned, ad hoc work during their beta phases alongside the more common new feature/iterative continuous delivery work it can plan in advance
8. Iterate and improve frequently
Decision
The service did not meet point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the team has clearly iterated the prototype design based on feedback and insights gathered from usability testing and further user research
What the team needs to explore
Before the next assessment, the team needs to:
- conduct more usability testing to inform service iteration. The team has conducted research with ten end users in the alpha phase on four iterations of the UCD prototype. However, this sample size does not form a large enough basis for identifying how intuitive the service is and should ideally test each iteration with a minimum of five users. By addressing the recommendations in point 1, the team will be in a better position to conduct research that supports frequent iteration
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the team will create a system that is secure and private by design
- the team has coordinated with the National Cyber Security Centre and other cryptography experts
- the team has thoroughly considered any risks of each interaction
- the team has automatic detection and notification of vulnerabilities in third party libraries
What the team needs to explore
Before the next assessment, the team needs to:
- continue to keep up to date with security-related best practices
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- the team has really thought about what success looks like for this particular service above and beyond the mandatory four KPIs that the Service Standard asks for
- the team has identified the areas where it needs to baseline data and has realistic expectations about what data it will actually be able to gather and what might prove beyond them
What the team needs to explore
Before the next assessment, the team needs to:
- do it’s best to gather and produce baselines for existing processes, so that it can be clearly demonstrated where this service/product has successfully improved things for users
- potentially reconsider whether certain metrics and success criteria can ever truly be attributed to use of the tool - retention rates for example, might be difficult to establish a link between changes in this metric and the role the service has played in these changing
- explore what its performance data publishing strategy will be and whether it is justified to not publish certain metrics, which whilst designed to help local senior management make decisions; if an FOI or Parliamentary Question would mean they could be placed in the public domain anyway, reduce the logic for not publishing
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the team considered a variety of architectures that could potentially address user and organisational needs, including a central database and federated communication model
- the team uses knowledge gained from the interim COVID-19 Digital Staff Passport service to inform technological decisions
- the team has consulted with the Chief Technology Officer
- the team has chosen an appropriate architecture within a complex space
- the team has planned mechanisms for frequently and quickly deploying changes across NHS organisations
What the team needs to explore
Before their next assessment, the team needs to:
- carry on reflecting on technical choices in response to feedback during private beta
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the team has released source code
- the team has designed the system so that with the source code, other organisations could easily stand up their own version of a Staff Passport system, noting parallels between the NHS and the Department for Education
What the team needs to explore
Before their next assessment, the team needs to:
- make sure it’s clear which Open Source Initiative licence applies to released source code
- include usage instructions with source code
- more regularly release source code, ideally as part of the deployment workflow
- explore releasing the source code alongside other open source code of a larger organisation
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the team has chosen the W3C Verifiable Data Model
- the team will use standard authentication mechanisms such as OAuth 2.0 and standard cryptographic methods based on elliptic curve public and private keys
- the team will host the system on a cloud provider using standard hosting patterns, with blue/green deployments without downtime
What the team needs to explore
Before their next assessment, the team needs to:
- implement and reflect on the planned technological patterns
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- the team will have both testing and monitoring in place
- the team will have a pipeline in place to deploy both frequently and quickly
- the team understands which components must be available 24/7 and prioritises accordingly
What the team needs to explore
Before their next assessment, the team needs to:
- explore having a team in place to respond to production issues of critical components in a timely manner
- explore whether automatic horizontal scaling is appropriate for components
Next Steps
In order for the service to continue to the next phase of development, it must meet the Standard and get CDDO spend approvals. The service must be reassessed against the points of the Standard that are not met at this assessment. Speak to your Digital Engagement Manager to arrange it as soon as possible.