How NHS Digital used API Management to support APIs at scale
NHS Digital implemented an API management platform to help them roll out digital healthcare services more quickly and consistently.
The Personal Demographics Service (PDS) is the national electronic database of NHS patient details. It is a secure database containing demographic information such as name, address, date of birth and NHS Number to help healthcare professionals identify patients and match them to their health records. It also allows them to contact and communicate with patients.
PDS has been around for about 15 years and was previously using a SOAP API. One of the big pain points was with integration, as it was technically difficult to connect to NHS Digital services. In a survey, external developers rated NHS Digital services 2 out of 5 for ease of integration.
Objective
The mission for this project was to “make integration easier”. The team at NHS Digital set an API management vision with goals to make it easier for developers to:
- onboard, by only asking developers to do things that are needed
- learn, by making documentation available online
- design and build, by making APIs internet-facing and using open standards
- test, by providing developers with sandbox environments for early testing
- get help and support with self-serve support and reliable help resources
In addition to these general principles, the team would be redeveloping the PDS API as an exemplar API. This would help show the capabilities of the platform and validate the governance process.
Getting approval and working with stakeholders
The team recognised that the lack of an API management platform meant external developers did not have the resources they needed to easily integrate with NHS APIs. The project was set up to deliver a central platform gateway and associated governance which would help internal API teams provide these resources to their users.
The team had senior sponsorship for the programme from the Cabinet Office, which helped with securing budget and prioritisation for the project.
Areas like commercial and information governance presented some challenges. Purchasing software can be a time-consuming process, so the team made use of a free trial of their chosen platform before they went live. This allowed them to get started before going through procurement.
The team needed to reassure stakeholders that the project aligned with requirements around information governance and security. APIs must go through assurance processes, and the central API management platform would allow API teams to do this in a consistent way. This would help projects get started quickly without having to start administrative processes from the beginning every time.
Improving the developer onboarding experience
The team recognised that API producing teams need to be supported from the beginning of any project. They put in place a process to support teams from the pre-alpha stage. This included holding initial design meetings to explain and demonstrate how the API life cycle fits into the new platform.
Making documentation discoverable
The team put a lot of effort into making documentation very clear and available online. Previously, NHS Digital documentation was created as Microsoft Word files. This meant that anyone needing documentation had to request files by email, and there was no central place to discover and view documentation.
Using an API management platform has allowed the team to centralise their documentation on a developer hub. Documentation is now easy for anyone to view, whether that’s product owners or developers.
The API management platform has also helped establish a publishing process for documentation. This has made it easier for the team to create, publish and maintain documentation alongside development.
A vital part of overhauling the documentation process involved having the right technical writing skills on the team. The team now has a pool of technical writers in the API Platform squad, and assigns them to projects based on needs. This helps to balance consistency and availability so projects always have a technical writer to work on documentation from the start.
Sandbox
Another important part of creating a seamless developer onboarding experience was the sandboxing function. This provides a self-service environment developers can access without needing any authorisation, allowing them to quickly start integrating with the API.
The sandbox returns placeholder patient data in a standard RESTful format with the correct responses for the API. This helps external teams start building services using the API in sandbox while they are still in the process of being authorised to use the live API.
Assurance
The team also built and documented processes for API teams to use while building APIs, including a separate staging platform for central teams to help evaluate and assess APIs in their early stages. This resulted in an assurance process which helps teams follow best practices, as well as improving their time to production.
Setting up 24-hour support
NHS Digital has bronze, silver, gold and platinum service classifications. Early in the project, the PDS API was classified as a bronze service which meant, among other things, support was limited to within office hours. One NHS team using PDS for a critical healthcare service needed 24-hour support, so this requirement pushed the team to upgrade the platform from bronze to platinum level.
Rolling out 24-hour support involved some operational changes. The existing backend already had 24-hour support, but this needed to be extended to the platform itself.
The team set up an on-call rota. The rota makes sure someone is available to respond to emergencies and provide backup support. As such, members of the platform team had their contracts extended to include out of hours work. This involved some logistics with HR, Legal and service operations.
Developers also received a minimum level of training on diagnosing issues and standard operating procedures when working out of hours. This prepared developers in case they needed to do things like shut the platform down or revert a release. Pair programming was also encouraged as a way to support knowledge sharing and upskill developers.
The team also set up an incident management process to identify, prioritise, resolve and report technical faults and failures. NHS Digital uses incident severity classification, also known as “sev” levels, to categorise incidents.
Using modern technology to balance flexibility and security
Previously, the PDS API was served over the Health and Social Care Network (HSCN), a data network for public sector health and care organisations across England. This presented challenges for citizen-facing services, as it limited access to these services to NHS premises. The new API uses Fast Healthcare Interoperability Resources (FHIR), a global industry standard for passing healthcare information between systems.
The API is also now delivered over the open internet to give more flexibility to support services used outside NHS premises, including those used by citizens directly. To reduce any increased security vulnerabilities of this approach, the API management team rolled out a new pattern to handle security and authorisation.
Using an API management platform as a proxy allowed them to apply this security pattern to all the APIs. It also allowed them to connect with an Identity Platform (IDP) and Identity and Access Management (IAM) module to handle authorisation.
They knew that simply having an API key would not be secure enough given this was personal data on a national scale. To increase security further, the team also implemented signed JSON Web Tokens (JWTs), an open standard. The API management platform allowed them to do this quickly and easily.
Outcomes of the project
Improved lead time
From start to go-live, it took the team about 7 months to set up the API management platform, and build the associated governance, to support the initial exemplar API. Following this, teams were able to roll out 2 APIs - the PDS FHIR API and the Ambulance Data Submission FHIR API - in just a few months. Previously, the lead time was around a year.
Improved developer rating and engagement
When deciding on what success metrics to look at, the team focused on improving the developer rating as a clear metric to measure how well the service was meeting user needs.
The team set a target to increase the developer rating from 2 out of 5 to 4 out of 5. They felt that building a minimum viable product (MVP) would get them to 3, but knew it would take a lot of work to get to 4. In January 2020 they conducted a survey, which showed a 3 out of 5 rating. By early 2021, similar surveys showed results of 4 or 5 out of 5.
The survey has also seen an increase in engagement, with the response rate approximately doubling each year. To maintain engagement with developers as the project continues, the team has an interactive roadmap where developers can suggest and upvote features.
Improved team motivation
In addition to the measurable success of the project’s deliverables, the team has seen positive improvements internally, as well. Team motivation has improved significantly since the project went live. A dashboard showing traffic has helped with this, and team members feel like they’re part of something special.
Lessons learned
The team found beta testing particularly valuable for the project, as it helped to reveal issues that the team overlooked in testing. For example, 3 developers got in touch about getting failed responses with names with apostrophes, where curly apostrophes were being rejected. This would not have come out in the team’s own testing as this very specific use case was not one they had thought of.
A model for future APIs
The PDS FHIR API was intended from the start to be an exemplar service on the NHS Digital API management platform. Since it went live, a number of other APIs have been added using both the technical and governance approaches that this API has validated. This has therefore created a framework or template for future integrations. This will make things easier for teams doing similar projects, not only for consumers of APIs, as was the initial objective here, but also for API producers.