NHS Digital Weight Management Referral Hub
This is the report from the cross-government beta service assessment of the NHS Digital Weight Management Referral Hub, on 7th July 2022.
Service Standard assessment report
NHS Digital Weight Management Referral Hub
| From: | Central Digital & Data Office (CDDO) |
| Assessment date: | 07/07/2022 |
| Stage: | Beta |
| Result: | Not Met |
| Service provider: | Midlands and Lancashire NHS Commissioning Support Unit (MLCSU) and NHS England and Improvement (NHSE/I) |
Previous assessment reports
Service description
The NHS Digital Weight Management Programme (DWMP) supports the delivery of NHS Long Term Plan commitments relating to obesity. It focuses on providing weight management support to people living with obesity and co-existing long-term conditions.
The DWMP provides access to commercially available digital weight management services that can be delivered flexibly, without the need for service users to travel to particular locations. This provides a viable alternative to traditional face-to-face interventions. Services are provided by Weight Management Service Providers (WMSP) commissioned by NHSE/I.
Service users
This service is for:
- External users: GP / Primary Care Healthcare professionals – those referring patients into the Hub via the e-RS
- External users: Community Pharmacy / Community Care Healthcare professionals – those referring patients into the Hub via the on-line web-based referral forms
- External users: Patients (adults) and carers
- Internal users: Midlands and Lancashire NHS Commissioning Support Unit Referral Management Centre (RMC) staff and business intelligence analysts
- Internal users: NHS England and Improvement Statistical Analyst
1. Understand users and their needs
Decision
The service did not meet point 1 of the Standard.
What the team has done well
The panel was impressed that:
- the team has conducted a range of research methods across their user groups
- the team has understood during private beta some of the ways that their service is helping users get better health outcomes and reducing inequalities
- the team understands the pain points faced by users trying to access weight management services, and the role that inclusion and exclusion play for different user groups
What the team needs to explore
Before the next assessment, the team needs to:
- explore conducting end-to-end user research across the entire patient user journey. The panel understands concerns about creating bias on patient outcomes but believe that this concern can be mitigated by an embedded user researcher in the team with careful management of consent and participation
- conduct iterative research and design cycles, where the service is regularly tested with real users and iterated based on their experiences
- develop more qualitative insights on patient outcomes to help define success for the service, rather than relying purely on quantitative improvements
- show a deeper understanding of the motivations, contexts, goals, behaviours, beliefs and pain points of all the user groups of the service
- test the end-to-end service with users with access needs and using assistive technologies, including sections using third party suppliers like chatbots
2. Solve a whole problem for users
Decision
The service did not meet point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the team has been working with service providers on a monthly basis to review the performance of the service and make improvements. For example, by signposting types two diabetes patients to other follow-up programmes in their discharge
- the team is engaging with GPs, clinicians, and relevant stakeholders that refer patients into the weight management programme
- The team has reused the existing referral pathways and digital tools available to create the service
- the team recognised the challenges of obtaining outcome data from service providers as the programmes take 12 weeks, and not all users will finish the programme. They are encouraging providers to get feedback data in real time
What the team needs to explore
Before the next assessment, the team needs to:
- explore ways in which patients don’t have to enter the same details provided in the referral service into the service provider
- conduct user research interviews with GPs and clinicians to better understand their needs and test the end to end service with them. Engagement sessions and focus groups are helpful but they don’t replace in depth interview and testing the user journey
- find more ways of gaining qualitative data to better understand the patient’s needs and experience throughout the end to end journey. Work with service providers to conduct user research and testing with patients in order to understand the effectiveness of the service
- investigate more thoroughly the outcomes experienced by users that complete, or part complete the journey, and use that knowledge to create improved iterations that increase the numbers who finish their journey
- ensure their research follows those who have finished the programme to see what lifestyle changes have taken hold and which have not, and why not
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the team has designed the service from referrals to handover to the service provider - including text messages, email, referral digital user journey
- data flows between referral systems and the service have been designed and enabled through APIs
- the team has looked at ways to communicate with all types of users across multiple channels so the same messages are received
What the team needs to explore
Before the next assessment, the team needs to:
- explore ways in which the online journey sign-posts or links to alternatives services (for example face to face support, guidance on the NHS website) to ensure users have options of getting support from the weight management programme
- explore ways in which patients don’t have to enter the same details provided in the referral service into the service provider
- consider users who may live in areas with little or no internet service and how they will participate fully in the programme and follow the sign up processes
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the online service uses the NHS design system components and patterns
- the service uses patient records and information via their NHS number and GPs, making it easier for users to receive relevant options for service providers
- the service is plugged into existing referral systems and forms in order to help Primary Care Healthcare professionals, GPs, and Community Care Healthcare professionals connect to the NHS weight management referral triage service
What the team needs to explore
Before the next assessment, the team needs to:
- conduct user research on the way service provider information is displayed and if patients feel confident they have the right information they need to make an informed decision. Explore if the design is suitable (Expander component) to compare service providers or if it needs iterations or another design solution
- explore ways in which to measure patient confidence when selecting service provider
- conduct usability testing with users with access needs (patients and healthcare professionals)
- conduct accessibility audit for third party providers for chatbot and any other parts of the service that are brought in externally
- publish an accessibility page for the service
- ensure users are aware of what to expect once they’ve selected a service provider. At the moment the end of the service gives no indication on how long it will take for them to contact the patient
5. Make sure everyone can use the service
Decision
The service did not meet point 5 of the Standard.
What the team has done well
The panel was impressed that:
- the team has implemented ways in which patients can get support on the digital user journeys via the “help” button. Clinicians and users that are submitting the referral for patients also have a way to access support
- the team has tackled some of the inequality of the services by expanding the reach to parts of the country that didn’t have access to these services, referrals have increased more male uptake (80% was female) and 50% of referrals are from the most deprived quintal users (less affluent)
What the team needs to explore
Before the next assessment, the team needs to:
- improve ways in which to reach and conduct user research with more diverse users (race, ethnicity, demographics, class). Currently the majority of people signing up to participate in research through the service are still white British females
- conduct usability testing with users with access needs (patients and clinicians)
- understand the needs of the people that may wish to engage with an assisted digital route, but have additional needs. Ensure there is a plan to provide an assisted digital route. Fully investigate the route and test the service with this user group. It was not apparent what work has been done in this space
- continue to test the viability and success of the non-digital routes being offered
- research and test a service name that resonates with users. The service name should describe what the user is trying to achieve. This will enable it to be more findable by users. Follow the guidance in the Service Manual about naming your service
6. Have a multidisciplinary team
Decision
The service did not meet point 6 of the Standard.
What the team has done well
The panel was impressed that:
- the team has successfully coordinated a very large project group over multiple departments and specialisms and enabled regular stakeholder interaction
- the team has used the various professional opinions from within that group to drive forward the service and iterate the offering to better meet the users needs
- proper use has been made of feedback received to drive forward the development of the service during private beta and they are seeing high numbers interact with the product giving valuable insight
What the team needs to explore
Before the next assessment:
- the team must employ all of the right professions throughout the whole of the public beta phase to gain the fullest and most complete information on the effectiveness of impacts of the new service. This includes design, user research, and content design
- the team must reexamine the previous thinking in light of the lack of the full multi disciplinary team to guarantee the service is indeed moving along the right lines
- have a plan to accommodate changes needed to suppliers who either do not meet, or begin to fail to meet, their full requirements and for new suppliers coming on board with the right offering
- ensure analytics work drives continuous improvement in measuring the longer term impacts of the service on users’ health
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- serious commitment has been evident throughout to open and frank conversation both within the direct team and to the wider stakeholder group
- the team use varied methods to communicate across their wide landscape to ensure there are no delays to messages or discussions
What the team needs to explore
Before the next assessment, the team needs to:
- demonstrate they work in an agile way, for example by conducting user research with patients, testing the design, and iterating based on that feedback
- ensure that the knowledge they gain from user insights is shared with all stakeholders with a view to ensuring that in beta the product develops further the drivers that make participants stick with the programme and effect real changes in their health lifestyle
8. Iterate and improve frequently
Decision
The service did not meet point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the findings in private beta were driving the changes in the service
- the stakeholders were engaged with the user needs demonstrating how and where to iterate the service
What the team needs to explore
Before the next assessment, the team needs to:
- test the end to end user journey across all channels with patients, evidencing design iterations based on user research. For example, testing if users feel confident they have the right information to make an informed decision on what service provider to choose
- ensure a focus is maintained on those users reaching the end of a cycle to research what happens to them next and seek to understand those who may repeat the process
- ensure user centred design disciplines are involved in the work. Design, user research, and content are key roles in order to take an iterative approach
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the team has carried out a Data Impact Assessment
- the team has used appropriate security controls (web application firewall) and ensured all data stored is encrypted in transit and at rest
- reviews were in place for data that was shared with providers
What the team needs to explore
Before the next assessment, the team needs to:
- carry out appropriate threat modelling using guidance from NCSC and others around the use of one time links and SMS delivery to ensure the system is fully robust
- assess third party code (introduced through Nuget packages) for any vulnerabilities as part of pipelines consider further restrictions where appropriate to verify the message delivery has been to the right candidate. Continually monitor and observe activity on the platform using the application insights and logging mechanisms deployed, sharing this with a security operations centre to ensure they can observe unusual behaviour
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that the team:
- had identified a number of key performance indicators
- was using a range of metrics from different sources including costs data and data from providers
- had implemented analytics on the digital interface, allowing them to see how users select providers (for example, whether users choose the first provider, how many options they view before selecting, what validation errors are shown)
- was able to segment data so they could evaluate each provider’s performance for different types of user, for example
What the team needs to explore
Before the next assessment, the team needs to:
- ensure that KPIs relate back to the overall purpose of the service and high-level user needs – the metrics that the team initially discussed were useful in understanding how users move through the process, but more should be done to measure the service’s success in improving weight loss and addressing inequalities in existing programmes
- give more examples of how data has been used in the development of the service to understand user behaviour and improve service performance
- ensure that the data being collected through their partnership with the Oxford Research Group continues to be available to the team as the service develops
- publish data to data.gov.uk
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the team had taken a public cloud approach with an active-passive highly available architecture
- the team had separated concerns into frontend web servers and API nodes running on platform as a service provision
- the team had chosen to use the latest version of .NET technologies to complement the team skillset the team has ensured they have chosen standard authentication patterns (Oauth) for the API layers.
What the team needs to explore
Before the next assessment, the team needs to:
- continue to share best practice between the two development teams
- move towards staging slots for the API as well as the frontend applications
- consider ways to optimise costs further as the platform evolves
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the source code has been made available at a number of locations https://github.com/mlcsu-digitalinnovations/wms : The main repository containing the API’s. https://github.com/mlcsu-digitalinnovations/wms-additional-ui : The User Interface repository for Pharmacy, MSK and Self-referrals
https://github.com/mlcsu-digitalinnovations/wms-templates : The GP Practice interface elements.
- the team clearly understands the value of coding in the open
What the team needs to explore
Before the next assessment, the team needs to:
- ensure each open source repository has an appropriate descriptive readme and licence information
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the team makes good use of the NHS design system components and patterns
- the team is using the GOV.UK Notify platform ensuring that SMS messaging within the platform is robust and have written additional integration code with which they can share with other projects
- the solution uses multiple authentication methods including the integration with NHS Login for one self-referral pathway
- the team has used standard patterns using Oauth for API authentication and authorisation
What the team needs to explore
Before the next assessment, the team needs to:
- consider how they can share their work across teams and the departments
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- users could access and make choices 24/7
- the team had designed for disaster recovery and high availability from the outset
What the team needs to explore
Before the next assessment, the team needs to:
- continue on the path to fully automated no downtime releases
- ensure recovery time objectives and recovery point objectives are clearly documented and agreed with the relevant departments
- ensure plans are robust to handle any interruption from integrated systems (Chatbot / other NHS systems and provider systems)
Next Steps
In order for the service to continue to the next phase of development, it must meet the Standard and get CDDO spend approvals. The service must be reassessed against the points of the Standard that are not met at this assessment. Speak to your Digital Engagement Manager to arrange it as soon as possible.