Apply for HM Armed Forces Veteran’s Card beta assessment
Service Standard report Apply for HM Armed Forces Veteran’s Card 07/12/2023
Service Standard assessment report
Apply for HM Armed Forces Veteran’s Card
| From: | Assurance team |
| Assessment date: | 07/12/2023 |
| Stage: | Beta |
| Result: | Amber |
| Service provider: | Ministry of Defence |
Previous assessment reports
Subsequent service standard reports
Service description
The service is for UK armed forces veterans and provides recognition of their service. It will allow veterans and providers of beneficial services to confirm that someone has served in the UK armed forces in a simple process, so veterans can quickly access the services they are entitled to.
Service users
This service is for:
- UK Armed Forces veterans, principally those who left service prior to 2018
- Third-party service / support providers (such as charities)
Things the service team has done well:
- the panel was very impressed with the team’s storytelling and their ability to clearly convey the overall problem they’re solving, how they’ve addressed it, how it is making a difference and the constraints that they are working within. They were also able to describe the plans for future development of the service, and how they are jointly working with other government departments to solve the problem.
- the team has a full multi-disciplinary team with all job roles covered, and could demonstrate that they are working in an agile way with standard tools and methodology. The team has secured funding so that they can have consistency as they move into public beta.
- the team’s user research approach has been thorough, and they demonstrated work on assisted digital, users with accessibility needs and paired users. They considered the journeys for veterans who needed support, and have a robust offline channel, which has also been thoroughly researched.
- the panel were pleased that the team had explored journeys for a wide variety of users, such as homeless veterans and veterans whose gender has changed since they left the service. The panel commend the team for considering the trauma that many veterans have suffered and ensured the research and content design approaches have been trauma-informed, and examined how user needs were influenced by this.
- the range of research methods used and how they each built on each other was impressive, as was the depth of contextual research conducted to give a big picture of the environments and situations that veterans would complete their applications in.
- the team have worked closely with the GDS One Login team to share findings from user research, and were able to share successful changes through this.
- the team has prioritised user research with veterans during private beta because they are the primary user group, this makes sense. We recommend that the team research with other user groups, such as charity partners, during public beta as they stated they plan to.
- the number of participants when broken down by research methods, neuro diversities and disabilities was not clear. When the team are involved in future service assessments we recommend they clearly state how many participants were involved with these.
- the team have adopted the GOV.UK Design System and are reusing common components where possible so that users are confident the service is legitimate. This also helps to lessen the impact of utilising Gov One Login within the service as they share a common look and feel. The team should be sure to continue to work closely with the Gov One Login team, exchanging findings and discoveries to benefit both services
- there appears to be a solid process of iterating content based on research and reviews
- the team have a strong communications plan for when the service is opened up to more users. This includes gov.uk launch page, third parties being informed and involved and the service being ready to be released publicly
- the panel was pleased to see that the team have identified a strong need for the guidance on the Start page, giving users the information they need beforehand in order to be successful. They should consider the suggestions for this page given in the Content Review in addition to guidance provided in the GOV.UK design system on start using a service
- the team have had a clear plan for undertaking research with the widest range of potential users of their service. They continue to search out less involved groups and consider their needs
- the team have a clear understanding of the landscape in which they’re working, and a passion for providing continued routes for those users needing digital assistance
- the team have involved back office users in their research so that the full user journey is designed to be simple and inclusive, as recommended in the alpha assessment report
- the use of a “built” environment as a prototype can have benefits in beta - enabling testing with assisted technologies, and highlighting any accessibility issues, for example. The team should, however, ensure that there is a dedicated research/testing prototype available in order to continue research and design iterations through the beta phase. This could be using the GOV.UK prototype kit or a dedicated environment
1. Understand users and their needs
Decision
The service was rated green for point 1 of the Standard.
2. Solve a whole problem for users
Decision
The service was rated green for point 2 of the Standard.
3. Provide a joined-up experience across all channels
Decision
The service was rated green for point 3 of the Standard.
4. Make the service simple to use
Decision
The service was rated amber for point 4 of the Standard.
Recommendations
- whilst the service is simple, the information that the veterans need to provide may not be to hand. The team has done a lot of work on managing expectations and letting veterans know what is needed, but also referred to veterans leaving the computer to find it. The panel recommends researching save and resume as a hypothesis, using performance data and user research to inform.
- the team should action all suggestions labelled “MUST” in the Content Review completed by GDS before progressing to the next phase.
5. Make sure everyone can use the service
Decision
The service was rated amber for point 5 of the Standard.
Recommendations
- further work is needed for the team to clarify whether the service should be offered in Welsh (the panel’s view is that it should be) and to plan to build this if needed. The panel accepts that this was added to the team’s backlog and prioritised during the assessment.
- the team has assured the panel that they have fixed all issues found in the initial accessibility audit. They must now have those changes re-audited and “pass” in order to progress to the next phase.
- the team should ensure they can respond nimbly to user research findings and have a clear research environment or prototype in order to ensure the iteration cycle can effectively respond to new information and evidence.
- as reflected in Point 11, the use of a Javascript framework prevents users without Javascript enabled from using the service and does not provide progressive enhancement in line with the service standard. Our recommendation is that this is something that should be done for components used by veterans directly, in order ensure all users can use the service
6. Have a multidisciplinary team
Decision
The service was rated green for point 6 of the Standard.
7. Use agile ways of working
Decision
The service was rated green for point 7 of the Standard.
8. Iterate and improve frequently
Decision
The service was rated green for point 8 of the Standard.
9. Create a secure service which protects users’ privacy
Decision
The service was rated green for point 9 of the Standard.
10. Define what success looks like and publish performance data
Decision
The service was rated amber for point 10 of the Standard.
Recommendations
- the team were able to show the panel a dashboard with main KPIs and describe some ways they are using this to iterate and improve. The panel was not entirely clear on how the presented stats linked to the narrative and would recommend more work is done to be clear on the story that these metrics are telling and would be interested in hearing more about how data is being used to inform decision making on the team.
- the assessors also noted that many of the metrics presented were based on data for fewer 100 requests via the digital channel and exhibit considerable variability. This includes data on session lengths and completion, which support significant design choices such as not providing a save and return feature. The assessors think it appropriate to review the data with an assessor with a larger dataset.
- the team is gathering a broad set of relevant metrics; a notable gap is a metric that clearly captures the end-to-end delivery of the card. The team should ensure they are tracking a metric such as the proportion of cards being sent out and received in the timescales stated on the email or the mean time from request to delivery. An end-to-end measure should be clearly described/labelled as such.
11. Choose the right tools and technology
Decision
The service was rated amber for point 11 of the Standard.
Recommendations
- the choice of front-end framework, Next.js, prevents users without Javascript enabled from using the service and does not provide progressive enhancement in line with the service standard. Our recommendation is that this is something that should be done for components used by veterans directly, however the panel recognises that changing this for the whole service could be a large amount of work at this stage in the service’s lifecycle.
12. Make new source code open
Decision
The service was rated amber for point 12 of the Standard.
Recommendations
- the team have identified that there are repositories as part of the service that could be reasonable candidates for publishing the source code. Our recommendation is that the team should open as much of their codebase as is feasible with services used by veterans directly, like frontends, being the highest priority.
13. Use and contribute to open standards, common components and patterns
Decision
The service was rated green for point 13 of the Standard.
14. Operate a reliable service
Decision
The service was rated green for point 14 of the Standard.
Next Steps
This service can now move into a public beta phase, subject to addressing the recommendations given for the amber points in three months time and CDDO spend approval.
This service now has permission to launch on a GOV.UK service domain with a Beta banner. These instructions explain how to set up your *.service.gov.uk domain.
The service must pass a live assessment before:
- turning off the legacy service
- reducing the team’s resource to a ‘business as usual’ team, or
removing the ‘beta’ banner from the service
Updates to this page
Published 14 October 2024