Pensionwise Digitalisation alpha assessment
Service Standard assessment report Pensionwise Digitalisation 6/7/2023
Service Standard assessment report
Money and Pensions Service (MaPS) Pensionwise Digitalisation
| Assessment date: | 6/7/2023 |
| Stage: | Alpha |
| Result: | Met |
| Service provider: | The Money and Pensions Service |
Service description
Helping people understand their options with accessing their pensions and the tax implications of their choices.
Service users
People over the age of 50 with a UK-based defined contribution pension pot who want more understanding of the options available for accessing their pensions, the associated tax implications and / or the long-term impact of the decisions they make. They may have been directed to the service by Stronger Nudge to Pensions Guidance from their pension provider.
1. Understand users and their needs
Decision
The service met point 1 of the Standard
What the team has done well
The panel was impressed that:
-
the team has identified different user groups based on findings from review of recent large-scale surveys and alpha
-
the team has conducted user research interviews with 35 users, also 24 moderated research sessions across six rounds of usability testing
-
the team has undertaken three rounds of unmoderated research engaging with 65 users, allowing variations of the nine prototype iterations to be tested. This approach was used due in part to the limited budget available for research
-
the team has shown they are thinking about how they are going to understand the needs of users with accessibility needs by doing informal research with a Neurodiversity Network. Also, it is good to understand that as an organisation, MaPS is exploring relationships with third-sector organisations, such as Good Things Foundation, Citizens Online, and We Are Digital which helps to inform their choices
What the team needs to explore
Before the next assessment, the team needs to:
-
prior to the service being tested with live users in private beta, speak to a wider range of users to identify their needs, such as those previously overseas, renters, those in mortgage and non-mortgage debt, which when combined could be significant numbers. The team has created six high level personas, which felt limited to the panel as the number of citizens over 50 could be around 25% of the population. The team has considered its users from different viewpoints, using attitudinal segmentation. The team suggested most of the users are in the ‘avoiders’ or ‘cautious planners’ segments. Speaking to a wide range of users, not just focusing on the segmentation, will help the service team to better understand the nuances of needs around the complex area of pension guidance
-
prior to the service being tested with live users in private beta, reconsider how they approach the required sensitive questions. For example, personal circumstances such as being diagnosed with a terminal illness, living overseas which can have many and varied implications on a user’s pension circumstance
-
prior to the service being tested with live users in private beta, consider the terminology used in the service. There is a considerable amount of technical and possibly confusing terminology surrounding pensions. The team is aware the users of this service are going to be making significant decisions based on the guidance provided, a greater understanding of how users interpret the wording is needed. If it emerges users do not understand the pension terminology used, this will support endeavours of your content designer to provide additional information to explain the terms
-
speak with representative users across the devolved nations to ensure they understand that the guidance is also for them. The 2022 pension legislation is UK wide. It was unclear if the team had spoken to users across all countries of the UK
-
consider the needs of people who had previously lived abroad (both inside and outside of the EU) as that will possibly impact their retirement planning
-
consider how users understand the power of attorney aspect of the journey. This is a complex concept that a user could potentially interpret in several ways, for example, power of attorney for themselves, or someone else
-
better understand the impact to users of state-related circumstances, such as receipt of benefits and state pension. As the Pension Wise service is an arm’s length body of DWP, consider working with teams at DWP in related areas to gain insight into users and their needs to benefit from their learning
-
consider the needs of users who are in debt. The panel observed there was too little acknowledgement of the debt aspect of this service. The team you should gain a specific understanding about what debt is to people, including mortgages, so the team will understand the needs of those in debt and is confident a user will understand their situation
-
put users at the centre of their focus against the Digital Inclusion Scale. The team had considered the service, rather than their users and acknowledging the multi channel approach to Pension Wise has the potential to accommodate users across the scale. It is important to record users’ digital confidence when conducting research to demonstrate that the digital service can move users to a different channel if required
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard
What the team has done well
The panel was impressed that:
-
the team is aware of the legal requirements to provide accurate pension guidance to the users
-
the team has used different moderated and unmoderated methods to understand more about the users
-
the team has provided a combined Design and Research plan until the end of the year
-
the team acknowledged there are parts of the service which they cannot influence or change right now but have plans to do so in future
What the team needs to explore
Before the next assessment, the team needs to:
-
conduct further accessibility testing. Creating a service that is accessible to all is a legal requirement, and for a service that is in place to support users through complex decision making, testing with users who have access needs is a priority
-
demonstrate what ‘save and return’ means in the service. Will this require setting up an account, and how this might be possible using codes, and not providing personal information. It would be worthwhile considering how other services have approached this. A Data Protection Impact Assessment should be conducted to ensure the team and stakeholders are fully aware of what that functionality will entail
-
conduct several rounds of both moderated and unmoderated usability testing to help accurately inform multiple design iterations and improve the team’s knowledge of what users need when completing required tasks
-
conduct and showcase the team has conducted additional research with a diverse group of users to better understand their needs
-
consider what the user journey is for this service and not just the portion of the service shown in the demo. This includes touchpoints before the service within the Money and Pensions Service and how people may find the service through other means
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard
What the team has done well
The panel was impressed that:
-
the team outlined how they had looked at analytics from the current live service and received feedback from users, allowing the team to identify the majority of users complete the Pension Wise journey using a computer, rather than a mobile device
-
the team has a plan to understand how online, offline, and assisted digital routes should work for their users
-
the team has thought about the awareness of the service and the profiles of the main types of users
-
the team mentioned their assumptions about the service
-
when asked about other channels, the team understood the limitations of using tools like chatbots to achieve the same goal as the service. But they are considering how that technology can be used for other means of supporting users
What the team needs to explore
Before the next assessment, the team needs to:
-
consider how the save and continue and returning user journey works and ensure this is tested with users. It wasn’t demoed and it was unclear what this journey would be like for users, i.e., creating an account, signing in, and continuing where they left off
-
understand the benefits and effects of the digital solution on contact centre volumes, including whether the service does support those who need support the most
-
routinely ask users of the service and the offline journey during research what their preferred device would be. Any service created should also be tested on mobile devices
-
identify users with accessibility needs to conduct research with. The panel recommends the team reach out to digital teams that also have users who are looking at pension information (HMRC) or pension decisions. Large government departments, such as HMRC have user research panels, where users sign up through the digital services to be part of research. The team should endeavour to contact the Research Operations team to investigate whether it’s possible to identify access needs users. If that is not possible, they may have alternative recruitment approaches to recommend that is not an agency
-
map out user journeys for multiple different user types (for example first time user, return user, user returning halfway through submission). The team has considered what the legislation needs a user to understand, but not enough about what it means in the short and long term for users
-
ensure users see the option for a telephone appointment throughout their journey, as they may feel overwhelmed with the process and would like to switch to an offline journey and speak with a person instead
4. Make the service simple to use
Decision
The service met point 4 of the Standard
What the team has done well
The panel was impressed that:
-
the user journey itself was simple in terms of journey and users could complete tasks in any order they chose
-
the team used affordance and established patterns such as the task list to enable the user to clearly understand what they could and could not do yet
What the team needs to explore
Before the next assessment, the team needs to:
- consider providing clearer guidance for users as to whether they’d need documents and information available or need additional support to complete the service. This is particularly important for users with memory or cognitive impairments who may forget why they are using the service or think they need to have precise details of pensions available to them to complete the service
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard
What the team has done well
The panel was impressed that:
-
the team considered the user journey using GOV patterns and it’s consistent with the rest of the service
-
the team considered the devices people will use, and they have engaged with the Plain English Commission for Plain Language standard
What the team needs to explore
Before the next assessment, the team needs to:
- revisit the design and test with users to better understand how the design patterns perform as some of the design patterns were confusing. This includes the use of the down arrows on the appointment options page, and the accordion page later in the journey. They are similar in design, but different in interaction
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard
What the team has done well
The panel was impressed that:
- the team is comprised of most of the roles you’d expect to see, including User Researchers, Service Designer, Content Designer, Interaction Designers, Performance Analyst, and technical roles
What the team needs to explore
Before their next assessment, the team needs to:
-
continue to work with a fully resourced team
-
consider whether they have enough user research resource and ensure they are bringing in enough support/expertise in at the right time in private beta to implement the recommendations in service standard point one (understand users and their user needs)
7. Use agile ways of working
Decision
The service met point 7 of the Standard
What the team has done well
The panel was impressed that:
-
the team has used a Scrum agile approach
-
the team has held all the expected ceremonies, such as sprint reviews, daily stand ups, retros, show and tells
-
the team has used ADO for backlogs and Kanban and Miro boards for collaboration
What the team needs to explore
Before the next assessment, the team needs to:
- continue to implement good practice agile ways of working with a multidisciplinary team
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard
What the team has done well
The panel was impressed that:
- the team has looked at several approaches to the service, and has iterated through several options for the service
What the team needs to explore
Before their next assessment, the team needs to:
- consider the feedback in points 1, 2, 3, 4, and 5 as they progress with the design and development of the service
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard
What the team has done well
The panel was impressed that:
-
the team has engaged fully with the relevant data protection processes within their organisation and are clear on when future iterations may involve revisiting document such as their data protection impact assessment
-
the team has good awareness of what data they are collecting, whether it currently constitutes personally identifiable information (PII) and what implications that has for how they can use that data
-
the team has made a plan for security testing their service with an external provider and have a good awareness of the particular risks of the technology they have chosen to use
-
an appropriate approach to “cookie consent” is being developed, which will allow users to opt out of unnecessary cookies
What the team needs to explore
Before the next assessment, the team needs to:
-
investigate whether allowing users to opt out of cookies entirely is overly permissive and confusing to users who would usually expect necessary cookies to be a minimum level, where they can still access the main of functionality of a service.
-
understand the data privacy implications of integrating with a hub service in the future. As this will centralise data for use across multiple different MaPS applications, it needs to be clear to users how their data will be shared across these services and informed consent given
-
determine an appropriate level of authentication for providing a save and return pattern, which is proportional to the sensitivity of data that is being stored. For example, whilst a magic link approach may be appropriate initially, as the service asks for further information it may be more important to use a username, password and two-factor authentication approach instead. Ideally, the approach should be consistent with other MaPS services / products
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard
What the team has done well
The panel was impressed that:
-
the team has a plan in place to measure three of the four mandatory KPIs (user satisfaction, completion rate, and digital take up), and is working toward being able to provide the information for the fourth mandatory KPI (cost per transaction)
-
the team will be using an analytics programme to measure success with user activity such as time spent on a page, time to complete the journey, and where users drop out of the journey to help inform them of the success of the service
What the team needs to explore
Before the next assessment, the team needs to:
-
have plans in place to measure the fourth mandatory KPI (cost per transaction) for the private beta phase
-
for the mandatory KPI of digital take up, the team plan to use the existing service of Explore your Options as the baseline. The team may wish to consider the offline telephone appointment journey as a measurement baseline as it will cover the same information as the online appointment journey
-
reconsider what a completed journey looks like. The team has stated the user journey is considered complete only when the citizen downloads the output of the online appointment. Not all users who step through the entire journey and review the suggestions will decide to download the information provided. Those users will have completed the journey, but it won’t be recorded as a complete journey
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard
What the team has done well
The panel was impressed that:
-
the need for ease of editing content without specialist technical knowledge has been identified as a key requirement when choosing the technology to base their service on
-
the team has a good understanding of the wider IT landscape in the organisation and has identified a suitable technology given the organisational constraints
-
the team has engaged well with architectural oversight both from within their organisation and from Cabinet Office, to validate their technology choices. They also have a good awareness of the Technology Code of Practice and how their approach adheres to it
What the team needs to explore
Before the next assessment, the team needs to:
- demonstrate that their initial choice of technology has remained appropriate considering the findings of their user research and any design changes made as a consequence, and that they haven’t continued with their current choice simply due to “sunk cost”
12. Make new source code open
Decision
The service met point 12 of the Standard
What the team has done well
The panel was impressed that:
- the team has committed to releasing reusable code artefacts in a publicly available, open-source repository to allow them to be reused where appropriate
What the team needs to explore
Before the next assessment, the team needs to:
- identify what code artefacts can be made public under appropriate open-source license to use, make them publicly available and ensure these are updated regularly as they are updated in the service. Ideally, the publicly accessible repository should be the only source for this code, rather than being a “mirror” of an internal repository
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard
What the team has done well
The panel was impressed that:
-
the team is focussing on providing a reusable set of visual components that can be used to build and iterate the service without the need for specialist technical skills. These have been iterated as the team have learned from their user research
-
the intended technology choice makes use of open-source software that implements open standards for storing content, as well as the packaging and running of software artefacts
What the team needs to explore
Before the next assessment, the team needs to:
- demonstrate how they are sharing their design and research findings with the wider organisation and across government, especially novel patterns that they are establishing around how to increase user engagement with large amounts of content
14. Operate a reliable service
Decision
The service met point 14 of the Standard
What the team has done well
The panel was impressed that:
-
the team has identified technology choices that will allow them to release changes to users with little-to-no downtime
-
the team has a good awareness of what monitoring and alerting tooling they will need to operate a live service, and have a plan around how this will fit into the incident management process within the wider organisation
-
the team has a good understanding of relevant testing practices they need to apply to ensure their service is functional. They have applied automation where appropriate, including for deployments of changes through test environments, which reduces the risk of manual errors
What the team needs to explore
Before the next assessment, the team needs to:
-
investigate the possibility of using automated accessibility and security testing, to reduce the number of issues identified by manual audit processes outside of the team and reduce the risk of bugs being introduced by changes perceived “too small” to follow these
-
develop a clear plan for what steps would be taken were the service unavailable for hours, days and weeks, and test any recovery process that would be applied
Next Steps
This service can now move into a private beta phase, subject to implementing the recommendations outlined in the report and getting approval from the CDDO team. The service must pass their beta assessment before launching their public beta.
To get the service ready to launch on GOV.UK the team needs to:
-
get a GOV.UK service domain name
-
work with the GOV.UK content team on any changes required to GOV.UK content