Manage and Register Pension Schemes - alpha
The report from the alpha assessment for HMRC's Manage and Register Pension Schemes service on 17 April 2018.
| From: | Central Digital and Data Office |
| Assessment date: | 17 April 2018 |
| Stage: | Alpha |
| Result: | Not Met |
| Service provider: | HMRC |
To meet the Standard the service should:
- Conduct additional user research and create “behaviour-driven” user personas to ensure that the service meets user needs.
- To ensure that users can meet their needs quickly and succeed first time, the service team should redesign this as two separate services, making it more intuitive for users.
- Show how user research has driven the technical decision making process so that the technical approach for Beta is evidenced by user needs.
About the service
Description
The service enables individuals and organisations to register as a Pension Scheme Administrator (PSA), and to manage pension schemes. It replaces one part of a service already in place, known as TPSS.
Service users
The users of this service are Pension Scheme Administrators, which may be SME or self-employed users, or large organisations with many schemes to manage.
Detail
User needs
The team have spoken to a reasonable amount of users, focusing on in-depth interviews and testing iterations of the prototype. However, the panel had a number of concerns about the way users were identified, and the breadth of insight they gained as a result.
They spoke about changes they had made to the content as they tested to encourage users to complete the required details, showing a good understanding of their behaviours at some points of the journey.
There are few areas where research needs to be focussed, so the team have assurance they have made the right design decisions, and that they thoroughly understand the users needs, motivations and behaviours. This includes; focussing on the data that is being captured during the registration process, and being clear if the user understands why this is being ask for, and, understanding the issues they face in collecting it, and how this may affect their journey.
Included in this is researching any inform or start pages, to understand if users can find the service and understand what they need to provide before they start.
The team didn’t clearly articulate an understanding of when users need to leave the journey, taking things away to refer to colleagues for example, and how the service could support that. The service team also described how some users were concerned about the wording used in declarations, being unsure if they should submit, more compelling evidence should be sought to understand this further.
Through research, the team identified that public sector users would have different needs and a different journey. The team described a “work-around” in order to manage this, but the panel were disappointed to learn this. The team must concentrate additional effort on understanding the user needs of the public sector and amend user journeys accordingly.
Personas should be clearly segmented by behaviour, goals and attitudes that are not tied to the solution. Some of the presented needs felt like functional requirements.
A test for a user need is whether an actual user would recognise it as something they would say, whether it helps to prioritise backlog and make decisions, describes behaviour not a solution, and if they would still be a need if the policy or technology changes.
The panel would encourage the team to actively look for users to test with, the panel felt that the service team were utilising a “friendly pool” which will be quickly exhausted. The panel were concerned that users conducting testing may have ‘learnt’ the service, giving a biased view.
The panel encourage the team to continue to shadow the pension support team as a way to inform the direction of research, and to access users who may have Assisted Digital (AD) needs.
The team must focus on finding these users; if the service is designed for these users, the service should automatically support the needs of all users. The panel encourage the team to think about what their assisted digital triage process may look like, including any offline support, and start to work with whoever will provide this to make sure they can differentiate between AD and general queries.
The panel were extremely disappointed to learn that the service team had not yet found any users with accessibility needs and planned to test with users of this type only at Beta stage.
The service team spoke about using a charity to find users with access needs and the panel support this approach. In addition, the team spoke about reaching out to the network of people they have already researched with and to look for people in their organisation with accessibility needs. They should consider using a recruitment agent to support brining in not only users who may have accessibility needs, but also assisted digital users.
Team
The team demonstrated that they have the roles the panel would expect for a service in Alpha.
The team have 50% of a Content Designer who works on another, similar service, the panel recommend that this is increased to full time in order to be able to properly address issues around declarations and page information in the Beta phase. The team should also consider whether they would benefit from addition of an Assisted Digital lead.
The team advised that they plan to make no changes to the team as they move into Beta apart from replacement of Lead Developer who will be managing a 1:1 handover to a replacement.
The team is currently made up of a mixture of civil servants and contractors, and should consider now what impact this may have on delivery and ongoing iterative improvement to the service as it moves into Beta and Live.
Policy colleagues attend sprint reviews though the service team described some issues with having a geographically disparate team. The team spoke compellingly about the relationship between policy and content designer who are working well together. The panel described designing specialised one-off workshops on particular topics to improve relationships and understanding with their policy colleagues.
However, the service includes requests for lots of information for risking purposes and the service team could not clearly articulate the need for that information, this suggests that more work should be done with policy and risk colleagues to understand how their requirements may affect user experience.
The team run agile ceremonies and the full team are involved in sprint planning but did not provide detail on how work was prioritised as a team. The service is currently well governed with weekly checkpoints, review boards every two weeks, and a risk-based monthly board. The team meet with senior policy stakeholders every two weeks.
Technology
The team provided the assessment panel with access to their existing prototype which showed that a number of iterations have been trialled throughout the Alpha stage. However, the assessment panel were extremely disappointed to see that significant technical development had already taken place which we would not expect to see at Alpha assessment.
Alpha phase should be used by the service team to understand user needs and make informed technical decisions to inform a plan for development at Beta. Alpha prototypes and any development work should be “just enough” to test hypothesis rather than run a full transaction.
The assessment panel were not clear what user needs had driven technical decisions from what was presented at the assessment, and the service team did not present any other distinct options for technical delivery that they had considered and discounted.
The assessment panel are concerned that there are likely to be significant risks to delivery as the technical decisions made to date may not meet user needs. It may be difficult to change the approach based on emerging user needs which could lead to increased costs in the longer term.
The Alpha service assessment also provides the service team with a good opportunity to review their proposed approach for development with the assessment panel, who may be able to provide additional support, or advice about the approach. By starting technical development ahead of assessment the service team have missed out on a valuable opportunity to review this with peers before committing to an approach.
The service team do appear to have made some good technical decisions, for example, the team are coding in the open, publishing to GitHub, and using appropriate open-source tools to manage planning and collaboration.
A separate note on the technology choices made will be provided by the panel, separate to this Alpha report.
Design
The service is moving from a current online service to a new service, with the move staggered over several years. The two transactions that are changing soon are “register as a pension service administrator” and “register a new pension scheme with HMRC”. These will not need to be used by the majority of users until data has been transferred from the old system and the new service becomes the main point of transaction for users (in 2019). This must be highlighted to users in all touch points so they do not feel they have to re-register now - a map of all start pages, guidance pages and other communications will help manage this.
These two transactions should be built as separate transactions with different names, as they support very different user needs (and may be being used by different people within a company). This would remove the need for grey sub-heading text on every page to orient the user (this style is also depreciated on GOV.UK). The start pages need to be clear about what information a user will need to collect to complete the transaction.
The hardest task is getting users to understand what each of the old and new services does, and which should be used for what during the transition period. There needs to be more prototyping, content design and research on the guidance and start pages for the different transactions, the movement from the old service to the new and vice versa. This should be worked through with the HMRC publishing team and GOV.UK, as there is a current start page for the old service and many guidance pages.
It is good to see that the same Government Gateway login can be used in the two services, and that users will not have to re-login to use the new and old services when moving between them.
It is good to see the “check your answers” pattern being used in the service prototype. This should also be used on the review pension scheme page.
There is no clear ending for the “register a new pension scheme” transaction. The team mentioned a page detailing where in the process each scheme was mentioned but this did not appear to be prototyped and iterated.
Both transactions currently ask for a lot of personal information. The team mentioned that this is for risking purposes, but that the risking process is a black box and is not understood by the service team. It was also mentioned that risk assessors often ask for more information by letter during the process. The team must understand what information is actually needed for these transactions and only ask for the information that is required. It is unacceptable to ask for information that may be useful or interesting at a later time, or by a different service or the department in general.
The service is also putting a burden on individuals in companies to source and store personal information from company directors, and it is not currently clear that that is acceptable.
The service should not require users to provide information to Government multiple times. Company director information is stored in Companies House and available via API. This should be used in the service, with users checking information and adding any additional required information (such as NI number).
Information collected by the service should also flow through to other parties in Government, such as the Pensions Regulator. The team mentioned that there had been discussions about this, but there needs to be a good reason why this can’t happen now.
One of the objectives of renewing the service is to make clearer and more enforceable the declarations and assessment of being “fit and proper” to run a pension scheme. The current wording of the declarations is unclear and lengthy, and are likely to be agreed to without the user understanding what they are declaring. It is also unclear who the user is declaring for, given that the login is often per company rather than per individual in a company. The content designer should pair write the declarations with policy and legal teams so there is joint ownership and fulfills the objective.
Analytics
The team have recently brought in a Performance Analyst to help them in this area. They are using a range of data sources to inform their work in this area; Google Analytics from the existing service, GOV.UK page information, information from the customer contact centre team. The team were confident about publishing to the Performance Platform and were in early discussions with the GOV.UK Performance Platform team.
The team were at early stages of designing additional KPIs which will enable them to iterate and improve the service including; the number of users, number of “successful” completions, number of saved and returned registrations, and error messages. The team have identified limitations in the data they have been able to get from the existing service and plan to use this to design KPIs for the new service.
Recommendations
To pass the next assessment, the service team must:
- Consider using a recruitment agent for accessing all types of users (access needs, Assisted Digital, normal users)
-
Increase numbers - use mix of quantitative and qualitative (use call centre data or surveys to help direct qualitative research, or use the quantitative to validate something that has been found in qualitative research)
- Focus additional research on public sector organisations, to understand the public sector organisations’ journey
- Develop a more concrete understanding of when users need to leave the journey and take things away, including who they refer things to, and how the service could support that
-
Reframe personas to be behavioural based, clearly segmented by behaviour, goals and attitudes that are not tied to the solution
- Map out Assisted Digital users triage based on the existing support models of users, and when they will require support from HMRC
- Undertake research focussing on the emails or letters sent to users, and their needs from these
- Have a map of entry points, guidance and start pages for the different transactions and movement between them, with clear content design so users know what they need to use
- Design the two transactions as standalone transactions meeting different user needs
- Understand the risking process on each transaction, working with case officers, so that the minimum amount of information can be collected
- Complete the register a pension scheme transaction, including summary pages whilst the case is being reviewed and notifications sent during the process
- Prototype using information gathered from HMRC and Companies House sources so that users do not have to re-enter information that is already known
- Rework the declarations within the two transactions, so they are consistent with GOV.UK style and also legally enforceable
- Map out the traceability of the end technical solution to the specified user needs so that it is clearer for whom the platform is serving
The service team should also:
- Work with the HMRC publishing team and GOV.UK content designers to decide on a strategy for getting users to the right transaction during this transition period, making it clear that most users will not have to re-register
- Work with the Pensions Regulator to ensure information does not have to be provided twice
- Have enough content designers in the team to make sure the transaction content is written to GOV.UK style and that questions and declarations are understandable, and that the guidance pages point users to the right transactions for their needs
Next Steps
In order for the service to continue to the next phase of development it must meet the Standard. The service must be re-assessed against the criteria not met at this assessment.
Please contact the Service Assessment team at least 4 weeks before the date you’d like to hold a reassessment.
Submit feedback
Submit feedback about your assessment.
Get advice and guidance
The team can get advice and guidance on the next stage of development by:
- searching the Government Service Design Manual
- asking the cross-government slack community
- contacting the Service Assessment team