Manage your student loan balance beta assessment report
The report of SLC's ‘Manage your student loan balance’ beta service assessment on the 21st of August 2019.
| From: | Central Digital and Data Office |
| Assessment date: | 21/08/2019 |
| Stage: | Beta |
| Result: | Met |
| Service provider: | Student Loans Company |
Service description
The ‘Manage your student loan balance’ service will allow customers at all stages of their student loan journey to view their balance and make repayments towards their loan. They will also be able to update their contact details and view any correspondence sent out.
Service users
Mainly customers who have completed their study but we’ll be promoting the service to those before they apply for university and whilst they are at study.
1. Understand user needs
Decision
The service met point 1 of the Standard.
What the team has done well
The panel was impressed that:
- the team has carried research through their Discovery into the BETA. The user needs the team wrote have been iterated and they were very visible on the wall during the Alpha. The team has a really good knowledge of these needs and has done regular research with assisted digital users
- the team has written personas for the service. These are based on people in repayment, both domestic where repayments come out of pay packet, and overseas students, where the onus is on the user to make payment; then students thinking about their growing balance and graduates - not thinking about paying off, but aware of the loan. And finally people not repaying. The panel was impressed with the knowledge the whole team has of their users
- the team has thought about how to include users without internet access, those with limited access, and those who lack confidence online, and done research with these groups. They’ve added the contact us option to support. This shows how the team is thinking about how the service can meet the needs of people with low digital confidence
- the team has evolved its understanding of people working overseas who need to pay off their student loan manually. This has been a particular area of learning since Alpha and the panel was impressed by the knowledge the team has built up
- the team is building out the UX team to look at the internal users. They sit down with internal users and do research with them. It’s part of a bigger strategy at SLC which will continue. It is really great that the team are thinking about internal users as users of the service and will be thinking more about their needs in the future
- in terms of needs they found hard to meet, the team said it had been hard to meet the need for people to make the right decisions. People want to make an informed decision, but it’s quite hard to support this given that the service can’t give advice. It’s great that the team are aware of this problem and are trying to meet the need within the constraints of the service
- the team addressed accessibility recommendations from accessibility audit and has a mechanism in place to monitor and address any accessibility concerns
What the team needs to explore
Before their next assessment, the team needs to:
- demonstrate that the team has been doing research to test that the service performs well for the full range of needs that they’ve identified in research. At the moment the service has been designed and researched to work for the top user needs. So the team should be ensuring they are meeting all user needs they’ve identified by testing how well the service meets each user need
- keep updating their understanding of their users and their needs. Keep exploring how well the service meets the needs and testing edge cases should provide new insights about what users need from the service. The team should demonstrate how they’ve developed their understanding of these user needs
- provide evidence that the planned ways of getting users to the service is meeting user needs
- continue research with tasks which users find difficult and demonstrate that they’ve continued accessibility and assisted digital research, particularly on any changes to the service. Demonstrate how the support the team designed is meeting the needs of users who require it
- demonstrate that the team has tested more edge cases. Such as different repayment scenarios, and scenarios when the information might not be up to date, and what happens when things go wrong (like the data link with HMRC etc)
- show how they’re doing research based on real-life situations users find themselves in by giving participants scenarios which are realistic to them. The team should try to understand how well the service meets the needs of people in these different situations
- show how the research has used real or realistic data in their research to get a better understanding of how well the service is meeting user needs. The research to date hasn’t yet prioritised this
2. Do ongoing user research
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the team has run a private beta with 2,500 users. They were recruited by notifying a group of existing users who signed up. the team has collected in-service feedback, support and they have analysed Google Analytics data. None of this pointed to areas of concern with the service
- the team has done accessibility research with a range of users but hasn’t found research that would suggests they’re not meeting the needs of people with physical or cognitive impairments. The team also did research with people with a range of assisted digital needs
- the team’s knowledge of people working overseas is an example of where the team has specifically looked to support a need which is not well served by the existing service
- the team gave evidence of a research finding from assisted digital research: when people were tabbing through, they wanted to be able to go to the main piece of content to update contact details, but it kept jumping to the top of the page throughout the journey. Also having to tab through the service - the worldpay service - they missed the next field. There are problems with autotab which are being addressed. It’s great that the team has fixed this in the service and are working with Worldpay to improve the payment experience for assisted digital users
- in the private BETA the team has done usability research in Scotland and London. They’ve recruited people from each of the personas
- the team talked about logging in as a sometimes difficult task. “I need my CRN (Customer Reference number) - looking through old correspondence. I can use an email.” Some people do just call to reset all their information. If they can’t remember they’ve been asking in research “do you know what you’d put in here” and have passed findings to the login team which has informed improvements
- the team has a lot of call information and done analysis of this. They introduced the “contact us” which means they can support people in this task. They worked a lot with internal users. This was a problem identified from speaking to support staff. Now when a customer calls and they identify that they’re an assisted digital they can support them
- the team has made significant changes including improvements to the Balance summary, which has been iterated based on feedback. An early change was to add a link to information about repayments and show pending repayments, which meets the need of being able to see money paid in
- the team has added a feature allowing users to make a quick repayment. This feature was added following research with overseas users. And the panel was really impressed with the teams thinking about how to better meet this user need. The team has added a new payment option through a partnership with Flywire - which provides payments without any additional fees. This supports someone who wants to repay monthly
- the team added content to explain what repayments are displayed and why. For UK customers it displays and explains the UK tax year but for overseas customers that doesn’t make sense as a start date. Overseas customers get a repayment schedule which references when people complete the form to declare overseas earnings. They have to submit their details every year. The panel was impressed with how the team are thinking about how meeting the needs requires a different design depending on the situation the user is in
- the team has tested how well users understand that interest can change based on income. When usability testing the table of interest rates it wasn’t clear for users what interest they pay if earning under £25k. The table was unclear, so the team added £25k and under. A good example of the way the team has iterated the design of the service based on research
- the panel were impressed with the team thinking about a range of channels to communicate with users. Research highlighted a problem with correspondence via letter from Student Loans Company as it can sometimes be slow. A new feature was to also send an email is currently in progress
The panel were really impressed with everything the team has done and plan to do to get users to the service. This includes:
- from Alpha into Beta the team has been looking at the wider service. They’re working with Partner services in schools and colleges to link to the service from the student account. This will introduce them to repayments (they don’t need to re-login). Trying to reach people in university. For people in repayment the team understand the search terms to be: Moving overseas - what do I need to tell the UK. They’ve updated content elsewhere. The team are in discussions for a link from personal tax account. For self-assessment - thinking about their repayment it’s the tax side which springs to mind first for users
- the panel was impressed with the team’s consideration of the wider user journey. For example, some graduates have a different perspective of who they’re repaying to. There’s a link from the student account to this account. The team found that people don’t search for things related to: planning to move overseas. This has helped them think about how and where to tell people about this service. They will be doing a lot of advertising and campaigns so that customers can easily find the service. One message that has really hit home is that there’s a threshold for starting to pay
- another way the team is planning to address this is step by step guides. Such as get student finance, when users are at the start of their journey. The fact that the team is considering where their service fits with this wider journey is also great to see
- another way the team is planning to address this is in the student finance portal. The team plan to expand the link into student finance. The team has learned that people who have been repaying for a while are more focused on Student Loans Company, and will search for things related to this. Whereas people earlier on in the journey search for things related to student finance. One student account for all this is out of scope for now
What the team needs to explore
Before their next assessment, the team needs to:
- have done regular research and iterative testing on the service
- have done more exploration into the tasks which users find hard to complete and needs which the service finds hard to meet and considered how the service could be improved to support users in these situations
- demonstrate how the team are using analytics data to help assess how well the service is meeting user needs, and how well it’s performing for all the tasks which users can do
- provide examples of problems the team has found in research and how the service has addressed those problems
- provide evidence of how well all the tasks a user can do with the service perform. The team have focused most of its efforts to date on researching the primary tasks of understanding balance, understanding interest and making a repayment. But more evidence should be provided about how these and other tasks are performing
3. Have a multidisciplinary team
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the service had a well-formed team, comprising all the relevant roles for the given phase, including external and permanent staff
What the team needs to explore
Before their next assessment, the team needs to:
- ensure that a multidisciplinary team remains in place, past the launch of public beta, to continue to learn from how users use the service to meet their needs, and to iterate the service to meet those needs
4. Use agile methods
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that:
- the team demonstrated key agile practices. The service is being delivered in sprints with the key ceremonies present, including ‘3 amigos’, stand ups, showcases etc
What the team needs to explore
Before their next assessment, the team needs to:
- maintain agile practices as the constitution of the team changes in public beta and beyond
5. Iterate and improve frequently
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that:
- the service team had done excellent work iterating aspects of the service as outline above under point 2 (the Balance summary page stands out)
- since the previous assessment workshop, significant progress has been made iterating the key features of the service in a way that demonstrates some of the users needs being met
What the team needs to explore
Before their next assessment, the team needs to:
- continue to return to the user needs, remaining clear when certain needs are out of scope, and also maintaining a clear view of how each of the identified needs are met
- the utilisation of weekly HMRC data can be expected to have a major impact on the service. The service needs to be able to understand how this potentially changes the way people use the service, and to respond to that where needed
6. Evaluate tools and systems
Decision
The service met point 6 of the Standard.
What the team has done well:
- the project team has decided to use Java with Spring Boot for the development of the application and use IntelliJ for their integrated development environment (IDE)
- the already existing Oracle DB will be interacted via ORDS while partially completed transactions and new data from the service to be stored in Mongo
- Rabbit MQ will be used to queue requests if Oracle is down
- the team uses Thoughtworks products such as GOCD for the continuous integration and Mingle for the backlog management of stories
- SLC as an organisation uses Atlassian products elsewhere e.g. other teams use Jenkins for CI. Other products such as Confluence are used in the organisation
7. Understand security and privacy issues
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- N.B. This evaluation criteria was already met in previous assessment
8. Make all new source code open
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the team now has a strategy for moving towards coding in the open
What the team needs to explore
Before their next assessment, the team needs to:
- demonstrate how they have successfully made progress in executing their coding in the open strategy
9. Use open standards and common platforms
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the team now has a strategy for moving towards cloud migration
What the team needs to explore
Before their next assessment, the team needs to:
- demonstrate how they have successfully made progress in executing their cloud migration strategy
10. Test the end-to-end service
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- N.B. This evaluation criteria was already met in previous assessment
11. Make a plan for being offline
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- N.B. This evaluation criteria was already met in previous assessment. SLC have a call centre available to handle queries in case the website goes offline. The call centre also assists in taking card payments
12. Make sure users succeed first time
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the service in general seems simple, accessible and straightforward with good evidence from private Beta that users can complete their tasks
- the team is iterating significantly on the design of the service based on user research. An excellent example is the feature to allow users to make some payments without having to sign in first
- the team is working on other touch points for their user’s journeys, for example adding a link to the GOV.UK Moving or retiring abroad guide
What the team needs to explore
Before their next assessment, the team needs to:
- look further into the issue around users trying to use credit cards to make payment. If it’s not legally possible to allow them, try to find a simpler and quicker solution than the current approach which has multiple steps and affects all users even if they’re not trying to use a credit card
- look into making the language as clear and simple as possible, for example there might be a simpler alternative for ‘correspondence’
- continue to look into meeting user needs wherever they might be - for example we heard about discussions with HMRC to add content or links to the tax service, which sounds good
- make sure that the navigation bar on the right side is the best approach. There is a risk some users might miss it there, especially people using magnifiers. It might work better on the left, or as a navigation menu at the top, for example
- the ‘Understanding repayment’ link now has a better name, thanks to your research. However it might be worth looking into now moving it to be closer to the Repayment section, rather than the Balance section of the page
- it’s great that the team has uncovered Payment as a possible separate need to signing in to view the balance. However the team should make sure that this and any other separate needs you uncover can be found from search engines. This may mean separate Start pages
13. Make the user experience consistent with GOV.UK
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the service is simple and clear with good use of standard GOV.UK design patterns
What the team needs to explore
Before their next assessment, the team needs to:
- look into some minor points, for example numbers in tables should normally be aligned to the right
14. Encourage everyone to use the digital service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- research undertaken included people with low digital skills and support needs. Including feedback from support channels
- support channels are in place
What the team needs to explore
Before their next assessment, the team needs to:
- ensure support remains adequate across all user groups, and iterated based on feedback where needed
15. Collect performance data
Decision
The service met point 15 of the Standard.
What the team has done well
The panel was impressed that:
- performance data for users making direct payments, including from overseas, has been established and trialled
- data sources including online analytics and call classifications are being used, including potential improvements to the breakdown of reasons for phone calls
What the team needs to explore
Before their next assessment, the team needs to:
- distinguish between different user needs, and the performance of the service in meeting each of them. The need to make direct payments is important, but only for a minority of users. A broader set of performance data might cover: users needing to make payments; users close to repaying in full via HMRC who want to avoid overpaying; users updating their contact details; and users wanting to check or understand their recent repayments or the interest they are being charged. Performance in these areas might best be measured by changes in the volume of users resorting to phone calls, especially as the team knows their users are, as a group, digitally capable
16. Identify performance indicators
Decision
The service met point 16 of the Standard.
What the team has done well
The panel was impressed that:
- indicators have been developed to measure the success of users wishing to make direct payments
What the team needs to explore
Before their next assessment, the team needs to:
- see point about further user needs under criteria 15
17. Report performance data on the Performance Platform
Decision
The service met point 17 of the Standard.
What the team has done well
The panel was impressed that:
- the team has already engaged with performance platform experts to make sure they can report on the indicators for users wishing to make direct payments
What the team needs to explore
Before their next assessment, the team needs to:
- see point about further user needs under criteria 15
18. Test with the minister
Decision
The service met point 18 of the Standard.
What the team has done well
The panel was impressed that:
- the service has remained visible to the Senior Responsible Owner
What the team needs to explore
Before their next assessment, the team needs to:
- N/A