Digital First Careers - Alpha Assessment
The report from the alpha assessment for Digital First Careers service on 30 January 2017.
| Stage | Alpha |
| Result | Met |
| Service provider | Department for Education |
The service met the Standard because:
-
The team has clearly done user research with a variety of users and identified a clear range of user needs that aren’t being met well at the moment.
-
The team has designed a number of prototypes and iterated on the design to find one that works to meet some of the demonstrated user needs
-
The team has done the technical work needed to understand the risks for technical integration during the beta and is aware of the technical work ahead
About the service
Service Manager: Angela Scale
Digital Leader: Alan Parnum
Description
The service provides a simple and clear way for citizens to understand career options available to them.
Service users
The users of this service fall in a few common groups, including: Young people embarking on a career for the first time; People returning to work after a career break and seeking a new career; People who have been made redundant and seek an alternative career.
Detail
Lead Assessor: Michael Brunton-Spall
User needs
The team have a good understanding of the primary user need - people searching for high quality, impartial information about career options. They have gained this through a good range of research activities in a variety of locations.
However, the panel is concerned that this user need is part of a long journey for the user from inspirational advice around careers in general all the way through to applying for a job, apprenticeship or enabling training course.
The edges of this service are difficult to determine, and as such the scope of the service is very hard to define. The team were unable to clearly outline the scope of where the user journey would end, and which specific needs were being addressed.
In particular, we noted that the prototype demonstrated a clear jump from browsing and finding information about a potential career into a CV building or profile building functionality. The panel does not believe that the same user needs are being met in that point of the transaction and that there are other alternatives, such as applying for a course, apprenticeship, traineeship or locating a job.
The team have gained a good understanding of the needs of people who may need support to use the service, including people who may have good digital skills, but are in emotionally difficult circumstances and lack confidence in their broader job finding skills.
During beta the team need to do more to understand how users who need support will move between digital, face to face and phone.
The core of the service will be the job profiles. Keeping hundreds of job profiles high quality, up to date, and building on them to meet expanding user needs (the team mentioned ‘youtuber’) will be a huge task. It wasn’t clear how this will be achieved. The current job profiles seem to suffer from being outdated in many places.
The team mentioned improved content management tools, but this is only part of the story - there needs to a be a clear strategy to gather new job profiles and produce accurate trusted content.
Many users will start with a google search for their particular job profile, and the current service does not seem to perform well for all of the searches we’ve tried. Again this will need a solid strategy to meet needs.
We strongly recommend that the team conduct scoping exercises to ensure that they are delivering an understandable minimum viable service in the beta.
In some areas the team describe a possible solution as a user need - for example, the ‘need for imagery’. In that case the need might be to “understand what it would be like to do a job” and an image or video is a potential solution to meet that need. The team will produce a better solution if it more clearly separates their understanding of users and their needs, from their ideas for potential solutions.
Some users will need to report their use of the services - example to show that they are meeting their Universal Credit Claimant Commitment. Through beta the team should learn more about this need and how they can support it.
The team have started learning about users with access needs, but need a clearer plan to discover and remove barriers during beta.
Team
The team during the alpha was clearly very strongly user research and prototype development focused, with multiple user researchers, interaction designers and the alpha outputs showed the great effort put in here.
In particular, the team had done an excellent job of finding and interviewing candidates and understanding their needs, and had done great work in trying a variety of prototypes, various design decisions and determining which worked and which didn’t. The team had discarded ideas that seemed interesting but didn’t hold up under testing, which impressed the panel.
There was much less involvement from technology specialists, but still the team had the ability to create some spikes to prove the feasibility of the CMS system and the data extraction capabilities.
The team clearly worked well and in an iterative manner, regularly involving the team in the user research and iterating the designs based on the feedback gathered.
Technology
The code produced for the alpha comprises a prototype built on Heroku that will be retired and not constitute the technical basis for the beta codebase. The team has also taken several opportunities during the alpha period to spike anticipated integrations to appropriately de-risk beta architecture and technology choices, as well as isolate themselves sensibly from internal dependencies like NCS’ Content Management system.
The team also works closely with those maintaining the interim website containing similar information that is already linked from GOV.UK. This has allowed the team to further de-risk the beta through a strong understanding of current use and traffic patterns which will inform architectural choices and performance/load testing.
The team is yet to decide how much (if any) of the transactional ‘skills profile’ journey will be needed to satisfy user needs.
This decision this will have a significant technical impact on the beta, as the skills profile forms may collect sensitive personal information, including health conditions and/or disabilities in save-and-return scenarios. If the team decides this is necessary, the panel advises follow-up with GDS, NCSC and/or CESG to share best practice on how to manage storage and user identity in these scenarios, and engagement with the GOV.UK Verify team on user identity in LOA 1.5 scenarios.
The team should weigh risks of this significant complexity against their genuine user needs at MVP. It may be acceptable to users and lower risk, technically, if they were to deliver the careers information content site first, and the transactional skills profile elements later. This would lead to a shorter beta build, and quicker more useful feedback from more real users. Alternatively, if the skills profile is genuinely identified as a strong requirement, work should be prioritised to solve these challenges early on in the build.
The team is currently planning to use the same release process as related teams on the interim site. This involves consideration from webops, regression testing, and approval from the product owner before deployment to production which can take some time to complete. The panel suggests this level of release control is unnecessary for beta, and would suggest seeking to streamline it until the live phase so work can progress more quickly.
The team is working in the open on GitHub and will continue to do so during the beta. The alpha prototypes will be entirely open sourced and the designers have also engaged with the cross-government community through channels including the hackpad.
Design
The prototype makes good use of GOV.UK design patterns to present information clearly, and the consistency with GOV.UK to give users confidence that this is the official source - a need identified by the team.
It was good to see clear iteration of previous approaches, and research to show why they didn’t work for users. However, it’s not clear that the current design really meets user needs.
-
There’s no way to browse from one profile to others
-
The search hasn’t been tested with users, even in a mocked-up state with some different search terms working
-
All users are funnelled into creating a ‘skills profile’ when this is not the need for all users
Analytics
Due to the broadness of scope, and the unique challenge of the user journey, the team has not been able to solve the difficult problem of measuring “completion” for users engaged in an information gathering exercise. The team had given some thought to this and had good idea for areas to explore in the beta process.
Again, the scope of the service caused significant issues here, because without strong scope boundaries to define the purpose of the system or transaction, it is hard to define whether a journey is successful or completed.
Recommendations
To pass the next assessment, the service team must:
-
Clearly define and be able to explain a tight scope for the site, including which needs are met, and how the team can measure and deliver improvements
-
A strategy for creating and maintaining the content, including evidence on how the team would measure the accuracy and quality of the content in the content system
-
Define performance indicators so the team can demonstrate the effectiveness of the service and what difference it is making in meeting the user needs
-
Be able to clearly explain what parts of the overall user need is met by other organisations or services, and how the coordination and hand-off between the service and the organisations work.
The service team should also:
-
Look to streamline their release process during the beta where possible, to iterate faster during the early part of the beta when it’s important to respond quickly to changes in needs and acceptance criteria, or bugs.
-
Decide early whether or not to build a user journey that supports storing personal data. Without a clear user need for storing personal data, the risk can be significantly reduced and the system simplified
-
Reduce the scope for private beta (for example just to ‘I need to find a job I feel confident I could do’) and clearly meet that need across your full range of users.
-
Have a clearer plan for beta to discover and remove barriers for users with access needs.
-
Do more to understand how users who need support will move between digital, face to face and phone, to make sure that support routes are clearly signposted and easy to access.