Apprenticeship Applications (Employer Posting) - Alpha Assessment
The report from the alpha assessment for SFA's Apprenticeship Applications (Employer Posting) on 20 August 2015.
Department / Agency:
BIS / SFA
Date of Assessment:
20/08/2015
Assessment Stage:
alpha
Result of Assessment:
Pass
Lead Assessor:
T. Scott
Service Manager:
G. Tucker
Digital Leader:
E. Stace
Outcome of service assessment
After consideration, the assessment panel has concluded that the Apprenticeship Applications (Employer Posting) service is on track to meet the Digital Service Standard at this early stage of development.
About the service
This digital service provides greater ownership over the advertising of a vacancy and selection of a candidate to employers who want to recruit apprentices/trainees, whilst also facilitating collaboration with providers who support them in the process. It will provide:
-
clear information about apprenticeships/traineeships and the employer’s responsibilities in offering them
-
a simple and fast way to create, advertise and manage apprenticeship/traineeship vacancies
-
tools to help manage candidate applications effectively and support outcome notification
-
a faster and more cost effective process for quality assuring vacancies
Reasons
User needs
This new service, which allows employers and providers to post jobs, will compliment the existing application service. The evidence presented showed that the service will make it easier for providers and employers to advertise and collaborate on available apprenticeship and trainee roles. User research has shown that both groups have different needs and the service is being designed to meet them.
The well researched personas have been validated through user testing which is opened up for the entire service team to observe and feedback on. Testing starts with a hypothesis based on the personas, leading to a rough design that gets built, allowing for rapid real feedback. During the alpha, the team have taken an adaptable approach to research and testing that will become increasingly regular during the beta phase.
The team have made a good start in identifying needs for support, by speaking to users from all user groups, by working closely with the existing helpdesk, and through an offline survey to providers and employers. The team has proposed the use of existing support routes, but understands that they must identify the needs of potential users, not just current users, and may need to iterate the support to bring it into line with those needs, and Digital Service Standard requirements. The team confirmed that the support will be sustainable and free to the user. The team has identified suitable actions for the next phase, including an increased focus on support needs for employers.
The current service is digital only; providers must upload through an existing website, but not all providers use the current service because of the high quality control and ease of publishing on external platforms. The team are aware of the need to build a service that is easy to use and requires minimal support through other channels. There are many improvements to the current journey that will help users.
The team
The team have the ensured there is flexibility to allow for the different user tasks, whilst not designing the service around particular groups. The team are ensuring the service is being built to allow providers and employers to collaborate through a non-linear process and improve the capacity to change or edit the data, while at the same time reducing the need for quality checks that might fail later in the process.
The team are well placed to deliver the service with all the expected roles in place. There is some dependence on external suppliers but this is shifting as recruitment continues, and the service has provision in place if this changed suddenly. The product manager has a wide depth of knowledge from the agency, and experience through the delivery of the Apprenticeship Applications exemplar service. The team have used an adaptable approach to agile in the alpha, based around spikes for testing proofs, but will be moving to a Scrum based sprint cycle in the beta. Regular show and tells engage stakeholders, and individual team members share best practice as part of clans within the agency. The service team is also responsible for maintaining of the current service, which will help with the many dependencies between the different aspects of the service.
The team share information through a wiki, managing the Proof of Concepts (POCs) and defining the success criteria for spikes and stories. Work is prioritised against releasing a minimal viable service as soon as possible along with availability of team members. The team are co-located in a single room enabling quick feedback. The regular meetings with the SRO ensure that the service is focused on delivering user needs.
Security, privacy, tools and standards
The team has considered the technical integrations needed for the service, and during the alpha, have carried out a number of technical proof of concepts to derisk the next phase of the beta.
The panel were pleased to note that the team are looking at the platforms used on the exemplar service, and not only reusing the appropriate ones, but were looking to use platform as a service (PaaS) and software as a service (SaaS) tools where appropriate.
Identity is the biggest security issue likely to face the service during beta, and the team has worked to understand the landscape and the issues the service is likely to face. A focus should be maintained on appropriate protection of the data stored as well.
The team are making code open, keeping private only the details that open up the service to risk. The service is reusing code drawn from the current application service. It was encouraging to see discussions with other government services that are providing similar services.
While the prototype has mainly been focused on the desktop users using Chrome, the team have the means to test the service on a wide variety of devices and conditions in the beta. Observing users in their own environment will reveal a wide range of devices and browsers need to be supported. The plans for service outage follow those used on the application service, mitigation is in place if individual components do not function as expected through graceful failure.
It was good to see that testing with users was tasked based, and focused on derisking and know pain points. Early testing of the service’s design has led to changes in the presentation of current vacancies, moving from a tabbed list to a filter with categories based on user feedback while also highlighting any important tasks to publish vacancies. The design of the vacancy page allows providers and employers to see how the details will be presented to the candidate early on in the process. Plans are in place to begin to automate the quality assurance process to ensure a higher quality of postings are placed on the site, and to reduce some of the frustrations users find in the current workflow. The use of a WYSIWYG editor is new to GOV.UK services, and tested well with employers and providers (showing similarities to job application sites).
Analysis and benchmarking
The team have gathered analytics from the current web service and call data relating to posting a role, matching any insights to the user research and testing. This has highlighted pain points around the Quality Assurance (QA) process and the differing needs of employers and providers. The service has engaged with the performance platform team, and discussions have started on the additional measures for the service to present on the platform.
Recommendations
During the next phase, there is more work to be done on the end-to-end user journey for users who don’t have digital skills or access, for example, considering how email approvals and confirmations will be handled. The providers are supporting some employers through their own processes; this needs careful examination to ensure that as employers are encouraged to post directly, they have the appropriate support they need. The plan to undertake research with potential SME and micro employers should help to test assumptions that proposed support will meet the needs of new users in future.
Further thought needs to go into encouraging current users who are used to and expect telephone support, to shift to self-serve through the new service.
There are some shared roles across the services within the agency, which has worked during the alpha period. The team need to ensure that shared resources, such as dev-ops and performance analysts, that are not dedicated part of the team, are available when needed.
Sharing a backlog with an already live service can be difficult. The flexibility in owning the entire service can help quick development, but thought should be given to how to prioritise and communicate the priorities to the different stakeholder groups.
The new WYSIWYG design for inputting data for applications may help infrequent service users, but will need testing with frequent users who upload multiple applications. The service will likely need to work on non-JavaScript enabled browsers; you must ensure that this is possible.
Summary
The approach the team has taken to the alpha phase of this new service demonstrate that the learnings from the application service are central to the delivery. It’s very encouraging that the team are have a strong focus on meeting their users needs. It was good to see the strength of relationships between the team members in the assessment and the enthusiasm for delivering the service.
Digital Service Standard criteria
| Criteria | Passed | Criteria | Passed |
|---|---|---|---|
| 1 | Yes | 2 | Yes |
| 3 | Yes | 4 | Yes |
| 5 | Yes | 6 | Yes |
| 7 | Yes | 8 | Yes |
| 9 | Yes | 10 | Yes |
| 11 | Yes | 12 | Yes |
| 13 | Yes | 14 | Yes |
| 15 | Yes | 16 | Yes |
| 17 | Yes | 18 | Yes |