Visas - Beta Assessment
The report from the alpha assessment for Home Office's Visas service on 20 June 2014.
Department / Agency:
HO / UKVI
Date of Original Assessment:
03/06/2014
Date of Reassessment:
20/06/2014
Moving to:
Beta
Result of Original Assessment:
Not passed
Result of Reassessment:
Pass
Lead Assessor:
M. Sheldon
Service Manager:
T. Bruck
Digital Leader:
M. Parsons
About the service
If you need a visa to visit the UK you’ll be able to apply using a simple online service. https://www.gov.uk/transformation/apply-visa
Outcome of service reassessment
The Visit Visa service, for mainland China, is seeking permission to launch on service.gov.uk as a Beta service.
We have concluded that the service has shown sufficient progress and additional evidence to launch as a Beta on the service.gov.uk domain. We have made this decision based on the evidence the service provided for criteria 1, 2 and 9 of the Digital by Default Service Standard, which were the sections the service did not pass at its original assessment.
The further information submitted by the team and explored at reassessment provided sufficient evidence of user research with a range of potential users of the service. This complimented the evidence provided at assessment which showed a thorough approach to user research, with a focus on quality.
Further evidence on the team’s data analysis capability was reassuring and sufficient to assure the assessment panel. With their emphasis on building capability within the team and working with GDS on recruitment of core digital roles across the Home Office, the assessors are happy to pass the service on criteria 2.
At reassessment the service team demonstrated significant engagement with content designers since the assessment, and the service under discussion at reassessment showed very significant improvement from the original assessment. The service team evidenced a clear commitment to continuing to iterate and improve content in response to user feedback, and the changes they had succeeded in making to content, in conjunction with policy and legal colleagues were very welcome.
Summary
We were pleased to see the immediate improvements made to the content of the service and the approach to prioritise further improvements. We were also grateful to the team for the additional evidence provided on the scope and scale of user research. We are therefore pleased to see the service proceed to Beta. We look forward to the team sharing feedback from the user research trip to China when they return for Live assessment.
Outcome of original assessment
After consideration the assessment panel have concluded that the Visit Visa General Visitor service should not launch on the service.gov.uk domain as a Beta service, for the reasons given below.
In line with current plans, the service should continue to test with real users, in a controlled way, so that the team can monitor and iterate the service. Over this time, through this process, the service team will be able to address the points made below. The assessment panel will then be able to reassess the service so that it can move to Beta on the service.gov.uk domain.
Reasons
The assessors were pleased to see that the Visas service team were meeting most of the 26 criteria of the Digital by Default Service Standard, using the same multidisciplinary team, platform and product catalogue built for the Tier 2 Visit Visa service. However, the assessors agreed that service did not fully meet the following criteria:
Point 1
Understand user needs. Research to develop a deep knowledge of who the service users are and what that means for digital and assisted digital service design.
Although the service team have carried out regular user research whilst developing the service, the assessment panel felt that this was not completed with enough real users. The current planned user research trip to China should be completed. This will provide the team with a deeper understanding of the population demographics and user proficiency profiles of China.
Point 2
Put in place a sustainable multidisciplinary team that can design, build and operate the service, led by a suitably skilled and senior service manager with decision-making responsibility.
The service team realise there is a skills gap around content design and are grateful for the support GDS has provided. There was some concern over how frequently the GDS Content Designer has been able to iterate and improve the content based on user feedback during the service development (see Criteria 9).
The service team also need to find a suitably skilled member to focus on service analytics and analysis. With the future planned rollout to other countries it will become an increasing challenge for the team to analyse service analytics and translate findings into service improvements. This was a recommendation from the Tier 2 Visit Visa assessment and there was insufficient evidence of a plan to act on this recommendation.
Point 9
Create a service that is simple and intuitive enough that users succeed first time, unaided.
During the assessment some verbose and confusing content was spotted, with noticeable errors. The assessment team felt that given the length and complexity of the form its content needs significant improvement. This is especially important for a service intending to launch in a country where the users’ primary language is not English. The assessment panel understand the service has immediate plans to do this but suggest the testing with real users also informs content improvement before reassessment.
Recommendations
For the service to pass and proceed to Beta on service.gov.uk the assessment panel recommend that the service team:
-
continues to work closely with a GDS Content Designer and complete a thorough review of the service’s content, which must include fixing errors and improving the language used. It should be as simple and easy to follow as possible, incorporating improvements based on user feedback and these changes must be reflected in the Chinese translations.
-
put in place a plan to find a dedicated Content Designer and transfer the knowledge and skills of the GDS Content Designer.
-
put in place a plan to find a dedicated Product Analyst to focus on service analytics and analysis.
-
continues with the planned trip to China and test the service with real users. Use their feedback to inform improvements to the service features and content.
-
continues to work with the GOV.UK User Formats team to plan how to recruit a growing number of users via the GOV.UK start page once the service passes a Beta assessment.
Next Steps
The service team should follow the recommendations made in this report and see the Government Service Design Manual for further guidance. In order for the service to proceed GDS will require a review of the failed criteria; 1, 2 and 9.
Summary
The assessment panel was encouraged by the knowledge and commitment of the service team to ensure that the best service is delivered to its users. We were pleased to see continued improvements to the agile ways in which the team are working, the visas platform and the product catalogue. With the planned user research trip to China the team have an ideal opportunity to test with real users, gather their feedback and use this to improve the service. We look forward to running a reassessment so that the Visit Visa service can be confidently launched as a public Beta in China.
Digital by Default Service Standard criteria
| Criteria | Passed | Criteria | Passed |
|---|---|---|---|
| 1 | Yes | 2 | Yes |
| 3 | Yes | 4 | Yes |
| 5 | Yes | 6 | Yes |
| 7 | Yes | 8 | Yes |
| 9 | Yes | 10 | Yes |
| 11 | Yes | 12 | Yes |
| 13 | Yes | 14 | Yes |
| 15 | Yes | 16 | Yes |
| 17 | Yes | 18 | Yes |
| 19 | Yes | 20 | Yes |
| 21 | Yes | 22 | Yes |
| 23 | Yes | 24 | Yes |
| 25 | Yes | 26 | Yes |