Licensing for International Trade - Alpha Assessment
The report from the alpha assessment for BIS's Licensing for International Trade service on 27 May 2016 and reassessment on 28 June 2016
| Stage: | Alpha |
| Service provider: | Department for Business Innovation and Skills |
About the service
Service Manager: Chris Parish
Digital Leader: Emma Stace
The new Licensing for International Trade and Enterprise service, part of the One Government at the Border programme, will provide a single service for import and export permissions across multiple departments.
The service is currently focussed on the licensing of controlled (military and dual-use) goods, subject to the Export Control Act 2002.
Assessment Result - 27 May 2016
Result: Not Met
To meet the Standard the service should:
- Develop a detailed research plan addressing the gaps in the existing user needs research, including those who have no prior experience of licensing, and particularly addressing the lack of any Assisted Digital research
- Evidence a commitment to use the outcomes of this research to inform the design of the private beta, ideally by starting to undertake this research and incorporating the results into the prototype designs
This is vital to ensure that the team doesn’t start developing a private beta service that subsequently fails to meet these needs.
Detail of the assessment - 27 May 2016
Lead Assessor: Adam Maddison
User Needs
The team conducted a significant amount of user research - 34 interviews, during the Alpha stage. There was a healthy emphasis on offsite contextual work, and the regular attendance at sessions by the wider team was a strength. Personas were developed based on earlier Discovery research, augmented by the later Alpha work. The personas encapsulated the user needs that had been identified thus far - for example the need for simplicity and speed in the process (large business user) and the need to understand the process before beginning an application (small business user)
While the volume of research was appropriate, the panel felt that the research sample was unbalanced:
- it consisted mainly of larger corporates (with little SME presence in the sample), with many full-time licensing specialists among those seen
- it was almost exclusively users of the existing system (ie people who have applied for a licence in the past, and who therefore have prior knowledge of the process)
- no intermediaries were involved (eg freight forwarders)
- no users or potential users with low digital skills were included
The service team acknowledged that the licensing process is complex and demanding. They mentioned that there is a body of existing exporters who are exporting in error without a licence. They mentioned that SMEs and ‘one man bands’ are subject to this licensing process And they reminded the panel that perceived misuse of the system can result in a jail term.
These factors would suggest a heightened need for a well balanced research sample, with strong representation of the least knowledgeable and least skilled users and potential users. The very low level of such representation in the sample thus far creates a high level of risk in the product.
A related gap in the sample is around Assisted Digital (AD) support. There has been no specific research into the needs of less confident users for any AD support. While the service has a phone line, the nature or level of any AD need (and hence the appropriate solution) has not been established. The service team recognised this, and did say they would welcome advice regarding AD from GDS.
The emphasis on conducting research at the respondent’s workplace is commendable, as is the attendance of other members of the team at these research events. The team’s proposal to increase the mix of research methods by also including lab research in the forthcoming period is ideal.
The material presented to the panel did not include a detailed research plan for private beta. The service team recognised some of the gaps in the research mentioned above, but have no concrete proposals at this stage for addressing those gaps.
Against this background of significant additional forthcoming work, the research team seems light on experience. The principal user researcher is new to BIS and the two fast-streamers she is mentoring have no prior research experience.
The Team
The service team is currently very well structured, with all necessary skills covered. The team obviously works extremely well together, and the team’s working practices are excellent. The team is clearly empowered to make decisions about the way that it works.
Regarding governance, there has been significant progress in moving from a more traditional approach to a model where the board provides guidance and assurance to the service, without impacting delivery. This is exemplary, and the team should be proud of the work done.
However, the panel had some concerns about both the size of the team, and the sustainability of the team.
The panel would like to see an increase in the depth of user research capability. The need to research areas not yet covered (such as users not yet on SPIRE, users in smaller companies, and users with low digital skills) indicates to the panel that user research capability should be expanded. The panel were pleased to hear that the two faststreamers supporting user research would be in place for 12 months. But as they’re new to the discipline, currently the panel agree that there should be more experienced support in place.
The panel also agreed that an increase in content design capability would benefit the team as it progresses development, particularly in a private beta phase. The service team recognised that the “arcane language” of licences, especially the Terms and Conditions, had a significant impact on the service’s users. Given that the team identified that “copy is a huge risk”, the panel would like to see an increase in content design capability, to help make licences as simple as possible to understand.
The panel had concerns about the sustainability of the team. The team is largely comprised of contractors. While the service team recognised the need to transfer skills to civil servants, with only the delivery and service manager, and the user research faststreamers currently being civil servants, the opportunities for such transfer are limited.
The panel would like to see the service team develop plans for building and growing a sustainable team - particularly before moving into a public beta phase. This will be especially important as the programme moves towards replacing SPIRE. There will be considerable business and technical change to manage, and the team will need to be protected, such that it can continue to iterate the service. However, the panel is aware that the service’s ability to create a sustainable team is highly dependent on BIS’ ability to attract and recruit capable civil servants. The panel therefore recommend BIS addresses limits on civil servant headcount in order to mitigate this.
Technology
The team have chosen appropriate technology to quickly prototype the service in alpha and the Dropwizard/JAVA/docker/kubernetes/openstack tech stack and microservice based architecture proposed for beta is sensible and portable between hosting providers.
Early on in private beta, the team needs to work with the rest of BIS to determine what the medium term model for hosting and supporting BIS services will be, as this may affect the choice of technology and the makeup of the team needed to deliver it.
The panel were pleased to see that the team have embraced continuous delivery in alpha and will be using an automated build pipeline based on Jenkins and gradle in beta. The team described a sound approach to vulnerability scanning and patching.
The team intend to allow users to authenticate against the existing SPIRE system via SAML and are looking into the use of GOV.UK Notify to send messages to users at such time as this functionality is migrated from SPIRE.
The panel were pleased to hear that the team intend to open source all code from the project under a BSD licence, with the likely exception of code intended to integrate with external systems with closed interfaces.
The team explained that SPIRE will be decommissioned over the next 2 years, and that extracting the licence application UI is an initial step towards this. The decommissioning work is a large and risky project which has the potential to interfere with the work of this service team. At the beta assessment the team will need to explain how this risk has been mitigated.
The team explained that the service to be built for beta will not store any personal or application data beyond the session lifetime. Data will be held within the existing SPIRE system, accessed via SOAP. The team explained that there will be no mechanism for the service to retrieve application data from SPIRE in scope for beta - this would carry an additional information disclosure risk.
Design
The service team has had success when researching the current prototype, but it’s unclear what the scope of the service being developed is, and acknowledged that parts of the transaction had not been focussed on during the alpha.
The team should be clear when starting the beta exactly what part of import/export licensing they are building, and develop a clear end to end service for this. The service should be realistic for the size of team making it.
The team have implemented the frontend toolkit and generally follow the patterns for GOV.UK services. However there are several key pages, such as picking licence type and goods classification, where different patterns have been used - these must be extensively iterated, researched against current patterns and discussed within the Government design community and with the design patterns team.
Whilst the team has tried to remove complicated language around importing and exporting, there are many terms (such as “dual use”) that are not clear and should not be papered over using notes and guidance - eg there should not be links or answers that say “I’m not sure”.
The team should look at prototyping journeys that start from many of the information pages on GOV.UK about importing and exporting (and potentially from tools like the Trade Tariff), as this may make the journeys simpler to navigate and understand.
The team has not yet considered the internal services that will need to be built for departments to access the information within the system. This needs to be prototyped and researched alongside the citizen-facing service.
The team should start to iterate the design based on a mobile-first approach. It is important that all devices can be used to access services, and design and research should be conducted on mobile and tablets as well as desktop computers.
The service is currently only available online so does not need to think about channel shift to digital. The team will need to investigate and plan how to migrate (currently pretty happy) users from SPIRE to the new set of services.
Analytics
At this early stage, the team have a sound starting approach to gathering data on performance. They have access to the SPIRE database, and are making good use of existing data to inform the direction of the prototype.
They have an existing platform, Piwik, for collating analytical data, and in-house analysis capability, which they plan to use to benchmark future development.
They have also established a relationship with the performance management team in GDS.
Recommendations - 27 May 2016
To pass the alpha reassessment, the service team must:
- Develop a detailed research plan addressing the gaps in the existing research described above, including the lack of any AD research
- Evidence a commitment to use the outcomes of this research to inform the design of the private beta. Ideally this should be done by starting to undertake this research and using the results to inform the existing prototype design
To pass the subsequent beta assessment, the service team must:
- Conduct all the necessary research to fill in the gaps in the existing research described above, including the lack of any AD research
- Ensure the team is staffed with researchers of sufficient experience to deliver this research to an adequate standard
- Create a clear description of the service to be developed initially in the beta phase
- Develop plans to create a truly sustainable team, capable of building and iterating this complex service without being impacted by migration from the existing SPIRE system
The service team should also:
- Use incentives, recruiters and departmental contacts (HMRC, UKTI) to find a wider range of user research participants
- Work with GOV.UK to determine entry points for the service and prototype and test journeys using these
- Prototype and iterate the internal services that will use the license data
- Consider the needs for large exporting organisations and intermediaries, such as APIs
- Design mobile first and use progressive enhancement based on differing device capabilities and screen sizes
Digital Service Standard points - 27 May 2016
Reassessment Result - 28 June 2016
Result: Met
Detail of the reassessment - 28 June 2016
Lead Assessor: Adam Maddison
The reassessment focussed on the 3 points not met at the previous assessment: points 1, 2 and 3.
The first assessment and the reassessment has focused on the permissions finder and obtaining an OGEL licence aspects of the Licensing for International Trade service.
Researching and understanding user needs [point 1]
The panel agreed that the initial research the team had undertaken with the two subjects had helped the service team understand that there are undoubtedly significant user needs amongst those users who, for example, have little understanding of importing and exporting, or who need additional support.
Some of these user needs, particularly around assisted digital, may well be met by existing support process in place, including the phone service, and the ability for members of the support team to undertake site visits to support users.
The service team remains in a position of having incomplete research data, as described in the previous assessment report, but the first steps in creating a research plan to address the gaps in the research have been taken.
A plan for ongoing user research and usability testing [point 2]
The team presented an initial research plan for the private beta, which proposes a significant widening out of the research sample to address the gaps discussed at the previous assessment. The target sample for beta includes the following sectors or dimensions:
-
intermediaries, including
- consultants,
- lawyers,
- freight forwarders
- accountants
-
type of applicant
- commercial
- leisure
-
type of goods
- military
- civilian (dual use)
-
industry segment - also broken down by military vs. civilian
- military contains 39 sub-categories.
- civilian contains 21 sub-categories (there may be a heightened focus on some high profile segments like nuclear)
-
size of organisation
- SME/micro
- large
-
experience with ECO
- experienced SPIRE users
- novice SPIRE users
- potential LITE users
- people who may not be aware of the regulations, who have goods which are potentially subject to control, who might at some point need to confirm whether this is the case or not, and who might then need to be handed on to another government department.
-
AD needs
- SMEs without dedicated licensing staff
- private individuals applying for licences for leisure purposes
- highly skilled individuals who have no need or interest in in engaging in common online tasks
- older staff members in industry sectors which do not invest in technology
-
accessibility needs
- sub sectors, to be specified, but which at a high level includes people who use assistive technologies and people with cognitive impairments
The panel agrees that relatively complex and multi-layered sample that is proposed above seems appropriate to the diversity in the audience for this service. The panel agree that more work on the size and structure would provide the numbers of respondents to be interviewed against each dimension, illustrating how the different dimensions overlap, and therefore a providing a framework for recruitment.
There was limited information on the timing or staging of the research, for example, in relation to any of the 5 components of the product proposed for beta, or in relation to any need to fill high-risk gaps in the sample early in the next phase. However, the panel was pleased to hear that the team was open to front-loading the research at beta stage with those parts of the sample where high-risk gaps exist.
The team propose to conduct usability testing with 5-8 users per sprint, using a mix of face-to-face and remote methods and to include a range of users in each sprint, including some with low skills or confidence. This would mean that the filling of the main gaps in the sample would be gradual - for example assuming 1 low digital skills user per sprint, it would take 4 months to build up a sample of 8 in that category. There may be some contextual research conducted in parallel with the usability testing, albeit there were no detailed plans or numbers for this.
Have a sustainable multidisciplinary team [point 3]
The panel was pleased to hear that team are in the process of recruiting an experienced interim lead researcher, to manage and take forward the complex research programme, with the support of the existing complement of one researcher and two fast-streamers. The panel agree that the addition of the experienced lead researcher is a positive and necessary step, and should provide the team with the leadership necessary to deliver the appropriately ambitious research programme which they have begun to define.
There is also a pool of qualitative user researchers within BIS that could be drawn upon - but the team would prefer to be clear that there is a need before recruiting more qualitative researchers.
The team have realised that a lot of the language used within the service needs to be researched and reworded, not least because recent user research shows that users misunderstand even basic terms such as Import and Export.
Whilst no additional content designers were felt necessary by the service team at the moment, the panel agree that the team should carefully monitor this, make sure content design is not a bottleneck, and ensure that language has been researched and iterated. The content designer must work very closely with the interaction designer to design and iterate the potentially complex decision trees that users will encounter.
There is a clearer breakdown of the project into smaller products that can be delivered and iterated independently of each other. The panel still believes that the service is very ambitious with timescales that will be hard to fulfil with the current size of team.
In terms of sustainability, whilst the panel still has concerns that the team has a high number of contractors, there is no dependency on a single supplier. The panel would like to see more use of civil servants but are aware of the constraints that team are working under within BIS. The panel was pleased to hear that the department is focussing heavily on recruitment, capability, and skills transfer across the Digital, Data and Technology profession.
Recommendations - 28 June 2016
The service team should:
-
Look to front-load the research at beta stage with those parts of the sample where high-risk gaps exist:
- This will allow more time to consider the implications of any findings in relation to AD, inexperienced users, SMEs etc which might affect the basic proposition or form of the service.
-
It will allow the team to more quickly arrive at a position where they can be confident about the nature of AD needs and how they can be appropriately met.
- Show at the next assessment the structure of the sample, and what and how much research was conducted against each of the dimensions identified by the team in the research plan presented for this reassessment.
- Describe at the next assessment what was discovered at beta stage across different parts of the sample and how the findings affected the design of the service (this should include evidenced Assisted Digital needs and accompanying tested Assisted Digital solutions).
- Show one well-defined part of the service in private beta that has been extensively user tested and iterated, with learnings from a private beta, before starting development of all products within the service.
- Reduce the dependency on contractors by pushing the department to help recruitment of and transfer of skills over to civil servants.
Digital Service Standard points - 28 June 2016
| Point | Description | Result |
|---|---|---|
| 1 | Understanding user needs | Met |
| 2 | Improving the service based on user research and usability testing | Met |
| 3 | Having a sustainable, multidisciplinary team in place | Met |