Research and analysis

Evaluation of the Regulators’ Pioneer Fund (round 1) – final case studies, April to July 2020 (web)

Published 15 March 2021

The high-level objectives of the Regulators’ Pioneer Fund

The Regulators’ Pioneer Fund (RPF) is an initiative set up by the Better Regulation Executive (BRE), part of the Department for Business, Energy and Industrial Strategy (BEIS), to help create a regulatory environment that gives innovative businesses the confidence to invest, innovate and deploy emerging technologies for the benefit of consumers and the wider economy. 

Key aims of the funding (which will invest up to £10 million over 2 years in 15 regulator-led projects across 12 sectors) are:

• Enabling economic growth
• Boosting value for consumers
• Projecting a pro-innovation image internationally
• Fostering a pro-innovation business culture

Aims of the RPF programme evaluation

• Assess the extent to which RPF grants have enabled regulators to become more innovation-enabling
• Identify the effects on business innovation in the sectors impacted by the projects
• Enable BEIS to improve the RPF design and competition process for any future funding rounds
• Contribute to BEIS work to create best practice guidance on innovation-friendly regulation and inform future policy decisions

The policy logic model for the RPF

The policy logic model is in four stages:

1. Outputs
2. Outcomes
3. Impacts
4. Goals

Outputs

RPF regulator(s):

• permits new business innovation (e.g. products, services, processes, business models), for example through new licensing or sandbox regime
• stimulates new business innovation (e.g. products, services, processes, business models), for example by setting challenging outcomes or releasing new information
• reduces time or cost of introducing business innovation (e.g. products, services, processes, business models), for example by providing better advice or simplifying processes
• improves business or investor confidence in how business innovation (e.g. products, services, processes, business models) will be regulated, for example through comms
• improves consumer confidence in business innovation (e.g. products, services, processes, business models), for example through improving protections or enhancing comms
• influences other UK regulators to take a pro-innovation regulatory approach, for example through forming partnerships or disseminating findings
• influences other administrations to align with its regulatory approach.

Outcomes

• Increased generation of ideas (e.g. products, services, processes, business models) by businesses operating in the UK.
• Increased investment in innovation in the UK.
• Increased competition (i.e. increased entry and exit to UK markets).
• Increased consumption of innovation (e.g. through greater consumer confidence) – i.e. increased UK market size.
• Lower regulatory barriers to trade (e.g. through global regulatory influence), with increased overseas market size.

Impacts

Increased quality/quantity of business innovation (e.g. products, services, processes, business models) that benefits the economy, society and the environment.

Goals (outside of scope of the evaluation)

• Faster, more sustained growth in productivity and wages.
• Increased resilience to economic shocks.
• Faster delivery of wider benefits of economic development (e.g. wider consumer choice, advances in medicine, transport, etc.).
• Better outcomes for consumers, workers, citizens and the environment.

Case studies: selection and methodology

Five final case studies were selected to draw lessons from the programme.

Qualitative interviews were conducted with regulators and their external stakeholders to explore their perspectives on progress, impact and lessons learnt. These were:

• The project team involved in the bid and delivery
• A strategic lead on the project
• Two external stakeholders identified to benefit from the project

The case studies with Scottish Environment Protection Agency (SEPA), Care Quality Commission (CQC), Information Commissioners’ Office (ICO), Intellectual Property Office (IPO) and Ofcom followed five interim ones conducted with the Solicitors Regulation Authority, Medicines and Healthcare products Regulatory Agency and Civil Aviation Authority (between July and October 2019) and Oil & Gas Authority and Financial Conduct Authority (November 2019 – January 2020) in the same format.

For each case study, early lessons have been captured from the following phases and themes:

• Project set up
• Internal engagement
• External engagement
• Enabling innovation through regulation *Value of the RPF

Case studies methodology

Participants’ selection and data collection:

Regulators interviewed for the case studies were selected by BEIS and Kantar based on their progress and nature of their projects to reflect a range of views and experiences.

For each project, the research team conducted: 1x 60 mins semi structured qualitative interview with key members of the project team (e.g. project and programme manager or bid manager, depending on team composition); 1x 60 mins semi structured qualitative interview with a strategic lead overseeing the project; 2x 30 mins interviews stakeholders (e.g. businesses, other regulators, other organisations in the sector) identified by the regulators and selected by BEIS as beneficiaries of the projects.

Fieldwork was conducted between April and June 2020 (once their projects had finished). The final case studies follow five interim case studies conducted between June and January 2019 and four rounds of a quarterly information management questionnaire delivered by Kantar. This questionnaire was used to understand how far projects were achieving the outputs and outcomes in the RPF logic model (see slide 3). Topics explored are included below.

Project team / strategic lead - the same topic guide was used flexibly to explore different perspectives:

• Regulators’ understanding of the RPF and motivations to apply for funding
• Vision for the project and connection with organisational mission
• Experiences of main stages of work (e.g. set up, internal and external engagement, delivery of project specific activities)
• Expected and unexpected challenges encountered
• Lessons they learnt about enabling innovation in their sector, engaging with businesses, regulators and other stakeholders
• Any outcomes and impact of their project on innovation in their sector (in relation to focus outcomes agreed with BRE)
• Perceptions of RPF support, any impact the Fund had on their ability to enable innovation in their sector, and ways the RPF could be improved for the future

Beneficiaries:

• Beneficiaries’ exposure to and perspective on regulators’ work;
• Nature of their involvement and views on their engagement with regulators;
• Ways in which regulators can more effectively engage with stakeholders to enable innovation in their sectors;
• Any perceived outcomes and benefits deriving from regulators’ projects;
• Views on future outcomes and how regulatory activity could be improved to encourage innovation
• Analysis of the information collected through the case studies interviews:
• Material collected in interviews (e.g. audio files, notes) were organised through a thematic framework developed in Excel, informed by evaluation objectives.

Individual and joint brainstorming sessions were carried out by researchers in the Kantar team to review and consolidate insight, and draw key overarching themes.

Cross cutting lessons

Topline overarching lessons across the case studies

All regulators interviewed felt the RPF accelerated and built momentum for their innovation work. They explained that the Fund allowed them resources and capacity to test and develop knowledge of innovative solutions to sectoral issues, and to build lasting networks with a range of organisations both within and across sectors. This was consistent with the thoughts of regulators involved in case studies at the interim stage.

While a few areas for improvement were identified for future funding (e.g. more flexible monitoring, reporting and spending requirements, more consistent communications about the Fund administration and access to escalation procedures, and improved support from monitoring officers), regulators described their experience of the Fund as positive and beneficial to support them in enabling innovation in their sector.

Most regulators interviewed felt that tangible outcomes connected with enabling economic growth and boosting value for consumers would be detectable as innovations informed by their work are developed and released. However, they highlighted achievements contributing to these key RPF goals, which suggest a positive indication of future impact.

All regulators interviewed successfully established sandboxes, trialled and tested solutions and proof of concepts with the potential to move into implementation, worked to effectively share advice tackling issues identified in their sector and making innovation more efficient.

They explained that their work enabled a better understanding of their sectors, and how innovation can be applied to tackle issues which will likely support their future regulatory activity.

Regulators also noted that their projects enabled direct links and improved collaboration with sector stakeholders and innovators they didn’t previously have a relationship with. They explained they increasingly sought feedback from the industry and most planned to continue to do so past the RPF. They felt they started adopting a more proactive approach to regulation (i.e. supporting compliant innovations’ development rather than detecting issues at the end) and it should be noted such ongoing, open conversation with sector innovators presents a possibility for more efficient development of solutions which protect consumers in line with regulation.

Regulators’ also reported activities and outcomes aligned with RPF’s key aims of fostering a pro-innovation business culture and projecting a pro innovation image internationally – also consistently with findings from previous case studies.

Regulators explained their RPF funded projects allowed them to expand their reach and collaboration with other regulators, forming personal relationships stretching across sectors to resolve shared issues.

Projects explored in these case studies also attracted notable international attention, with many (e.g. ICO, IPO, Ofcom) starting conversations about their work with international regulators, contributing to positioning the UK at the forefront of innovation.

Regulators also explored applying lessons on effective collaboration and effective working practices internally as they noted the value of openly engaging their own organisations in their innovative work and tapping into the varied expertise within them to bring value from different functions.

In order to achieve their projects’ outcomes, regulators reflected on the importance of consistent communication both internally and externally to promote their work, secure different stakeholders’ buy in while managing expectations, and build beneficial contacts and partnerships. They learnt to adopt an open approach to collaboration, taking chances to advance their projects and then reviewing what could be done better going forwards.

Across interviews, it appeared that regulators perceived the innovation work they started or advanced with the RPF as ongoing ^{[footnote 1]}, as they planned to develop approaches to innovation started with their projects and continuing collaborating with the sector / other regulators part the RPF lifetime.

A more comprehensive description of the overarching lessons and findings from individual projects can be found in the evaluation report.

Case study 1, SEPA DecomRegHub

Project at a glance – DecomRegHub

Role as the regulator

The Scottish Environment Protection Agency (SEPA) is Scotland’s principal environmental regulator, protecting and improving Scotland’s environment. SEPA’s role is to make sure that the environment and human health are protected, to ensure that Scotland’s natural resources and services are used as sustainably as possible and contribute to sustainable economic growth. 

Oil and gas decommissioning is an important emerging sector for Scotland, with potential to re-circulate valuable resources back into the economy.  

SEPA’s oil and gas decommissioning sector plan focuses on how SEPA will work with the sector to help it put environmental protection and sustainable resource use at the heart of its development, creating opportunities for both business and environmental success. 

SEPA’s interest in decommissioning to date has largely been focused on the transfer and disposal of waste associated with decommissioning and preventing environmental harm through these activities.

Project vision

DecomRegHub aims to bring together the main regulators involved in regulating the decommissioning sector: SEPA, Oil and Gas Authority (OGA), The Environment Agency (EA), Health and Safety Executive (HSE) and the Offshore Petroleum Regulator for Environment and Decommissioning (OPRED).

The project focuses on establishing a formal collaboration between these regulators so they can ensure a consistent and coordinated regulatory effort on decommissioning. Through this they can apply relevant, existing legislation and provide a clear response to industry who are trying to navigate multiple regulators to be compliant.

Furthermore, through the creation of an online website, DecomRegHub presents a centralised, single point of information for industry stakeholders to find the guidance and help that they need to understand their obligations and for innovators to understand gaps in the market.

Motivations for RPF bid

SEPA recognised that oil and gas decommissioning is an inherently complex issue, spanning a range of regulators (see previous box). In addition, existing legislation being applied by regulators has typically been written in isolation from other regulation. This has led to confusion in the sector and times when regulatory application of legislation generates competing demands for sector stakeholders.

The difference in regulatory approaches is exacerbated by a lack of clarity in industry about where to find relevant guidance and support. SEPA understood that stakeholders want to comply but have often found it difficult to know what to do.

The RPF presented an opportunity to resource the creation of a collaborative regulatory effort and an online portal to address these issues.

The ultimate aim of the project is to make it easier for industry stakeholder to achieve compliance.

Main activities involved:

• Initial user research to understand internal and external stakeholder problems and perspectives
• Creation of a face-to-face collaboration between staff at five different regulators across the UK
• Industry engagement through events and direct feedback
• Creation of a DecomRegHub website to serve as a centralised source of information and contact for stakeholders

Focus RPF programme outputs:

• Reduced time or cost of introducing business innovation
• Improved business/investor confidence in business innovation regulation

“[Stakeholders currently] have to look in multiple different directions towards different regulators to try to understand who is regulating what, and what those companies need to do with each regulator in order to ensure that they understand the obligations that they have.” - SEPA

Project set up

Understanding of the problem and RPF purpose

Kantar judged that SEPA had a clear understanding of the problem that their project was aiming to address, and that this problem was similarly understood by the interviewed stakeholders.

The core problem being addressed by the project is the complexity of a fragmented regulatory landscape for decommissioning leading to industry confusion and frustration both for established operators and new innovators. For the latter group this can deter market entry.

The second problem is that decommissioning is a growing issue in need of attention – UK offshore oil and gas infrastructure is aging and, in many cases, nearing the end of its useful lifetime. Falling oil prices also make more infrastructure uneconomical for operators to maintain. The cost of decommissioning to UK industry and taxpayer is estimated by the Oil and Gas Authority in 2019 to be in the region of £48bn. As such, there is a need to ensure that it is done efficiently, safely and without harm to the environment.

SEPA understood the RPF to be “a catalyst to help innovation take place in the regulatory domain where that innovation might not otherwise take place. That innovation needed to focus on solving problems, not just for regulators, but solving problems for business.” – SEPA

Challenges in setting up the project

Project set-up took more time than anticipated. Firstly, the project lead was already working on a number of other pieces of work and had other responsibilities. Therefore, there was a delay before they could start fully on the project, requiring a period of transition.

Secondly, it took time to build the project team, with the need to create job descriptions, go through recruitment processes and get people in place.

Finally, it took time to translate the work packages, objectives and schedule of work from the bid into a full, detailed, achievable and resourced project plan. Overall, this setup process took 3-4 months.

These time pressures made it challenging to spend the first financial year’s split of the budget as most of the team weren’t in place until January 2019. SEPA felt that the funding split was structured to suit the financial timings of the funders rather than the project team and this meant that SEPA had to work to make their workplan fit. This affected their resource plan and the way that they had to engage with external resources and partners.

SEPA felt that this financial pressure was particularly challenging for a programme that was trying to do something innovative and novel: “When you’re doing something creative and looking at imposed financial restrictions, […] I’m not entirely sure that assists the creative process.” - SEPA

Lessons learnt on project set up and planning

SEPA felt that more time to get the project team in place would have been beneficial, in terms of transitioning existing resource, recruiting new staff and securing arrangements with partners.

Less restrictive financial requirements (specifically the removal of the need to spend half the budget before the first financial year end) would have better supported a creative and innovative project.

One other lesson learnt was that the reporting templates provided by Innovate UK seemed to be designed around commercial entities involved in developing new products and intellectual property rather than public sector entities involved in improving their service offerings. They therefore found some of the reporting protocols quite challenging and needed to reword and adapt their reporting to fit the requirements. A more fit-for-purpose reporting regime, designed for the public sector would have been preferable. Otherwise it would have been appreciated if the reporting regime could have been more clearly explained up front so that projects were better able to cope with it.

Internal engagement

Internally, SEPA engaged:

• Regulatory officers
• Senior staff

“SEPA as an organisation have got an incredible amount of learning and lessons learnt out of this that we want to capture and bottle the essence of. [We want to] replicate, where there’s going to be business value in doing so, into other sectors that we regulate where there are other regulators regulating their respective parts of that sector as well (e.g. agriculture).” - SEPA

Nature of internal engagement and response from the organisation

SEPA noted that the challenges surrounding decommissioning were already well understood across their organisation and that these challenges are highlighted in their sector plan (SEPA have created plans for key sectors detailing their regulatory vision including a specific plan for oil and gas decommissioning). Therefore, gaining support for the project was not difficult. 

SEPA reported that they made a decision from the outset to be driven by user research to identify internal and external stakeholder needs and design their outputs around this, rather than deciding what the outputs should be first. 

They hired an independent user researcher during the first stage of the project to design and execute a research plan that included a survey and face-to-face/telephone qualitative interviews. This included interviews with staff across SEPA and also their regulatory partners. Questions explored internal stakeholder experiences and opinions e.g. ‘Do you encounter any barriers when trying to collaborate with other regulators?’ and ‘What are the gaps that from your perspective that need to be filled to smoothen the regulatory journey for operators?’

SEPA felt that the whole process of engaging with stakeholders, internally and externally, at the start of the project was very positive and helped inform the work that followed. Similarly they sought internal feedback on the beta website towards the end of the project that helped them improve it with users in mind. 

They reported that they would definitely do user research of this kind again and recommended that other regulators’ projects would benefit from a similar approach. They noted, however, that user research can be difficult, particularly considering timing. They felt that carrying out research too early risks talking about problems and solutions in the abstract (i.e. rather than presenting tangible solution ideas or concepts). Similarly, leaving research until later in the project risks spending time developing a solution that fails to meet the needs or expectations of the target stakeholders. Finding a balance within the available resource was seen as an important challenge to overcome.  

SEPA reported that as a result of the project they are noticing an increase in interdepartmental collaboration and a greater degree of confidence about who to speak to in other regulatory bodies when needing to engage across regulatory boundaries. They have also seen an increase in confidence and support internally for an agile approach to service improvement including culture change, collaboration and user engagement. 

External engagement

Externally, SEPA engaged:

• Project team regulators (OGA, EA, HSE, OPRED)
• Operators
• Supply chain stakeholders
• Trade bodies (e.g. Decom North Sea)
• Other RPF regulator project teams (e.g. OGA team)
• Other stakeholders (e.g. Atkins)

External stakeholder engagement

SEPA’s user research (mentioned on previous page) extended to industry stakeholders across the decommissioning sector including operators and supply chain services. At the beginning of the project this involved interviews and surveys to better understand industry challenges and perspectives.

Towards the end of the project SEPA engaged external stakeholders to show them what the project had achieved, particularly the beta website. SEPA wanted to show stakeholders something tangible that they could review. Through this, SEPA gained useful and constructive feedback from industry that helped identify areas to improve, clarify and strengthen in the final output. This also helped build confidence with key industry stakeholders in the final output.

SEPA also collaborated with the trade body Decom North Sea, which helped build networks with other regulators and industry stakeholders, particularly through Decom North Sea events such as conferences, webinars and with direct interaction with Decom North Sea specialist members.

Each regulator had well-established existing networks with industry. These networks overlapped in places but also were complementary to each other and sharing contacts supported the impact of industry engagement.

SEPA also shared knowledge with other RPF projects (e.g. OGA).

Finally, SEPA engaged informally and formally with existing stakeholders such as Atkins who helped provide input and feedback throughout the project.

Lessons learnt on stakeholder engagement

SEPA initially found the differences between regulators (their ways of working, cultures and priorities) difficult to overcome, particularly in the context of a devolved regulator working with centralised, Whitehall-based regulators. However, they noted that this challenge has been overcome during the project through all partners having equal power and by taking time to build relationships, leading to a strong collaboration:

“What’s been really good, I think, is seeing that spirit of cooperation or collaboration developing between the people in the projects that are in the different regulatory organisations. Each organisation has its own particular statutory purpose, firstly, but secondly its own way of doing things, its own corporate culture. And what’s been really good, I think, is seeing a little sub-culture developing within the project team of collaborative problem solving, doing so with the end-user businesses in mind and wanting to make sure that no one organisation is dominating the collaboration, ensuring that the collective efforts meet the objectives of all the participating organisations towards that common goal.” - SEPA

They also felt that the value of the user research, both at the beginning and end of the project, was of critical value to them in helping them understand their stakeholders’ problems. By better understanding the needs of stakeholders this helped them ensure that the project’s solution addressed these problems in the most useful way for stakeholders (for example, in making the website easier to navigate and have the most important information to stakeholders in easily accessible places).

Enabling innovation

Initial positive outcomes and achievements

SEPA felt that the project has successfully laid the foundations and established a forum for the relevant regulators to collaborate and collectively address challenges associated with decommissioning in a joined-up way. This has helped generate a joint regulatory perspective across the sector. This will lead, they hope, to greater clarity for industry stakeholders, easier compliance and new opportunities for innovators to support the decommissioning sector. Atkins noted that this could make decommissioning activity more efficient, make compliant solutions quicker and ultimately save money for the taxpayer (aligning with the RPF goal of boosting value for consumers and focus output 3).

SEPA also felt that there is now an increased level of opportunity for this new approach to support innovation in the field of regulatory cooperation by being more responsive and clearer with innovators and through better highlighting gaps in the current decommissioning sector that can be filled by new technologies or services. This supports the RPF goals of enabling business growth and fostering a pro-innovation business culture.

SEPA felt that the DecomRegHub website (decomreghub.org.uk) has been critical to establishing a link with industry stakeholders and has given the decommissioning sector a very clear signpost to the most up-to-date and most relevant information (as the other regulator websites cover broad ranges of areas, they can be difficult for stakeholders to navigate). The new website provides a roadmap for the decommissioning journey, highlighting the role of each regulator at each stage and provides drill-down layers of detail to access whatever information a user needs, from whichever regulator is responsible, providing a one-stop-shop for industry stakeholders, so they don’t have to look in multiple locations. 

Decom North Sea also note that the collaboration demonstrated by SEPA and the other regulators is improving industry confidence in the regulation of decommissioning and already making it easier for them to comply with their obligations. This delivers against focus output 4.

“The website provided us with a means of being able to get our messages out, being able to respond too and being, above all, a reliable source of regulatory information to the sector.” - SEPA

“We are seeing the start of a knowledge base that now informs our strategy for regulation that now involves DecomRegHub as a tool to be used.” – SEPA

Beneficiaries of the project:

SEPA and the other regulators are benefiting from an improved ability to respond collectively to industry and supporting industry to comply with regulation more easily.

Industry operators could benefit through a clearer route to compliance, supporting more efficient decommissioning and better decision-making and planning.

Supply chain stakeholders could benefit from a better understanding of the areas where they can support industry to meet obligations including innovative technology and services.

The UK public could benefit in the future from a more efficient decommissioning sector that costs the taxpayer less and is less likely to result in environmental damage from non-compliance. 

Stakeholder feedback, Decom North Sea

Decom North Sea is a trade body that focuses on decommissioning activity in the North Sea and aims to act as an interface between regulators, supply chain companies and operators to support compliance. They have worked with SEPA on multiple projects in the past and supported the DecomRegHub project by providing user feedback and supporting industry networking.

Decom North Sea agreed that the fragmented nature of decommissioning regulation is a significant problem that DecomRegHub can help address. Having been involved throughout the project, they reported positive opinions of all aspects of the project. They felt that SEPA have demonstrated a desire to collaborate and make compliance easier and more straightforward for industry, sending a strong message to the sector:

“It’s been a huge step change to actually pull the regulators together and see that interaction. I think that’s really significant. I think the key part is that if industry sees the regulators collaborating and listening to the needs of industry it sends a really strong message out to the industry that you need to engage and you need to look at the whole picture because the regulators are talking to each other and are pretty united in the way they are approaching things.” (Decom North Sea)

They noted that one way to improve the impact of the project may have been to increase clarity for the sector through more interactive engagement - e.g. through free webinars, collating and sharing industry feedback on issues they are having in the sector, or a forum to share these issues.

Stakeholder feedback, Atkins

Atkins is an engineering and design consultancy with a stakeholder interest in the decommissioning sector. They have an existing relationship with SEPA and other regulators and were involved in the DecomRegHub project initially through informal feedback and then more formal review and critique of the beta website.

As with Decom North Sea, Atkins similarly recognised the problem that SEPA were trying to address with their project. They added that the decommissioning sector is growing rapidly, but with a mismatch between different organisations’ levels of understanding – noting that some are quite advanced in their thinking, while others are in the early stages. They also felt that some organisations have aims that are contradictory to other organisations’ goals (i.e. the aims and activities of one may directly oppose the aims and activities of another). They felt that SEPA recognised that they needed to work to bring the sector together.

Atkins valued being involved in the project’s early stages as well as in reviewing the beta site: “That was quite refreshing, because I could see how some of the things I’d been discussing, some of the thoughts that we’d shaped, had come into reality and had been further developed.” - Atkins

Atkins felt that the project addresses all of the RPF’s stated objectives: enabling business growth through clarifying regulatory expectations, boosting value for consumers by increasing efficiency in decommissioning activity, projecting a pro-innovation image internationally by being a global leader in collaborative decommissioning and fostering a pro-innovation business culture by providing greater clarity to allow businesses to develop new business plans to address decommissioning challenges.

Lessons learnt on enabling innovation in the decommissioning sector

SEPA reported that a major lesson from the project has been the value of user research specifically in terms of understanding stakeholder needs and in gaining feedback on the usability of the website. In terms of improving this, SEPA felt that it would be beneficial to engage with internal and external stakeholders throughout the project, updating them on project progress rather than leaving most of this until the end. They felt that a more iterative process would ensure that users can feedback as the output develops.

SEPA have also learnt that a cross-regulator project can be a challenge due to ‘culture clash’. They felt that taking the time to build relationships between regulators through regular face-to-face interactions, equal power and a collaborative framework such as DecomRegHub can help overcome this. In doing so, they felt that they achieved a shared way forward, better understood each other and ultimately built the foundations to provide better regulatory support to industry.

SEPA also noted that there are challenges in leading a UK-wide project as a devolved authority. During the early part of the project they felt, at times, that there were some difficulties in gaining senior, executive sponsorship from their centralised regulator partners. This affected the time taken to secure resources and commitment from the partners.

SEPA also noted differences between regulators in cost recovery models that can have a bearing on decisions about resource allocation.

Plans to disseminate learnings

SEPA recognise that the key next step of their work is to share their website and raise awareness of DecomRegHub in the sector. They also aim, in future, to share and apply best practice from the project to other market sectors where there is a need for multi-regulator collaboration to streamline the regulatory journey for businesses.

To achieve this, they had intended to launch the site at several events including at the Scottish and UK Parliaments. However, with the COVID-19 lockdown, these have all been cancelled. They are proceeding to promote and share online but feel that their dissemination and awareness-raising activities have been significantly affected by the lockdown.

Beyond promoting the website, they plan to conduct further stakeholder engagement and dissemination activities both internally and externally in partnership with trade bodies (e.g. Decom North Sea) and share knowledge with other regulators:

“What we’re also keen to do is help other regulators in other regulatory domains where they have similar business drivers and challenges themselves and/or businesses that they regulate have similar needs requiring problem solving.” – SEPA

Value of the RPF

The RPF supported the SEPA effort to enable innovation in their sector, which has and could provided value in a range of different ways:

Greater clarity for industry and other stakeholders around the regulatory requirements associated with decommissioning.

Improved collaboration between regulators to improve their ability to address sector-wide issues.

In the future this could create efficiency savings in decommissioning activities by ensuring clarity on requirements and avoiding wasted efforts.

Improving understanding in the sector around gaps in the market that can lead to innovative technical solutions or services.

“What we’ve tried to do in our project was try to be a game-changer, try to make a transformational change in how regulators collaborate together and how regulators engage with industry.” - SEPA

Thoughts on how the RPF could be improved and better support regulators if the programme were repeated:

SEPA reported high levels of satisfaction with the administration of the fund, particularly identifying the Innovate UK monitoring officers as being very supportive and helpful.

They also reported high levels of satisfaction with the level of support provided and the lack of restriction on what SEPA could do, perceiving this to be due to the funders being very flexible in terms of project content.

SEPA’s suggested areas for improvement include:

Greater flexibility around the phasing of the funding to avoid early pressure to spend money while trying to set up the project.

More time afforded for project setup, particularly where there is a need to reallocate resource or recruit staff.

Consider how project outputs can be embedded into business as usual to maximise return on investment. At the project bidding stage encourage bidders to carefully consider how the project design and phasing can include securing senior buy-in from all regulatory bodies to facilitate a smooth transition of project outputs to business as usual, thereby ensuring delivery of an ongoing improved service to industry.

Case study 2, Care Quality Commission (CQC)

Project at a glance – supporting innovation in health and social care

Role as the regulator

The Care Quality Commission (CQC) is the independent regulator of the health and adult social care sector in England.

CQC’s duties include monitoring, inspecting and rating services to make sure they meet fundamental standards of quality and safety. They publish their findings to people choose care and protect them in case they received care below the fundamental standards.

CQC’s strategy for 2016-2021 sets out 4 priority areas:

• Encourage improvement, innovation and sustainability in care
• Deliver an intelligence-driven approach to regulation
• Promote a single shared view of quality
• Improve efficiency and effectiveness

Project vision

The CQC project team explained that they recognised the need for regulation to keep pace with the rapid development of innovative and technologically-enabled models of care. 

CQC said they wanted to increase their increase their understanding of what good quality innovation looks like in the health and social care sector as well as trial new regulatory methods. To do so, they planned to compose a set of innovation principles to inform an update to their inspection procedures and run a series of regulatory sandboxes. 

The team explained that, according to their vision, this would give more confidence to the sector to innovate in line with quality standards so that they can innovate and ultimately improve quality of care.

“Improving quality, and innovation has been a big factor in [CQC’s vision for the project], and that’s bound to be the same in the future.” – CQC

Motivations for RPF bid

The project team explained that, at the time of their bid, innovation was already a strategic priority for CQC, and that the Fund presented an opportunity for them to speed up the work they were doing in this area.

The team wanted to set out a shared vision of how health and social care organisations can use technologies and innovation to improve quality of care and test out new ways of engaging with innovative providers, such as regulatory sandboxing.

“CQC have always had in its strategy to … encourage innovation. The chief executive and chair were very interested in areas of tech and innovation and what CQC could do to not be seen as a barrier.” - CQC

“We had these priority areas and we wanted to go faster. So [applying to RPF funding] was a chance for us to really accelerate what we were doing.” – CQC

Main activities involved

The project team explained that their project involved:

• Setting up and running three regulatory sandboxes
• Conducting research into what ‘quality’ means and how it can be applied to the sector
• Engaging stakeholders through individual discussions, workshops and events
• Publishing findings through a report and disseminating their ‘Innovation Principles’

Focus RPF programme outputs

1. New business innovation through new licencing or sandbox regimes

2. Reduced time or cost of introducing business innovation

3. Improved business/investor confidence in business innovation regulation

Project set up

Understanding of the problem and RPF purpose

CQC explained that they believed that there is a perception among health and care services of them not supporting innovation. This, along with the lack of guidance on how to recognise and assess innovation for providers and CQC workers, resulted in widespread hesitance within the sector to introduce new innovation for fear of being penalised.

The project team had identified (through firsthand experience with previous innovations and feedback from providers and inspectors) two key problems that they needed to address: lack of clarity around what ‘good’ innovation means; and a need to be quicker to react in dealing with new and innovative types of services.

“We’d never been super clear what we mean when we say innovation, which makes it difficult for our inspectors to know whether organisations are good at innovation.” - CQC

The Kantar team understood that CQC clearly understood the RPF’s purpose from the outset, and that discussions with BEIS had facilitated this.

“It was very clear to me that it was all to support the industrial strategy. People at BEIS were really willing to talk about it and explain it very clearly to me.” - CQC

Challenges in setting up the project

CQC explained that the recruitment process was long and was complicated further by lead times for new recruits. In particular, waiting for the Policy Manager to start presented a challenge because the team was unable to get as much input as they would have liked from him in the research phase of the project.

“Being regulators we have to go through a long recruitment process, and then there were the lead times for getting people to start as well.” - CQC

CQC hired a consultancy firm to conduct research and compile a summary of literature on good innovation and sandboxing, but found that what they got back was less comprehensive than they were expecting, and they therefore needed to do follow-up work to fill the gaps.

In addition, CQC found the reporting requirements from Innovate UK complicated, which meant they were time consuming to understand. They also felt the requirements were designed for commercial organisations, and therefore not suited to CQC’s way of working.

Lessons learnt on project set up and planning

The project team explained they learnt the importance of appointing their Policy Manager in time to feed into the research stage of the project and work directly with the researchers to shape the project.

Through completing the research phase, CQC learnt about the importance of collaboration with the sector and getting their input into innovation, after building a solid basis through a review of the literature.

“It’s about pulling together the literature and then working with people in the sector and building a consensus rather than doing a hard evidence-based research exercise.” - CQC

CQC carried out regulatory sandboxing for the first time which taught them the importance of early engagement with government stakeholders. They found that doing so helped to cement strong relationships which were helpful in holding stakeholders accountable for carrying out sandbox recommendations.

They also realised that they could have been more ambitious straight away. In hindsight, they said that they should have spent more time on looking at how sandboxing would work in CQC rather than whether it would work at all.

Internal engagement

Internally, CQC engaged:

• Wider organisation
• Technological Innovation Steering Group
• Sandbox Committee
• Internal Working Group

“I think the senior leadership team and people in policy have realised that there are other ways of getting across new and emerging pieces of technology quickly.” - CQC

“Having that multidisciplinary team was really important, not just for delivering it with our different expertise, but also if we needed to answer a question.” - CQC

Nature of internal engagement and response from the organisation

CQC explained that they engaged widely across their organisation in order to get the sandboxes up and running, and then to deliver the work and the final report. They involved colleagues across the organisation, including policy, strategy, legal, finance, HR, and engagement teams, as well as all their inspection directorates. The project team engaged other teams by providing them with regular updates on the project via intranet communications, bulletins and the executive team.

CQC said that governance of the project was through the Technological Innovation Steering Group and the Sandbox Committee to ensure organisational buy-in to the work. The team had regular meetings with senior project sponsors who gave advice and guidance to ensure the project ran smoothly. The team also set up an Internal Working Group, consisting of operational staff and other relevant areas of CQC, to offer guidance and practical input.

CQC explained that it was sometimes challenging to fit the project around existing initiatives in the organisation. Whilst they said that their Technological Innovation Steering Group and senior project sponsors helped them to navigate this, Kantar understood that this may have had a negative impact on the project in terms of planning their time. Another challenge was securing the time needed from operational staff, due to their busy schedules. The project team explained that RPF enabled them to buy out staff time for the undertaking, but engaging them was difficult nevertheless. Similarly, getting input from their Engagement staff was difficult due to other pressures on the team, meaning that the support they gave was often down to individual goodwill and interest in the project.

CQC’s engagement activities taught them about the importance of bringing together staff from various parts of the organisation and with a range of expertise. As a result of internal engagement, there is now an increased awareness of innovation across the organisation, and senior level buy-in to sandboxing as a way of understanding new and emerging technology, and a recognition that this should carry on after the RPF funded project. Ultimately, CQC learnt from this experience that pulling together cross-functional teams for a limited period of time to do new, innovative projects is a viable approach to regulation.

External engagement

Externally, CQC engaged:

• 22 innovators e.g. eConsult and Behold.ai
• 2 other UK regulators - MHRA and ICO
• Public bodies e.g. DHSC and NHS Digital
• Academia e.g. UCL
• Charities e.g. Skills for Care
• Professional associations e.g. Royal College of Radiologists
• Consultancy organisations e.g. Community Catalysts

External stakeholder engagement

CQC engaged a wide range of organisations as part of their project. They did so by using existing relationships between organisations, asking their networks to connect them to others, with BEIS support and by connecting to people through conferences they attended. For example, they held a ‘learning day’ with Imperial College London prior to the AI sandbox, bringing together 70 people working in this area across government, which they found very helpful.

CQC developed their innovation principles in conjunction with partners in the health and social care sector, and these partners have all agreed the content of the report and will have their logos on the front page. They will also be involved in the dissemination, and discussions that follow.

In addition, they set up an external reference group with industry leaders in innovation, which they found to be a good way of getting regular input from a core group. They also involved the most relevant and engaged people in the sandboxes themselves.

CQC noted a high level of professionalism and commitment from everyone involved, and highlighted the more informal and personal approach they had used to connect. This helped to lay the foundations for strong relationships and enabled workshops to run smoothly.

“I think it was much more informal and more personal… You could see the benefits when we got them together for the workshops in having built those relationships beforehand.” – CQC

Lessons learnt on stakeholder engagement

CQC found it challenging to get some stakeholders to commit the time and resources for their project, as they often had other priorities. They also said that getting to the right person was sometimes difficult, as many were part of large organisations and they needed to find people with the appropriate technical expertise.

“People’s time was already committed to doing other things, or their priorities were already in a particular area, so it can be difficult. People were doing it in the margins of their time.” - CQC

CQC learnt several lessons on how to overcome challenges with stakeholder engagement. They emphasised the importance of a personal approach (as opposed to going through formal structures) and the need for persistence, kindness and genuine involvement in forging strong relationships. They made calls to all workshop participants prior to taking part, and found these important to explain the purpose of the project and to put people’s minds at ease.

They learnt about the value in involving the sector to identify priorities, risks and issues, finding that collaboration helped them pull together knowledge and avoid mistakes.

CQC also reported that their collaboration with government agencies allowed them to work with a broader scope than had previously been possible. For example, by involving NHSX they were able to land recommendations about what central government needs to do to support hospitals in taking up high-risk technologies. This would usually have been outside of their remit.

Enabling innovation

Initial positive outcomes and achievements

CQC successfully developed a set of principles for what constitutes good innovation in health and social care, in conjunction with NHS England and other health and care organisations. The project team explained that people have been in touch with them asking to see it, indicating that the project has been successful in projecting a pro-innovation image and that there is an appetite for this information. They were planning a consultation on their assessment framework (which informs inspections), but this did not go ahead due to COVID-19.

CQC’s achievements went beyond the initial scope of their project. Their initial idea was to explore how a sandbox would look in their sector and potentially do a test run, however they successfully delivered three sandboxes in different innovation areas and published the final reports on two of them. The project has generated international interest, with three international regulators (in Singapore, Germany and USA) expressing interest in CQC’s findings from two of their sandboxes, thus providing another example of them projecting a pro-innovation image.

Findings from the sandboxes have already led to some policy and commissioning changes in other national bodies that will help accelerate the adoption of technology. As an example, the project team said that their AI sandbox “landed almost perfectly into some policy needs of NHSX and is being used for immediate COVID-19 deployment of [AI] diagnostics.” CQC also said that there was increased demand for certain innovative technology thanks to the AI sandbox in particular, which they felt gave healthcare providers more confidence in the technologies and which will ultimately boost value for consumers.

CQC found that they have been able to advance their knowledge more quickly than they would have done otherwise. They said that they have received excellent feedback from stakeholders, and now have stronger relationships with the sector and other regulators than previously. The strong relationships forged meant they were able to quickly publish information when working with NHSX on the COVID-19 response.

“We’ve given confidence to the sector… if you look now at the use of the things we were looking at in the sandbox, it’s increased massively.” - CQC

“We were particularly pleased with the success of our sandboxes, as this was something that has never been tried by CQC before. The work was done at pace, and the outputs have been well-received both internally and externally.” - CQC

Beneficiaries of the project:

The CQC could benefit through improving their ability to engage directly with industry and being more proactive when it comes to innovation

Once innovation principles are published, health and social care service providers could benefit by feeling more confident about adopting innovative technology that can help them provide a better service

Technology providers could benefit by gaining a better understanding of the regulatory environment surrounding technological innovation

If the project stimulates uptake of innovation, users of health and social care services could benefit by increased access to care and better clinical decision making

Specific stakeholder feedback, Behold.ai

Behold.ai is an AI start-up company that took part in CQC’s AI sandbox. Their involvement was motivated by wanting to increase their understanding of how CQC involvement might be required as AI algorithms are able to perform more autonomous functions (such as diagnosis). They expressed being very satisfied with the experience. They found it useful to have a range of organisations with different functions in the same room, enabling participants to learn “from real world examples rather than learning from theory”.

Their product sits at an intersection of regulators because their product is a medical device (relevant to MHRA) that can provide a diagnosis (relevant to CQC), and needs access to personal data to perfect the algorithm (relevant to ICO). For this reason they particularly appreciated the interaction with other regulators.

Following the sandboxes, they have met with an NHS trust to discuss implementation of autonomous AI algorithms for diagnosis, , which they found positive and which gave them validation that they were on the right track. They are currently in the process of applying for registration with the CQC and are happy to be trailblazers and to be setting high standards within the industry.

“We’re very happy to work with them…working with CQC to set that high standard of care is definitely something that we’re keen to do.” - Behold.ai

They think this is especially important as public perception towards AI might be a challenge due to the need for access to personal data, and that regulation by CQC would help give assurance to the public about such concerns.

Specific stakeholder feedback, Care UK

Care UK is the largest independent sector health provider to NHS and took part in the digital triage sandbox. They are not a developer of technology, but rather a finder of technology and provider of clinicians that use it. Their objective was to work out how they could make better use of some of their infrastructure to deliver more virtual and digital health services.

They found the sandbox workshops very effective for stimulating conversation and came out with an increased level of understanding of digital services and how they need to be viewed. They, like Behold.ai, saw it as a mutual learning exercise, with all parties gaining important insight. Most importantly, they felt that this was a sign of CQC wanting to bring together different perspectives and avoid making decisions “in a darkened room” without input from relevant parties.

“Our view is you need to be involved to influence how things are done. CQC wanted to get insight from people that are on the receiving end of inspections, rather than just going to a darkened room and create an inspection regime because they thought that’s what it needed to be like. They’ve quite rightly put themselves out there and asked who wants to be involved.” - Care UK

Care UK contend that ultimate outcomes remain to be seen, but have a positive view of their experience, and have had a CQC inspector come to one of their clinical call centres for a ‘test’ visit, incorporating some of the initial sandbox findings. This visit was beneficial for CQC in helping them solidify thinking around how future inspections can reflect innovative services used, while helping Care UK to prepare for future inspections.

Lessons learnt from effort to enable innovation in the health sector

The biggest lesson CQC took away from their project was about sandboxing being a viable tool to encourage innovation in their sector, as they explained that “the main thing we’ve learnt is this sort of thing is possible” and they are thinking about how to continue sandboxing after the RPF.

Through their sandboxes, CQC established a new way to engage with developers of new services and technology, and learnt that you need to focus on a particular topic or issue in order to get the most from the engagement. This new internal methodology has also shown CQC that there is merit in collaborating more with innovators, as it enables them to be more proactive, as opposed to a traditional approach of reacting once a product/service is in the market. As a result, CQC leadership has an improved understanding of how to approach regulatory reform work related to innovation, and has contributed to the overarching RPF goal of fostering a pro-innovation business culture.

Another learning for CQC was that innovators need more practical frontline experience in order to improve their products/services and ultimately to encourage quality, safety, roll out and adoption. They found being open and collaborative and engaging with technology stakeholders in a personal, less formal way to be helpful, and highlighted the importance of including people in discussions.

Finally, there was a learning about the importance of clarity and consistency from the regulator in order to establish trust in CQC’s recommendations. They explained that healthcare providers need to have a clear understanding of what needs to change and that CQC will need to follow up with providers to ensure that they are adhering to recommendations.

“Clarity and consistency from the regulator is extremely important. Being clear that innovation is good, we care about it, it’s part of good leadership. Being able to consistently apply that so the organisations that we regulate trust [us].” – CQC

Plans to disseminate learnings

CQC have disseminated two sandbox reports (one on the use of machine learning in clinical diagnostics and one on digital clinical triage tools) and the third report on umbrella organisations that support individuals and groups delivering care in people’s home is due later in the year, subject to delays caused by the COVID-19 pandemic.

CQC plans to publish the innovation principles that they have developed later in the year and are thinking about ways to ensure that their work is as impactful as possible.

“It’s a really strong thing we’ve done, but for it to work we’re going to have to do lots of engagement - workshops, regional consultations…” - CQC

Their evaluation team, with input from the University of Nottingham, has conducted an assessment of the sandboxes intended for internal use, but copies have been made available to BEIS. CQC are seeking funding from NHSX to continue sandboxing and to develop an advice portal. Innovation will continue to be a strategic priority for CQC.

Value of the RPF

The RPF supported the CQC effort to enable innovation in their sector, providing value in a range of ways:

Accelerating the speed of CQC’s work in one of their strategic priority areas (encouraging improvement and innovation)

Bringing together stakeholders with different knowledge, learning from their insights

Building strong stakeholder networks that will remain after project end (exemplified by work on COVID-19 response with partners)

Allowing them to try new methodology that facilitates collaboration and a more proactive approach

“Without the funding, we would not have piloted sandboxing or written the principles for good innovation.” - CQC

“By doing policy in this way and involving the stakeholders so deeply early on, we’re able to use that network to do good things and carry it forward afterwards.” - CQC

Thoughts on how the RPF could be improved and better support regulators if the programme were repeated

Overall, CQC explained that they were very pleased with the support they received from BEIS throughout the Fund’s process and appreciated the flexibility in allowing them to change their approach to include the sandboxes. The following suggestions for improvement were offered:

The Kantar team understood that clearer communications about the timeline for awarding funding would be helpful. CQC felt that clearer communication would help with planning and to avoid the rush in setting up that CQC experienced. CQC suggested that more clarity from BEIS around whether there will be another round of funding would provide more certainty about longer-term plans and help with continuity.

Reporting and monitoring could be more flexible. The project team found that the Innovate UK platform was better set up for commercial organisations than an organisation like CQC. They also felt that, as the Innovate UK monitoring officers’ attention was split between BEIS and the regulators, it was challenging for them to reach a high enough level of understanding of regulators’ projects.

Facilitating more sector-specific collaboration. CQC suggested that BEIS could potentially do more to facilitate collaboration within sectors. They referred to supporting innovation in the “health family” and emphasised the value of working together with MHRA during their project. They felt this was particularly valuable because the innovations they discussed cut across traditional health regulation boundaries (see Behold.ai example on slide 23), so regulators “needed to act together to get it right” by convening around a particular technology.

Case study 5, Ofcom

Project at a glance – using blockchain for telephone number management

Role as the regulator

Ofcom is the regulatory and competition authority for the broadcasting, telecommunications and postal industries of the United Kingdom.

They help to make sure people across the UK are satisfied with what they see and hear on TV and radio, and that programmes reflect the audiences they serve. They provide advice and information to people through their website and call centre, registering complaints from people and businesses. This helps them to take action against firms when they let their customers down.

In regard to the telecoms sector specifically, Ofcom’s work covers issues of infrastructure and competition as well as protecting consumers from harmful practices such as nuisance calls and unfair contracts.

Project vision

Ofcom explained that their project is a proof of concept to demonstrate whether blockchain technology can be used to solve problems around telephone number portability and nuisance calls and fraud.

Number portability is a regulated facility which allows customers to keep their numbers when changing telecom providers and is currently inefficient. Nuisance calls and fraud refer to calls where scammers target consumers by pretending to be somebody they’re not.

Blockchain offers increased transparency between users and is resilient because the number database can be replicated. Updates can be seen in real-time, by all users. It therefore has the potential to improve customer experience when moving a number between providers, and reducing incidents of nuisance calls and fraud.

“It’s a proof of concept project, so an early look rather than an implementation of a solution.” – Ofcom

Motivations for RPF bid

The project team explained that they recognised the RPF as an opportunity to address the aforementioned problems within the sector.

They said they wanted to use the UK number management and UK porting process for landline telephones as a user case reference model with a view to increase their understanding of the capability of the technology and understand how they and their stakeholders might use it in the future.

“We had a conversation with BT …and they noted that it was probably an appropriate time to revisit some of the ways that numbers are managed in the UK - particularly how they are ported.” - Ofcom

“I think broadly now everybody agrees that it’s necessary. The question now is what is the right approach to use.” – Ofcom

Main activities involved

Presentations and meetings with stakeholders to secure buy-in and active collaboration

Developing and testing two concepts:

Telephone management platform – to address problems around number portability

Digital identity wallet – to address problems around nuisance calls and fraud by creating mechanism to authenticate calls

Ofcom also set up two internal sandboxes (that they may widen out to external organisations in future). The purpose of these was to look at uses and implementation of blockchain technology

Focus RPF programme outputs

1. New business innovation through new licencing or sandbox regimes

2. Reduced time or cost of introducing business innovation

3. Improved consumer confidence in innovation

Project set up

Understanding of the problem and RPF purpose

Kantar understood that there are two key challenges Ofcom is trying to address with their project.

The first key issue is inefficiency of number portability. Numbers are moved between service providers using an old fashioned way of routing traffic through networks, which is inefficient as it means that consumers often won’t be able to use their number during the switching period. This causes inconvenience for individual consumers and has a large economic impact on business consumers who may be unable to operate at full capacity during the switching period.

“This is not just a bit of a pain - it’s actually got significant economic impact on some business customers.” - Ofcom

Ofcom also explained they have been noticing an issue of scammers targeting consumers, as a result of insufficient methods for authenticating calls. Tackling nuisance and scam calls is an ongoing priority for Ofcom to protect consumers from harm.

Kantar understood that Ofcom found the purpose of the fund clear at the point of application, but that there were aspects of the fund that they felt didn’t fit their project. These are discussed on the last slide.

Challenges in setting up the project

The project team explained that a challenge in the set up of their project was around resourcing. Ofcom needed programmers to develop the software behind the ‘telephone number management’ platform, and as they do not have programmers that are based internally they had to outsource the task.

Further, Ofcom are bound by IR35 legislation, which makes contracting less attractive because it means that contractors pay the same amount of tax as permanent employees. This makes Ofcom a less attractive destination “for the resources [they] need to build technical, innovative platforms” and consequently securing the right people took longer than anticipated.

“We don’t have programmers in Ofcom - we have to go out and get them…getting the skill sets and resources under IR35 is a problem.” - Ofcom

Another challenge that Ofcom described experiencing was related to the stability of the blockchain needed to build the platform itself, causing delays as a new code base had to be released. This issue was due to the “immaturity of blockchain technology”. Because it’s a relatively new technology, new versions and updates are regularly released, with older versions no longer being supported.

Lessons learnt on project set up and planning

Through project set up, Ofcom explained that they learnt about how to collaborate internally to meet the aims of the project.

Ofcom talked about the fact that they collaborated more broadly and less formally than they usually would. They brought together colleagues from different teams and, rather than relying on a structured governance process, were able to bring together an informal network of people from various teams across the organisation based on their interest.

Kantar understood that this was beneficial for Ofcom’s progress as it allowed them to pull people in to help on an ad-hoc basis and receive input from staff members with different areas of expertise.

“We still had all our governance boards and everything else, but if we needed to pull people in we could do so unofficially and informally, just through people’s interest and desire to help the project. That felt right for the project. ” - Ofcom

“It didn’t need to be so formal as some of our policy projects.” – Ofcom

Internal engagement

Internally, Ofcom engaged:

• Wider organisation
• Staff members from various teams in the organisation through informal network and sandboxes

“Everyone is keen to see how we can use technologies to improve regulation, and that runs through the entire organisation in Ofcom. So getting people engaged is not a problem as long you’re open, you listen to them and you start embedding some of that feedback into the process and people see it.” - Ofcom

“It’s about bringing everyone on the journey.” Ofcom

Nature of internal engagement and response from the organisation

The project team explained that they engaged their own organisation in a range of ways. They engaged widely with other teams (such as policy, competition and consumer protection) within Ofcom by giving presentations and running workshops to explain their project. They found it easy to engage people as staff were enthusiastic to learn about the ways in which technology can be used to improve regulation, and found that the presentations and workshops they ran were effective in educating colleagues about the proof of concepts being developed and the technology behind them. The project team emphasised the importance of “bringing everyone on the journey” by giving people a chance to input their thoughts and ideas to improve their work and secure internal buy in. They said that the response from within the organisation was overwhelmingly positive and that internal engagement played a part in changing mindsets within Ofcom about the potential of technology within the regulatory space.

“We will be introducing this technology and how it can underpin and support the tech consumers… in that regard we are educating and changing mindsets within Ofcom about what the art of the possible is, and how we can extend that art across many different things.” (Ofcom)

A senior member of the project team explained that Ofcom successfully applied for additional funding to create two sandboxes when it became clear that the technology they were exploring had other relevant applications to their sector, such as digital identity. These sandboxes pulled together staff from different teams with the purpose of getting input from across the organisation on uses and implementation of technology. As a result of the blockchain sandbox (the digital identity sandbox is yet to launch), Ofcom felt that they have a better understanding of the capability of the technology, which has helped them in developing the platform and will be helpful also when addressing future problems where technology is relevant.

So far, all sandboxing has been internal to Ofcom only (no external stakeholders have been involved), and they are using some of the outputs to inform their policy thinking. They are currently formulating plans for how to include external stakeholders in their discussions, however the strategy lead highlighted that it has been more difficult to get engagement with external parties recently due to COVID-19.

In addition to the formally appointed project team, Ofcom said that they pulled together an informal network of staff from various functions (e.g. legal, strategy, finance). They did this by discussing and presenting the work they were proposing, and staff were brought in based on their interest and desire to help.

External engagement

Externally, Ofcom engaged:

• A range of UK telecom providers
• ITSPA (trade association
• NICC (UK telecoms technical standards organisation)
• Office of Telecommunications Adjudication
• Other UK regulators (e.g. ICO and FCA)

“We got their active collaboration.. we’ve developed a proof of concept system that is now undergoing testing, firstly internally, but then on a distributed basis.” - Ofcom

External stakeholder engagement

Ofcom said that they engaged with about 20 external stakeholders throughout their project. These were all companies from the UK telecoms industry that use telephone number resources, and included several large companies such as BT and Vodafone, as well as smaller ones such as Gamma and Magrathea Telecom. Ofcom highlighted the importance of engaging smaller organisations as these tend to be “more aligned to the new technology” due to their recent entry to the market, and therefore increased tendency to make use of the latest technology to deliver voice services.

A trade association (ITSPA) provided Ofcom with a list of telecoms providers to contact. Through engagement, Ofcom was hoping to gain support for reviewing processes by testing the products, and support for adopting new technologies in the future. Ofcom felt that they secured an active collaboration with stakeholders, giving advice, testing the platform and providing feedback.

The project team said that they engaged with stakeholders through various avenues: a forum set up by a standards body (NICC), through individual meetings, and through a liaison group.

The liaison group functioned as a steering group where Ofcom would present project progress to various representatives from industry to hear their views and get feedback form them, which they found useful to help work through technical challenges. They would also ask representatives from industry to champion the project within their own organisations.

Ofcom worked with ICO and FCA on the significance of data protection rules and their application to new technologies and found this helpful in furthering their understanding of data management and blockchains.

Lessons learnt on stakeholder engagement

Ofcom said that the biggest challenge they faced in regard to stakeholder engagement was around managing expectations. Stakeholders had various expectations around the project. Some misunderstood Ofcom to be working towards implementing a solution rather than developing a proof of concept, while others incorrectly believed that Ofcom were going to make the adoption of new technology mandatory – an approach that they did not like. However, Ofcom found that stakeholders were generally keen to listen to their ideas and cooperate due to the fact that number portability is a long standing problem in the sector that affects several organisations.

“At the beginning there were these expectations on it, both positive and negative, and we had to change and get people thinking a bit differently.” - Ofcom

Ofcom said that the best way to manage stakeholder expectations and gain their trust was by being honest and upfront in communications about the scope of their project. They explained that engaging with stakeholders at an early stage helped enable them to learn and develop together and promote a willingness to look at this ‘old’ problem in a new way. Ofcom felt that this approach also enabled a more open relationship between them.

However, one stakeholder interviewed said that, although they were engaged early on, they were disappointed to find that collaboration decreased after the initial stage. This is explained on the stakeholder feedback slide.

Enabling innovation

Initial positive outcomes and achievements

Ofcom said that they have built a initial platform to enable number management that has been tested and received feedback, and have built a second version that will be undergoing testing imminently. By doing this they have shown to the sector that innovation through technology is possible, which may open avenues for implementation and ultimately fulfil RPF evaluation outputs 3 and 5 and overall goal of enabling economic growth. The project team explained that they have secured high-level buy in for testing the latest version of the platform by sending out a letter to CTOs at four key stakeholder organisations.

Ofcom also reported having talked to international regulators who are keen to see their work (primarily Mexico and Cyprus), which suggests that the project has started promoting a pro-innovation image for a UK regulator, one of the RPF goals. A senior member of the team explained that Ofcom have also used the work they’ve been doing on the project to feed into a formal consultation process that they launched last year to look at the options for technological solutions to the number portability issue. However, they emphasised that, as the project is still ongoing, there is not yet consensus within the sector on whether technology using blockchain will be the solution that is adopted to improve number portability. The project team mirrored this by highlighting that, as a sector, telecommunications is still in the early days of implementing blockchain technology and that the next couple of months will be instrumental in solidifying thinking around how this can deliver impact and boost value for consumers.

The project team talked about having built strong relationships with sector stakeholders over the course of their project, contributing towards the RPF goal of fostering a pro-innovation business culture in the UK. They described stakeholders as being enthusiastic about contributing to the project and this was reflected in Kantar’s discussions with stakeholders. However, the project team also suggested that it might have affected how Ofcom’s capabilities are viewed within the sector, but Kantar’s interpretation was different. One of the stakeholders expressed some doubts over whether Ofcom will be able to encourage enough organisations to adopt new technology, and worried whether the momentum built up during the project will be maintained long-term.

“It makes people look at regulators and say actually wow they’re doing something different, they’re trying to fix a problem in a way that isn’t using regulatory tools. So maybe there’s a little bit of something else there about what our capabilities are.” - Ofcom

“We’re trying to fix a problem for them, they can recognise that. At the same time we’re building something we can use and the sector can use, so it’s a win-win for us.” – Ofcom

Beneficiaries of the project:

Ofcom will benefit by increased knowledge of innovative technology, influencing their future work. They also expect to benefit from stronger relationships with stakeholders

If the proof of concept leads to a solution being adopted, consumers could benefit from more seamless switching process and a reduction in nuisance calls and fraud

The project lead said that telecom providers could benefit in the future by their knowledge of technology from having been involved in testing, and if proof of concept leads to adoption, by gaining access to the platform

Regulators in other countries may benefit from learnings once project has been concluded, by applying these to their own organisations

Specific stakeholder feedback, Magrathea Telecom

Magrathea Telecom are a UK supplier of managed, next-generation voice services to business clients. Their General Director sits on the council of ITSPA (trade association), and through them got involved in Ofcom’s RPF funded project.

They explained that they have been involved throughout the duration of the project. They contributed to concept development at the beginning and have helped to test and give feedback on the first version of the ‘telephone number management’ platform, and are currently helping to test the second version. They lamented the fact that development took longer than hoped which illustrates the point Ofcom raised about the importance of managing stakeholder expectations.

“It’s been much slower than I would have hoped.” - Magrathea

Magrathea emphasised that the project team had been very communicative and proactive, and were very happy with the contact with Ofcom. They said that it has been effective in opening up discussions and has raising awareness within industry, having been highlighted at several industry events, and praised the fact that Ofcom are seeking to address an issue with a big impact on consumers.

However, they did express some concerns about whether this proof of concept will ultimately lead to a solution. Ofcom does not intend to mandate a solution, and Magrathea questioned whether the process will carry enough weight to create change within the industry without a stricter approach.

“Industry might not have the impetus for change.” - Magrathea

Specific stakeholder feedback, BT

BT is one of the world’s leading communications services companies. Ofcom have a long standing relationship with BT, and said that their decision to explore blockchain technology was a result of discussions with them about how communication providers use numbering resources.

BT funded their own project on blockchain in 2018, and as a result of that project a lot of interest was generated within the telecom industry towards use of technology as a means of addressing the issue of number management, portability and telephony fraud.

For the Ofcom initiative, BT explained that they provided knowledge and expertise in the initial phase of the project, however they felt that they were not consulted or involved as much as they would have expected after that, but did not know why this was the case. They would have expected and liked to have fed into the design and functionality aspects of the platform development as they would have been able to do so based on their own experience in this area.

“It’s been very much Ofcom’s project. We haven’t had much influence since September last year…” (BT)

They said that their CTO recently received a letter from Ofcom inviting them to evaluate the latest version of the technology, and will be starting this shortly. However, they said that meaningful testing will be difficult as the timeline for completing testing is too brief and the viability of the PoC environment as a test platform is likely to be questionable.

Overall, BT said that the project had certainly been a collaborative exercise and made some progress. However, they thought that more collaboration would have been beneficial through the design and development phases, and they expressed some concern that momentum might have been lost due to delays to the project caused by technology challenges.

Lessons learnt from effort to enable innovation in the telecom sector

Ofcom said that the key lessons they learnt were around external collaboration and the technology itself.

The project team reiterated the importance of engaging stakeholders at an early stage and working on things together. They did this within the project by involving stakeholders through both development and testing of the platform. The team talked about the fact that they made sure to be easily contactable and communicative to maintain positive working relationships, and about the importance of being open and honest about their progress to manage expectations.

Ofcom explained that they had worked with stakeholders in a slightly different way than before. Normally they would be liaising with regulatory affairs departments and through formal channels such as information requests and established forums. However, in this project, the project team had worked directly with the technology counterparts within stakeholder organisation, and they found that this was beneficial in giving them a more well-rounded view of the organisation, and therefore more knowledge that they can use when making decisions around policy.

“I think you get a more holistic view of how an organisation operates and what its priorities are, which then informs your approach to wider policy issues.” - Ofcom

The project team explained that they learnt how to engineer the software for the ‘telephone number management’ platform. Due to the complex nature of the technology and it being unfamiliar to Ofcom, they learnt through trial and error, Kantar understood that, through building and testing the platform with stakeholders, they have learnt about important functionality and design aspects. A senior member of the project team highlighted that the sandboxes set up to look at uses and implementation of technology had given those involved a better understanding of the capability of the technology – knowledge that could be useful for other applications besides the ‘telephone management platform’ and ‘digital identity wallet’.

Plans to disseminate learnings

Internally, the project team is presenting the platform to Ofcom staff, and they will be looking at how to apply the learnings from the project to other responsibility areas.

“We haven’t made a final decision on what we will do. We certainly will write something up for general consumption for best practice and to write up what we did. [Team member] is also internally presenting the platform to people. We haven’t really decided yet what we will do externally.” -Ofcom

However, they have not yet made a decision on external dissemination, although they confirmed that they will compile a write up of the process. They said that they believe they are committed to publishing public papers on their blockchain work, in which they will cover performance, costs, findings and learnings.

They explained that they have already delivered design documentation to the NICC (UK technical standards body) in order for the technical community to learn about the platform Ofcom have built, but that this will not be publicly published. In addition, the project team said that they would like to open source the platform i.e. make it freely available to download on the internet.

Value of the RPF

The RPF supported the Ofcom effort to enable innovation in their sector, providing value in a range of ways:

• Increasing understanding of innovative technology and how it can be applied to problems in the sector
• More holistic understanding of stakeholders, meaning more aligned approach on policy issues in future
• Enabling collaboration with the sector by involving stakeholders in the development and testing process
• Building momentum within the industry to find a solution to a long standing problem.

“We have changed people’s thoughts and ideas and got them moving. From a regulatory innovation project that’s fantastic, so the whole industry now is engaged and regardless of whether this platform is successful they want to resolve the problem and they want to resolve it in the next two years. Everyone’s enthusiasm and drive to resolve the problem is kicking in and we’re going to work hard to maintain that.” – Ofcom

Thoughts on how the RPF could be improved and better support regulators if the Programme were repeated

Ofcom were very pleased with the fact that RPF funding allowed them to work towards a possible solution to important issues within the sector and set up sandboxes. The following suggestions for improvement were offered:

The award structure caused some issues. For example, Ofcom explained that not being able to claim VAT which means 20% of the award is lost straight away, and that they would have liked more flexibility in terms of how the funding was split so as to avoid having to give back money that they couldn’t spend.

Ofcom also felt that the evaluation metrics could be updated to more accurately reflect their intentions. They felt that the metrics relied too heavily on commercialisation, and therefore did not adequately take into account how Ofcom works as an organisation (i.e. they are not a go-to-market organisation) and the scope of their project (as a proof of concept).

Ofcom felt that the communication with BEIS could be organised differently. They highlighted that their contact with the BEIS official was helpful, but that they would have wanted easier access to someone who could empower them, for example in their discussions about the funding structure. They suggested having a straight line of communication with the project board where they could ask questions in written format and receive a written response. They also said they would like more communication about future editions of RPF and suggested giving people a quarter’s notice so regulators can prepare by running internal innovation competitions.

1. It should be noted that COVID-19 represented a disruption for progress and affect capacity and collaboration opportunities for some projects. ↩