FOI release

Government Finance Function technology review - 28 February 2020

Published 23 September 2020

Terms of review

I was requested by Emily Ackroyd, Director Strategy & Engagement at GDS to carry out a light touch critical friend review of the GovTech Catalyst Phase 2 Progression Options for the Northern Ireland Audit Office’s Innovation Challenge to develop a data enabled public sector audit approach.

This paper and the views expressed are not intended to be comprehensive or fully researched or informed, but are provided in my professional capacity as [role redacted] for the Government Finance Function, leveraging my experience in leading of the strategy, design and implementation of financial, commercial and HR transformation programmes to implement enterprise-grade data analytics and planning solutions for UK central government departments.

I have spoken with both [name and role redacted] at GDS and [name and role redacted] at NIAO as well as reviewed the documentation supplied by GDS.

Focus of review

The focus of the review was determined by scoping conversations with GDS, analysis of the documentation and key findings. The review findings and my recommendations are structured in three key areas:

1. Identifying the Edges of Innovation
2. Sufficient Capacity, Capability and Resources for Success
3. Ensuring Wider Benefit for HMG and the public sector

Identifying the Edges of Innovation

The Steering Group options paper rightly flagged that there are commercially-available of the shelf (COTS) solutions which could potentially meet most of NIAO’s needs. Cloud-based COTS solutions can be configured, not customised, to meet most government standards and requirements, however some edge cases always remain. NIAO have stated that COTS products have significant gaps and cannot meet the needs of public sector audit.

To address this friction, there needs to be a detailed analysis to understand how these gaps erode the day one benefit of the solution, and then work with the system vendors to close the functional gap. This approach would be compliant with the Government Shared Services strategy to adopt, not adapt, commercially available cloud-based solutions. In addition, there would be value in further investigating the products and developments in areas such as financial services, where the investment in data analytics for management and audit purposes has probably been most prevalent.

NIAO have provided market research across NIAO, and the Welsh Audit Office (WAO) and Audit Scotland (AS) of the COTS products they have assessed, which supplied in Annex A.

The most common COTS core audit systems and additional data analytical capabilities are:

• TeamMate+ (http://www.teammatesolutions.com/audit-management.aspx) has been discounted by all three audit organisations.
• ECAT (https://ecat-group.com/software-features/) is an industry leader used globally as a holistic Audit Management solution.
• Analytics tools such as Alteryx and Tableau would provide the data staging, analytics and data visualisation but have been ruled out on the grounds of cost. These tools are being used elsewhere in government.

The key issue regarding innovation in this context is about the more advanced capabilities to deploy leading edge data analytics with an advanced machine learning capability, potentially paired with intelligent automation to support rapid analysis and workflow. These capabilities are on many product roadmaps of the enterprise-grade suppliers, but these emergent technologies had not yet left the product development/R&D teams into productised solutions. This means that we would be able to leverage vendors’ more substantial R&D budgets rather than having to directly invest in these in the public sector.

In addition, there is an underlying concern about the management of data quality in a proactively manner. Especially when data is sourced from multiple organisation and datasets. Even if the full scope can be achieved data quality management needs to be fully addressed as part of any data-led approach. Whilst technology can be implemented by the Audit Organisations, this does not fully mitigate the data quality from the customer organisation they are auditing. NI Civil Service and the Scottish Government in the early stages of programmes to transform their own corporate systems.

Recommendation #1

Market testing has been carried out, however greater clarity is needed from NIAO, the Welsh Audit Office and Audit Scotland on the specifics why standard Audit COTS products cannot be used to meet the majority of the requirement and why the gaps present critical disbenefits. There is also a need to understand and explore the wider changes necessary in terms of data design and access that may allow large-scale analysis. Addressing these aspects would help focus the next phase. I would recommend that GDS actively supports this assessment as part of the next phase. This would be helpful to shape the innovation component of the solution.

Sufficient Capacity, Capability and Resources for Success

The steering board paper noted a capacity issue in NIAO to develop and deploy a functioning solution. This concern does not just relate to the support and development of a solution, but also to maintain but continuously improve the solution once developed and deployed.

It may be worth tendering for an innovation and transformation partner who can both aid the development and implementation, identify and draw on best practise and build capabilities in NIAO and/or across NIAO, and the Welsh Audit Office (WAO) and Audit Scotland (AS). This can be tendered via G-Cloud 12 in a multi-lot approach to enable SMEs and larger consultancy organisations to bid equitably.

In addition, NIAO may want to consider additional, alternative sources of funding to increase their project team’s capacity such as the Small Business Research Initiative funding (SBRI).

Recommendation #2

Conduct light-touch market testing of third-party partners with established credentials in audit systems and/or transformation to further clarify requirements and potential solutions ahead of any procurement.

Recommendation #3

NIAO engages with SBRI regarding funding additional resource on the core project team.

Ensuring Wider Benefit for HMG and the public sector

Whilst NIAO should be highly commended for fostering stronger collaboration with WAO and AS on the overall approach. Direct joint working could unlock greater efficiencies. A co- investment consortium between NIAO, WAO and AS could together procure, build and deploy the solution would help alleviate the capacity issue. It could also mean the COTS products could potentially be funded between the three organisations. Allowing GovTech Catalyst Funding to focus on the more challenging problems and drive R&D via a dedicated innovation and transformation partner.

There is also a clear link to transformation of the departments and organisations, which NIAO audit, which should be explored further and as a minimum aligned. These departments provide key data to NIAO on an ongoing basis. The NICS as part of the Central Government Transformation Programme is currently preparing to procure new financial, procurement and HR systems (ERP/HCM) which will very likely include data analytics capabilities. Understanding the strategic alignment between the NICS and NIAO programmes could help drive greater value.

Recommendation #4: If Option 4 was funded by GovTech Catalyst Funding, I would recommend creating a NIAO-led consortium fund and resource the programme of work which is out-of-scope of GovTech Catalyst Funding.

Final recommendation on GovTech Catalyst Funding

There is still significant risk that releasing funding for the full phase 2 programme of work will not provide significant return on investment to meet the stated outcomes of the innovation challenge. Risks around the capacity of NIAO to deliver the full outcomes are clearly evidenced by the outcome of phase 1 and could affect the delivery of future work.

However, this risk needs to be sensitively balanced against the prevailing social, political and economic factors; as well as the potential that innovation opportunities can still be realised to address the challenge.

In addition, all the three organisations are unable to obtain leading edge cloud technologies on the basis of cost, which is a major barrier faced by smaller organisations.

I therefore recommend that reduced GovTech Catalyst Funding for Option 4 of the Steering Board Paper is released on the caveat that NIAO address and implement the recommendations included in this report. These recommendations must be closely tracked by the GovTech Catalyst Challenge Governance in addition to any other key tests.

[signature, name and role redacted]

Government Finance Function 28 February 2020