Independent report

Appendix 2: Data sources and Methodology

Published 17 September 2020

SSRO functions

The SSRO must keep under review the extent to which persons subject to requirements under the Act are complying with them.

The SSRO’s compliance methodology directly supports two of our statutory functions:

• the requirement under section 36(2) of the Act to keep under review the extent to which persons subject to reporting requirements are complying with them; and
• the requirement under section 39(1) of the Act to keep under review the provision of the regulatory framework established by the Act and the Regulations.

In carrying out these functions, the SSRO must aim to ensure that:

• good value for money is obtained in government expenditure on qualifying defence contracts (value for money); and
• that persons who are parties to qualifying defence contracts are paid a fair and reasonable price under those contracts (fair pricing).

Submission requirements

Defence contractors are required to submit two types of reports, as summarised in Figure 1.

Figure 1: reports required under the regulatory framework

Our analysis refers to three types of contract reports, being the initial, update and completion reports:

• ‘initial reports’ being the Contract Pricing Statement (CPS), the Contract Reporting Plan (CRP) and the Contract Notification Report (CNR), known collectively as the Contract Initiation Report (CIR); and
• the ‘update reports’ being the Interim Contract Report (ICR) and the Quarterly Contract Report (QCR); and
• the ‘completion reports’ being the Contract Completion Report (CCR) and the Contract Costs Statement (CCS).

We also refer to two types of supplier report:

• ‘overheads reports’ being the Qualifying Business Unit Estimated Cost Analysis Report (QBUECAR), the Qualifying Business Unit Actual Cost Analysis Report (QBUACAR), the Estimated Rates Agreement Pricing Statement (ERAPS), the Estimated Rates Claim Report (ERCR), the Actual Rates Claim Report (ARCR) and the Rates Comparison Report (RCR) which is only triggered by a written notice from the Secretary of State; and
• ‘strategic reports’ being the Strategic Industry Capacity Report (SICR), and the Small or Medium Enterprises (SME) Report.

Overheads reports may be required for a Qualifying Business Unit (QBU) in some years and not others, depending on whether the ongoing contract condition^{[footnote 1]} and QBU threshold^{[footnote 2]} are met. The SSRO does not have independent access to the information required to assess whether these requirements are met for a QBU and is dependent on notifications from the MOD and the contractor. The analysis therefore considers the timeliness of those submissions which have been received.

The reporting requirements are set out in Parts 5 and 6 of the Regulations. The SSRO supplements those requirements with reporting guidance, which contractors must have regard to when completing the reports.

Information included in this report

The SSRO has developed DefCARS, with input from the defence industry and MOD users, to provide an easy to use and secure means of submitting the reports. The majority of the analysis presented in this report is drawn from the data submitted into the on-line version of DefCARS, as it focuses on 2019/20, however some analysis is based on information held outside of the system.

The contract data in this report is sourced from the latest of the CPS, CNR, QCR, ICR, CCR or CCS. Data related to quality and timeliness of submissions are sourced from the DefCARS, and the SSRO’s internal compliance monitoring logs.

This report analyses reports submitted on or before the cut-off date of 30 June 2020. It considers:

• QDCs and QSCs entered into between 1 April 2015 and 31 March 2020 and notified to the SSRO by 30 April 2020; and
• associated contract and supplier reports due for submission by 30 April 2020.

As of 30 April 2020, the SSRO had been notified of 296 QDCs and QSCs that had been entered into between 1 April 2015 and 31 March 2020 and reports had been received by the SSRO for a total of 278 of these contracts by 30 April 2020. The key statistics relating to these contracts have been reported in the SSRO’s Annual qualifying defence contract statistics: 2019/20 and the detailed messages from that publication are not repeated here.

The 278 QDCs and QSCs for which reports had been received were placed with 120 individual contracting companies, across a total of 85 GUOs^{[footnote 3]}. The MOD placed 60 per cent of qualifying contracts (equating to 167 contracts) with the top ten GUOs or companies subsidiary to them. Figure 2 sets out the distribution of the 278 QDCs and QSCs placed by the MOD, showing the numbers of contracts awarded to the top ten GUOs by year.

Figure 2: Number of QDCs / QSCs, where reports have been received, by Global Ultimate Owners

Reporting on compliance issues

The SSRO reviews the reports submitted by contractors and seeks to understand the information provided, relying on automated validation checking in its DefCARS system. More detailed, manual investigations around validation warnings that have not been addressed by the contractor are also considered. The SSRO focuses its manual reviews on issues that can be linked to validation warnings. It may base future targeted or thematic reviews on issues identified from validation warnings.

Ensuring the accuracy of reported information depends on the MOD also checking reported information and taking action where appropriate. We have enabled the MOD to raise queries with contractors in DefCARS and for contractors to respond, and this information is reviewed by the SSRO as part of the implementation of its compliance methodology.

The SSRO’s review process starts after a contract submission has been made and, depending on the timeliness of contractor responses to issues, can continue over a period of several months. The SSRO queries potential errors with report submissions that impact data quality, such as:

• internal inconsistencies;
• arithmetical errors; and
• matters that appear to be erroneous, for example incomplete information.

To keep the provisions of the framework under review, the SSRO may also raise issues with contractors in order to understand relevant explanations relating to the pricing of contracts. In line with the Act and Regulations, contractors are obliged to report the facts, assumptions, and calculations relevant to each element of the Allowable Costs and to describe the calculation used to determine the contract profit rate, including all adjustments to the baseline profit rate.

The compliance approach includes querying obvious errors (for example internal reporting inconsistencies) as well as raising any issues if completed reports seemed to be erroneous (for example reports containing incomplete or limited information).

We consider the extent to which persons subject to reporting requirements have complied with their obligations and what this tells us about how the regime is operating. To deliver our statutory aims and functions, we seek to achieve the following:

• good quality data from contractors, that is relevant, comparable and reliable;
• identification of issues related to meeting reporting requirements (reporting issues), to data quality and to the application of the regulatory framework, for example pricing control;
• a shared understanding with the MOD and contractors about identified issues; and
• appropriate action by the SSRO, the MOD and contractors to address issues. Action by the SSRO to address issues may include revised support to contractors, updated guidance, development of DefCARS and recommendations for legislative change.

Additionally, we have reviewed the reports submitted by contractors to understand the operation of the provision of the Act and Regulations with respect to the pricing of contracts. While we have sought to understand the operation of the pricing provisions of the regulatory framework by reference to information reported on individual contracts, we have not audited reported costs or profit rates on a contract by contract basis, nor provided any assurances that individual contracts have been priced in accordance with statutory requirements.

As part of the compliance methodology, the SSRO typically raises queries arising from report submissions directly with contractor. If the contractor does not respond to issues or provides a response that does not address the issues raised, the SSRO passes these matters to the MOD. Issues raised with the MOD may involve both compliance with reporting requirements and the way in which the system of pricing contracts is being applied.

Where the SSRO raised concerns with the MOD on pricing issues, particularly as to how the price control provisions of the Act and the Regulations were being applied, these concerns were raised for the following circumstances:

• the facts, assumptions and calculations relevant to an element of the Allowable Costs suggested a breach of the Act and the Regulations or deviation from the statutory guidance which was neither reported nor explained;
• the calculation made under Regulation 11 of the Regulations, including any adjustment under the six steps, to determine the contract price of a QDC appeared to be a breach of the Act, the Regulations or a deviation from the statutory guidance but was neither reported nor explained;
• an unsatisfactory explanation was provided for an apparent contravention of the Act or the Regulations; or
• a deviation from the statutory guidance was reported by a contractor; and other information material to the pricing of the contract was reported and this appeared to suggest a failure to comply with the Act, the Regulations or a deviation from the statutory guidance.

Data revisions

All historic data has been revised since the previous annual compliance report, due to a change in methodology to measure the quality of submissions. Previously, all issues raised against a report were included, but this year an additional criterion has been applied. Only reporting issues (not pricing issues) that result in a contractor providing additional data or resubmissions are counted as issues that affect the quality metric. Issues that were raised, but after discussion with the contractor turned out not to be issues do not count against the report when measuring quality of submissions. This has had the effect of reducing the number of issues raised by report in the analysis.

Adjustments to data

All data is as reported to the SSRO. Some adjustments have been made in circumstances where there are known, and significant, data quality issues so that the analysis is not misleading. In summary, the following adjustments were made in a small number of cases:

• some report due dates were amended where the due date reported by the contractor was known to be wrong; and
• some contractors had incorrectly submitted reports into the system, or had submitted reports as new submissions instead of corrections (and vice versa).

Additionally, we have manually added to the analysis where the contractor submitted a report outside of the DefCARS system. Mostly, this was for Strategic Industrial Capacity Reports (SICRs), which cannot be submitted in DefCARS.

Definitions

A QDC is a non-competitively procured defence contract with a value of £5 million or more. If a sub-contract of a QDC is also awarded without competition, has a value of more than £25 million and is assessed by the prime contractor, it becomes a QSC.

A contracting company is defined as a UK or non-UK company based on the registered address of the contracting company, sourced from the contract report submissions. The SSRO has grouped contracting companies into their respective GUO by considering whether the GUO controls a majority (greater than 50.01 per cent) of the voting rights of the company in question. Where a company has no single entity with a controlling majority, the company itself is considered the GUO of the corporate group. One contracting company has submitted its own strategic reports separate to its ultimate parent undertaking. For the purposes of the analysis, we have counted it as its own separate GUO.

Analysis

Analysis looking at the timeliness of report submissions uses the report due date to group the analysis by the financial year of 1 May to 30 April each year. Contractors have one month after the contract becomes a qualifying contract to submit their reports. For example, a contract entered into on 30 March 2020 will have an initial report due date of 30 April 2020 and therefore the initial report would be included in the analysis for the 2019/20 financial year. The analysis aims to be consistent with the Annual Qualifying Defence Contract Statistics 2019/20, which reports on contracts by the government financial year in which they became QDCs/QSCs. References to ‘initial reports’ includes any on-demand CPS and CRP submissions.

Our analysis looking at the quality of report submissions uses the report submission date to group the analysis into the relevant financial year, this is different to our analysis of timeliness which is currently undertaken outside of the system and uses the report due date to group the analysis by financial year. There may therefore be a variance in the numbers of submission considered for assessing quality and timeliness.

Queries are raised directly with contractors and if an issue arises on a CIR submission and is applicable to each of the three initial submissions, it is counted as three individual issues raised.

Totals are calculated on unrounded figures, before being rounded for presentational purposes.

1. The “ongoing contract condition” is met in relation to a financial year if, at any time in that year, obligations relating to the supply of goods, works or services under one or more of the qualifying defence contracts referred to in subsection (4)(a) or (b) (as the case may be) are outstanding (s25(5) of the Act). Regulation 31(2) notes that this is subject to a minimum value of qualifying defence contract for reporting requirement to be imposed, and the amount specified for the purposes of that subsection is—
   (a) for the financial years ending on 31 March 2016 and 31 March 2017, £20,000,000;The “ongoing contract condition” is met in relation to a financial year if, at any time in that year, obligations relating to the supply of goods, works or services under one or more of the qualifying defence contracts referred to in subsection (4)(a) or (b) (as the case may be) are outstanding (s25(5) of the Act). Regulation 31(2) notes that this is subject to a minimum value of qualifying defence contract for reporting requirement to be imposed, and the amount specified for the purposes of that subsection is— (a) for the financial years ending on 31 March 2016 and 31 March 2017, £20,000,000;
   (b) for subsequent financial years, £50,000,000. ↩

2. The total value of what it provides for those purposes in that period is at least £10,000,000. ↩

3. The legislation refers to the ‘Ultimate Parent Undertaking’ to be consistent with the Companies Act 2006 which defines ‘parent undertaking’ and ‘subsidiary undertaking’, however contract report submissions do not identify the Ultimate Parent Undertaking. The SSRO uses the Global Ultimate Owner (GUO) definition from the Orbis database, provided by Bureau van Dijk. ↩