Consultation outcome

Proposed compliance and review methodology

Updated 31 January 2020

Compliance and review methodology

1. Introduction

1.1 The Single Source Regulations Office or SSRO is an executive non-departmental public body, sponsored by the Ministry of Defence (MOD). We play a key role in the regulation of single source, or non-competitive defence contracts. It is vital that single source contracts efficiently deliver the goods, works and services the UK government needs for defence purposes.

1.2 The Defence Reform Act 2014 (the Act) and the Single Source Contract Regulations 2014 (the Regulations) together create a regulatory framework for single source defence contracts. The framework places controls on the prices of qualifying defence contracts (QDCs) and qualifying sub-contracts (QSCs). It requires greater transparency on the part of defence contractors, who must submit reports to the SSRO and the MOD about qualifying contracts.

1.3 The SSRO is at the heart of the regulatory framework, supporting its operation. We keep the framework under review, monitor reporting compliance, give guidance and answer questions about its operation. We analyse data and provide reports and recommendations to the Secretary of State.

1.4 The SSRO’s compliance and review methodology (the methodology) identifies how the SSRO exercises its function, under section 36(2) of the Act, to keep under review the extent to which persons subject to reporting requirements are complying with them. It also identifies how the ongoing review and associated findings link to the SSRO’s wider functions, particularly how its duty under section 39(1) of the Act to keep under review the provision of the regulatory framework established by the Act and the Regulations may be informed by information obtained from compliance monitoring..

1.5 The statutory reports are a fundamental component of the regime, providing details of prices that can be used to support the MOD’s procurement decisions and contract management to achieve value for money and fair and reasonable prices. The methodology has been prepared having regard to the SSRO Data Strategy, which aims to see reported data fully utilised in support of the regulatory framework. Where data is submitted on time and of a sufficient standard to meet the purposes intended by the Act and the Regulations, this will help to ensure that:

• good value for money is obtained in government expenditure on qualifying defence contracts (‘value for money’); and
• that persons who are parties to qualifying defence contracts are paid a fair and reasonable price under those contracts (‘fair pricing’).

1.6 The SSRO’s compliance review function does not involve providing assurance that individual contracts have been priced in accordance with statutory requirements and this is not a feature of the methodology. The SSRO may be asked by the MOD or contractors to give opinions or make determinations about matters related to the pricing of individual, but these are separate functions that depend on a referral being made. The SSRO has published Guidance which identifies how it deals with referrals.

2. The SSRO’s general approach

2.1 Keeping under review the extent to which persons subject to reporting requirements are complying with them involves:

• identifying whether reporting requirements are being met;
• understanding any issues with meeting the reporting requirements;
• linking the review and its findings to appropriate actions; and
• reporting the review findings.

2.2 It is helpful to have some broad measures of whether reporting requirements are being met that can be monitored over time. There are three key performance indicators that the SSRO will apply when measuring compliance against statutory reporting requirements:

1a) “All required reports have been submitted within the relevant deadlines”;

2a) “Reporting obligations have been met first time for all reports submitted, in accordance with the Regulations and relevant statutory guidance”; and

2b) “Reporting obligations have been met either first time or in subsequent submissions for all reports submitted, in accordance with the Regulations and relevant statutory guidance”.

2.3 This methodology aims to contribute to achieving good quality data from contractors. As set out in our data strategy, data that is relevant, comparable and reliable^{[footnote 1]} can be utilised in procurement decisions, contract management and in the development of the regulatory framework. Our assessment, under this methodology, of the quality of data submitted by contractors will focus on its relevance and reliability.

2.4 The primary means by which the SSRO identifies whether reporting requirements are being met is by reviewing the report submissions made by contractors into the SSRO’s Defence Contract Analysis and Reporting System (‘DefCARS’ or ‘the system). Section 3 of the methodology deals with how the SSRO will go about examining whether the reports are submitted on time and whether they contain the required information.

2.5 The SSRO will work in partnership with stakeholders to identify whether reporting requirements are being met and to understand any issues with meeting those requirements. This will provide a more complete picture than could be obtained from reviewing the report submissions alone. Sections 3 to 5 of the methodology identify how we will interact with contractors and the MOD to inform our compliance reviews. Our aim is to develop a shared understanding with stakeholders about issues raised and how these should be addressed.

2.6 Where issues are identified from a compliance review after feedback from contractors or the MOD, appropriate action should be taken to address them. Action may include:

• correction of report submissions by contractors;
• enforcement action by the MOD; or
• development of reporting guidance and DefCARS by the SSRO, or recommendations to the Secretary of State for legislative change.

2.7 Section 6 of the methodology specifies the circumstances in which the SSRO will refer issues to the MOD in relation to individual reports so that it is aware and can take action. The links between compliance reviews and the SSRO’s wider functions are set out in section 7.

2.8 The SSRO will report the findings of its compliance reviews, as set out in section 8. This transparency provides an evidence base that informs discharge of the SSRO’s functions and the operation of the regulatory framework.

3. Review of report submissions

3.1 The SSRO will routinely examine report submissions made by contractors on DefCARS to identify whether reporting requirements are being met. This review will consider the timeliness of each submission, whether it contains the information prescribed in the Regulations, and whether it is consistent with statutory guidance issued by the SSRO.

3.2 Where reasonable, the SSRO will rely on automatic checking within DefCARS to carry out the primary review of report submissions. The system has been developed to carry out a range of automatic validation checks at the point of report submission, providing information on whether reporting requirements are being met. Automatic validation checking has the additional benefit of assisting users to make good quality submissions, because validation warnings are flagged prior to final submission and there is an opportunity for the contractor to address any issues raised. The SSRO will keep the set of validation warnings under review to ensure they target reporting requirements in a proportionate way. Consideration will be given to the extent to which issues continue to be raised on report submissions that are not linked to validation warnings. The SSRO has published a list of the validation warnings to inform stakeholders and proposes to continue this practice.

3.3 The SSRO will routinely carry out manual reviews of reports after they have been submitted. This will generally involve a limited, risk-based review of the report, based on the available information, including:

• unresolved validation warnings, excluding any matters that are not material, such as obvious rounding differences;
• issues raised on the submission by the MOD and any response from the contractor; and
• inconsistencies between the submitted report and the supporting information.

3.4 The SSRO aims to commence such reviews after 15 working days have elapsed from submission of the report (to allow time for the MOD to carry out its own reviews, as set out in section 5), and to complete the reviews within a further 15 working days.

3.5 The SSRO may raise an issue with a contractor to inform its understanding of the extent to which reporting requirements are being met. The SSRO will generally raise an issue with a contractor if it appears that a report:

• has not been delivered within the legislative timeframes; or
• appears to be incomplete, inconsistent, erroneous or lacking in detail, having regard to the reporting requirements.

3.6 This provides the contractor with an opportunity to clarify whether a reporting requirement has been met and whether there is any inherent difficulty with DefCARS, reporting guidance or the Regulations.

3.7 In some instances, the SSRO’s review of a submitted report will raise an issue as to whether the pricing requirements of the regulatory framework have been met. For example, the reported calculation of the contract profit rate may appear not to follow the six steps required by section 17 of the Act and regulation 11. In such cases, an issue may be raised by the SSRO to inform its understanding of how the provision of the Act and the Regulations is operating, in line with its duty under section 39(1) of the Act to keep under review the provision of Part 2 of the Act and the Regulations.

3.8 When raising an issue, the SSRO will categorise the issue into ‘reporting’ or ‘pricing’ matters. An issue raised by the SSRO in relation to its section 36(2) function will generally be categorised as a ‘reporting’ issue, and as a ‘pricing’ issue if it is in relation to its section 39(1) function.

3.9 The SSRO will generally raise issues using the compliance functionality provided in DefCARS. This allows issues to be efficiently raised and responded to within the report submission in DefCARS. The functionality was introduced in 2018 and will be kept under review.

3.10 The key performance indicators 2a) and 2b) will consider whether any ‘reporting issues’ have been raised on a submission by either the SSRO or the MOD. Any issues categorised as pricing issues will not impact these indicators. We will count the submission as a ‘pass or fail’ of an entire submission once made, regardless of the number of errors that may be apparent in the initial submission. We will report these indicators as a twelve-month rolling average proportion of report submissions.

4. Additional compliance monitoring activities

4.1 The SSRO may carry out detailed targeted reviews of selected report submissions, in addition to the routine reviews under section 3. It may also conduct thematic compliance reviews to understand how a specific aspect or aspects of the reporting requirements is being dealt with across all reports or a sample of reports.

4.2 The aim of an additional review is to enhance the SSRO’s understanding of the extent to which reporting requirements are being met. A targeted or thematic review may provide insight into the effectiveness of automatic and routine compliance reviews, and how the provisions of the regulatory framework are being applied in practice. The SSRO may proactively target areas where issues have historically been identified, or which have not been the focus of previous review.

4.3 The SSRO will carry out focused engagement with stakeholders, as necessary to complete a review. This may include discussing the processes that a contractor has in place to meet reporting requirements. The SSRO will aim to provide constructive feedback on its reviews and report its findings as appropriate.

5. MOD reviews of report submissions

5.1 The SSRO has developed DefCARS to facilitate review of reports by the MOD. Officers with access to the system can review reports within their area of authorisation and raise issues with contractors about the content of the reports. These issues can be raised and responded to within DefCARS in relation to a report submission. If issues are identified and addressed, then this will improve the quality of data in the system. The intention is to maintain this functionality, subject to appropriate review.

5.2 The SSRO will rely on the findings from the MOD’s reviews of reports to enhance its understanding of the extent to which reporting requirements are being complied with and how the regulatory framework is operating in practice. The MOD may identify issues and provide information that would not otherwise have been available to the SSRO. For example, the MOD can check not only whether reports are complete, but whether they are accurate and consistent with the qualifying contract. The information gathered from analysing issues raised by the MOD will inform the SSRO’s prioritisation of further compliance activities and its wider work.

5.3 The SSRO will seek to avoid duplication between the MOD’s reviews and its own compliance reviews, with the aim of avoiding any unnecessary burden on industry. After a report has been submitted, which involves automatic validation checking in DefCARS, the SSRO will allow time for the MOD to review a report submission before it undertakes its own review. The period of time allowed is currently 15 working days from the date of submission, in line with the MOD’s own internal target for reviews.

6. Raising matters with the MOD

6.1 The SSRO will work with contractors to encourage timely and good quality submissions. If an issue is not satisfactorily addressed through engagement with a contractor, the SSRO will consider whether to raise it with the MOD. Bringing issues to the attention of the MOD allows the MOD to:

• provide feedback that informs the SSRO’s understanding;
• resolve the issue in cooperation with the contractor; or
• take appropriate enforcement action.

6.2 Feedback is an important part of compliance reviews and the SSRO will work with the MOD to obtain its input on any issues raised. In each instance, the SSRO will be interested in the MOD’s views on whether there is an issue, the underlying causes, its relative importance, and how the issue can, or should, be addressed. The feedback should assist the SSRO to understand whether reporting requirements are being met and how the regulatory framework is being applied.

Timeliness of submissions

6.3 Where an expected contract^{[footnote 2]} or overhead report^{[footnote 3]} is not submitted, the SSRO will attempt to contact the contractor within five working days of the expected submission date and notify the MOD of the delay at the same time. Where the delay relates to the submission of the strategic reports^{[footnote 4]}, the SSRO will inform the MOD only. The SSRO will monitor the status of outstanding submissions and provide an update to the MOD at regular intervals.

Quality of submissions

6.4 The SSRO will refer unresolved reporting and pricing issues to the MOD that fall within the following categories:

• submission incomplete or otherwise fails to meet the requirements of the Regulations, either individual elements or the entirety of the submission;
• information is internally inconsistent or inconsistent with other supporting information provided by the contractor;
• information included in the submission or as supporting documents is not reasonably sufficient to meet regulatory requirements;
• information appears out of line with the Act or the Regulations;
• information appears out of line with statutory guidance (guidance on Adjustments to the Baseline Profit Rate or Allowable Costs).

6.5 The SSRO will further note whether it considers that the issue forwarded to the MOD is of a high, medium or low priority. The priority assigned to an issue will be based on its potential impact on operation of the regulatory framework. This involves an assessment of relative priority. The SSRO takes seriously any apparent non-compliance with the requirements of the regulatory framework.

6.6 Based on the MOD’s responses, the SSRO will log referred issues into the following categories:

• there has been no failure to comply with the Regulations or deviation from the Statutory Guidance;
• the issue has been resolved in cooperation with the contractor, for example by requiring a correction report, on-demand report, or amendment to the contract;
• a compliance notice has been or will be issued;
• a referral has been made to the SSRO for an opinion or a determination;
• there was a clear reason for deviating from statutory guidance;
• the issue should be dealt with through changes to guidance, DefCARS or the legislation;
• other responses, including where the MOD considers no action should be taken for some specified reason;
• no response.

6.7 In each case, the SSRO will rely on explanations from the MOD to enhance its understanding of the issues. The SSRO will use these categories for reporting purposes and the feedback will be used to plan the SSRO’s future work in support of the regulatory framework.

7. Supporting compliance and improving the regime

7.1 The SSRO will coordinate its compliance review processes with its support functions. Early engagement will assist the SSRO to identify difficulties that contractors may have with understanding, or complying with, reporting requirements. Support can be targeted towards identified difficulties, for example through on-boarding sessions, help desk assistance and training on reporting and the use of DefCARS.

7.2 The SSRO will consider the findings from the compliance review process when prioritising its future work. Compliance reviews may disclose areas of difficulty in complying with the pricing or reporting requirements of the regulatory framework that the SSRO can address through its wider functions, for example:

• revised support arrangements;
• new or updated guidance;
• developments in DefCARS; or
• recommendations for legislative change.

7.3 To support this consideration, the SSRO will log issues from compliance reviews that need to be addressed and will seek to identify repeat or consistent themes. The SSRO expresses its prioritisation principles in its corporate plan and will reflect these when addressing any issues identified through its compliance work.

8. Reporting findings and providing feedback

8.1 The SSRO will detail the findings from its compliance work in an Annual Compliance Report, which will consider the performance indicators on timeliness and quality of report submissions, supplemented with further analysis as appropriate. The Compliance Report will set out a factual commentary on the data, highlighting emerging themes and areas where changes or improvements are required.

8.2 The SSRO will take care that the Annual Compliance Report does not disclose information contrary to Schedule 5 of the Act. The SSRO will publish its report so that it is clear how it has discharged its functions, and so that the findings provide an evidence base for future work by the SSRO and engagement with stakeholders.

8.3 The SSRO recognises that some issues may take time to resolve but are capable of being corrected on DefCARS. Some other issues, however, relate fundamentally to how the provisions of the regime apply in practice and may be difficult to understand or address. In order to ensure that issues identified are not left without consideration, the SSRO will provide regular feedback to the MOD, identifying where matters have had an impact either on data quality or the operation of the regulatory framework.

8.4 The SSRO will provide feedback to contractors with multiple contracts on compliance themes identified from their report submissions, as well as on any issues that remain outstanding. We will consider how feedback could be delivered through the system in future.

1. “Data will be relevant if what is prescribed by the Regulations and submitted by contractors is that which is needed for the regulatory framework and no more. The data must also be submitted on time. To be usable, data must be comparable over time and will be standardised to aid comparability. Data will be reliable if it is accurate when submitted and complete.” ↩

2. As required by Part 5 of the Single Source Contract Regulation 2014 ↩

3. As required by Part 6 of the Single Source Contract Regulation 2014, excluding Strategic Industry Capacity Report (SICR) and Small and Medium Enterprise (SME) submissions ↩

4. SICR and SME submissions ↩