Consultation outcome

Response to the consultation

Updated 31 January 2020

1. Background

1.1 The SSRO launched a public consultation on a revised compliance and review methodology (‘the methodology’) in October 2019. This followed consideration of responses to a working paper that was issued to key stakeholders in relation to this topic in July 2019.

1.2 The consultation document was circulated widely and published on the SSRO’s website. This document summarises the key comments raised by respondents and the SSRO’s response to them.

1.3 We would like to thank respondents for sharing their views on the consultation. We have considered all submissions in drafting this consultation response document. This document summarises the main points raised by respondents and, where appropriate, the SSRO’s response.

1.4 Alongside this consultation response document, we have published the updated methodology. The methodology will take effect from 1 April 2020. This allows reasonable time for stakeholders to become familiar with the methodology before it takes effect.

2. The consultation process

Process

2.1 The SSRO sought feedback from stakeholders on its updated methodology through a public consultation. The consultation commenced on 14 October 2019 and closed on 6 December 2019, a period of eight weeks. Prior to consultation, the SSRO had issued a working paper on the topic during July 2019 and held a workshop with key stakeholders where we obtained feedback on the changes that were being considered.

2.2 The consultation set out the proposed updated methodology, alongside the SSRO’s reasons for making the update. It focused on the SSRO’s review obligations under sections 36(2) and 39(1) of the Defence Reform Act 2014 (the Act) and also on how, having gathered intelligence on the operation of the regime, we have continued to develop our approach to implementing the methodology published in 2017.

2.3 Consultees were asked to respond to a total of nine consultation questions, which are considered in detail in the sections below. Any changes made following the responses are also detailed. We have also taken the opportunity to make minor amendments to clarify some aspects of the methodology.

2.4 The responses gave the SSRO an understanding of views on the proposed methodology and suggested alternative wording to some paragraphs, as well as alternative approaches for the SSRO to consider. We have grouped the comments received into key issues by each question and responded to them in this document.

Breakdown of responses

2.5 We are grateful to all those who responded to the consultation. We received a total of seven responses to the consultation as set out in Table 1:

Table 1: Number of consultation responses

	Government	Industry	Trade Association
Number of responses	1	5	1

2.6 Four of the seven responses are published on our website alongside this response document. Three others were either marked confidential or did not provide affirmation that the response could be published. Respondents, other than the MOD and the ADS Group (‘ADS’), are not identified in the document but are referred to as ‘respondents’.

3. Key issues raised by respondents

3.1 The key comments raised in the responses are detailed below by question, alongside the SSRO’s response.

Question 1: Does the methodology clearly demonstrate how the SSRO will exercise its s36(2) function and how its s39(1) function may be informed through our work in this area?

3.2 The demonstration of how the SSRO will exercise its s36(2) function and how its s39(1) function may be informed through its compliance work was broadly considered to be appropriate and was understood by most respondents. Three comments were raised in relation to this:

• a respondent considered that by undertaking compliance work in this way, the SSRO may threaten its independence in the case of opinions or determinations, particularly if it was the SSRO who raised an issue that was later subject to referral, in the first place;
• ADS considered that while the methodology demonstrated how the SSRO fulfilled its obligations under s36(2) of the Act, it was more difficult to see how this work may link to its s39(1) obligations; and
• the MOD considered that the proposed performance indicators 2a) and 2b) should not refer to whether reports were in accordance with statutory guidance. This was on the basis that statutory guidance is not included in either of section 36(2) or 39(1).

The SSRO’s response

3.3 The legislation contains four clear roles in relation to reporting requirements, which combine to support a database of information that can be used for the purposes of achieving value for money and fair and reasonable prices:

Table 2: Roles in reporting compliance

Role	Responsibility
Submission	Contractors are required to submit the reports.
Review	The SSRO keeps under review whether reporting requirements are being met and keeps under review the provision of the regulatory framework.
Enforcement	The MOD is responsible for enforcing enforcement obligations.
Referral	The SSRO gives opinions and makes determinations if requested, including on penalty notices issued by the MOD.

3.4 The SSRO accepts that it should avoid exercising its review functions in a way that could prejudice its referral functions. It seeks to avoid any suggestion of predetermination and, with this in mind, has clarified in paragraph 2.3 of the methodology the basis on which it will measure its performance indicators 2a) and 2b), in essence relying on information provided by contractors and not arriving at an independent conclusion.

3.5 The SSRO’s view is that its approach to informing its s39(1) function through the methodology follows a similar process to that outlined above. An issue may be raised on a submission to allow the SSRO to better understand how the provisions of the legislative framework have been applied in practice, as set out in paragraph 3.8 of the methodology. Our view is that it would be inefficient and ineffective to apply the methodology in a silo fashion to exclude any intelligence gleaned from reviewing reports which may inform our s39(1) work.

3.6 The SSRO considers that our statutory guidance forms part of the reporting obligations as contractors must have regard to the guidance issued by the SSRO when preparing reports. As such, it is our view that it is appropriate for the two indicators to refer to the statutory guidance.

Question 2: Is the SSRO’s approach sufficiently clear from the methodology?

3.7 All respondents noted that the approach was generally clear. There were two comments raised in relation to this:

• ADS stated that the effectiveness of the approach would depend on the ability of the SSRO and the contractor to build the trust to allow for collaborative working arrangement. ADS’s view is that to do this, the SSRO needs to be able to be flexible and potentially tailor its approach for different contractors; and
• the MOD stated that the SSRO’s work should not be an audit of contractor submissions as it is not the responsibility of the SSRO.

The SSRO’s response

3.8 The SSRO welcomes the input from ADS and accepts that trust is essential. One of our corporate objectives is to maintain effective and comprehensive engagement with stakeholders. Our aim is to ensure that the engagement we have with contractors and the support that we provide aligned to the methodology is both positive and constructive. There is a balance to be struck between automation to achieve an appropriately risk-based approach that can be applied across a variety of contractors, and the flexibility to deal with each contractor in a tailored way. We state in the methodology that our support can be targeted towards identified difficulties, for example through on-boarding sessions, helpdesk assistance and training on reporting and the use of DefCARS and in this way we may be able to tailor our approach, and our support, for different contractors.

3.9 We agree that care needs to be taken to ensure that the work under our methodology is not viewed as an audit of the submission, particularly as this is not one of our functions and as noted in the methodology our work is not an assurance exercise. We have updated paragraph 1.6 of the methodology to emphasize these limitations and note that our compliance review function does not involve providing assurance that individual contracts have been priced in accordance with statutory requirements, nor is it an audit of individual submissions.

Question 3: Is the SSRO’s review process sufficiently clear from the methodology?

3.10 All respondents noted that the review process was generally clear. There were three comments raised in relation to the review process:

• a respondent considered that the SSRO needed to ensure that it adhered to these timescales when reviewing reports;
• ADS considered that significant input from suppliers (referring to paragraph 5.2 of the methodology consultation document) would only be required if reports were substantially non-compliant when submitted; and
• the MOD’s view was that there needed to be a clear differentiation between reporting and pricing matters, and that the review process should not become an exercise in querying contract prices or become an audit of submissions. The MOD also considered that a diagram to outline the process would be helpful.

The SSRO’s response

3.11 The SSRO has implemented a process which actively monitors when submissions were made and when the initial 15 working days have passed. This should avoid issues being raised in advance of expected timescales. It should be noted that these timescales only apply to initial submissions and not to any correction reports that have been submitted which may give rise to additional queries. While the SSRO endeavours to review supplier reports on a timely basis, these are not currently subject to the 15 working day timescale, although we may endeavour to do so in the future. We have updated paragraph 3.5 of the methodology to make this clearer.

3.12 The SSRO accepts the aspiration from ADS that significant input from contractors should only be required if a report is substantially non-compliant. We have sought to promote a high degree of compliance with reporting requirements which, if achieved, will reduce the need for the SSRO to raise issues with the MOD or contractors. Issues are also now raised and responded to in DefCARS, which should help to streamline the level of input required from industry.

3.13 As stated in response to the comments raised for question 2, we have updated paragraph 1.6 of the methodology with respect to the review process not being an audit. We agree with the MOD that the line between reporting and pricing issues may be ambiguous, but our view is that it is important that issues are raised with contractors in the first instance to allow the SSRO to understand how the provisions of the legislative framework have been applied in practice.

3.14 We have clarified an aspect of our manual review process, making clear that it includes a manual ‘sense-check’ of the submission. We have updated bullet two of paragraph 3.3 of the methodology to state that our process considers issues that may have passed the validation check and have stated in paragraph 3.4 that our manual review will involve consideration of whether changes need to be made to its set of automatic validation checks.

3.15 We agree that a diagram may help to understand the review process and have added this as Figure 1, below paragraph 5.3 of the methodology.

Question 4: Do you agree that the methodology appropriately identifies how the findings from compliance reviews will inform the SSRO’s other work?

3.16 Six respondents noted that the methodology was clear in how findings from compliance reviews would inform the SSRO’s other work. The following additional comments were raised:

• one respondent’s view was that a pragmatic approach should be adopted in cases where there may be some inflexibility in the reporting required in line with the Regulations; and
• ADS noted that there was complexity involved in meeting reporting requirements and that the SSRO should recognise this when considering findings from its compliance work. ADS suggested that simplifying and streamlining reporting should be considered by the SSRO as part of its work to review the legislation.

3.17 The MOD, however, considered that the process for how findings from compliance reviews may inform the SSRO’s other work was not clear with the approach appearing to be more of ‘tick-box’ exercise.

The SSRO’s response

3.18 The SSRO seeks to understand, as a matter of course, why reporting or pricing issues arise. In cases where the regulations do not allow flexibility in reporting or where the complexity of reporting becomes evident, these are the types of issues that need to be considered when making recommendations for legislative change or be considered as part of our work on reviewing reporting requirements.

3.19 Our approach is not intended to be a ‘tick box’ exercise, and findings from our work to date have informed development of our statutory reporting and profit rate guidance documents, as well as our recommendations for legislative change. To make the linkage clearer, we have updated paragraph 3.7 to state that where there is an issue that the SSRO needs to address, it will consider and log the issue and prioritise action in accordance with its prioritisation principles, as set out in section 7 of the methodology.

Question 5: Do you agree that these additional activities can be reflected in the methodology without setting out the detail of how each may be undertaken?

3.20 There were a wide range of responses to this question. A respondent and the MOD agreed that additional compliance activities could be undertaken without setting out the detailed process in the methodology.

3.21 Three respondents, however, stated that the section on additional compliance activities could be deleted. ADS, supported by two other respondents, considered that this was a natural progression in the review process and that it was expected that some aspects of the information provided would require further investigation or clarification to understand what was being reported. As such, ADS believe Section 4 of the methodology could be deleted without impeding the transparency of the review process.

3.22 Another respondent, however, considered that if the SSRO was to carry out such activities that it would be useful for it to inform the relevant stakeholders that a report had been selected for review, prior to the detailed review taking place.

3.23 Finally, two other respondents stated that setting out some of the detail behind the additional activities would be beneficial in order to ensure consistency, as at the moment the proposal appeared to be open ended for inclusion in the methodology and should be evidence based.

The SSRO’s response

3.24 The SSRO accepts that these activities represent normal incremental improvements and quality assurance activities that would be expected with this type of process, but we are committed to transparency and believe the methodology should reflect the various ways in which the SSRO discharges its functions. This will allow for incremental developments to take place in future, without having to amend the methodology.

3.25 The SSRO’s targeted reviews have, to date, focused on the work of the MOD delivery team and their review of a contractor’s submission, rather than seeking to involve the contractor any more than would be required under the standard review process. However, in any instance where additional compliance activities impact the contractor directly, we would seek to involve them at the earliest opportunity. We have updated paragraph 4.3 of the methodology to state that the SSRO will make contact with relevant stakeholders at the earliest opportunity to enable reasonable timescales for review and feedback.

3.26 We agree that additional reviews should be evidence based and have updated paragraph 4.1 of the methodology to state that before undertaking any such work, the SSRO would consider the evidence available from the reviews of individual reports.

Question 6: Do you agree that the SSRO should only review a submission having given the MOD sufficient time to undertake its own review in the first instance?

3.27 All respondents agreed that the MOD should undertake a review in the first instance. There were three comments raised in relation to this:

• a respondent considered that the 15-day window for the MOD to conduct a review may not be an achievable time frame and if it was not achieved, the SSRO should not step in to perform what it believed to be the MOD’s job;
• ADS considered that for the process to be successful, an undertaking was required from the MOD to complete its review in a timely manner. ADS also noted that its view was that it was the MOD’s role to review the content and accuracy of the reports and the SSRO’s role to manage the reporting regime; and
• the MOD stated that it should be made clearer that the MOD should also be picking up on reporting issues as part of its process of reviewing reports.

The SSRO’s response

3.28 The SSRO accepts the point made by a respondent that it should not seek to perform the MOD’s job. The role of the MOD in reviewing reports has been emphasised in the methodology and the SSRO cannot substitute. The SSRO nevertheless has its review functions to discharge and any review carried out by the SSRO will be in discharge of those functions, as outlined in this methodology.

3.29 As reported in its commercial toolkit, the MOD seeks to adopt a consistent policy with respect to the collection and review of data on qualifying contracts. Chapter 5 of the toolkit makes clear that the MOD must review QDC reports submitted on DefCARS within 15 working days of their submission.

3.30 The SSRO agrees that the MOD should pick reporting issues as part of its use of the reports to manage the contracts and we have updated paragraph 5.2 of the methodology to state the MOD may identify reporting issues when reviewing reports for contract management purposes.

Question 7: What is your view on the general approach to the notification of issues to the MOD as reflected in the methodology?

3.31 There was a range of responses to this question. The MOD and two other respondents agreed that the general approach of the SSRO notifying the MOD of issues was reasonable. There were two comments raised in relation to this:

• one respondent considered that only notifying the MOD of a non-submission of the ‘strategic reports’, may not be the most effective course of action as it would be the contractor who would be better informed as to the status of the report; and
• another respondent considered that the SSRO needed to remain independent in this respect, particularly considering its role in issuing opinions and determinations.

3.32 One respondent did not agree that the SSRO’s approach as drafted in the methodology was appropriate. It considered that the SSRO appeared to be undertaking a detailed operational compliance role, on behalf of the MOD. This respondent’s view was that the SSRO’s role should be more independent, identifying broader compliance themes, rather than individual issues for review.

3.33 ADS, and two other respondents supporting the ADS submission, considered that the tone of this section suggested that the SSRO would be extending its role into enforcement, something which should be left to the MOD. These respondents considered that the SSRO should not recommend to MOD the action it should take. There were two additional comments raised in relation to this:

• a respondent queried what the definition of a compliance issue was; and
• linked to this, in response to a previous question, a respondent noted that it would welcome a review of the measurement of non-compliance to confirm that it was not solely based on the number of questions raised on submissions.

The SSRO’s response

3.34 The responsibility to submit the strategic reports (the Strategic Industry Capacity Report ‘SICR’ and the Small or Medium Enterprise Report ‘SMER’) falls to the ‘designated person’, who may be the contractor or the Ultimate Parent Undertaking if the contractor is part of a group of companies. The SSRO agrees there may be instances where the contractor should be contacted if a strategic report has not been received as it may be able to provide a response on behalf of the designated person. We have updated paragraph 6.3 to state that where the delay relates to the submission of the strategic reports^{[footnote 1]}, the SSRO will seek to contact the designated person due to make the submission if known and will then inform the MOD.

3.35 As noted in our response in relation to comments made against question 1 of the consultation, the SSRO’s role is not to become an active participant in enforcement. Paragraph 6.1 of the methodology notes that the SSRO’s focus is on encouraging timely and good quality submission. The SSRO may bring matters to the attention of the MOD to gain a better understanding of the issues arising, but it is for the MOD to decide if any enforcement action is appropriate. We accept that care must be taken when raising issues to avoid expressing concluded views. As such, paragraphs 6.4 and 6.5 have been updated to make clear that it is the MOD who will consider whether any follow up action is required for any issues that may be raised by the SSRO.

3.36 We have reviewed section 6 of the methodology for tone and have updated the following paragraphs to recast them as follows:

• paragraph 6.4 “…The SSRO will refer unresolved reporting and pricing issues to the MOD, to allow it to consider whether any follow up action is required”; and
• paragraph 6.5 “…The SSRO will further note whether it considers that the issue forwarded to the MOD is of a high, medium or low priority to allow the MOD to consider the matter further. The priority assigned to an issue will be based on its potential impact on operation of the regulatory framework. This involves an assessment of relative priority. The SSRO takes seriously any possible non-compliance with the requirements of the regulatory framework and would likely mark such issues as ‘high priority’.

3.37 The SSRO’s review of submissions considers the timeliness of each submission, whether it contains the information prescribed in the Regulations, and whether it is consistent with statutory guidance issued by the SSRO. An issue may be raised where a submission appears to be incomplete, inconsistent, erroneous or lacking in detail, having regard to the reporting requirements. In raising issues, the SSRO also tries to maintain a standardised set of data and seeks to apply categories to issues as appropriate.

3.38 We agree there is merit in providing greater clarity regarding when the SSRO will consider a reporting requirement has not been met for the purposes of monitoring its performance indicators 2a) and 2b). As noted above, responses may inform either of the SSRO’s 36(2) or 39(1) responsibilities, however only an issue in relation to our 36(2) responsibilities will impact the quality rating of a submission. We have updated paragraph 2.3 of the methodology to state “…A reporting obligation will be identified as not being met for the purposes of indicators 2a) and 2b) if the SSRO or the MOD has raised an issue on a submission which has resulted in the contractor:

• submitting a correction report to rectify an error;
• providing additional information required by the legislation; or
• failing to respond to the issue raised.

3.39 Paragraph 3.11 has also been updated to state that for the purposes of applying the indicators, the SSRO will rely on the contractor’s response rather than forming its own conclusion about compliance. Any issues categorised as pricing issues will not impact these indicators nor will the SSRO form any concluded views about those matters. We will count the submission as a ‘pass or fail’ of an entire submission once made, regardless of the number of errors that may be apparent in the initial submission. We will report these indicators as a twelve-month rolling average proportion of report submissions.

Question 8: Are there any other elements of support or engagement that should be reflected in the methodology?

3.40 All respondents noted that the elements of support or engagement reflecting in the methodology were appropriate. There was one additional comment in relation to this:

• a respondent considered that the work undertaken with the help of the SSRO’s Reporting and IT Sub-Group should be reflected within the methodology as feedback was often provided at these meetings from which improvements were made.

The SSRO’s response

3.41 The SSRO agrees that the SSRO’s Reporting and IT Sub-Group of its Operational Working Group provides a forum for stakeholder feedback and is an arena to provide support to stakeholders. We have updated paragraph 7.3 to state that we will engage with stakeholders, for example through the Reporting and IT Sub-Group, to help us identify, prioritise and take forward any required action.

Question 9: What are the key areas of feedback for the SSRO to provide to industry?

3.42 Following publication of the 2019 Annual Compliance Report, the SSRO wrote to 14 companies who were included anonymously in the report under their Global Ultimate Owner to show their relative position on submission timeliness and quality. Respondents found these individual compliance letters useful and noted also that the Annual Compliance Report itself was useful.

3.43 One respondent also suggested that individual feedback on supplier report issues would be useful. One other respondent stated that it would welcome regular reports for those companies who have multiple DefCARS accounts.

The SSRO’s response

3.44 The SSRO states in the methodology that it will provide regular feedback to the MOD, identifying where matters have had an impact either on data quality or the operation of the regulatory framework as well as to contractors with multiple contracts on compliance themes identified from their report submissions.

3.45 We are grateful for the suggestions put forward by respondents on what the key areas of feedback should be. We have reordered the paragraphs in this section and have updated paragraph 8.1 to state that the SSRO will provide information about reporting issues at a contractor or designated person level, in addition to the information provided to the contractor’s administrator or the person submitting the report, if it is practicable to provide the information and it would assist with compliance.

3.46 The analysis of compliance issues is an area that the SSRO will keep under review as it considers future developments to DefCARS.

Other comments made by respondents

3.47 Respondents made two other specific comments linked to the SSRO’s compliance work:

• a respondent stated that issues are often reported against the contractor, but that the reporting would benefit from ownership attribution as not all actions were for the contractors to resolve; and
• the MOD stated that the information gathered through the compliance process should be used to identify and correct issues with DefCARS and the Statutory Guidance in a more timely fashion.

3.48 The SSRO thanks both respondents for this feedback. We will consider what action we can take to make it clearer within DefCARS to whom issues are attributed and whether the processes in place to update DefCARS and the Statutory Guidance can accommodate changes within a shorter timescale.

3.49 There were other comments made by respondents which do not directly relate to the methodology. These comments related to reporting functionality within DefCARS and a specific reporting issue in relation to provisionally priced contracts. The comments have been logged separately for consideration.

4. Next steps

4.1 The SSRO has commenced implementation of the updated compliance and review methodology. The published 2020 methodology is available on the SSRO website. The SSRO has produced internal guidance to ensure that the methodology application is objective and consistent.

4.2 Data produced from the application of the methodology will be collated ahead of the publication of the next applicable Compliance Report and will be used to analyse compliance trends.

4.3 The SSRO will therefore publish its next Annual Compliance Report in the summer of 2020 after completion of the 2019/20 financial year, using the 2017 methodology. This methodology will be implemented from 1 April 2020.

1. SICR and SME submissions ↩