Independent report

SSRO Opinion 1

Published 30 November 2015

1. Introduction

1.1 The Single Source Regulations Office (the SSRO) was referred a request for an opinion in respect of a proposed Qualifying Defence Contract (QDC). The referral sought the SSRO’s opinion on the extent to which specified costs were Allowable Costs, as well as additional guidance to that contained in the SSRO’s Single Source Cost Standards^{[footnote 1]}. The referral was made under Section 35(1) of the Defence Reform Act 2014 (the Act) and Regulation 51(1) (d) of the Single Source Contract Regulations 2014 (the Regulations).

1.2 Following receipt of the request for an opinion, the SSRO acknowledged the request and formally accepted the referral, having conducted due diligence in line with its guidance.

1.3 The SSRO appointed a Referrals Committee, a sub-committee of the SSRO Board comprising two nonexecutive members of the Board and an independent member who is not a member or employee of the SSRO, to consider the referral. The Referrals Committee gathered and analysed written evidence from both parties, conducted a site visit and held an informal information gathering hearing with both parties.

1.4 In forming its opinion, the SSRO had regard to:

• its duty under the Act and the Regulations to express an opinion on the extent to which the costs are Allowable;
• the relevant sections of the Act, the Regulations, its referrals guidance^{[footnote 2]} and its Single Source Cost Standards guidance; and
• its general obligation under Section 13 of the Act to aim to ensure that good value for money is obtained in government expenditure on QDCs and that persons (other than the Secretary of State) who are parties to QDCs are paid a fair and reasonable price under those contracts.

2. Opinion overview

2.1 In accordance with the procedure published in the SSRO’s referrals guidance, this document provides an anonymised summary of the SSRO’s opinion. Recognising the commercial sensitivity for the parties, this summary provides an overview of the key principles which emerge from the opinion, rather than providing details of either the opinion or the parties involved.

2.2 The SSRO noted that the process of reviewing and challenging whether costs are Allowable can itself reveal potential savings and uncover the scope for efficiencies that were not otherwise available. The discipline of meeting the legal requirements to demonstrate that costs are Allowable also provides a solid basis for negotiations between the parties. For example, the discussion of various estimates for certain contract costs provides an opportunity for one party to introduce external or benchmarking data to challenge the other party’s costings.

2.3 The SSRO was asked for its opinion on:

• the extent to which the price risk for direct labour, materials, and technical risks included within the proposed Target Cost was an Allowable Cost, and a view on an appropriate confidence percentile range to contract at for future contracts;
• the extent to which the cost for re-work and defects and deficiencies included within the proposed Target Cost were Allowable Costs; and
• the extent to which the price risk for indirect costs included within the proposed Target Cost was an Allowable Cost.

2.4 In addition, the SSRO was asked for general observations or points of principle that might be applicable to other contracts. In order to provide the requested opinion, the SSRO expressed some principles that we believe may be more generally applicable and are therefore important for the consideration of Allowable Costs and price risk in future QDCs.

2.5 The SSRO’s findings are set out in the following sections:

An opinion on the extent to which the price risk for direct labour, materials, and technical risks included within the proposed Target Cost is an Allowable Cost.

2.6 In response to this question, the SSRO first considered whether it was appropriate, in principle, to include an adjustment for price risk in a Target Cost estimate.

2.7 Paragraph 11.1, bullet point 12, of the SSRO’s Single Source Cost Standards includes the following statement:

“Provision for cost contingencies and cost risk are not Allowable as contractors are expected to manage actual costs and cost estimates as part of contract delivery. This is separate from, and does not apply to price risk in relation to a risk based contract where the contract accepts risks on behalf of the MOD, and where both parties accept this as being a necessary component of the contract.”

2.8 The effect of the guidance is that, for a contract that uses a regulated pricing method that transfers most or all cost risks to the MOD, such as the cost plus method, the cost contingencies or cost risks are not Allowable Costs. By contrast, appropriate provision for risk can be an Allowable Cost in contracts that use a regulated pricing method where the contractor bears a substantial element of cost risk. In this case the proposed contract uses the Target Cost method, and both parties agreed on the inclusion of price risks in the contract. The SSRO therefore concluded that this contract can include a provision for risk in the Target Cost, where risk is borne by the contractor and where it is adequately quantified.

Consideration of whether the contractor will be bearing risk

2.9 Secondly, the SSRO considered whether the contractor would be bearing risk in relation to direct labour, materials, and technical risks under the proposed contract. The SSRO concluded that, since the contractor would be bearing risk in relation to direct labour hours, materials and technical risks under the proposed contract, it was reasonable for the Target Cost to include an amount that reflects the quantum of those risks.

Consideration of methods used for assessing a value for risk

2.10 Thirdly, the SSRO considered the methods used for assessing a value for risk to include in the Target Cost. In order to estimate a value for price risk, the parties used a Monte Carlo analysis. This is a simulation technique that contributes to an assessment of the combined impact of uncertainty about the estimates of individual cost items on variation in the estimate of Target Cost.

2.11 The main output of the method is a probability distribution for Target Cost and the main inputs are ‘minimum’, ‘most likely’ and ‘maximum’ estimates for each cost item. As with all analysis of this nature, the outputs of the analysis can only be as robust as the inputs.

2.12 Monte Carlo analysis is a widely used method for assessing risk in the context of major projects. HM Treasury makes reference to the method in the ‘Green Book’ guide on appraisal and evaluation in central government. As with any other method, it has limitations and the outputs of the analysis need to be used in that context.

2.13 The SSRO’s view was that, while Monte Carlo analysis was an appropriate technique to use, it might be appropriate in large and complex projects to use more than one technique for the purposes of cross-validation.

Consideration of proposed value of risk

2.14 Fourthly, the SSRO considered the proposed value of risk for inclusion in Target Cost. The proposed value was derived by the parties from a Monte Carlo analysis and, as noted above, the quality of the inputs to the Monte Carlo analysis is a key consideration in the assessment of the outputs.

2.15 The SSRO sought and received written assurances from the contractor that, in its opinion, all of the costs, including price risks, which are included in the Target Cost are AAR.

2.16 The SSRO also sought and received written assurances from the MOD that, in its opinion, the elements of risk in the Target Cost are AAR.

Consideration of learning curve benefits

2.17 The SSRO’s research identified that, given the nature of manufacturing production units concerned in this referral, the taxpayer should benefit from learning curve improvements. A specified percentage improvement would be appropriate, while properly recognising the manufacturing challenges involved. The application of this learning curve would indicate that a share of the direct labour costs in the price risk would not be an Allowable Cost.

2.18 Therefore, the SSRO’s opinion was that a portion of the price risk labour cost was not an Allowable Cost.

Consideration of wage inflation uprating

2.19 The referral noted that an adjustment to the Target Cost in 2014 Economic Conditions (ECs) was agreed between the parties in order to reflect that value in 2015 ECs. The adjustment involved inflating labour-related cost elements by specified percentage. We understood that the final Target Cost in the contract would be in 2015 ECs.

2.20 Having reviewed the evidence, the SSRO reached the view that calculating labour costs at 2015 ECs using a relevant and current inflation index would be a more suitable method for calculating Allowable labour costs than using a pre-determined per cent uplift factor. Should this lower uplift rate be applied, there would be a reduction in both the labour and price risk cost included in the Target Cost. The SSRO’s opinion was that an amount equal to the price risk cost corresponding to the higher percentage was not an Allowable Cost.

A view on an appropriate confidence percentile range to contract at for future contracts

2.21 The SSRO was asked for its view on the appropriate confidence percentile at which to contract on future projects. The SSRO did not consider it appropriate to provide an opinion on the confidence percentile range to apply to future contracts for a similar product. Each contract must be considered in the light of its particular circumstances and any percentile derives from its own specific input assumptions. The SSRO clarified that it was appropriate that the MOD and the supplier agree on the most appropriate methodology or methodologies to be applied to establish the level of risk on each contract. The burden of proof rests with the contractor to demonstrate that costs meet the requirements of an Allowable Cost. It is therefore appropriate for the contractor to define those methodologies that best define the risk using only those costs as a base.

2.22 Additionally our view was that any percentile starting point of risk, out of context, is equally flawed. The result of any analysis needs to be put into appropriate context, taking into consideration the risk profile that applies to the contract to which the risk is applied.

An opinion on the extent to which the cost for re-work and defects and deficiencies included within the proposed Target Cost are Allowable Costs

2.23 The second matter for consideration related to the cost of re-work and defects and deficiencies. The referral asked the SSRO’s opinion on the extent to which the cost for rework and defects and deficiencies included within the proposed Target Cost were Allowable Costs. The SSRO’s Single Source Cost Standards states that: “Costs that are generally unallowable” include those “for the remedy of faulty workmanship or the consequences that result”. In addition we recently issued the following as an SSRO answer on re-work:

“The cost of re-work will be allowable if it meets the principle of being appropriate, attributable and reasonable and is agreed between the contractor and the MOD, who will have regard to our guidance on allowable costs. For example:

(1)First in Class – re-work which occurs during the process of manufacturing an item for the first time would generally be considered allowable.

(2)Re-specification – re-work which occurs due to a change in design specification from the MOD would generally be considered allowable.

(3)Faulty Workmanship - the costs of re-work due to faulty workmanship or the consequences that result from faulty workmanship are generally nonallowable. Similarly, re-work that is a result of a contractor not delivering a specification would generally be nonallowable.

2.24 Making the cost of re-work and defects and deficiencies Allowable under the Act, and reflecting this in the Single Source Cost Standards, would by definition enable these costs to attract profit based on the formula for determining contract price specified in Regulation 10 of the Regulations (i.e. price = AC + (CPR x AC)). Changing the definition in this manner would theoretically reward inefficiency, which was not the intent of the Currie Report and the resultant legislation, Regulations and published SSRO Single Source Cost Standards. Any production inefficiency should represent a concomitant reduction to the bottom line, and in Target Cost contracts should constitute an impact to the share-line.

2.25 The SSRO’s opinion was that cost disallowed by the parties for faulty workmanship and defects and deficiencies was within the bounds of reasonableness. However, the SSRO would have expected the contractor to have collated historic re-work data from which to make an accurate estimate of the costs that should be allowed. We recommended that new and accurate measurement processes are put in place.

2.26 Re-works do not add value to the product and are factors that reduce the productivity of a company and increase the lead time for implementing the final product, making it uncompetitive compared to its main competitors^{[footnote 3]}. To demonstrate that re-work is appropriate, attributable and reasonable the contractor will clearly need to be able to measure reliably how much re-work falls into the various categories outlined.”

An opinion on the extent to which the price risk for indirect costs included within a proposed Target Cost is an Allowable Cost

2.27 The third matter related to the price risk for indirect costs. The referral asked the SSRO its opinion on the extent to which the price risk for indirect costs included with the proposed Target Cost is an Allowable Cost. The Single Source Cost Standards states that:

“overhead and indirect costs are defined as those costs which, through necessarily having been incurred during the performance of the qualifying defence contract… and qualifying sub-contract for the conduct of the contractor’s business, in general, cannot be identified and measured as directly applicable to the performance of that contract.”

2.28 The SSRO’s opinion was that fixed indirect costs should be treated as fixed, and only the proportion of the indirect costs that are variable should be identified, proven and uplifted for direct hour uncertainty. In addition, in order to comment on the extent to which price risk for indirect costs should be allowed, for example due to possible overrecovery of overhead costs, the SSRO would require more evidence than was provided in the referral. The SSRO had no evidence to validate the assurance given on this point.

3. General observations

Application of the SSRO’s Single Source Cost Standards

3.1 The SSRO was asked to clarify, based upon the information provided to us, whether the approach taken by the parties in negotiating the proposed QDC was consistent with the SSRO’s Single Source Cost Standards. The referral requested the SSRO to draw out general conclusions or points of principle that might be applicable to other contracts.

3.2 The SSRO noted that it was not straightforward to verify the price risk components presented to it in the referral. Both parties were unable to explain their own view of the relevant inputs and how they were at variance with the other party’s perspective, and placed undue reliance on their view that they had reached a satisfactory outcome. The negotiation of a settlement that is believed to be beneficial to both parties does not release either from their obligation to comply with Section 20(4) of the Act and paragraph 7.4 of the Single Source Cost Standards.

3.3 Referrals must contain sufficient, relevant and detailed evidence to support matters on which parties seek a referral. Parties that are unable to supply sufficient evidence to justify that costs are AAR are not compliant with the Act.

3.4 Section 20(4) of the Act “places the onus upon the primary contractor of a qualifying defence contract to demonstrate to the Secretary of State (if required) that costs meet those requirements set out in this guidance or otherwise as being Allowable. The burden of proof rests with the contractor and it is essential that the MOD operates as an intelligent client and has the ability to verify, challenge and agree the costings that are submitted as being Allowable”^{[footnote 4]} .

1. Single Source Cost Standards: Statutory Guidance on Allowable Costs January 2015. ↩

2. Guidance on the SSRO’s referrals procedures under the Defence Reform Act 2014 and the Single Source Contract Regulations 2014. ↩

3. Sustainable Maritime Transportation and Exploitation of Sea Resources – Rizzuto & Guedes Soares (eds) © 2012 Taylor & Francis Group, London, ISBN 978-0-415- 62081-9 ↩

4. SSRO’s ‘Single Source Costs Standards’. ↩