Transparency data

Details of process for determining performance scores

Updated 17 December 2024

The emergency measures agreements (EMAs) signed in March 2020 stated that operator performance would be assessed against the scorecard set out in the EMA schedule (schedule 8B) for the periods:

• 1 April 2020 to 20 September 2020
• 20 September 2020 to 31 March 2021

The emergency recovery measures agreements (ERMAs) signed in September 2020 stated that operator performance would be assessed against the scorecard set out in the ERMA schedule (schedule 8.1B for the majority of train operators) for the period 20 September 2020 to 31 March 2021.

Both Great Western Railway and London South Eastern Railway were placed on longer-running EMA contracts than other train operators in March 2020 due to commercial negotiations at that time.

The national rail contracts (NRCs) signed from May 2021 onwards stated that operator performance would be assessed against the Scorecard criteria set out in Chapter 7.2 from the start of the contract to 31 March 2022. The performance assessment period for financial year 2022 to 2023 is from 1 April 2022 (or the start date of the operator NRC if later) until 31 March 2023.

All 10 operators, excluding those under DfT OLR Holdings Limited (DOHL) ownership are now on NRCs, with the last 2 – West Coast and Cross Country – being transferred in October 2023.

Scorecard criteria

The 3 EMA scorecard criteria were:

• operational performance
• customer experience
• acting as a good and efficient operator

The 4 ERMA scorecard criteria were:

• operational performance
• customer experience
• financial performance
• collaborative behaviours:
  • with Network Rail and other industry partners
  • with the Secretary of State for Transport
  • with other stakeholders

Each operator was awarded a score and corresponding payment individually for each criterion of either:

1. below acceptable standard – no performance payment
2. acceptable – partial performance payment
3. good – full performance payment

Only full integer scores of 1, 2 or 3 were awarded.

For ERMA there were 6 individual evaluations in total with the 3 elements of collaborative behaviours each scored separately.

The assessment process for the NRC is comprised of 4 qualitative scorecard assessments and the operational performance regime (6 quantifiable target-based regimes), the combination of all will determine the fee given to operators.

The 4 NRC scorecard criteria are:

• operational performance
• customer experience:
  • customer satisfaction
  • service quality standards
• financial performance:
  • cost
  • revenue
• business management:
  • business plan quality
  • business plan delivery
  • collaboration

On East Anglia and West Midlands, bespoke scorecards apply which additionally assess performance of the cost and major milestones of the capital works programmes. This was also the case for Transpennine Express to the point it transferred over to DOHL ownership in May 2023. Transpennine Express also had an additional performance assessment against the Transpennine Route Upgrade programme.

How scores relate to performance payments

For train operators under EMA contracts a maximum of 0.5% of franchise historical costs (effectively the operating cost base of the franchise before the COVID-19 pandemic began) were available as performance payments, equally divided across the 3 criteria. A fixed management fee of 1% was also available under EMA.

As such, the maximum EMA fee for a score of 3 (good) on a single criterion was a third of the 0.5% historical cost figure. A score of 2 (acceptable) received 40% of this amount and 1 (below acceptable) did not receive any payment.

EMA contracts were replaced by ERMA contracts for the majority of train operators from September 2020. ERMA contracts increased the proportion of the total fee, which was performance based from 0.5% up to 1%, with a corresponding reduction in the ‘fixed’ management fee from 1% down to 0.5%, better aligning fees with performance and acting as a more effective incentive for train operators.

Performance payments were divided across 4 weighted scorecard criteria for:

• operational performance
• customer experience
• financial performance
• collaborative behaviours

The maximum ERMA fee for a score of 3 (good) was 100% of each individual weighted criteria. A score of 2 (acceptable) could receive 75% of this amount except for financial performance, which could be 50%. A score of 1 (below acceptable) would not receive any payment.

For train operators under NRCs, a maximum of approximately 1.5% of historic franchise costs are available as performance payments divided across the 4 weighted criteria, as follows.

Assessment area	Minimum weighting	Maximum weighting
Operational performance	15%	25%
Customer experience	15%	25%
Financial performance	25%	40%
Business management	25%	35%

Assessment process

The assessment of the scorecards was made after the end of the respective period on the basis of evidence that was collected by DfT and operator during the period.

The evaluation was undertaken by independent evaluators who all received compulsory pre-evaluation training. This ensured each operator was fairly evaluated and an objective view on performance was determined based on the review of evidence in comparison to the scorecard criteria. The evaluations were then subsequently moderated and subject to independent assurance to ensure all franchisees had been treated consistently.

See the scorecard criteria for each train operator for more information.

Scoring was undertaken ‘in the round’ for each criterion. For example, although there was a single overall score for the customer experience this involved several elements (station and train cleanliness, maintenance, staff deployment and the passenger journey experience).

The evaluation score, therefore, reflected the overall score for those elements taken together in each case using the judgement of the independent evaluators.

The performance scores and fees data (ODS 45.4KB) show management fees and any performance payments that are payable.

Subject to other conditions, actual payment is made to the TOCs after the DfT has received annual audited accounts and other financial reporting for the franchisee year.

NRC assessment process

The operational performance regime (OPR) includes 6 quantifiable performance measures that are individually measured for each of the 13 rail periods. The payment due for each period is accrued and made at year end as per the contract.

Operational performance targets are set to ensure that train operators are incentivised to deliver punctual and reliable services for passengers. The department uses a basket of measures to ensure it can effectively reward and hold operators to account for areas over which they have most control as well as wider whole-industry measures that reflect the whole service experience of their passengers and impact upon other network users.

For each element of the regime, the department sets a range or area in which the results shall be deemed to have delivered acceptable performance, the ‘expected’ range.

Train operators will receive the same expected range fee regardless of where within/on this range the results land. Train operators will earn a lower fee for performance that is worse than the expected range, tapering down to a nil (zero) fee level.

Train operators will earn an improved fee for performance that is better than the expected range, tapering up to a maximum fee level.