Transparency data

Details of process for determining performance scores under EMAs and ERMAs

Updated 27 April 2023

The emergency measures agreements (EMAs) signed in March 2020 stated that operator performance would be assessed against the scorecard set out in the EMA Schedule (Schedule 8B for the majority of train operators) for the period 1 April 2020 to 20 September 2020 and for the period 20 September 2020 to 31 March 2021.

The emergency recovery measures agreements (ERMAs) signed in September 2020 stated that operator performance would be assessed against the scorecard set out in the ERMA Schedule (Schedule 8.1B for the majority of train operators) for the period 20 September 2020 to 31 March 2021.

Scorecard criteria

The 3 EMA scorecard criteria were:

• operational performance
• customer experience
• acting as a good and efficient operator

The 4 ERMA scorecard criteria were:

• operational performance
• customer experience
• financial performance

• collaborative behaviours, split by:

  • with Network Rail and other industry partners
  • with the Secretary of State for Transport
  • with other stakeholders

Each franchisee was awarded a score and corresponding payment individually for each criterion of either:

1. Below acceptable standard – no performance payment.
2. Acceptable – partial performance payment.
3. Good – full performance payment.

Only full integer scores of 1, 2 or 3 were awarded.

How scores relate to performance payments

For train operators under EMA contracts, a maximum of 0.5% of franchise historical costs (effectively the operating cost base of the franchise before the COVID-19 pandemic began) were available as performance payments, equally divided across the 3 criteria.

For train operators under ERMA contracts, a maximum of 1% of franchise historical costs (effectively the operating cost base of the franchise before the COVID-19 pandemic began) were available as performance payments, divided across the 4 weighted criteria, as follows:

• 25% operational performance
• 22.5% customer experience
• 30% financial performance
• 22.5% collaborative behaviours

As such, the maximum fee for a score of 3 (good) is on a single criterion is a third of the respective EMA or ERMA historical cost figure. A score of 2 (acceptable) would receive 40% of this amount and 1 (below acceptable) would not receive any payment.

The EMA and ERMA assessment process

The assessment of the scorecards for both EMAs and ERMAs was made after the end of the respective period on the basis of evidence that was collected by the Department for Transport (DfT) and franchisee during the period.

The evaluation was undertaken by independent evaluators who all received compulsory pre-evaluation training. This ensured each franchisee was fairly evaluated, and an objective view on performance was determined based on the review of evidence in comparison to the scorecard criteria. The evaluations were then subsequently moderated and subject to independent assurance to ensure all franchisees had been treated consistently.

See the scorecard criteria for each EMA and ERMA operator.

Scoring was undertaken ‘in the round’ for each criterion. For example, although there was a single overall score for the customer experience, this involved a number of elements (station and train cleanliness, maintenance, staff deployment and the passenger journey experience). The evaluation score, therefore, reflected the overall score for those elements taken together, in each case using the judgement of the independent evaluators.

The EMA and ERMA performance scores and fees data show EMA and ERMA management fees and any performance payments that are payable.

Subject to other conditions, actual payment will be made to the TOCs after the DfT has received annual audited accounts and other financial reporting for the franchisee year.