Apprenticeship training provider accountability framework and specification

Question 1

Accountability policy for apprenticeship training providers

Accepted Answer

It’s important to protect the experience of all apprentices, increase achievements and drive-up quality. We also want to ensure that pockets of poor provision do not undermine the performance of apprenticeship provision more broadly and brand of the programme. Apprenticeships rely on the commitment and effective collaboration between an employer, an apprentice, and a training provider. While all parties have a key role to play in the success of an apprenticeship, we know that the quality of training is a major factor in whether apprentices successfully complete their apprenticeship.

This framework provides a holistic, timely and data-led approach to training provider accountability. The framework provides a wide range of quality indicators with specified threshold that providers can use to support self-improvement. These will reduce the likelihood that we’ll need to intervene. We’ll use this information to determine when and where intervention may be needed.

Now that the framework is established, we’ll be taking a more robust approach in its application. We’re retaining the same indicators, but we’re introducing higher thresholds for 3 of the 10 indicators (withdrawals, apprentices past planned end date, and breaks in learning) and removing the 250-apprentice threshold that applied to 3 of the measures.

The thresholds in this framework reflect our minimum expectations. You should not use them as targets or aspirations. Providers with large volumes of learners should expect close monitoring and performance management designed to effect continuous improvement.

Our stronger approach will take immediate effect. We’ll use the new thresholds in performance conversations from June 2024, when the supporting data dashboard is published with the updated thresholds and applied to current and historic data. We’ll use the framework to assess performance throughout the academic year to ensure that, where we need to take intervention action, this is timely and proportionate.

At our discretion, we’ll make full use of the range of existing contractual measures available that are detailed in your provider funding agreement. We’ll evaluate each case according to its own circumstances. This may include acting in cases where performance across the different indicators compares poorly with other providers delivering similar provision, or where there have been ongoing performance conversations but agreed improvements have not been realised for reasons within the provider’s control.

We’ll continue to consider relevant contextual factors that may be impacting a provider’s performance, where these can be appropriately evidenced.

Question 2

Indicators we use to review provider performance

Accepted Answer

Reviews will assess the latest available data for each indicator for the current academic year and the previous academic year (where applicable).

We’ll continue to keep the indicators and their thresholds under review to ensure they’re set at the right level.

Question 3

Quality indicators and thresholds

Accepted Answer

Outcomes from Ofsted inspection

This is the outcome from your most recent Ofsted inspection. We’ll consider organisations to be ‘at risk’ if Ofsted grades them:

inadequate for ‘apprenticeships’
inadequate for ‘overall effectiveness’ under its further education (FE) and skills remit, where there is no separate apprenticeship grade

The contractual action we may take in these circumstances is set out in your funding agreement.

We’ll prioritise providers under the accountability framework if they’ve not yet been inspected by Ofsted, or if they’ve been inspected by Ofsted and received:

a ‘requires improvement’ grade at full inspection
an ‘insufficient progress’ assessment at a new-provider monitoring visit

Apprenticeship achievement rates

Apprenticeship achievement rates are calculated as part of qualification achievement rates (QARs).

We’ll assess organisations with an all-age apprenticeship achievement rate:

of less than 50% as ‘at risk’
greater than or equal to 50% and less than 60% as ‘needs improvement’

Apprenticeship retention rate

Apprenticeship retention rates are calculated as part of qualification achievement rates.

We’ll assess organisations with an all-age apprenticeship retention rate:

of less than 52% as ‘at risk’
greater than or equal to 52% and less than 62% as ‘needs improvement’

Employer feedback

This is collected through a questionnaire and displayed on the find apprenticeship training service.

Organisations with average employer feedback of less than 2.5 will be assessed as ‘needs improvement.’

Currently, the feedback rating shown on the find apprenticeship training platform is cumulative. However, to ensure we only review recent performance, the framework will use the most recent academic year rating for assessment against the threshold. You should therefore refer to the data shown on the accountability framework dashboard.

Apprentice feedback

This is collected through an apprentice’s ‘my apprenticeship’ account and displayed on the find apprenticeship training service.

Organisations with apprentice feedback of less than 2.5 will be assessed as ‘needs improvement’.

You can review apprentice feedback in the find apprenticeship training service, once 10 apprentices have responded.

Currently, the feedback rating shown on the find apprenticeship training platform is cumulative. However, to ensure we only review recent performance, the framework will use the most recent academic year rating for assessment against the threshold. You should therefore refer to the data shown on the accountability framework dashboard.

Supplementary indicators and thresholds

‘Total number of apprentices’ means all your apprentices within the academic year, regardless of their completion status. It includes:

new starts
existing apprentices
apprentices on both apprenticeship standards and frameworks recorded in the individualised learner records (ILR) through funding models:
- 35 (‘adult skills’)
- 36 (‘apprenticeships’)
- 81 (‘other adult’)

Apprentices who do not meet the qualifying period of a minimum 42 days are excluded. This is set out in the apprenticeship funding rules.

We may monitor the volume of leavers in the first 42 days of apprenticeships in future updates.

Off-the-job training

This is collected through the FRM37 report of financial assurance: monitoring post-16 funding.

We’ll assess organisations as ‘at risk’ if they have any of the following:

more than 20 records with errors
one or more apprentices reported with zero planned hours
one or more apprentices with zero actual hours on apprenticeship completion

We’ll assess organisations as ‘needs improvement’ if they have either:

more than 15 records for planned hours
one or more records for actual hours

Withdrawals

This is where the ILR ‘completion status’ field is ‘3, the learner has withdrawn from the learning activities leading to the learning aim’.

We’ll assess organisations as ‘at risk’ if their withdrawals are greater than 20% of the total number of apprentices.

We’ll assess organisations as ‘needs improvement’ if their withdrawals are less than or equal to 20% and greater than 15% of the total number of apprentices.

Apprentices past planned end date

This indicator refers to apprentices who are continuing training past their planned learning end date or were past it when they completed their apprenticeship (it does not include the end-point assessment period).

This includes apprentices where the ILR ‘completion status’ field is recorded as either:

code 1 – the learner is continuing or intending to continue the learning activities leading to the learning aim
code 2 – the learner has completed the learning activities leading to the learning aim

We’ll assess organisations with more than 15% of the total number of apprentices past their planned end date by 180 days or more as ‘at risk’.

We’ll assess organisations as ‘needs improvement’ if they have more than 15% of the total number of apprentices past their planned end date by 90 days or more but less than 180 days.

Break in learning

This is where the ILR ‘completion status’ field is ‘6, learner has temporarily withdrawn from the aim due to an agreed break in learning’.

We’ll assess organisations with more than 10% of the total number of apprentices on a break in learning by 365 days or more as ‘at risk’.

We’ll assess organisations as ‘needs improvement’ if they have more than 10% of the total number of apprentices on a break in learning by 180 days or more but less than 365 days.

End-point assessment organisation (EPAO) data

This relates to apprentices:

on apprenticeship standards only
with no EPAO in the ILR

It includes apprentices who are past their planned end date.

We’ll assess organisations with apprentices identified as having no EPAO in the ILR:

within 3 months of the planned end date as ‘at risk’
within 3 to 6 months of the planned end date as ‘needs improvement’

The EPAO in the ILR must be valid for the standard being delivered. It is the provider’s responsibility to ensure they select an appropriate EPAO for the apprenticeship standards they’re delivering. Where our validation checks highlight instances of invalid EPAOs being selected, we may take this into account as part of your performance review.

Question 4

Review process

Accepted Answer

We’ll continually monitor provider performance against the indicators. We may contact providers at any point in the academic year to discuss where their performance falls below the thresholds.

Before a review

Case managers will identify any provider that is ‘at risk’ or ‘needs improvement.’ They will contact you to set out where data indicates provision has fallen below any thresholds.

The case manager will invite you to have a management conversation. We expect you to respond promptly.

Consider whether there are mitigating factors and share any supporting evidence with your case manager before the conversation.

During the management conversation

The case manager will review provider data against the framework’s quality and supplementary indicators.

The discussion or correspondence is focused on where your data indicates performance is below required thresholds. We’ll discuss:

evidenced reasons why performance is below our specified thresholds
your track record
capacity to improve, including evidence of action you may have already taken to improve and when you expect to see the impact, and progress in relation to improvement plans or targets we have previously agreed with you
how your performance compares with sector or standard benchmarks (where appropriate)
any other relevant contextual evidence

The case manager may ask for further information. You should provide this within the agreed time period.

After the review

Following the meeting, the case manager will inform you in writing about whether any follow-up action is required.

Question 5

Range of interventions

Accepted Answer

After the management conversation, if we believe that poor-quality provision is due to provider failure, we’ll intervene and take further action.

Our interventions will prioritise protecting apprentices’ interests. We may support you to improve your provision, where you demonstrate this is possible in a timely manner.

Your funding agreement sets out the full range of intervention actions we may take. These include but are not limited to:

requiring you to produce an improvement plan with associated targets
restricting your ability to recruit new apprentices for one or multiple standards
restricting your sub-contracting arrangements
withholding or suspending funding for a fixed or indefinite period
capping funding for delivery of new standards, for a fixed or indefinite period
terminating your contract, where necessary
putting in place other contractual conditions as appropriate

No intervention

If you are ‘on track’ against all indicators, we’ll not contact you to arrange a review.

There may be instances where providers do not meet one or more of the quality indicators, but we decide that an immediate management conversation is not needed. For example, a provider may have an ‘outstanding’ Ofsted grade and we expect them to have the capacity to improve.

Question 6

Contextual factors

Accepted Answer

We’ll take relevant contextual factors and a holistic view into account when we review your performance.

Benchmarked QARs

We have introduced benchmarked QAR data to the apprenticeship accountability framework (AAF) dashboard. This will be reviewed as part of the management conversation. It will support the evaluation of each case through more transparent comparison of performance with providers delivering similar provision and therefore facing similar sectoral challenges.

Benchmarked data will not lead to automatic removal of provision. Recognising that providers delivering similar provision may be impacted by different provider-level challenges, we’ll consider wider contextual information – for example, delivery location, learner and employer profiles, and cohort or provider size – when reviewing benchmarked data with you.

Further detail on benchmarked QAR data can be found in Annex 1.

Apprentices with protected characteristics

Every apprentice deserves excellence in their training provision, and all those considering an apprenticeship need to know that apprenticeships are accessible and that they will be supported to achieve.

We expect providers to offer opportunities for training and progression that meet the needs of a range of apprentices and businesses. This is in line with your duties under the Equality Act 2010 not to discriminate against apprentices with protected characteristics. Some apprentices will need support to achieve full occupational competence on their chosen standard.

We’ll consider the profile of a provider’s cohort when we review their performance.

The provider guide to delivering high-quality apprenticeships provides guidance on the additional funding and support available to support different cohorts to achieve.

Small or new apprenticeship provision

When deciding on intervention action for underperforming providers, we’ll consider whether they:

have small cohorts
offer new or immature provision

We expect these providers to set realistic improvement targets as a priority. We’ll challenge them on reasonable progress and evidence of impact.

Data timeliness and accuracy

It’s important that your data accurately reflects your apprentice population and performance at any point in the year. Your funding agreement sets out your obligations with regard to the accurate and timely reporting of data.

You can use the following tools to test its credibility:

funding information service
provider data self-assessment toolkit (PDSAT)
post-16 monitoring reports dashboard in view your education data
analyse FE data tool (AFED) – this includes an overarching dashboard for the quality view

Question 7

Complaints and feedback

Accepted Answer

You can complain about an intervention through our customer help portal, if you’re unable to resolve this with your case manager directly.

Question 8

Framework policy principles

Accepted Answer

The framework is underpinned by 5 principles.

Data-driven

We’ll use a wide range of quality indicators to give us a rounded overview of a provider’s delivery.

Risk-based

We’ll take a risk-based approach, using a range of quality indicators. These will focus on providers where there may be quality issues so we can intervene to protect apprentices’ interests.

Encourage self-improvement

We aim to:

identify risks to quality early
protect apprentices’ interests
support self-improvement

Timeliness

We’ll monitor your performance data throughout the academic year. This ensures that management conversations and intervention happen earlier, where necessary.

We expect you to monitor and review your performance data throughout the academic year.

Proportionality

We’ll only take interventions as a result of a management conversation. Interventions are not automatic and we’ll consider previous or ongoing management conversations and providers’ track record.

We’ll take proportionate action to support providers to address quality issues, where they demonstrate they have the capacity to improve in a timely manner or that some performance issues are evidently beyond their control.

Question 9

Annex 1: Guidance and technical notes for benchmarked QAR data

Accepted Answer

Benchmarked data can be found by selecting ‘check here to see how your stats compare to national comparison’ on the QAR section of the AAF dashboard.

It includes comparative benchmarking data, setting out providers’ overall, sector subject area and standard-level QARs against national equivalents.

It also includes detailed benchmarked standard-level QAR data, based on quartiles – this shows providers in more detail how their delivery compares to providers delivering the same standard, by setting out whether they are performing better than 25%, 50% or 75% of other providers delivering the same standard, or are in the lowest 25%. As this is a new way of presenting data, we have provided technical notes to support providers’ understanding.

AAF indicator thresholds will remain the route for identifying which providers will be contacted for an AAF management conversation. The indicator thresholds for QARs are based on providers’ overall QAR. Benchmarked QAR data will not be used as an indicator threshold.

Benchmarked data will be updated once a year with full-year data, once national achievement rates tables (NARTs) have been published. The front page of AAF achievements data will continue to be updated in-year alongside in-year QAR updates.

Benchmarked data will remain within each provider’s private AAF dashboard and will not be shared publicly or published on Explore Education Statistics (EES). The quartile benchmarked data is confidential information belonging to DfE that is not in the public domain and therefore should not be used in public-facing materials.

We welcome feedback on the first iteration of this data, especially on how we can improve its presentation in the AAF dashboard and the explanation in the technical notes. Email provider.strategy@education.gov.uk.

Quartile benchmarking: technical notes

How quartile benchmarking is calculated

Quartiles divide an ordered list of providers’ qualification achievement rates for a given standard (that is, ordered from lowest to highest) into 4 equal-sized groups to reflect the relative performance of providers.

We calculate that if a provider is achieving better than 25%, 50% or 75% of other providers for a given standard, based on respectively falling into the second, third or fourth quartile of this ordered list of achievement rates for that standard.

The QAR value is calculated at the 25th, 50th (median) and 75th percentile of an ordered list of achievement rates for a given standard. Each provider’s QAR is then compared to these values to calculate the quartile they fall into and the proportion of providers they are performing better than for that standard, as follows:

providers with a QAR less than or equal to the value at the 25th percentile: their QAR for this standard is within the lowest 25% of all QARs for this standard (the lowest quartile)
providers with a QAR less than or equal to the median, and above the 25th percentile: their QAR for this standard is better than that of at least 25% of other providers delivering this standard
providers with a QAR less than or equal to the 75th percentile, and above the median: their QAR for this standard is better than that of at least 50% of other providers delivering this standard
providers with a QAR above the 75th percentile: their QAR for this standard is better than that of at least 75% of other providers delivering this standard (the top quartile)

The minimum and maximum quartile boundaries for the quartile a provider falls into for a standard are set out in the dashboard under the columns ‘quartile boundary (minimum)’ and ‘quartile boundary (maximum)’.

The quartile a provider falls into is determined only by the value of their QAR. This calculation is based on the exact QAR value, calculated as achievers ÷ leavers (× 100), without rounding to the single decimal place QAR is usually displayed with.

Scenarios

Scenario 1

The quartiled approach reflects the relative performance of providers. For standards where most providers have high QARs, the boundaries between quartiles will also be high. For standards where most providers have low QARs, the boundaries will be low. This means that:

providers securing high QARs in high-performing standards may not fall into the higher quartiles when benchmarked against other providers delivering the same standard because their performance is good but relatively lower than other providers delivering the same standard
providers delivering standards that usually achieve low QAR rates may fall into the higher quartiles despite having a low QAR, because their performance is relatively better than other providers delivering the same standard

Scenario 2

In any instance where a provider has exactly the same QAR for a standard as another provider (a tie), they will both fall in the same quartile.

Scenario 3

Where the 25th, 50th (median) and 75th percentile value does not fall exactly on an existing provider, a calculated value is used. This means the boundary value may not be a QAR achieved by a specific provider.

For instance, in a standard offered by 12 providers, each quartile would ideally contain 3 providers if no ties existed. Calculated values would be used to create boundaries falling between the third and fourth, sixth and seventh, and ninth and 10th provider to split the providers into 4 groups of 3.

Scenario 4

Where the QAR of a provider is exactly equal to a boundary, the provider falls into the quartile below to ensure the statement ‘For this standard, your QAR is within the lowest 25% of other providers or better than at least 25% , 50% or 75% of other providers’ holds true.

Scenario 5

For standards where many providers achieve a 100% or 0% QAR, the calculated quartile boundaries may be 100% or 0%, so, in some standards, a 100% QAR may not place a provider into the highest quartile. In this instance, enough providers have 100% QAR for this standard that 100% is not an achievement better than a full 75% of other providers, as ties are not counted.

Scenario 6

At present, there must be at least 2 providers delivering a standard for quartile benchmarking data to be generated for that standard. We’ll explore setting a further minimum in future iterations of this data if user testing suggests this is appropriate. Quartile benchmarked data has therefore not been generated for standards delivered by only one provider. This means providers that are the sole deliverers of standards will see comparative benchmarked data but not quartile benchmarked data for these standards.

Scenario 7

The use of calculated quartile boundaries, as explained in scenario 4, will have a greater effect on standards offered by fewer providers, as follows.

Where a standard is delivered by only 2 providers, the calculated boundaries would fall 25%, 50% and 75% of the way between the 2 providers’ QAR values. This means one provider will fall in the lowest and one in the highest quartile.

Having more than 2 providers on a standard gives more data for the boundary calculations, increasing the accuracy of the calculated boundaries.

The quartile given where numbers are too small for even quartile splits represents the proportion of providers you would be achieving better than if the standard was more widely offered and achievement followed the current pattern.

As quartiles are a measure of the spread of current data, quartile benchmarking is a more accurate indicator where there is more data for comparison, as this allows for more even quartile splits and more accurate projections.

Cookies on GOV.UK

Applies to England