Policy paper

EIF evaluation plan and theory of change

Published 22 August 2023

Applies to England

In the Ofsted strategy 2022–27, we committed to evaluating the education inspection framework (EIF). We are accountable to Parliament on how we inspect and how we spend taxpayers’ money. This evaluation supports our commitment to transparency. It will help us learn from and improve our policy, practice and training to make sure we have the best impact we can, both on those we inspect and on children and learners.

Our programme of evaluation

We are carrying out a programme of evaluation spanning 2 to 3 years. We will start with a process evaluation in 2023 and follow this with an impact evaluation.

Our evaluation programme covers:

• the early years
• schools, including provision for pupils with special educational needs and/or disabilities (SEND) and independent schools
• further education and skills, including provision for learners with high needs

Through the evaluation, we want to understand:

• the impact of the EIF on those we inspect – childminders, nurseries, schools, colleges and other education providers
• the value the EIF has for parents, central and local government, academy trusts, governing bodies and other education sector stakeholders
• whether we are implementing the EIF as we intended

Although we are ultimately interested in the impact of the EIF, we first need to assess whether we are implementing it as we intended in our process evaluation.

Our whole programme of evaluation will look at the main things we designed the EIF to achieve and the processes that are distinct to it. We will not look at aspects of the EIF that apply to inspection in general, such as inspection grades and notice of inspection.

Theory of change

Our approach to the evaluation is theory-based.^{[footnote 1]} We have developed a theory of change that identifies how we expect the EIF to work and the impact we expect it to have.^{[footnote 2]} Our approach is to evaluate whether the assumptions and mechanisms in the theory of change hold true.

To develop the EIF theory of change, we have worked with an academic adviser with expertise in inspection research and evaluation. We have also held extensive conversations within Ofsted. We spoke to those involved in developing the EIF and its current implementation, as well as to His Majesty’s Chief Inspector, Amanda Spielman. We also collated evidence from various internal and external documents and training materials.

The theory of change clarifies how the features of the EIF are expected to contribute to its desired outcomes. We are just one of many actors in education and training. As an inspectorate, we are only responsible for diagnosis. We are not an ‘improvement agency’ – improvements are made by the institutions themselves, with or without external intervention – but our inspection, regulation and insights can contribute to these improvements.

The diagram below shows a high-level theory of change that can be applied differently in different inspection remits. It should be interpreted as an initial guide to considering the contributions that the EIF could make to the education sector.

The process evaluation will assess:

• whether we are carrying out inspections under the EIF as intended across each of the inputs and outputs in the diagram below
• any barriers to us carrying out effective inspections under the EIF

The impact evaluation will assess:

• the extent to which the mechanisms in the diagram below are happening
• the contribution the EIF has made to the short-term outcomes in the diagram below
• the value it has for stakeholders
• any unintended consequences

Overall, we designed the EIF with the intention of putting curriculum – the real substance of education – at the heart of education. The theory of change identifies how we expect this to happen, through inspection handbooks, inspectors’ professional dialogue, inspection judgements and inspection reports.

The EIF is underpinned by an evidence-based conception of what constitutes high-quality education. This is communicated through our published research, methodology and inspection handbooks. As part of the EIF, inspectors make judgements and hold conversations about education that give leaders the information they need to develop and improve.

For the EIF to contribute positively to change, inspection needs to inform leaders, parents, government and education sector stakeholders about the things that matter. In response, they need to have the ability to act on what inspection tells them.

We expect inspection to contribute to changes in behaviour. As a result, we expect children and learners to receive a higher quality of education and training and be better prepared for their next steps.

In the longer term, we hope to see a shift towards a shared conception of quality across the education sector.

EIF theory of change diagram

Inputs

Inputs are the features of the EIF that are needed in inspection:

• evidence, research and insights
• the EIF handbooks
• inspector training
• inspectors

Outputs

Outputs are the actions and activities that we deliver as part of the EIF:

• professional dialogue and feedback to providers
• inspection judgements
• inspection reports

Change mechanisms

Change mechanisms are the actions that are needed for the EIF to result in the intended outcomes:

• the sector is informed about quality
• providers act on the feedback they receive
• stakeholders and regulators are informed about quality
• parents and learners are informed about quality and challenge providers

Short-term outcomes

Outcomes we expect the EIF to contribute to first, such as:

• other regulators can take action after receiving information from us
• parents’ and learners’ choices are informed
• providers focus on curriculum quality
• providers are ambitious for all
• learners are safe and equipped to make well-informed decisions
• learners have high-quality education and training and are ready for their next steps
• providers use assessment and performance data appropriately and proportionately
• unnecessary inspection-related workload is reduced
• the sectors focus on evidence-based curriculum quality
• providers have improved overall effectiveness

Longer-term outcomes

Outcomes that we hope the EIF will contribute to, such as:

• learners’ achievements improving
• putting the curriculum at the heart of education
• a greater choice of quality providers
• a shared concept of quality in the sector

1. Theory-based evaluation is a methodology used to draw causal conclusions about a programme’s effectiveness in the absence of any comparison group. In this instance, there is no comparison group due to the universal availability of the framework since its publication in 2019. ↩

2. Theory of change is a comprehensive description and illustration of how and why a desired change is expected to happen in a particular context. ↩