Guidance

Summary evaluations of multi-academy trusts

Updated 31 March 2023

Applies to England

Introduction

1. This page contains guidance for inspectors on carrying out multi-academy trust summary evaluations (MATSEs). It sets out the process and range of evidence-gathering activities that inspectors carry out from the start of the MATSE until the publication of the summary evaluation letter.

2. MATSEs are carried out in line with the policy agreed between the Department for Education (DfE) and Ofsted see the legal basis of MATSEs section of this operational note. They are not inspections and are carried out with the agreement of the trust being reviewed. They do not result in graded judgements.

Purpose of MATSEs

3. The aim of a MATSE is to recognise a trust’s positive impact on the quality of education in its academies, and to give the trust helpful recommendations on aspects of its work that could be improved.

4. During a MATSE, inspectors explore the extent to which the trust is delivering high-quality education and improving pupils’ achievement across the academies in the trust. They consider a range of information about the trust, including academy inspection outcomes and information gathered through discussions with trust leaders and visits to some of the trust’s academies.

5. The focus of the MATSE is the quality of education provided and how well pupils as a whole are achieving across the trust, rather than how individual academies, or pupils within them, are performing.

Legal basis of MATSEs

6. We collect evidence about the trust’s work through inspecting a sample of their academies. These inspections will be carried out when they are due (that is, we will not schedule them solely because of the MATSE), and will follow the procedures set out in the school inspection handbook and the monitoring handbook.

7. By carrying out inspections of academies within a single trust and carrying out a summary evaluation of the quality of education provided by the trust, His Majesty’s Chief Inspector (HMCI) is able to:

• better fulfil her general duty to keep the Secretary of State for Education (SoS) informed about matters connected with her remit
• perform her functions for the general purpose of encouraging and promoting improvement within her remit, as set out in sections 118(3) and 119(1) of the Education and Inspections Act 2006

8. Ofsted is commissioned by the SoS to carry out MATSEs. The terms of this commission are set out in a letter from the SoS to HMCI.

9. HMCI has a duty to inspect academies (including academies within a trust) at prescribed intervals under section 5(1) of the Education Act 2005. Further, under section 8(2) of the same Act, HMCI has the power to inspect schools, including academies, in circumstances in which she is not required to do so under section 5(1). HMCI/Ofsted has other general duties/functions that are relevant to these summary evaluations.

10. Throughout this operational note, any reference to a MAT or trust includes multi-academy companies.

The MATSE process

11. MATSEs have 2 stages:

Stage 1 – Routine inspections under the education inspection framework are carried out in the trust’s academies. When considering a trust for a MATSE, we will look at inspections that have taken place over a period of time not exceeding 2 terms. Typically, these inspections will have taken place over 1 or 2 terms, in accordance with our standard inspection windows, as set out in the School Inspection Handbook

At stage 1, the academy inspections may include:

• graded inspections, carried out under section 5(1) of the Education Act 2005
• ungraded inspections carried out under section 8(2) of the Education Act 2005
• monitoring inspections carried out under section 8(2) of the Education Act 2005

Urgent inspections carried out under section 8(2) of the Education Act are not routine inspections. However, if an urgent inspection should take place at the same time as stage 1, it may be included in inspections looked at as part of stage 1 if it provides relevant evidence on the impact of the trust’s work. Ofsted’s regional directors will decide whether to include the findings from urgent inspections in stage 1. Regardless of the circumstances, inspectors may consider the outcome of an urgent inspection during a summary evaluation in the same way as they would any other inspection outcome.

Stage 2 – On-site visit: once stage 1 is complete and all inspection reports have been published, we carry out the on-site visit to the academy trust. This will usually be during the term after stage 1.

The visit will take place over the course of a week and will involve:

• meetings with trust personnel, including trustees
• reviews of relevant documentation
• visits to the trust’s academies (the number of academies visited will depend on the size of the trust)
• surveys of academies not inspected at stage 1 or visited as part of stage 2 (these will take place through video or telephone calls)

The academy visits and surveys are to gather specific evidence on the points that emerged from the discussions with trust leaders and trustees. They gather the views of academy leaders and staff on the impact of the trust’s work.

Clarification for trusts

12. We aim to ensure that our activities do not create unnecessary workload for trusts. The information below confirms how these events will work and will correct any misconceptions about MATSEs.

Stage 1

We will:

• inspect the academies within the trust that are due for an inspection (each academy will receive its own inspection report)
• expect to see academies as they normally operate
• note whether any inspections included took place before the school joined the trust
• not usually include urgent inspections unless they provide relevant evidence on the impact of the trust’s work

We will not:

• carry out a graded, ungraded or urgent inspection of any academy within the trust just because we have decided to carry out a MATSE
• expect trusts to carry out any activity or preparations specifically for the purpose of the summary evaluation

Stage 2

We will:

• with the agreement of the trust, visit a sample of the trust’s academies
• when deciding which academies to visit, be mindful of how recent the last inspection was
• carry out surveys of a sample of academy leaders to gather their views
• meet with key personnel from across the trust, as agreed with trust leaders
• at all times adhere to the guidance on Ofsted’s conduct during Ofsted inspections
• only carry out inspections of individual schools within the trust during stage 2 when there is an urgent need to do so

We will not, except where the trust and the lead inspector agree it is necessary:

• visit any academies that were inspected during stage 1
• visit any academies that are in a category of concern, as defined in paragraph 162 of the school inspection handbook

Selection of trusts for MATSEs

13. We are not carrying out a programme of MATSEs that will see every trust evaluated within a certain window. In selecting trusts for summary evaluations, we aim to cover a broad range of trusts, not just those that may be a cause for concern. This is to give us an accurate and balanced understanding of the contribution that trusts make to the school system, to highlight areas of strength that may be disseminated more widely and to provide an insight into any weaknesses.

14. Each year, we aim to select from a wide range of trusts so that the overall sample is broadly representative of the sector.

15. In selecting a trust for a summary evaluation, we will take account of a range of information, including:

• the number of academies in the trust
• the make-up of the trust, for example whether it is mostly made up of primary, secondary or special schools, or alternative provision
• the geographical spread of the trust, for example whether it cuts across multiple Ofsted regions
• the pupils within the trust, including pupils who are disadvantaged and pupils with special educational needs and/or disabilities (SEND)
• the number of academies in the trust that are due for inspection in the coming terms, relative to the size of the trust
• previous inspection outcomes of trust schools, including changes in the performance of individual academies over time, reflecting where there have been significant improvements or a decline while a school has been in the trust
• previous MATSE and focused review outcomes (where relevant)
• progress measures and pupils’ attainment at academy level and across the trust
• the number and nature of any complaints received by Ofsted about academies within the trust
• any warning notices issued by the SoS to academies within the trust
• any other intelligence that indicates good practice within a trust, or that raises concerns, for example regarding the safeguarding of pupils
• any requests from the SoS

These requests may be submitted by the DfE regional director acting on behalf of the SoS. The circumstances in which the SoS may elect to make such a request through the DfE regional director could include seeking advice about a particular trust, or trusts more generally, or a number of academies within a trust.

16. All academies selected for inspection must be due to be inspected (under section 5 or section 8 of the Education Act 2005), in accordance with:

• our standard inspection windows
• the maximum permitted interval between inspections prescribed by regulations
• the monitoring inspection arrangements under section 8(2) of the Act for schools judged to be causing concern or as requires improvement
• our guide for inspecting academies and free schools

Stage 1 routine inspections

17. No academy is inspected solely for the purposes of a MATSE. All inspections considered as part of stage 1 are routine inspections. They are carried out as part of our normal inspection work, and in the usual way.

Before the stage 2 visit

Preparation for the MATSE

18. Before notifying the trust of a MATSE, the lead inspector will review the information we hold about the trust and any information that may be publicly available.

19. As part of their preparation, inspectors will review all the evidence and reports from inspections carried out during stage 1 and any other inspection outcomes or reports for academies within the trust.

Notifying the trust

20. The lead inspector will notify the trust of the MATSE (stage 2 on-site visit), typically up to 5 working days before it starts. Subject to the availability of the trust’s chief executive officer (CEO), the lead inspector will ask to speak to the most senior executive officer available. In this document, ‘CEO’ refers to the CEO or their equivalent.

21. MATSEs are carried out with the trust’s agreement. Once the trust has been notified of the MATSE, and the trust has confirmed it agrees, we will send formal confirmation to the trust by email. If the trust indicates that it does not agree to the MATSE, then we will share this information with the relevant DfE regional director.

22. Once the MATSE is confirmed, the lead inspector will hold a preparatory telephone conversation with the CEO. This may be in the same telephone call as the notification or in a separate call, as agreed by the lead inspector and the CEO.

23. The telephone call is an important opportunity to initiate a professional relationship between the lead inspector and trust leaders. As with academy inspection notification calls (detailed in the school inspection handbook), the call has 2 elements:

• a short planning conversation that focuses on practical and logistical issues

• a longer reflective conversation, focused on the trust, in which the lead inspector may discuss any particular areas of interest stemming from the information that they have reviewed in preparation for stage 2

The planning conversation

24. The purpose of the planning conversation is for the lead inspector to:

• make logistical arrangements (inspectors’ room, parking and other facilities)
• inform the trust that there will be some academy visits and surveys during the week, to be agreed with the trust
• explain the extent to which senior staff can be engaged in activities and make arrangements for the activities, including discussions with seniors leaders and other staff, trustees and members
• explain which documents inspectors are likely to need to see when they are on site and ask for relevant evidence to be made available during the on-site week, making clear that we do not expect information to be provided in any particular format or expect the trust to do work specifically for the MATSE
• discuss the arrangements for keeping in touch with the trust throughout the visit and agree the process for final feedback
• confirm arrangements/contact details for the purpose of our post-MATSE survey
• discuss the timetable for activities, including inspectors’ intended arrival and departure times, and remind the trust that the summary evaluation letter will be published on Ofsted’s reports website.

The trust-focused conversation

25. Inspectors will use this conversation to:

• clarify the structure of the trust and how it is governed
• discuss the findings of the stage 1 inspections
• discuss the CEO’s evaluation of the strengths and weaknesses of the trust
• discuss the emerging areas of interest for the summary evaluation

Requests for a deferral

26. We may, in exceptional circumstances, defer the stage 2 on-site visit.

27. Where trusts have concerns about the timing of a stage 2 visit, they may ask us to defer. As summary evaluations are carried out with the trust’s agreement, our usual policy on deferrals will not apply.

28. We will make every effort to accommodate the specific circumstances of the trust so that the summary evaluation can proceed. The decision to defer is at the discretion of the relevant Ofsted regional director. If deferral cannot be agreed and the MATSE cannot proceed, we will share this information with the relevant DfE regional director.

During the stage 2 on-site visit

MATSE timetable

29. On each day of the MATSE, inspectors will not arrive before 9am and will leave by 6pm except in exceptional circumstances. Inspectors will work within the trust’s standard office hours.

Arrangements for the stage 2 on-site visit to the trust

30. The lead inspector will meet briefly with the CEO or other representatives on the first day of the MATSE to:

• introduce team inspectors and other attendees
• confirm arrangements for meetings with key staff, as agreed with the trust, over the course of the week
• confirm arrangements for providing feedback during and at the end of the evaluation
• discuss practical issues, including plans for academy visits and surveys

31. There is no expectation that the trust will prepare evidence specifically for the benefit of inspectors. Inspectors are likely to encounter a variety of approaches and arrangements. They will accept existing evidence in whatever format the trust provides. They may take into account information such as:

• if available, a summary of any evaluation the trust has carried out of the effectiveness of its work, including the overall quality of education within its academies and arrangements for supporting and improving this
• documentation about arrangements for monitoring, challenging, intervening in and supporting the trust’s academies
• any documentation or strategies in relation to curriculum design and implementation (including details of any partnership arrangements, commissioning and brokerage), and any evaluation of impact
• any documents or strategies in relation to improving pupils’ behaviour and attendance, including the use of exclusion, managed moves and part-time timetables
• any documents or strategies in relation to the recruitment, training and continual professional development of academy staff at all levels
• any data about the trust’s performance and any analysis carried out
• case study material on targeted academy improvement work and its impact on the trust’s academies
• a list of key staff, with roles and responsibilities
• information regarding strategies used to improve the quality of the curriculum across the trust’s academies, and the evidence of their impact
• information on the strategies used to improve the standards of governance across the trust

32. The purpose of the meetings with key leaders and trustees is to share and discuss any areas of focus drawn from the lead inspectors’ preparation for stage 2 of the MATSE, including an analysis of the inspection outcomes of the trust’s academies. In addition, meetings will provide an opportunity for the trust to present, and inspectors to review, further evidence of the impact of the trust’s work. Discussions will focus on the quality of education across the trust and the trust’s arrangements for overseeing, challenging and supporting its academies.

33. Inspectors will hold separate meetings with trustees, without the CEO or other trust leaders present. They will also consider the outcomes of academy surveys carried out during the MATSE and evidence gathered during academy visits. They will give the trust every opportunity to present relevant evidence.

34. We do not require or expect trusts to use any particular structure or curriculum approach. Therefore, inspectors will comment on the effectiveness of the arrangements that are in place, based on the evidence they gather. The DfE has guidance on governance structures in MATs, which includes information about effective arrangements in relation to checks and balances.

What inspectors may consider

35. While discussions and the evidence presented may be wide ranging, inspectors will focus on the trust’s impact, both on pupils and in terms of raising standards and the quality of education offered, including through improving the performance of its academies.

36. Inspectors will explore these themes with trust leaders, trustees and staff at all levels, including local academy leaders and teachers. What is discussed in these meetings will be shaped by the emerging strengths and weaknesses from the routine inspections in stage 1 and any evidence from discussions with trust leaders and other activities in stage 2.

37. Inspectors will, in practice, tailor their areas of focus to the circumstances of the trust. In when exploring these areas of focus, as outlined in paragraphs 38 to 43.

Quality of education

38. Academies are expected to offer all pupils a broad curriculum that is similar in breadth and ambition to the national curriculum. Inspectors will evaluate the trust’s overall approach to curriculum design and how the trust assures itself of the quality and impact of this, including for disadvantaged pupils and those with SEND.

Behaviour and attitudes

39. Inspectors will explore the trust’s approach to behaviour management and whether this results in safe, calm, orderly and positive environments in its academies. They will explore the extent to which the trust’s challenge and support of its academies are helping to improve attendance. They will look at relevant policies, including attendance policies, and how they are developed, monitored and implemented. They will also usually look at how the trust uses and monitors suspensions and exclusions and its approach to alternative provision and managed moves.

Leadership and management

40. Inspectors will explore the impact of leaders and managers on the trust. They will want to understand how trust leaders ensure that they operate in a way that builds strength, resilience and accountability at all levels. They will want to consider how trust leaders support and sustain leadership in academies. They will discuss the support and training that the trust provides, why these were identified as being required and the difference that the training has made.

Governance

41. Trust structures are diverse, and Ofsted has no fixed view on what constitutes the best arrangement. Inspectors should therefore avoid advocating any particular structure or arrangement. Inspectors will instead understand and consider the individual trust’s scheme of delegation and governance structure. They will also consider the 3 core functions of governance as set out in the DfE’s governance handbook.

Reviewing safeguarding

42. In addition to the areas outlined above, inspectors will also consider safeguarding across the trust. Safeguarding children is a responsibility shared by all in the system, as set out in statutory guidance. This applies to all ‘practitioners’, whom the statutory guidance defines as those working with children and their families in any capacity.

43. All schools, including academies, should have a culture of safeguarding. This means they should have effective arrangements to:

• always act in the best interests of children, pupils and students to protect them online and offline, including when they are receiving remote education
• identify children, pupils and students who may need early help and who are at risk of harm or have been harmed. This can include, but is not limited to, neglect, abuse (including by their peers), grooming or exploitation
• secure the help that children, pupils and students need and, if required, refer in a timely way to those who have the expertise to help
• manage safe recruitment and allegations about adults who may be a risk to children, pupils, students and vulnerable adults

Inspectors will consider the trust’s overall approach to safeguarding. They will evaluate the extent to which trustees and trust leaders assure themselves that there is an effective culture of safeguarding in each of their schools.

Arrangements for the academy visits

44. During the evaluation, inspectors may also visit some academies in the trust. These visits are not inspections of the individual academies. Their purpose is to gather further evidence about the trust’s impact and to provide an opportunity for trust leaders to demonstrate this impact. The visits are not intended to gather information about the academy and will focus on discussions with academy leaders, including local governors (where relevant), staff and pupils about their experiences of being part of the trust.

45. The lead inspector will discuss and agree with trust leaders which academies will be visited. Normally, they will aim to exclude academies that were inspected during stage 1. These visits will not typically include academies that are in a category of concern. If trust leaders feel strongly that a visit by inspectors to such an academy would provide compelling evidence of the impact of the trust’s work, inspectors may visit it. The decision to do so is at the discretion of the lead inspector.

46. The visits are intended to be short: typically no more than 2 hours. They may involve discussions with academy staff and local governors, or other activities, as agreed between the lead inspector and trust leaders. Discussions will not normally take place with pupils unless there is a clear need for them.

47. Although the academies visited are not being inspected, in exceptional circumstances (for example, if a serious concern about safeguarding arises during the visit) we may have reason to inspect the academy at a later date. Depending on the circumstances, this may mean that the feedback to the trust and/or publication of the summary evaluation letter are delayed so that the outcome of that inspection can be considered fully as part of the summary evaluation.

Arrangements for the academy surveys

48. Evidence gathered from the routine inspections carried out as part of stage 1 and the academy visits at stage 2 will be supplemented, whenever possible, by a survey of academies within the trust.

49. Inspectors will carry out the surveys during the stage 2 visit. The lead inspector will, in consultation with trust leaders, select a sample of academies, usually those not inspected during stage 1 or visited as part of stage 2.

50. In the surveys, inspectors are likely to explore:

• how well the trust understands the quality of education it offers, and in particular its strengths and weaknesses
• the measures in place to further enhance strengths and address weaknesses
• how the trust knows it is having a positive impact on the quality of education provided for its pupils

Inspector team meetings

51. The team will meet briefly each day. This may be by telephone when inspectors are visiting academies that are a significant distance from the site at which the team is based.

52. During these meetings, the summary evaluation team will:

• share initial findings
• discuss hypotheses
• corroborate evidence
• consider any additional evidence-gathering that could be helpful to clarify the work of the trust

53. Wherever possible, a representative from the trust, typically the CEO, will be invited to attend meetings as an observer.

Providing feedback to the trust

54. Throughout the MATSE, inspectors will offer verbal feedback to senior leaders and other staff, trustees and members.

55. Before the end of the stage 2 visit, inspectors will ensure that the trust is clear:

• about the inspectors’ view of the relative strengths and weaknesses of the quality of education offered across the trust
• that the main points of feedback will also be referred to in the text of the summary evaluation letter, subject to quality assurance
• about the procedures for quality assuring and publishing the summary evaluation letter
• about the procedure for making a complaint

56. On the final day of the MATSE, inspectors will meet to draw their evidence together and agree the provisional findings. Inspectors will agree recommendations that will be recorded in the summary evaluation letter. Senior staff and trustees may attend this meeting as observers, as agreed between the CEO and the lead inspector.

57. After inspectors have reached their provisional findings, they will give a summary of these to the trust. While any feedback should be consistent with the findings, all findings will be subject to moderation and quality assurance by senior Ofsted staff. This will be clearly explained to trust leaders during feedback.

After the summary evaluation

The summary evaluation letter

58. The findings from the MATSE will be set out in a letter to the trust’s CEO and copied to the SoS, the chair of the board of trustees and the DfE regional director. The letter will recognise any specific areas of strength and highlight any recommendations for improvement. It will report the range of evidence considered and list the most recent inspection outcomes for all academies in the trust. It will be published on the Ofsted reports website.

59. Before publication, the lead inspector will share the draft summary evaluation letter with the trust. The trust will normally have 5 working days to comment on the draft letter, including on any matters of factual accuracy.

60. Inspectors will consider all comments received and address them before the letter is finalised and published.

Quality assurance

61. All inspectors are responsible for the quality of their work. The lead inspector must ensure that MATSEs are carried out in accordance with this operational note and Ofsted’s code of conduct.

62. We monitor the quality of MATSEs through a range of formal processes. HMI/Senior HMI visit some trusts or monitor remotely to quality assure these processes. We may also assess the quality of a summary evaluation evidence base. The lead inspector will be responsible for giving team inspectors feedback about the quality of their work and their conduct.

63. The draft MATSE letter will be checked to ensure that the wording is clear and coherent. In addition, the letter will be quality assured by the relevant regional director before publication.

64. All trusts are invited to take part in a voluntary post-MATSE survey in order to contribute to policy development.

Handling concerns and complaints

65. Any concerns and complaints will be handled in line with our complaints procedure.