Transparency data

16 June 2021 open paper: Changing the governance of the Standards Advisory Group National Reference Test Sub Group – accessible version

Updated 9 June 2022

Applies to England

Date

16 June 2021

Title

Changing the governance of the Standards Advisory Group National Reference Test Sub Group

Report by

Dennis Opposs, Standards Chair

Responsible Director

Michelle Meadows, Executive Director Strategy, Risk and Research

Paper detail

• Paper for decision
• Open paper
• Paper to be published
• Publication with meeting minutes

Issue

After five years of operation, it is appropriate to consider again how Ofqual can best receive expert advice in relation to the National Reference Test (NRT).

Recommendation

That the Board agrees that its Standards Advisory Group National Reference Test Sub Group is dissolved and replaced by a new group outside of the formal Committee structure that advises Ofqual’s Deputy Chief Regulator on the Test.

Background

Ofqual awarded a contract to the National Foundation for Educational Research (NFER) in 2015 to support the introduction of the NRT and to deliver it on Ofqual’s behalf. That contract is still in operation.

In 2016, the NRT Sub Group of the Standards Advisory Group was established to support Ofqual’s work in introducing and using the NRT that was then still in its development phase. The focus of the work was on the technical and statistical aspects of assessment such as the school and pupil sampling methodology, and the psychometrics around the analysis of the marking.

The Standards Advisory Group is formally a Committee of the Board and the Sub Group is a Sub-Committee of the Standards Advisory Group. It is timely to review the role and nature of the Sub Group.

Analysis

In the early years of its work, the Sub Group met two or three times annually. However, the tests have been running for five years and, with expert guidance from the Sub Group and much development work, the technical and statistical aspects of the test data are now well established.

Had this year’s GCSEs run as normal, the Sub Group would have met just once in 2021. The meeting would have reviewed the draft technical report from NFER and provided advice on the conclusions drawn from the outcomes of the 2021 tests. There is now little development work to do on the analysis on presentation of the data so one meeting is sufficient.

2024 is the last NRT in our current contract with NFER. We need to consider what we want from NRT in the subsequent five or so years and to manage carefully any transition in the supplier or in the design of the test to ensure continuity in trend data. Allowing time for any new supplier to shadow the administration of the 2024 test means that we should decide our requirements in the next six to 12 months, making now a good time to look afresh at how we receive advice and support.

In these circumstances we think it would help to have more flexible governance arrangements around the Sub Group, its membership and the focus of its meetings. The last set of appointments to the Sub Group have now expired. We therefore recommend removing the Sub Group from the Board’s governance structure, dissociating it from the Standards Advisory Group (although normally having an overlap of at least one member), and having the new group give advice directly to Ofqual’s Deputy Chief Regulator and Executive Director of Strategy, Risk and Research. The Deputy Chief Regulator would then provide assistance to the Chief Regulator as to how, if at all, the NRT outcomes should be used each summer as part of GCSE awarding.

Draft terms of reference for the new Group are attached at Annex A for the Board’s information.

Finance and Resource

The financial and resourcing costs associated with the proposed new Group will be very similar to those for the existing Sub Group and covered within Ofqual’s budget.

Impact Assessments

Equality Analysis

Refreshing the terms of reference of the Group gives us an opportunity to reconsider ways in which the NRT design takes account of the most relevant equality matters such as the representativeness of the sample and what personal data we collect on pupils. In appointing members to the new Group, we will aim for a diverse make up.

Risk Assessment

There is a risk that SAG and therefore the Board are not as sighted on technical matters associated with the NRT. We will manage that risk by committing to raise with them anything material.

Regulatory Impact and Growth

Not applicable.

Timescale

If the Board agrees to the change then it will be implemented immediately.

Communications

The decision will be communicated to the Standards Advisory Group at its next meeting which is scheduled for July. We will also ensure that we are prepared to manage any external enquiries that we receive about the new Group.

Internal Stakeholders

The same staff connected with the existing Sub Group would be associated with its proposed replacement.

External Stakeholders

The work of the Standards Advisory Group National Reference Test Sub Group has not been directly visible to external stakeholders.

ANNEXES LIST

ANNEX A – Draft Terms of Reference for the proposed National Reference Test Advisory Group

National Reference Test Advisory Group – Draft Terms of Reference

Purpose

The purpose of this Advisory Group (NRTAG) is to support Ofqual’s responsibilities in relation to the National Reference Test (NRT). The NRT is normally taken annually by a nationally representative sample of Year 11 students to provide additional information that can contribute to the award of GCSE English language and GCSE mathematics later that summer.

Membership

NRTAG shall consist of up to six independent members, at least one of whom is normally also a member of the Standards Advisory Group. One member of the Group will be appointed to be its Chair.

Appointments are made for a period of up to three years. Appointments are renewable.

Members are appointed to NRTAG to bring particular expertise and will be expected to attend all meetings unless otherwise advised. Alternates are not permitted.

Role

The role of NRTAG is to advise Ofqual’s Executive Director for Strategy, Risk and Research. The advice will focus on the analysis and presentation of NRT results, the use of NRT results in GCSE awarding, longer term plans for the development of the NRT, and on any other issues associated with the NRT at the request of Ofqual’s Executive Director for Strategy, Risk and Research.

Meetings

NRTAG shall meet at least once a year.

A quorum shall be considered to be three members.

Meetings of NRTAG will normally be attended by Ofqual’s Executive Director of Strategy, Risk and Research, the Director for Standards and Comparability, the Standards Chair, and relevant senior members of the Strategy, Risk and Research Directorate. Other Ofqual staff and members of organisations associated with the NRT such as the contracted delivery body may be asked to attend meetings as appropriate.

The Chair may ask any of those who normally attend to withdraw to facilitate open and frank discussion of particular matters.

Members will need to read and review documentation before each meeting in order to provide an appropriate level of insight. Members of the Group will be paid a fee of £300 for virtual, half-day meetings and £600 for full day meetings. In exceptional circumstances, where Ofqual requests a commitment well beyond the norm, it will provide an additional fee for preparation or follow up work.

Reporting

Ofqual will take notes of each meeting and circulate them to members.

Review

These terms of reference will be reviewed at least once every three years.