Ian Bauckham CBE’s speech to the Confederation of School Trusts annual conference
Ian Bauckham CBE’s speech to the Confederation of School Trusts annual conference, on the topic of assessment in 2021.
Good morning colleagues.
Thank you for that kind introduction – I am very pleased to have the opportunity to speak with you as Chair of Ofqual at what is obviously an important time for schools and colleges with candidates taking formal qualifications this summer.
The decision taken by the government back at the start of January to cancel examinations and move to teacher assessment for all qualifications was a tough call, made at the time when the lightning spread of infection was at its very worst and the differential impact of school disruption on learning was becoming a serious problem.
Although we are now as a country in a far better place than we were then, it was not possible to know at that point just how long it would take us to get infection securely under control and thus enable schools and colleges to open normally again. So it was a tough decision that probably did have to be taken to give some certainty to students, parents and schools.
It would, of course, be a grave mistake to assume that during these extended periods of disruption, nationally during the first lockdown and this last 2021 lockdown, and more geographically differentially during the autumn, the business of education has not been continuing.
Apart from all the pupils who attended throughout, we have all grown hugely in our confidence in remote teaching, and by the 2021 lockdown, in very many cases, it was actually happening very effectively indeed. Moreover, my own impression is that the capacity trusts have brought to bear has effectively been a ‘proof of concept’.
Repeatedly, I hear heads and others who are in schools in good trusts telling me that they have felt confident and secure in their approach, supported and up to date with information and the recipients of shared innovation to manage education in this time of crisis. Great credit is due to these trusts for the work they have done.
Despite all this, we are facing a cohort of young people taking qualifications this year, many of whom will not have covered either as much content as normal, or will have covered it in less depth than usual.
And the distribution of differential impact is very granular. It cannot simply be broken down by geographical area or region, nor even by school. Within one school there will be young people who have, despite strong remote teaching, had very different educational experiences, one from the other. Access to devices, connectivity, support within the home, a place to work – all of these are uneven outside school in a way which is simply not the case when everyone is in a classroom in front of the same teacher.
So an underpinning principle of assessment this year is that it should be based (within reasonable limits) on content that teachers have actually covered. Of course, that will vary from school to school but that is a variation we are prepared to accept this year in order to give a fair chance to everyone.
A second principle for assessment this year is that it is based on teacher professional judgement.
It is worth pausing for a moment to reflect on what the term professional judgement actually means. Firstly, it means judgement which is not inappropriately influenced by those outside who may have a vested interest. As I have said before, and will say again now: it is critical that teachers and schools are able to make these judgements without pressure from parents or others who may, albeit for understandable reasons, want grades to be higher than the teacher’s judgement and reasonable evidence indicate is right.
We have been clear that students should know on what basis grades are being determined, but knowing how a grade is being determined - ie what evidence is being used – is emphatically not the same thing as then pressurising a teacher or a school to award a higher grade, or indeed use different, potentially more favourable, evidence. Other than in exceptional cases, clearly set out in the guidance, the expectation is that the identified evidence on which grades are based is the same across a cohort within a school. It becomes impossible to effectively standardise, thus ensuring fairness, in any other way.
A second aspect of professional judgement is that it is undertaken following proper training and established policies, so that the exercise is transparent and stands up to scrutiny. That is essential for fairness, and certainly important in the event of later challenge. Most schools and colleges will by now have run some parts of the training provided, or be about to do so, (I know we certainly have, with my school leader hat on) and it is critical that that training happens before teachers begin the process of determining grades.
Teachers are of course amongst the most respected and trusted professionals in society, and the level of public respect for them has only increased over the past year. That integrity which is at the heart of teacher professionalism means that teachers overwhelmingly want to undertake this year’s important work fairly and honestly. But they will also quite reasonably want to know that every one of their colleagues in other schools is acting with the same degree of integrity.
It is part of our job, as the regulator, to put in place the conditions which make it possible for every teacher, every school, to feel supported in doing the right thing. That is the purpose of the quality assurance processes we have put in place for this year’s grading, about which I will say more in a moment.
But before I turn to quality assurance, let me first just say what I mean by doing the ‘right thing’. Qualifications serve a number of different purposes in normal years, including accountability for schools and for teachers. But their core purpose, and effectively the only one that matters this year, given that all qualifications-based accountability has been set aside, the only purpose that matters is what I will call the passport function.
Qualifications have currency because they are accepted and well-understood indications of the point students have reached in their learning which indicate the most appropriate next stage for them, whether that is an educational destination or a training placement or employment. Effectively, young people rely on qualifications to get them to the right next stage. That is why it is essential that qualifications are fair, and this is part of what we mean when we talk about validity. The qualification a young person holds must ‘tell the truth’ about them to the point that it gives them access to the right next destination.
Moreover, many qualifications continue to serve that purpose not just at the immediate point of transition but in later life too, which raises the stakes even further.
If qualifications do not tell the truth about their holders’ abilities, what we risk is not only eroding the value of hard-won outcomes for young people, but actually we risk sending them to destinations, be they FE, university, post 16 courses or training programmes, which are not right for them, risking later drop out, and potentially displacing others who would have been better qualified and better suited to those destinations.
That helps no-one.
So the mark of professionalism for us as teachers and educational leaders is to set aside our personal feelings, not allow ourselves to be influenced either by other individuals or by our own aspirations or sympathies for individual pupils, and make our judgements dispassionately and objectively. That is what doing the ‘right thing’ really means. It can, of course, feel hard, but it must be the right way.
So let me turn then now to quality assurance. I said earlier that quality assurance is part of the way in which we make it easier for teachers and schools to do the right thing, and that is both because it provides the assurance that other teachers in other schools will need to work on the same basis, and because it strengthens the position of teachers and schools should they come under inappropriate pressure to bend their judgements. ‘I am sorry, I know you are asking me to award a higher grade to this student, or look at work outside the range of evidence which we have decided to use, but my work may well be quality assured, so I am afraid I am not able to do that.’
As most teachers and schools will know by now, the quality assurance process this year will have three main stages.
The first stage is the requirement to create and submit a centre policy for awarding. This process is largely complete and the deadline is in fact tomorrow. Also in stage 1 is a direct contact between exam board and centre to ask about key aspects of the policy. In cases where the policy is unclear or raises concerns, these will be probed in more detailed discussions.
The second stage is the virtual visit which will take place in every case where there are unresolved concerns arising from the policy. Plans for internal standardisation of grades will be a particular focus here, but Boards reserve the right to test out the robustness of any aspect of the centre’s planned approach, and to agree modifications where necessary in the interests of fairness and consistency. Not every centre will get a virtual visit, only those where Boards deem there are questions or concerns which need resolving.
The third stage is the sampling of centre judgements. This will start immediately after the deadline for submitting grades has passed. The sampling process will begin with a universal sampling, covering all centres, regardless of size or type, and both GCSE and A Level, where both are offered.
All centres will be required to submit samples of evidence used to underpin judgements. Boards will select subjects and candidates, and the turnaround time for this will be short. This first stage is designed to be non-labour intensive and administratively straightforward for exams officers to manage. It is important for boards to have this universal sample bank from the start to enable them to move speedily through the QA process before schools start to finish for the summer break, which in some cases is by the end of June or early July. For the majority of centres, where there is no reason for further scrutiny, this will be all that they need to do.
For centres which are flagged for further scrutiny, the initial sample will be the starting point, but in those cases boards will sometimes need to ask for further evidence, either in a wider range of subjects or a larger number of candidates.
Being selected for further scrutiny is not a pre-judgement of a centre’s grades – they may of course turn out to be acceptable. But if wider sampling does support an emerging concern, then the board may ask the centre to make adjustments to its grades, in the interests of fairness to all candidates. In the last analysis, Boards reserve the right to withhold or delay the publication of results if it is not possible to agree an acceptable way forward in any such cases.
So what factors will flag a centre for this kind of additional scrutiny? That is certainly a question I would want to know the answer to, as a head of a school or college, or, indeed, as someone in charge of a trust.
There are a number of main ways in which this can happen:
- initial concerns over the approach the centre is taking as set out in its policy and follow up discussions
- concerns seen during a virtual visit at stage 2, whether or not these have been subsequently corrected
- a pattern or profile of results which looks surprising, given the centre’s results history in 2017-19
- a significant change in the pattern of entries from previous examined years, for example significant numbers of year 10 entries where this has not previously been the case
- being a centre with no previous examination results
- any centre where boards receive other intelligence relating to potential malpractice, for example from whistle-blowing
- if the centre is part of the random selection
It may be useful for me to pick up two of these in more detail: a changed profile of results, and the random selection. Let’s take the profile of results first.
We have been clear all along that it is perfectly possible for a centre’s profile of results legitimately to change over time, and specifically for it to look different in 2021 to how it looked in 2017-19. A legitimate change in results can happen, for example, because of a significant change in entry profile, turnaround in teaching quality, significant changes in curriculum structure.
Where, during the quality assurance exercise, this is found to be the case, results will of course be accepted. However, it is also the case that a markedly different profile of results in 2021 where the pattern up to 2019 was relatively stable may be a sign that 2021’s results are inconsistent with those at other centres and therefore potentially unfair. This is why they may need to be looked at in more depth.
Exam boards will make comparisons between every centre’s results profile in 2021 with their profiles in 2017-19 by looking at cumulative grades, in other words percentages of grades above the specific grades across the range.
Of course, the advice to heads of centre on this is to have done this sort of analysis in any case before results are submitted. It is covered in the JCQ guidance already published:
“centres are advised to consider the profile of their results in previous years in which exams have taken place … Centres can use this to undertake a high-level check once grades have been assigned to students, to ensure that they have applied a consistent standard in their assessment of the 2021 cohort compared to previous years in which exams took place.”
Speaking for a moment with a school leader hat on, what we will be doing is putting the provisional 2021 results into our results software, whether it is 4 Matrix, SISRA, SMID or whatever (other brands also available), so we can get a full range of standard comparisons between 2021 and previous years.
These packages usually generate the indicators used in performance tables such as Attainment 8, but that will do the job perfectly well of enabling us to see how 2021 compares with previous examined years. Any areas of substantial misalignment that are flagged in this way will then be able to be further checked or standardised internally before our grades are submitted.
The other point I wanted to comment on was the random selection. From the start, we have again been clear that we will require examination boards to carry out both risk based and randomised quality assurance. So a number of centres will come under scrutiny not because they fall into any of the risk categories but because they have been randomly selected.
This is a critical part of securing confidence in this year’s results, because the risk-based reasons for identifying centres for further checking may affect some types of school or college, some centres, much less than others. Any such imbalance will be corrected by the randomised quality assurance where Boards will be choosing centres from each of the main categories and types of school, for example independent, academy, comprehensive, post 16 only, selective etc.
I hope that paints a full and helpful picture of the quality assurance processes. By putting in place this robust quality assurance system we are able to give confidence to teachers making dispassionate and even-handed evidence based judgements that there are checking mechanisms and incentives in place for all other teachers and schools to do likewise. These measures will also give confidence to students, parents, and the wider public to help identify and challenge any cases where fair judgements may not be being made.
Because these quality assurance processes will be ongoing during the second half of June and into July, and because they may result in some cases in requests or requirements to review grades, the instruction to centres is not to reveal to students the grades that were submitted to Boards, though clearly if students know what evidence has been used they will in many cases have a reasonable idea. But it is important that the final grades are published on results day in the normal way by Boards, not least so as to give this cohort something resembling a normal experience of a results day.
As in every year, there will inevitably be some students who are dissatisfied with their grades when they open that envelope in the week of 9 August. It is, of course, absolutely right that in such cases students should have some recourse.
We listened to respondents to the consultation on this who said overwhelmingly that they wanted the final arbiter on appeals to be outside the school responsible for awarding the grade with which the candidate is dissatisfied. Once a basic accuracy check has been carried out within the school or college to identify any mistakes in assigning the grade, centres will be required, to support a student’s appeal, to submit the full evidence pack used to determine the grade to the Board for review. This is another reason why it is essential that that all evidence is in good order and ready to send in at short notice.
Unlike in a normal examined year, there will be no marks or grade boundaries available, as grades will be single holistic judgements. In considering whether to appeal, students will not know for example how ‘strong’ a grade B or grade 6 their result is, in other words whether the evidence ‘only just justified a grade 6, or easily justified a 6 and was getting close to a 7’. Clearly the school may be able to provide some advice on that, but ultimately the decision about whether to proceed with the appeal will sit with the student.
For appeals that do go forward, with their evidence pack, to the exam board for review, the decision will be whether the grade awarded represented a reasonable exercise of academic judgement on a review of the evidence in the round. While it will be possible for a student to challenge if they believe the evidence range was unreasonable, a successful appeal would be unlikely if the centre has followed its approved policy by using a reasonable and consistent set of evidence for the whole cohort, permitted exceptional circumstances notwithstanding.
Once the review of evidence is complete, there can be two outcomes.
Either the centre followed its policy and the original grade represented a reasonable exercise of academic judgement.
Or the original grade represented too lenient or too severe an exercise of academic judgement, on the basis of the evidence, in which case the Board will issue a higher or lower grade which will replace the original grade.
While we have built in more time to deal with appeals this year by bringing forward results days, it is likely that appeals will take longer to process than in normal years. In normal years, it is ‘simply’ a matter of reviewing the marking of an exam script which is already well known to markers and which the board already has on file. This year, there may be several pieces of evidence to review, the evidence will look different from one centre to the next, will need to be submitted to the board when the appeal is made, so the whole process will take longer. We clearly cannot predict how many appeals there will be but boards are modelling for various scenarios.
I mentioned at the start the remarkable contribution that trusts have made over the past year. Not only have they stepped forward to support the national effort in many remarkable ways, like for example founding the Oak National Academy, but also in innumerable smaller ways provided assistance, advice and solidarity for their own schools.
In the exceptional exercise we have been considering this afternoon for awarding qualifications to young people in the absence of examinations, trusts also have a very significant contribution to make, especially those which include a number of secondary schools. Wherever practical, agreeing approaches across larger groups of schools within trusts and providing consistent support for those responsible for this process from the trust will add a greater level of stability to grading, give teachers at the front end of this greater assurance, and take a leadership role in the modelling of professional expertise for which trusts are and should be known.
To all of you here this afternoon in leadership positions in trusts, thank you in advance for what you will no doubt be doing to support your schools to do the right thing this summer.
I should say at this point that we are most grateful to CST and your wonderful central team and members for your preparedness to engage with us both critically and constructively as we have worked up the approaches for 2021. It has been immensely valuable.
Finally, let me offer a few brief thoughts about the future. We have all learnt a lot about our systems as a result of the pressure the pandemic has brought. We certainly need to do some serious thinking about the resilience of our national qualifications, and delegates will want to be assured that this work is in train. Very often the innovations developed at speed during a crisis enable us to do things we were not able to do before the crisis.
While the Covid pandemic is a crisis of generational proportions, it is not the first to affect the awarding of qualifications. We will want to think hard about how we bring together the experience of the pandemic and the wider, fast developing evidence we now have in areas such as curriculum design, evidence-based pedagogy and assessment, our growing knowledge of the capabilities of technology, and how all these interact, to ensure we have the best possible system in the future.
So on behalf of the cohort of young people who will be gaining qualifications this summer, a debt of gratitude is owed for all you will be doing, in the challenging circumstances they and we have faced, to ensure that awards are professional and fair. And thank you also for your attention today.