Policy paper

2022 Primary Assessments: Lessons Learnt

Published 27 April 2023

1. Introduction

Statutory assessments are an essential part of a child’s education. They give teachers, parents and pupils vital information on pupil attainment and progression.

Key stage 2 (KS2) tests are the most complex of the assessments to administer and run to tight deadlines. Each year, 650,000 pupils in over 16,000 primary schools across England take six different KS2 papers, generating some 3.9 million KS2 test scripts. These scripts are collected from schools and sent to a scanning centre. Once scanned, scripts are externally marked over a 4-week period against rigorous quality procedures. The Standards and Testing Agency (STA) then undertakes a detailed standards maintenance process involving in-depth analysis of test data by specialist psychometricians. At the completion of this process, results are returned to schools in early July.

We know the administration of KS2 tests in schools is taken just as seriously. STA and its supplier Capita have a responsibility to ensure a clear and smooth process surrounds the administration of these assessments. Schools should have the information they require when they need it, and later our external markers should understand the process so they can complete their work to the right quality, and throughout STA and Capita should have the right systems in place to be able to quickly identify and respond to any issues.

In 2022, primary assessments were reinstated following a two-year gap as a result of the coronavirus (COVID-19) pandemic. A new approach was also taken to manage the assessment process, in particular end of KS2 tests, including a transition to online marker training and a change in the STA’s main delivery contractor to Capita.

Many aspects of assessment delivery went well. The reception baseline assessment (delivered through NFER), end of key stage (KS1) assessments, the phonics screening check and the multiplication tables check delivered directly by STA were all administered with few issues. KS2 test papers were distributed to schools without incident and most schools were able to administer the KS2 tests smoothly and received their results on Tuesday 5 July as scheduled.

Our processes were observed by Ofqual, who noted that marker training was delivered successfully. Ofqual’s analysis also found that the quality of the marking of KS2 assessments was also high across subjects and in line with previous years. We were also able to undertake our usual standards maintenance procedures. This meant that results for the KS2 tests in 2022 could be linked robustly to the standards as they were originally set in 2016. In its report on 2022 assessments, Ofqual stated “in the context of the challenges to the delivery of national assessment marking in 2022, it is noteworthy that there is no evidence that marking quality was impacted by the change in supplier or cessation of testing in 2020 and 2021”.

There were, however, other aspects that did not go as well, mostly in relation to end of KS2 tests and their associated processes. We know that those delivery issues caused significant frustration and inconvenience to the schools, school trusts, local authority staff, parents and pupils who were affected.

STA and Capita reiterate our sincere apologies to those schools who experienced difficulties in 2022. We do not want a repeat of these issues in 2023, and it was critical that we understood in full the issues that arose and the lessons we could learn from this experience. This report outlines the process by which we undertook that exercise, the key areas of improvement we have identified, and the approach taken by STA and Capita to address these for the 2023 test cycle and beyond.

2. Reviewing the 2022 key stage 2 assessment cycle

To gather a full picture of how the test cycle could be improved, we undertook a thorough lessons learnt exercise over the summer and autumn of 2022, including spending time listening to schools, local authorities and school trusts to get feedback and input on what mattered most to them.

This work has involved:

• structured internal ‘lessons learnt’ activity with different teams, involving colleagues at all levels within STA and Capita
• a thorough review of all stages of the full assessment process, not just where schools encountered issues, as well as a full analysis of the data, to ensure the whole process can operate efficiently
• a short ‘pulse’ survey to provide an opportunity for all schools, local authorities and school trusts to provide feedback on their experiences, which received 749 responses
• a series of lessons learnt workshops with representatives from schools, local authorities, school trusts and trade unions, where STA and Capita colleagues had the opportunity to hear directly from those impacted about the difficulties they faced and how they believed we could improve for the future

We are very grateful to all those stakeholders who participated in these processes for their time and expertise, which has been invaluable in ensuring that we pick up on the right issues and make the right changes for 2023 and beyond.

3. The key lessons learnt

The lessons learnt activity provided a range of useful insight into the issues schools and others experienced and the ways in which we could best address these.

A range of lessons have been taken forward, with the following three areas for improvement highlighted as critical to ensuring the 2023 test cycle runs more smoothly:

3.1 Improve the quality of, and access to, the national curriculum assessments helpline

Issues identified: During the 2022 test cycle some schools found it difficult to get through to an agent on the national curriculum assessments helpline in a timely manner. In test week, the average call response time was 53 minutes. Emails to the helpline were not answered quickly enough in many cases. When schools were able to reach agents, the information provided was not always of the right quality.

Actions taken: We know how important it is to be able to resolve queries quickly and accurately, especially in end of KS2 test week, so we are addressing the issues experienced by schools in 2022 in three ways.

First, we have recruited additional helpline staff to better provision for turnover and are providing full training on the system they will use. This training has also been reviewed and updated to ensure agents have the right information when they speak to schools.

Second, we have taken steps to improve the quality and speed of responses across our helpline. This has included bolstering the information available to helpline agents and using the insight from the lessons learnt process to ensure we can deal with the most common queries as efficiently as possible. This is an ongoing process, which will be reviewed throughout the test cycle to ensure our agents can provide consistent, high-quality updates. We have also improved the information and guidance provided to schools around the tests programme. This means there is less ambiguity for schools around the interpretation of the guidance, which should lead to fewer queries being raised via the helpline.

Third, we have put in place a new database system to help us to identify any problem areas quickly, as well as to highlight if there are multiple calls from one school, so we can address these in a timely and coordinated manner.

This package of measures will support our work to ensure schools have the information they need when they need it.

3.2 Improve system capacity

Issues identified: On return of results day, Tuesday 5 July 2022, many schools had issues logging in to the Primary Assessment Gateway (PAG) to download their results when the system opened that morning. While schools were able to access results by later the same day, we can understand the frustration this caused for the schools affected. Primarily, this issue was due to system capacity being exceeded.

Some schools and local authorities also had issues uploading their teacher assessment data earlier in the cycle, and this caused further inconvenience and frustration.

Actions taken: For 2023 we have made changes to the PAG, and are running a robust programme of performance tests, which cover multiple scenarios. This will ensure the system will cope with the large spike of users at the point the results became available for download in July.

During our investigations we also identified the systems issue with teacher assessment data uploads and have resolved it. We are confident this issue will not recur in 2023.

3.3 Take action to reduce late or missing end of KS2 results

Issues identified: While most pupils received their results on time, some 7,437 pupils, just over 1% the year 6 cohort, did not have an outcome for one or more subjects on return of KS2 results day.

In some cases, the scripts themselves were not missing but had not been marked in time. In these instances, pupils received their outstanding results by the start of the summer holidays or shortly thereafter. In other cases, the pupil had not taken the test and the coding on the system was inaccurate and the situation was resolved when this was updated.

Around 4,000 scripts were however unaccounted for on return of results day. STA and Capita worked together to investigate the situation and locate missing scripts, and by September we had identified, marked, and returned results for around 2,000 such scripts. Ultimately, 2,251 scripts were however deemed lost, affecting 1,900 pupils in total, significantly more than in previous years. We were able to provide a subject outcome for 516 cases because there were sufficient marks in the remaining scripts to assess that the pupil had achieved the standard.

We know how hard pupils work to prepare for their KS2 tests and any loss of test scripts is upsetting for them, their parents and their school. Reducing the incidence of missing and lost test scripts is a key priority for us this year.

Action taken: Our investigation into this set of issues showed that the primary cause for delayed or missing scripts could be tracked to the initial stages of the scanning procedure and how papers were processed on initial receipt at the scanning centre.

To address this, we have put in place clear reconciliation points at key stages of the process to track scripts throughout the process. This means that, at each stage of the process, we can quickly identify any issues, such as missing or unclear scripts, and raise these with the school in good time.

We have also developed clearer guidance for scanning staff on how to prioritise and manage the workflow of scripts. In addition, we have provided further training on key areas that we know could cause issues in the workflow, for instance, when scripts cannot be automatically matched to a pupil, usually caused by written details not being easy to read. In 2022, errors in the reconciliation process meant some pupils (372 in total) were matched to the wrong script and initially received the wrong mark. These errors were corrected and with the correct training in place, we do not anticipate a recurrence of this issue in 2023.

We also know that some stakeholders raised concerns that modified scripts may have been more likely to be missing and ultimately lost. Our investigations did not find this to be the case. While around 1,000 scripts on coloured paper were delayed during the scanning process, these scripts were not missing, and the results were returned to schools shortly after the due date. There is no evidence beyond this that modified scripts were more likely to be lost than other scripts.

In a small number of cases, when modified scripts were returned alongside blank, non-modified papers, the blank scripts were scanned in error. Where we identified this issue, the modified scripts were located and marked, and the results returned to pupils, albeit much later than should have been the case. We have updated our training for our scanning staff and have also improved our guidance to schools on how to return modified scripts to help reduce the recurrence of this issue.

As well as addressing the workflow processes and improving the training for our scanning staff, we have also put a new scanning system in place. This will help us embed the updated processes and will incorporate a new data reporting system, which will help to identify any issues quickly, so that they can be tracked and resolved faster.

4. Conclusion

This report has outlined the key changes we have made as a result of the lessons learnt process in 2022. As we enter the 2023 test cycle we will be regularly monitoring and reviewing each stage of the process to ensure it is working as required. We are committed to responding to any issues that arise and to keep our stakeholders updated in terms of activity and progress.

We are grateful to school, trust and local authority staff across England for their work to support primary assessments. STA and Capita are committed to delivering an improved service throughout the 2023 cycle and are confident the changes we have put in place will achieve this.