Key stage 1 tests: standard setting
Information for schools about the standard setting process for key stage 1 national curriculum tests in 2016.
The Standards and Testing Agency (STA) uses the internationally-accepted practice, ‘the Bookmark process’ to set standards for the 2016 key stage 1 (KS1) tests.
The same process was used to set the standards on the phonics screening check, the English grammar, punctuation and spelling tests and the level 6 tests when they were first introduced.
This process involves convening teacher panels to recommend where the standard should be set.
Standard setting process
For 2016 standard setting, there were 2 panels for each subject, operating independently from each other. They were comprised of experienced teachers, who have had no involvement in setting the questions.
Participating teachers completed a preparation activity to ensure they had fully engaged with the tests, test frameworks and performance descriptor for the expected standard. The first part of the standard setting meeting established a common understanding of the standard and what is required of pupils as indicated in the performance descriptor in the test framework.
Each panel looked at the test questions in order of difficulty (as determined by live test performance of pupils). They decided whether, in their opinion, two-thirds of the children working just at the expected standards would get each question correct.
This is a necessarily technical process – making judgements about performance on each question and the likelihood of a child working at the expected standard getting it right.
Analysis of responses of pupils involved in the early administration of the KS1 tests in April (approximately 26,000 pupils per subject) was used to determine the relative difficulty of the questions, so we can put questions in the correct order.
Information about pupils’ performance in the early administration sample was not used to determine whether pupils at the expected standard will be able to answer the questions. The teachers based their judgements on this on their own professional experience.
Outcomes
STA considered the recommendations of the 2 panels before setting final standards.
STA test development and psychometric experts, as well as the chair of its technical advisory group (a group of international experts that supports STA in technical decisions) attended the meeting. Representatives of the teacher and headteachers unions also observed the confirmation meeting.
We will publish further technical details about the process in test handbooks later in the 2016 to 2017 academic year. The handbooks will provide details of the full test development process.