Standard setting (Condition TQ8)
Rules and guidance on standard setting in Technical Qualifications
- TQ8.1 An awarding organisation must ensure that the specification for each Technical Qualification which it makes available, or proposes to make available, sets out specified levels of attainment which comply with any requirements which may be published by Ofqual and revised from time to time.
- TQ8.2 In respect of each Technical Qualification which it makes available, an awarding organisation must comply with any requirements, and have regard to any guidance, which may be published by Ofqual and revised from time to time in relation to –
- (a) the promotion of consistency between the measurement of Learners’ levels of attainment in that qualification and similar qualifications made available by the awarding organisation and other awarding organisations, and
- (b) the setting of specified levels of attainment.
- TQ8.3 In setting the specified levels of attainment for a Technical Qualification which it makes available, an awarding organisation must have regard to an appropriate range of qualitative and quantitative evidence.
- TQ8.4 In respect of each Technical Qualification which it makes available, the range of evidence to which an awarding organisation has regard for the purposes of Condition TQ8.3 will only be appropriate if it includes evidence of –
- (a) the Level of Demand of the assessments for that qualification,
- (b) the level of attainment demonstrated in those assessments by an appropriately representative sample of Learners taking that qualification,
- (c) employers’ expectations of the knowledge, skills and understanding necessary for Learners to reach the specified levels of attainment,
- (d) the level of attainment demonstrated by Learners taking that qualification in a –
- (i) prior assessment (which was not for that qualification), whether or not that assessment was for a regulated qualification, or
- (ii) prior qualification, whether or not that qualification was a regulated qualification,
- (e) following the first year in which the qualification is awarded, the level of attainment demonstrated by Learners who have previously been awarded the qualification, and
- (f) where a different awarding organisation has previously made the qualification available, the specified levels of attainment set by that awarding organisation for the qualification.
- TQ8.5 An awarding organisation must maintain a record of –
- (a) the evidence to which it has had regard in setting the specified levels of attainment for each Technical Qualification which it makes available, and
- (b) its rationale for the selection of and weight given to that evidence.
Condition TQ8.1 allows us to specify requirements and guidance in relation to the specified levels of attainment that must be used for Technical Qualifications.
Condition TQ8.2(b) allows us to specify requirements and guidance in relation to how those specified levels of attainment are set.
We set out our requirements for the purposes of Conditions TQ8.1 and TQ8.2(b) below.
Specified level of attainment in Technical Qualifications
- In relation to each Technical Qualification, an awarding organisation must ensure that the specified levels of attainment take the form of –
- (a) for the Core Assessments, a six-point scale from A* to E, where A* represents the highest level of attainment, and
- (b) for the assessment of each Occupational Specialism, a three point scale made up of Distinction, Merit and Pass, where Distinction represents the highest level of attainment.
A Learner who does not meet the criteria to be awarded a specified level of attainment on the relevant scale must be issued with a result of ‘unclassified’.
A Learner must be issued a specified level of attainment for the Core Assessments based on an aggregated mark for the Core Examination and the Core Project.
A Learner must be issued a separate specified level of attainment for each Occupational Specialism that he or she completes as part of the Technical Qualification.
Setting the specified level of attainment for Core Assessments
In setting the specified levels of attainment for the Core Assessments an awarding organisation must ensure that these are set for the Core Assessments as a whole.
(a) Key grade boundaries
- The key grade boundaries for the specified levels of attainment for the Core Assessments are –
- (a) A/B, and
- (b) E/U.
- In setting the key grade boundaries for the Core Assessments an awarding organisation must –
- (a) in the first year in which a Technical Qualification is awarded, comply with any specific requirements, and have regard to any guidance, that Ofqual may set, and
- (b) in compliance with Condition H3.1, in the second and subsequent years of awarding, set each key grade boundary so as to ensure consistency with the standard set for the same grade boundary in the first year in which the qualification is awarded.
An awarding organisation must publish grade descriptors for the two key grade boundaries, which in each case must reflect expectations for the Core Examination and Core Project.
(b) Remaining grade boundaries
- An awarding organisation must provisionally set the A*/A, B/C, C/D and D/E grade boundaries arithmetically as follows –
- (a) The grade B/C, C/D and D/E boundary marks are calculated by dividing the mark interval between the A/B and E/U boundaries by four. Where there is a remainder of one or more marks, one mark is added in turn to each of these grade intervals, working from the highest to the lowest.
- (b) The grade A*/A boundary mark is provisionally set the same number of marks above the A/B boundary mark as the B/C boundary mark is below the A/B boundary mark, or halfway between the A/B boundary mark and the maximum mark, whichever is lower.
- (c) Where a review of statistical and technical evidence leads the awarding organisation to judge that the A*/A boundary mark should be set at a different mark, consideration must be given to moving that boundary mark, and the reasons for any such move must be recorded.
Setting the specified level of attainment for Occupational Specialisms
In setting the specified levels of attainment for the assessments for an Occupational Specialism an awarding organisation must ensure that these are set for the assessments for each Occupational Specialism as a whole.
(a) Key grade boundaries
- The key grade boundaries for the specified levels of attainment for an Occupational Specialism are –
- (a) Distinction/Merit, and
- (b) Pass/Unclassified.
- In setting the key grade boundaries an awarding organisation must –
- (a) in the first year in which a Technical Qualification is awarded, comply with any specific requirements that Ofqual may set , and
- (b) in compliance with Condition H3.1, in the second and subsequent years of awarding, set each key grade boundary so as to ensure consistency with the standard set for the same grade boundary in the first year in which the qualification is awarded.
An awarding organisation must publish grade descriptors for the two key grade boundaries which in each case must reflect the relevant performance outcomes.
(b) Remaining grade boundaries
- An awarding organisation must determine the Merit/Pass grade boundary arithmetically as follows –
- (a) The Merit/Pass boundary mark is calculated by dividing the mark interval between the Distinction/Merit and Pass/Unclassified boundary by two.
- (b) Where there is a remainder of one, the extra mark is added to the Distinction to Merit grade interval.
Exemplification materials
- In relation to the assessments for an Occupational Specialism, an awarding organisation must publish for Pass and Distinction –
- (a) indicative exemplification materials before making the Technical Qualification available,
- (b) actual exemplification materials once relevant evidence generated by Learners is available.
- Those exemplification materials must comprise, as relevant, examples of actual or indicative evidence generated by Learners in assessments for the Occupational Specialism (anonymised as necessary) which –
- (a) reflect the minimum requirements for those specified levels of attainment, and
- (b) are produced in consultation with any relevant employers.
An awarding organisation must keep those exemplification materials under review, including in consultation with any relevant employers. It must revise them where necessary to ensure that they provide an accurate reflection of what Learners must demonstrate to obtain each of those specified levels of attainment.
Condition TQ8.2(b) allows us to specify requirements and guidance in relation to the setting of specified levels of attainment for Technical Qualifications.
We set out below our guidance for the purposes of Condition TQ8.2(b).
Evidence to be taken into account in setting specified levels of attainment
Condition TQ8.3 states that in setting the specified levels of attainment for a Technical Qualification which it makes available, an awarding organisation must have regard to an appropriate range of qualitative and quantitative evidence.
Condition TQ8.4 states that such evidence will only be appropriate if it includes evidence of –
(a) the Level of Demand of the assessments for that qualification,
(b) the level of attainment demonstrated in those assessments by an appropriately representative sample of Learners taking that qualification,
(c) employers’ expectations of the knowledge, skills and understanding necessary for Learners to reach the specified levels of attainment,
(d) the level of attainment demonstrated by Learners taking that qualification in a –
(i) prior assessment (which was not for that qualification), whether or not that assessment was for a regulated qualification, or
(ii) prior qualification, whether or not that qualification was a regulated qualification, and
(e) following the first year in which the qualification is awarded, the level of attainment demonstrated by Learners who have previously been awarded the qualification, and
(f) where a different awarding organisation has previously made the qualification available, the specified levels of attainment set by that awarding organisation for the qualification.
Without prejudice to any requirements that Ofqual may set in relation to the weight to be given to evidence in the first awards, examples of the evidence that may be used by an awarding organisation in setting the specified levels of attainment for a Technical Qualification which it makes available may include –
- grade descriptors for key grades in respect of Core Assessments and Occupational Specialism assessments,
- exemplification materials for key grades in respect of Occupational Specialism assessments,
- question papers/tasks and final mark schemes,
- senior Assessor input into decisions, for example comments on how the assessments have worked or are likely to work, and recommendations for the setting of specified levels of attainment,
- technical information about how the assessments, and/or any similar assessments previously and concurrently available, have functioned, for example mark distributions, mean marks, standard deviations and item-level statistics,
- samples of current Learners’ work selected from a range of Centres and assessed/Moderated by Assessors/moderators whose work is known to be reliable,
- details of changes in entry patterns and choices of options,
- archive Learners’ work exemplifying specified levels of attainment in previous assessments for the qualification, together with the relevant question papers/tasks and mark schemes,
- inter-awarding organisation evidence for Technical Qualifications,
- pertinent material deemed to be of equivalent standard from similar qualifications or other relevant qualifications,
- information on Learners’ performance in previous assessments for the qualification, and
- marking guides for assessments where the evidence is of an ephemeral nature.
In determining whether it has sufficient evidence of the level of attainment demonstrated in the assessments for a Technical Qualification by an appropriate percentage of the Learners taking that qualification, an awarding organisation should consider whether the marks on its system reflect –
- all possible routes through the qualification, and
- a representative proportion of Learners’ marks for the qualification.
Setting specified levels of attainment in the first year of awarding
In setting the specified levels of attainment for in the first year in which it awards a Technical Qualification, an awarding organisation may place greater weight on qualitative evidence, although not to the exclusion of relevant quantitative evidence where available.
In subsequent years we will expect qualitative evidence to be supplemented with quantitative evidence from previous assessment series as this becomes available, such as the comparison of mean marks in particular assessments over time to help track variations in Levels of Demand from one year to the next.