Assessment strategies (Condition PTQ3)
Rules and guidance about assessment strategies for Performance Table Qualifications
- PTQ3.1 In respect of each Performance Table Qualification which it makes available, or proposes to make available, an awarding organisation must –
- (a) establish and maintain an assessment strategy for that qualification,
- (b) ensure that the assessment strategy complies with any requirements which may be published by Ofqual and revised from time to time, and
- (c) have regard to any guidance in relation to assessment strategies which may be published by Ofqual and revised from time to time.
- PTQ3.2 In particular, an awarding organisation must ensure that the assessment strategy for a Performance Table Qualification sets out how the awarding organisation intends to secure, on an ongoing basis, compliance with its Conditions of Recognition in respect of the assessments for that qualification.
- PTQ3.3 An awarding organisation must ensure that all assessments for a Performance Table Qualification which it makes available, or proposes to make available, are designed, set, delivered and marked in compliance with its assessment strategy for that qualification.
- PTQ3.4 An awarding organisation must –
- (a) keep under review its assessment strategy for a Performance Table Qualification, and revise it where necessary, so as to satisfy itself that the assessment strategy meets at all times the requirements of Conditions PTQ3.1 and PTQ3.2,
- (b) review its assessment strategy for a Performance Table Qualification promptly upon receiving a request from Ofqual to do so, and subsequently ensure that its assessment strategy complies with any requirements that Ofqual has communicated to it in writing, and
- (c) promptly notify Ofqual of any revisions made by it to any assessment strategy.
- PTQ3.5 An awarding organisation must –
- (a) upon receiving a request from Ofqual to do so, demonstrate to Ofqual’s satisfaction that it has complied with its assessment strategy for a Performance Table Qualification in respect of any particular assessment for that qualification, or provide an explanation to Ofqual as to why it has not so complied, and
- (b) give effect to any recommendation that Ofqual may make in respect of its compliance with its assessment strategy.
Condition PTQ3.1(a) requires an awarding organisation to establish and maintain an assessment strategy for each Technical Award which it makes available or proposes to make available. Condition PTQ3.2 requires an awarding organisation to ensure that the assessment strategy for a Technical Award sets out how the awarding organisation intends to secure, on an ongoing basis, compliance with its Conditions of Recognition in respect of the assessments for that qualification.
Condition PTQ3.1(b) requires an awarding organisation to have regard to any requirements in relation to assessment strategies published by Ofqual.
We set out our requirements for the purposes of Condition PTQ3.1(b) below
General Requirements
There are three key properties of effective assessment strategies:
- A coherent and logical narrative
- The use of argument and justification with evidence
- A focus on the fitness for purpose of the qualification
1. A coherent and logical narrative
An assessment strategy should present a coherent and logical narrative demonstrating how an awarding organisation will seek to ensure that the qualification is fit for purpose. The detailed requirements section below sets out the aspects of the qualification that need to be covered within the assessment strategy. However, the assessment strategy should not be presented as a set of discrete answers to the questions and requirements set out below, as this approach risks lacking coherence. Rather, it must demonstrate consistency between different justifications and explanations, by –
- linking back to previously referred aspects (e.g. the content domain and assessment model should be demonstrably consistent with the purpose of the qualification), and
- demonstrating clear consideration and understanding of the consequences of decisions (e.g. of the implications of the assessment model used for the approach to setting and maintaining standards).
Many decisions entail trade-offs (e.g. between Validity and marking Reliability, for example when using selected-response item types in some written assessments). These should be addressed as appropriate, so that the thinking behind the key decisions that the awarding organisation has made in developing the qualification is clear.
2. The use of argument and justification with evidence
In an assessment strategy, all statements that may have implications for the fitness for purpose of the qualification should be underpinned with clear and concise argument and evidence. Such statements should not constitute only assertion. For example, the following statement provides no evidence or justification to support the assertion that multiple choice questions align best with the content being assessed in the qualification –
The two external assessments will comprise only multiple choice questions. This item type aligns best with the type of content we are assessing in this qualification.
The reasoning and evidence behind such assertions must be provided. An awarding organisation should focus on the inclusion of material that supports its arguments for the fitness for purpose of the qualification.
For example, an awarding organisation should not include descriptions of processes or include/refer to pre-existing process documentation if the relevance of this content to the Validity and fitness for purpose of the assessments has not specifically been drawn out.
3. A focus on the fitness for purpose of the qualification
The purpose of an assessment strategy is to demonstrate that the choices that have been made in the development and ongoing monitoring of the qualification ensure that the qualification meets the Conditions and is thus valid and otherwise fit for purpose.
In particular, where an awarding organisation seeks an exemption under Condition PTQ9.2 or Condition PTQ10.2 from any requirement in relation to assessment or specified levels of attainment it must highlight this intention and ensure that its alternative approach is fully justified within the assessment strategy, including consideration of the consequential risks.
Detailed requirements
We set out below our detailed requirements on the specific information and evidence an awarding organisation must include in its assessment strategy. The requirements are set out in eight sections, as shown below. These sections are intended to represent the key stages in the lifecycle of a qualification, and an awarding organisation’s assessment strategy must be structured so as to reflect them.
- Qualification origination
- Content domain
- Assessment design
- Assessment delivery
- Marking and standardisation
- Attainment and reporting
- Standard setting and maintenance
- Qualification monitoring
For each of the eight sections, indicative content is provided within a table that includes subheadings, corresponding questions, an explanation of the importance of those questions, and the Conditions to which they relate. In many cases, the column explaining the importance of information will also make clear where links should be made to other aspects of the qualification to aid the coherence of the document. For example, the content within the Assessment Structure subheading highlights the importance of demonstrating that the assessment model is consistent with both the purposes of the assessments and its content domain. Note, however, that the links highlighted in the table are not exhaustive and should be regarded simply as indicative of the types of dependencies between different aspects of the qualification that may need to be addressed.
The subheadings and corresponding questions in each section have been specifically written to reflect the key areas of qualification development that should be addressed; and to reflect their approximate sequence as part of the qualification lifecycle. Assessment strategies must address the subheadings and questions contained in them. They may be structured according to the sections in the table, notwithstanding the need for decisions and justifications to be linked back appropriately to previous decisions, and to demonstrate clear consideration and understanding of their consequences (see general requirements under ‘A coherent and logical narrative’ above). Depending on the nature of a particular qualification, there may also be additional areas that require focus. Similarly, some areas covered in the table may require more extended focus, while other areas may not be applicable.
Assessment strategies should refer, as appropriate, to the proposed or existing assessment materials (e.g. test items, tasks and mark schemes) to exemplify any points or arguments made within the document.
Section 1: Qualification origination
This refers to the initial approach to scoping the specification, including clearly defining the purposes of the assessment.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Purposes | What are the specific purposes of the qualification? • Where there are competing purposes (including general purposes specified under Condition PTQ5.1), what trade-offs have been considered, and how have these been resolved? |
The qualification should articulate meaningful and accurate purposes: • as key aspects of the qualification, such as the content domain and assessment structure, must be designed to be consistent with the qualification purposes • so that users of the qualification are not misled. |
Condition PTQ5.1 |
Section 2: Content domain
This refers to the skills, knowledge and understanding to be taught and assessed in the qualification.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Structure | What are the fundamental knowledge and skills inherent in the qualification, and how are they represented? Refer to the key elements of the content domain where applicable, such as: • learning outcomes • topics (and their weightings) • content statements • assessment objectives. |
The content domain should be aligned with the purposes, level and target group(s) of the qualification. A poorly defined domain can lead to: • Learners being insufficiently prepared for assessments • the development of assessments and criteria/mark schemes that fail to appropriately target the domain in a way that is fair and/or transparent • the qualification failing to meet one or more of its general or specific purposes. |
Condition PTQ8.1 |
| What is the size of the qualification? | The Total Qualification Time and number of Guided Learning Hours should be justified in relation to the purposes, level and target group(s) of the qualification. | Condition PTQ7 | |
| Demand | What steps were taken to ensure that the demand of the qualification content is set appropriately? | The demand of the qualification content should be considered in relation to the target group(s) and to other qualifications at the same level. | Condition PTQ8.1 |
| Optional content | Do Learners have to study all of the content? | The presence of optional routes may have implications for content coverage (where one or more routes may omit key elements of the content domain). |
Condition PTQ8.1 Condition G1.2(a) |
Section 3: Assessment design
This refers to all aspects of the structure and development of Assessments by Examination and Non-exam Assessments.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Assessment structure | What is the model of assessment used? • How many assessments are there in total, and what are the weightings between them? • What is the balance of assessment by Assessment by Examination and Non-exam Assessment? • What is the overall assessment time, and what is the rationale for this? • What is the approach to requiring Learners to draw together knowledge, skills and understanding appropriately from across the course of study? |
The key structural aspects of the assessment should be: • consistent with the purposes of the qualification • appropriate in relation to the content domain • conducive to the development of robust standard setting and standards maintenance processes. |
Condition D1 Condition E4.2 Condition G1 Condition PTQ5.1 Condition PTQ9.1 |
| Assessment development | What is the process for developing assessment materials? • What are the different stages involved? • Who is involved at each stage? • What is the approach to training individuals who will be responsible for setting assessments, including ensuring security and mitigating any conflicts of interest? |
The process of assessment development should facilitate the development of valid and fit for purpose assessments. |
Condition A4 Condition D1 Condition E4.2 Condition G1 Condition G3 Condition G4 Condition G9.1 – G9.2 |
| Item and task Validity | What steps are taken to ensure that assessment tasks elicit the assessment evidence they ought to elicit? For example: • that each task triggers its intended learning outcomes and no unintended learning outcomes • that Learners understand what each task is asking them to do • that bias and other construct irrelevant factors are minimised • that Learners have sufficient time to complete tasks. |
Poorly designed assessments could impact on their ability to elicit reliable information regarding the proficiency of Learners. |
Condition D1 Condition E4.2 Condition G1 Condition G3 |
| Sampling | Where applicable, how is content sampled from the content domain? | The sampling methodology should demonstrate adequate coverage of the domain both over time and in relation to individual assessments. |
Condition D1 Condition E4.2 Condition G1 Condition G9.3 |
| Assessment by Examination | What parts of the content domain are assessed through Assessment by Examination, and why? • What item/task types are used? • How has the balance between different item/task types been determined? • How are mark schemes designed, and what steps are taken to ensure they support reliable application? • What are the conditions in which assessments must be taken? • Why is the assessment appropriate to be taken simultaneously? |
Assessments by Examination should be justified in relation to their: • alignment with the purposes of the qualification • effectiveness in assessing the knowledge and skills laid out in the content domain • conduciveness to achieving reliable outcomes for Learners. |
Condition D1 Condition E4.2 Condition G1 Condition G9.3 Condition PTQ5.1 Condition PTQ9.1 |
| Non-exam Assessment | What parts of the content domain are assessed through Non-exam Assessment, and why? What type(s) of Non-exam Assessments are used, and why? |
The use of Non-exam Assessment should be justified, with a particular emphasis on: • why this type of assessment is required in order to maximise assessment Validity • what the key threats to Reliability and Validity are, and how these have been mitigated. |
Condition A6 Condition D1 Condition E4.2 Condition G1 Condition G9.3 Condition PTQ9.1 |
| Centre adaptation | What approach is taken to whether Centre adaptation of tasks will be permitted, and in terms of the extent to which Centres are, for example, otherwise required to interpret or expand tasks set by the awarding organisation? What is the rationale for the approach? |
It is important to demonstrate how Comparability is assured in relation to any Centre adaptation (or similar) of assessments set by the awarding organisation. |
Condition D1 Condition E4.2 Condition G1 Condition G9 Condition PTQ9.1 |
| Controls | What controls are specified? • Are assessment tasks to be performed within specified time limits? • Are Learners permitted to ask for support? • Are Learners permitted to access resources? |
It is important that any controls have been fully considered and specified to ensure assessments are Valid, Reliable and appropriate for the target group. |
Condition G8 Condition G9 |
| Generating outcomes (assessment level) | What is the approach to generating scores for individual assessments? Are scores based only on a Learner’s overall attainment within a Component? Or are there any areas that must be demonstrated, irrespective of performance elsewhere? |
It is important to ensure that the approach to generating scores: • discriminates effectively between Learners of different abilities • enables awarding organisations to monitor and control standards. • is in line with the purposes of the qualification and the aspects of the content that it is assessing. |
Condition PTQ5.1 Condition G1.2(a) |
| Optional assessment | What elements of the assessments are optional? | It is important to ensure that standards are maintained when there are optional routes. | Condition G1.2 |
Section 4: Assessment delivery
This refers to the available modes of delivery and the availability of assessments.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Modes of assessment delivery (all assessment) | What modes of assessment delivery are available (e.g. paper-based, computer-based)? | The modes of delivery should be: • appropriate for the target group • manageable for Centres • optimal in terms of eliciting evidence of the skills and knowledge described in the content domain. Prioritisation between these requirements should be detailed. |
Condition D1 Condition E4.2 Condition G1 Condition G9 |
| Assessment availability | What approach is taken to the availability of assessments, including: • number of assessments to be available • type of assessment (e.g. online and/or paper-based) • approach to Learners taking an assessment again. |
An awarding organisation must: • justify its approach in relation to the purpose of the assessment. • detail specific risks that have been identified (such as to the Comparability and predictability of assessments), and how these have been mitigated. |
Condition A6 Condition D1 Condition E4.2 Condition G1 Condition G9 Condition PTQ9.1 |
| Security | What approach is taken to ensuring the secure transmission, distribution and storage (including by Centres) of materials? Are there any requirements for Learners to have access to assessment materials prior to assessments being taken? |
Inadequate approaches to security can undermine Reliability – if, for example, assessment scores do not accurately reflect Learners’ attainment. |
Condition G4 Condition G9 |
| Centre feedback | What approach is taken to ensuring that Centres provide appropriate feedback to Learners and that the frequency of this is also appropriate? | If feedback from Centre Assessors to Learners is provided excessively frequently and/or is excessively directive about what must be included/addressed as part of the assessment, this creates the potential that assessment scores do not accurately reflect Learners’ attainment. |
Condition C1 Condition C2 Condition G9 |
Section 5: Marking and standardisation
This refers to the approach taken to the marking of all assessments.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Marking approach (all assessments) | What approach is taken to the marking of assessments? What steps are taken to identify the risk of any adverse effect which may result from the awarding organisation’s approach to marking assessments? |
An awarding organisation must justify its approach to marking in relation to the qualification purposes and the need for Reliable outcomes. |
Condition PTQ5.1 Condition A6 Condition D1 Condition H1 Condition H2 Condition H5 |
| Monitoring marking (all assessments) | What processes are in place to monitor accuracy and consistency of marking and issuing of results, and to take remedial action where necessary? | An awarding organisation must justify its approach to monitoring in relation to the need for Reliable outcomes. |
Condition H1 Condition H2 Condition H5 Condition H6.1(d) |
| Assessor standardisation (all assessments) | What approach is taken to training and standardising Assessors, including details of standardisation procedures and any wider training? | It is important to clearly articulate expected criteria for the judgements of success in order to ensure the Reliability of the judgements that are made. |
Condition G1.3 Condition H1 Condition H2 Condition H5 |
| Moderation (Non-exam Assessments) | What is the approach to Moderation, and the rationale for this approach? What are the risks with this approach, and how will they be managed? |
An awarding organisation must justify its approach to Moderation in relation to the need for Reliable outcomes. |
Condition A6 Condition C1 Conditions C2.1 and C2.3(j) Condition D1 Condition H2 Condition H5 |
| Centre training and monitoring (Non-exam Assessments) | What approach is taken to the provision of guidance and training to Centres around marking? How are Centres monitored? |
An awarding organisation must justify its approach to training in relation to the need for Reliable outcomes. |
Condition C1 Condition H1 Condition H5 Condition PTQ9.1 |
Section 6: Attainment and Reporting
This refers to the identified levels of attainment and the grading structures used to characterise these levels.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Levels of attainment | What characteristics differentiate learners who have demonstrated different levels of attainment? Are differences between adjacent levels of attainment easy to detect (i.e. the levels are clearly discrete) or hard to detect (i.e. achievement is more continuous)? |
The characteristics identified should follow clearly from relevant elements of the content domain. |
Condition D1 Condition E4.2 Condition G1.3 |
| Grading structure | What grading structure is employed, and what is the justification for it? How do the assessments generate Reliable information in line with the proposed grading? | The grading structure should be justified in relation to the purposes of the assessment, the identified levels of attainment and the assessments proposed. |
Condition D1 Condition E4.2 Condition PTQ5.1 Condition PTQ10.1 |
| Aggregation (qualification level) | What is the justification for the approach to aggregation, for example, the extent to which Learners’ results for the qualification reflect their overall attainment across all assessments, or their attainment in particular assessments? What is the approach to scaling marks from individual assessments for the purpose of aggregation? |
The approach should be justified in relation to the content domain and purposes of the qualification. An appropriate scaling and aggregation method is needed to ensure appropriate discrimination between Learners. |
Condition D1 Condition E4.2 Condition PTQ11.1(b) |
| Reviews | What steps are taken to ensure that concerns over inaccuracy of assessment results are addressed adequately and appropriately? For example: • that they are investigated fairly, thoroughly and in a timely fashion • that justified concerns over inaccuracy are resolved by following an appropriate due process. |
An appropriate process is fundamental to ensuring that appropriate inferences are drawn from results. | Condition I1 |
Section 7: Standard setting and maintenance
This refers to the processes for setting and maintaining standards, and the types of evidence that are used.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Approach | What is the approach to setting and maintaining appropriate grading standards within the qualification? Within an individual Component, are the requirements (such as number of marks) to achieve a given grade set out upfront or determined after the assessments have been taken? For the qualification as a whole, are there processes to adjust expectations at Component level to secure suitable qualification level standards? |
It is important the approach to standard setting and maintaining follows an appropriate technical methodology. | Condition PTQ11 |
| Evidence | What quantitative and qualitative information is used? How is the weighting of evidence determined when multiple sources of evidence are used? |
It is important the approach to standard setting and maintaining: • includes full consideration of the strengths and limitations of the different types of available evidence, and the weight given to different evidence • leads to securing fair outcomes over time and between Learners. |
Condition PTQ11 |
| Modes of assessment | Where there are multiple modes (e.g. paper and computer), how is Comparability maintained? | It is important to ensure that standards are maintained when there are multiple modes of assessment. |
Condition D1 Condition E4.2 |
Section 8: Qualification monitoring
This refers to the processes that are applied on an ongoing basis to monitor the Validity of key aspects of the specification, such as its assessments, susceptibility to malpractice and maladministration and standard maintenance processes.
| Subheading | Key questions | Why this is important | Examples of relevant conditions |
|---|---|---|---|
| Approach | Describe the techniques and measures used to monitor all key aspects of the qualification that may affect its Reliability and Validity. This should include, for example: • reviewing of functioning of previous assessments • security/malpractice issues/incidents • feedback through engagement with users of the qualification |
Failure to consider the on-going Validity of the qualification increases the likelihood of threats manifesting themselves and/or assessment processes ceasing to remain fit for purpose. |
Condition A6 Condition A8.1 Condition D1 Condition D3 Condition E4.2 Condition G1 Condition G4 Condition G9.3 PTQ9.1 |
| Data requirements | What systems and procedures are in place to ensure that any qualification or Component-level data that may be required by Ofqual can be provided within a reasonable time? Data required may include: • aggregate indicators of assessment functioning at either qualification, Component or subcomponent level as well as Learner-level performance in each item of each assessment taken. • details of the Learner and the Centre where the assessment/qualification has been taken. • Learners’ work at specified levels of attainment. |
Ofqual will closely monitor the delivery of these qualifications and the maintenance of standards in these qualifications. In addition, it is important to demonstrate that this data is used where appropriate by awarding organisations at relevant points in qualification development. |
Condition B4 |