Research and analysis

​​A review of the assessment of reformed Functional Skills qualifications in English and maths​

Published 29 January 2024

Applies to England

Summary

Functional Skills qualifications in English and maths have recently been through significant reform. The Department for Education (DfE) put in place new subject content for the reformed qualifications, which were introduced from 2019. Alongside this, Ofqual introduced specific new regulations and has now conducted an evaluation of the overall approach to assessment in the reformed qualifications.

As part of this evaluation, Ofqual has engaged widely with the sector, including awarding organisations and other stakeholders, such as teachers and students. Ofqual would like to thank everyone who took the time to contribute to this work. It has been important and helpful to understand how these qualifications are working for their users.

The evaluation has explored a range of important issues regarding the assessment of the reformed qualifications. The areas noted have been weighed carefully, and although the evaluation does not indicate the need for a change to the overall approach to assessment, it has identified aspects that require action from awarding organisations, as well as aspects that would benefit from further exploration. 

Overall, the level of demand in the reformed assessments appears to be appropriate. At the time of the reform, a comparative judgement exercise involving a number of Functional Skills practitioners found that the expected difficulty of the legacy and reformed Level 1 and 2 maths assessments was very similar. Further to this, a review of Level 1 and 2 maths papers once in delivery, which was conducted during the evaluation, found the level of demand of the reformed qualifications to be broadly appropriate against the subject content set by DfE. 

Stakeholders have, however, reported that they perceive the reformed qualifications to be more difficult than the legacy qualifications, particularly in maths. Ofqual has identified a range of potential contributing factors that may explain this. These include: 

• changes made by DfE to the subject content during the reform, including moving some content down a level (for example, from Level 2 to Level 1)
• the introduction of non-calculator assessment and the assessment of underpinning skills
• the time for providers to become familiar with the changes to the qualifications, particularly in light of the disruption arising from the COVID-19 pandemic
• changes in the cohort taking the qualifications

In addition to these general factors, Ofqual has also identified a potential issue specific to Level 1 and 2 maths. This relates to the assessment of problem solving. When reviewing papers as part of the evaluation, Ofqual found that awarding organisations’ approaches to problem solving questions may have contributed to an additional reading load. It may also have led to more questions being based around a context than necessary. Both of these may have contributed to stakeholder feedback that some students found it difficult to understand the questions. Ofqual has therefore initiated research to consider effective practice in the assessment of problem solving questions in FSQ maths. Where the evaluation has identified issues with awarding organisations’ approaches, Ofqual will require that these are addressed. 

Introduction

Ofqual has conducted an evaluation of the overall approach to assessment in the reformed Functional Skills qualifications (FSQs) in English and maths. 

The evaluation comprised 3 core strands: 

• stakeholder engagement
• market and data analysis
• a review of Ofqual’s regulatory approach

The evaluation has also been informed by Ofqual’s monitoring of the qualifications in delivery, including a review of FSQ Level 1 and 2 maths papers by subject matter experts in maths and English. 

Ofqual has worked closely with DfE throughout the course of the evaluation, as it conducted its own evaluation focused on its areas of responsibility, including the new subject content, and teaching and learning. Where stakeholders have referenced these areas, Ofqual has passed the relevant information on to DfE.

This report sets out:

• the background to the reformed qualifications
• a summary of the main potential areas for improvement identified in the stakeholder engagement
• Ofqual’s observations on the level of difficulty in the reformed qualifications, as well as the key aims of the reforms
• Ofqual’s planned next steps

Background

The reform

In 2015, Ministers decided to reform FSQs in English and maths to ensure they equipped students with the skills and knowledge to prepare them for work and life, and to improve the qualifications’ recognition and credibility with employers. In 2017, the then Minister of State for Apprenticeships and Skills wrote to Ofqual’s Chief Regulator, providing the Government’s steers for the reformed qualifications. Three key aims set out in the steer letter were:

• ensuring that provision remains flexible
• maximising reliability of assessment against the new subject content
• improving the comparability of grading standards over time and between awarding organisations

In February 2018, DfE published its new FSQ English and maths subject content. Ofqual then published its conditions, requirements and guidance (which adopted DfE’s subject content) in June, building on the steer given by the minister. In December 2018, DfE instructed Ofqual that there was to be an increase in the guided learning hours (GLH) for the reformed FSQs in English and maths from 45 to 55. This was consulted on and included in Ofqual’s regulations.

Awarding organisations developed qualifications in line with Ofqual’s regulations, which were then required to go through a technical evaluation process before being made available to schools, colleges and training providers. This process entailed reviewing and providing feedback on key qualification documents, such as specifications and sample assessment materials. The first qualifications were made available in September 2019.

The market

The cohort

FSQs in English and maths are taken by a diverse cohort of students for a wide range of purposes. They are currently taken by apprentices (both 16 to 18 year-olds and adults), students who have not obtained a grade 3 or above at GCSE, and adults seeking to improve their skills for the workplace, amongst others. Within these groups, there is significant diversity – for example, these qualifications can play an important role for students in prison, as well as those with special educational needs and disabilities. There is also diversity of characteristics such as ethnicity and socioeconomic background.

The age profile of FSQ students is also varied. The table below shows the age breakdown of students who were issued a FSQ certificate between July 2022 and June 2023.

Table 1: Age breakdown of students who were issued a Functional Skills Qualification certificate between July 2022 and June 2023, by subject and level

	English		Maths
	Level 1	Level 2	Level 1	Level 2
16 to 18	35%	13%	33%	11%
19 and older	65%	87%	67%	89%
All	100%	100%	100%	100%

There has been a long-term decrease in certification volumes for FSQs in English and maths, pre-dating the FSQ reform. Factors that have contributed to this reduced take-up include changes to the qualifications that appear on school performance tables, as well as to DfE’s condition of funding for students on study programmes. Since August 2015, students who obtain a GCSE grade 3 or D have been required to retake GCSE, whereas previously they may have taken an FSQ instead. The COVID-19 pandemic also caused significant disruption to the volume of certificates, with restrictions in place impacting the ability of students to take assessments from March 2020 to July 2021. These factors are also likely to have impacted on the type of students entering FSQs. 

The following graph shows Functional Skills certifications in English and maths over time, across all levels.

Graph 1: Functional Skills Certificates in English and maths, Q1 2012- Q2 2023

Awarding organisations

The number of awarding organisations offering FSQs in English and maths has also changed. In June 2018, prior to the introduction of the reformed qualifications, there were 15 awarding organisations offering FSQs, of which 6 included Entry Level in their offer. Post-reform, in 2021 to 2022, there were 9 awarding organisations offering FSQs in English and maths, with 5 including Entry Level in their offer.

The qualifications and Ofqual’s regulation

For the reformed qualifications, Ofqual put in place subject-level conditions, requirements and guidance. This included rules to support:

• consistent approaches to assessment design across awarding organisations, including subject-specific requirements and the adoption of DfE’s subject content into Ofqual’s regulations
• greater alignment of standards between awarding organisations
• Ofqual’s evaluation of the qualifications before they entered the market

Many features of the reformed qualifications were retained from the legacy versions. For example, maths qualifications still comprise a single component and English qualifications still comprise 3 components (Reading; Writing; and Speaking, Listening and Communicating (SLC)). There were several key changes to the qualifications which mainly centred around increasing or introducing the assessment of underpinning skills, which was a curriculum intention set out in the DfE subject content. In maths, Ofqual introduced weightings for the assessment of underpinning skills (25%) and problem solving (75%), as well as discrete non-calculator assessment (25%) either as a separate paper or a discrete section within one paper that covers both non-calculator and calculator assessment. In English, Ofqual no longer permitted access to spelling and grammar checks in the Writing assessment. Students were also required to take all 3 components at the same level as the overall award, in order to support standard setting.

As with the legacy qualifications, Ofqual chose not to put in place rules restricting the availability and delivery mode of the reformed FSQ English and maths assessments. This has enabled awarding organisations to offer flexible assessment of the qualifications that meets the needs of their users. All but one of the 9 awarding organisations offering FSQs in English and maths offer assessments on-demand and in both onscreen and paper-based formats, as well as offering remote invigilation.

Stakeholder perceptions

During the evaluation, Ofqual conducted a wide range of stakeholder engagement, including surveys to enable students, and staff in colleges, training providers and other centres offering FSQs (thereafter described as providers), to share their views about the reformed qualifications. Ofqual also conducted focus groups with students, employers and representatives from awarding organisations.

It is important to note that although the views of the survey respondents and focus group attendees provide us with an insight into the way that some providers and students view and experience these qualifications, it is not possible to know how representative they are of FSQ provider and student views overall.

Key themes - areas for potential improvement

In the survey, many providers noted the benefits of FSQs in supporting progression to employment or further education, and equipping students with necessary skills and knowledge. This section, however, focuses on the main areas for potential improvement noted by stakeholders. These are the key areas that Ofqual is considering as part of its next steps from the evaluation, as well as in wider work across the organisation. 

Ease of understanding the questions 

Throughout Ofqual’s stakeholder engagement, including when asked about perceptions of fairness, providers raised concerns that students can sometimes find it difficult to understand the questions in the reformed FSQ English and maths assessments. Providers raised concerns about unfamiliarity of the contexts used in some instances, as well as the length or complexity of wording of the questions. When asked about their experience of the assessments, students were more positive. This divergence in views may be because students were asked only about their most recent FSQ assessment, whereas providers were asked for their general views on the assessments. 

FSQ English for English as an Additional Language (EAL) students 

Despite FSQ English not being designed to facilitate language acquisition for EAL students, the most common change cited by providers in Ofqual’s survey when asked about changes to the type of student taking FSQs in English since September 2019 was an increase in EAL students. Concerns were raised by providers about the accessibility of the language and contexts used in FSQ English assessments for these students. 

Perceptions of fairness

Ofqual asked providers to what extent they agreed or disagreed that the FSQ assessments were fair for students in general, and in relation to specific groups. There were some negative perceptions around the fairness of the assessments among providers, particularly in relation to maths assessments, as well as for specific groups of students such as those with special educational needs or disabilities, and those from disadvantaged backgrounds. Students, however, had more positive perceptions of the fairness of the assessments, with most reporting that they felt they were assessed fairly in their most recent FSQ English or maths assessment. 

It is important to note when interpreting this feedback that the concept of fairness is complex and the manner in which it is likely to have been interpreted by different respondents will vary.

Perceptions of difficulty in maths

Most providers who responded to Ofqual’s survey reported that fewer of their students taking Level 1 and Level 2 FSQ maths qualifications were passing now, than before September 2019. When asked why they thought this was, roughly a third of respondents reported that they thought it was because the qualification is now more difficult for their students to pass. 

In the context of FSQs, comparing pass rates over time is challenging. This is because the make-up of the cohort of students taking the qualifications typically varies over time. For example, as outlined above, the size of the FSQ cohort has changed considerably in recent years, and it is likely that the type of students entering the qualifications has also changed. Any variations in outcomes will therefore reflect changes in the cohort. This theme, and Ofqual’s follow-up work, are explored further in the next section.

The Speaking, Listening and Communicating (SLC) component in English

The SLC component is assessed by teachers at Level 1 and Level 2, whereas Reading and Writing are externally assessed by the awarding organisation. More than half of provider respondents reported that the guidance provided by their awarding organisation in relation to the SLC assessments was useful, compared to just over a sixth who thought that it was not. A similar proportion also reported finding the assessment criteria for the SLC component easy to understand and apply, compared with just under a quarter who thought that they were not. 

That said, the most commonly cited concern in relation to the reformed FSQs in English was perceived issues with the SLC component. Examples of issues raised were that the assessments were overcomplicated, that the marking criteria were unclear and that the assessments placed a burden on teachers. Providers also noted that the requirement to speak in front of others caused anxiety for some students. 

Onscreen vs paper-based

Ofqual heard feedback that although most awarding organisations provide both onscreen and paper-based assessments to give students flexibility, some students feel as though they are not being entered for the format that is most appropriate for them.

Ofqual also heard feedback from students that they are being taught using paper-based formats but are then expected to take the assessment onscreen. Students reported being disadvantaged by this as they have had to spend time working out how to use the software, as opposed to completing the assessment.

These issues were reported as having a particular impact in maths assessments as certain techniques (for example for drawing graphs) are substantially different in the 2 formats.

Application of Reasonable Adjustments

Providers and students reported a lack of understanding around the availability and application of Reasonable Adjustments. These are changes made to an assessment or to the way it is conducted that reduce or remove a disadvantage caused by a student’s disability, and are needed because some disabilities can make it harder for students to show what they know and can do in an assessment than if the student had not been disabled. In some cases, providers reported that their students were not able to access appropriate Reasonable Adjustments, with some suggesting that the only option available to them was extra time, which was either not appropriate or insufficient for the student. Providers also reported that issues with obtaining a formal diagnosis of need amongst FSQ students at the start of the qualification were limiting their ability to obtain the necessary adjustments for their students in a timely way. 

Conclusions

This section sets out Ofqual’s observations on the level of difficulty in the reformed FSQ qualifications, recognising the extent of stakeholder feedback in relation to this theme. It also considers the extent to which the key aims for the reform have been realised in the approach to assessment for the reformed qualifications.

Level of difficulty in the reformed qualifications 

As noted earlier in the report, a major area of concern from stakeholders was that the reformed qualifications are more difficult than the legacy qualifications, particularly in maths. Although there were some views in this regard for English, it was not to the same extent. DfE’s overall intention was that the reforms should make the qualifications more trusted by employers, but their level of demand should not be increased. 

A comparative judgement exercise was conducted by Ofqual in 2019 looking at Level 1 and 2 maths qualifications as part of the technical evaluation process. This exercise involved a number of FSQ maths practitioners. It found that the expected difficulty of the legacy and reformed assessments was very similar. More recently, following stakeholder feedback received as part of the evaluation, Ofqual conducted a review of reformed FSQ Level 1 and 2 maths papers. The independent subject matter specialists conducting this review did not find the level of demand to be excessive. They also identified questions across all awarding organisations at Level 2 where the level of demand was considered to be insufficient. Though these pieces of work suggest that there has not been an increase in the level of demand, Ofqual has identified a range of reasons that may help explain stakeholder perceptions of increased difficulty.

Ofqual has identified a potential issue with problem solving questions in Level 1 and 2 maths assessments. Through its review of reformed Level 1 and 2 maths papers, Ofqual found that awarding organisations had potentially misallocated questions as problem solving, when they were instead underpinning skills questions framed by a ‘real life’ scenario. This may explain the feedback Ofqual received that some students found it difficult to understand the questions in Level 1 and 2 maths papers. In the paper review, Ofqual found that using contexts in this way has, in some circumstances, contributed to an additional reading load, due to the inclusion of additional information unrelated to the mathematical construct being tested. This may be the reason for stakeholder feedback that students have experienced issues with the length of the questions, as well as the complexity of the language used. 

In relation to the feedback that students are unfamiliar with some of the contexts used in the qualifications, it is important to recognise that FSQs are intended to give students transferable English and maths skills. This means that neither the skills being assessed nor the contexts should be sector-specific, but are instead intended to be broadly applicable. The paper review found that where questions were context based, the scenarios used were generally sufficiently real-life as to be familiar to students. Ofqual does, however, believe that the misallocated problem solving questions could be leading to more questions than necessary being set in context.

Changes made by DfE to the content during the reform process may also have contributed to stakeholder perceptions of difficulty. This includes moving subject content down a level, for example moving content that was previously in Level 2 qualifications to Level 1 qualifications. The introduction of non-calculator assessment and the assessment of underpinning skills into FSQ maths during the reform may also have made the qualifications seem more challenging to some stakeholders. 

Another contributing factor could be the time for providers to become familiar with the changes to the qualifications, particularly in light of the disruptions arising from the COVID-19 pandemic. Ofqual wrote to awarding organisations when the reformed FSQs were introduced, indicating that standard setting in the early awards should compensate for any dips in student performance caused by lack of familiarity with the new qualifications. This continued during the COVID-19 pandemic. Ofqual notes that awarding organisations are now providing more practice materials, which should help providers better prepare students for their assessments.

Finally, there have been changes over time in the type of students taking the assessments. The change in funding rules in 2015, meaning full time students who achieved a GCSE grade 3 or D are now required to sit GCSE English and/or maths again, instead of taking a Level 2 FSQ, led to a change in the 16 to 19 cohort taking the assessments. Arrangements put in place during the COVID-19 pandemic, such as the use of Centre Assessed Grades in 2020, may have led to further variability in the 16 to 19 FSQ cohort. This means that it is difficult to compare pass rates pre- and post-reform.

Regulating to achieve the key aims of the reform

As set out above, the ministerial steer set out 3 key aims for the reforms. Ofqual has reviewed the extent to which these aims have been realised in the approach to assessment for the reformed qualifications.

Ensuring that provision remains flexible

Ofqual chose to maintain the structure from the legacy qualifications, requiring maths qualifications to be one single component, and English 3 components (Reading, Writing, and Speaking, Listening and Communicating). In English, this provides some flexibility as a student only need resit the unit(s) that they have not passed, as opposed to the whole qualification. Ofqual received limited feedback from stakeholders through the evaluation on the structure of the qualifications, suggesting that there were not significant concerns about retaining this structure in the reformed qualifications. Ofqual did receive feedback from some stakeholders that they would prefer maths to be split into a non-calculator component and calculator component, to give schools and colleges (or teachers) a better understanding of in which of those 2 broad subsets of the qualification students may need more support, should they not pass.

To enable awarding organisations to offer flexible qualifications that meet the needs of their users, Ofqual chose not to put in place rules restricting the availability and delivery mode of the reformed FSQ assessments. Instead, Ofqual asked awarding organisations to consider certain risks around the use of on-demand assessment. Out of the 9 awarding organisations currently in the market, all but one offer FSQ assessments on-demand, in both paper-based and onscreen formats, and with remote invigilation. 

Maximising reliability of assessment against the new subject content

Ofqual put in place several requirements aimed at supporting the reliability of the qualifications. At Levels 1 and 2, the subject level conditions require maths FSQs, as well as the Reading and Writing components of FSQ English, to be assessed entirely by examinations that are set and marked by the awarding organisation. Ofqual requires this approach to maintain a high degree of awarding organisation control over setting and marking of examination papers. 

Ofqual also required all reformed qualifications to go through a technical evaluation process before being made available, to ensure oversight of awarding organisations’ approaches, and to assess any risks to validity and reliability. As part of this process, awarding organisations were required to submit to Ofqual materials including the specification, sample assessments and an assessment strategy (a document in which the awarding organisation justifies its approach to the design and delivery of its qualifications).

Ofqual continues to monitor the qualifications in delivery. The review of maths papers at Level 1 and Level 2 considered coverage of the subject content within the assessments. At Level 1 it was found that some papers repeatedly assessed the same subject content element in a paper using the same question type. This could indicate that an insufficient breadth of content is being assessed by the paper overall. As noted above, the paper review also found issues with the allocation of problem solving questions. This may impact on the reliability of the qualifications against the subject content as it suggests that the problem solving guidance, as set out in the subject content and Ofqual’s guidance, may not be being implemented as intended. Ofqual will require awarding organisations to address these issues to maximise the reliability of the assessments against the subject content.

Improving the comparability of grading standards between awarding organisations and over time 

To improve the comparability of the qualifications, Ofqual introduced new requirements for assessment arrangements, including more specific guidance around standard setting than the arrangements in place for the legacy qualifications. Ofqual also set minimum and maximum assessment times at each level of the qualifications. The requirement for qualifications to go through technical evaluation supported comparability by ensuring that Ofqual had oversight of awarding organisations’ approaches. Ofqual worked with awarding organisations to facilitate alignment in standards, including the development of common pass grade descriptors, to support consistency in grading. 

Concerns were raised by a small number of stakeholders about the perceived differences in the level of difficulty between awarding organisations, as well as between past papers and live assessments. Ofqual is considering this feedback as part of its ongoing monitoring of awarding organisation comparability.

Next steps

This work has explored important issues regarding the assessment of the reformed FSQs. At this stage the evidence does not indicate the need for a change to the overall approach to assessment but the evaluation has identified potential areas for improvement. Ofqual is:

• requiring awarding organisations to address issues identified in the FSQ maths paper review
• planning to conduct further paper reviews including for FSQ English, as part of its monitoring of qualifications in delivery
• conducting research focused on further exploring effective practice in the assessment of problem solving in FSQ maths
• considering stakeholder feedback on the application of Reasonable Adjustments as a part of its wider review of awarding organisations’ application of Reasonable Adjustments in vocational and technical qualifications, with a view to improving their practice
• considering stakeholder feedback on the use of onscreen assessment as a part of its wider work on technology in assessment