Research and analysis

Qualification price index: supplementary analysis for 2020 - Extraordinary Regulatory Framework context

Published 11 February 2021

Applies to England

1. Introduction

1.1 Background and objectives

In 2019 Ofqual published the first qualification price index (QPI) to highlight how qualification prices change over time. This aligns with our statutory objective to secure that qualifications are provided efficiently and the price paid for them represents value for money.

Our qualification price index 2020 contains qualification prices as if examinations and other assessments had gone ahead as normal. This is to allow for continued analysis of qualification prices over time. Here we explore the contextual analysis of the different activities which took place for qualification delivery in 2020, in light of the coronavirus (COVID-19) pandemic.

Our findings are mainly gathered from interviews with a small sample of organisations including awarding organisations, centres and representative bodies. We aim to capture the extent of the range of activity and broad cost implications of the coronavirus (COVID-19) pandemic across these organisations. The level of different activities, and therefore associated costs (both new and saved), varies across organisations.

We will continue to review prices and monitor price transparency within the sector. We intend to continue with our longitudinal price analysis and publish a qualification price index for 2021. Since January 2021 we have required awarding organisations to publish qualification prices in an accessible manner for review by potential purchasers.^{[footnote 1]} We will sample this information and use this expanded dataset to inform our 2021 qualification price index analysis and publication.

1.2 Methodology

To explore the cost implications of the changes to qualification delivery in response to the coronavirus (COVID-19) pandemic we used a focused case study approach. We focused on 2020 delivery from summer onwards, and included delivery of the additional 2020 autumn exam series for general qualifications.

We began by reviewing the regulatory impact assessments from our 2020 summer consultations.^{[footnote 2]} We then used this information to inform interviews with a range of organisations which were completed between November and mid December 2020.

We interviewed:

• 3 exam boards
• 3 awarding organisations delivering vocational and technical qualifications
• 2 sector representative bodies ^{[footnote 3]}
• different types of centres involved in delivering qualifications, including:
  • a school
  • a college
  • an independent training provider

The following 4 areas were the focus of the interviews:

1. New activities and costs for organisations.
2. Saved costs for organisations.
3. Impact across different qualification type.
4. Implications for future cost efficiencies.

We haven’t sought to directly link our findings to the qualification prices in the qualification price index 2020 because we do not have a sufficient level of financial detail. Nor have we included specific financial analysis as we do not have comprehensive or representative financial data. Instead, we have sought to capture the unique factors and activities involved in qualification delivery in 2020 and the associated cost implications for awarding organisations and centres.

2. Key Findings

Below are the ten main observations from our evidence review and interviews.

1. 1. It was necessary for many awarding organisations and centres to undertake materially different activities to deliver qualifications in 2020, compared with previous years. Awarding organisations and centres undertook these new activities across different types of general, vocational and technical qualifications, including across calculated, adapted and delayed assessments. Many of these new activities had significant cost implications.
2. 2. The level of cost associated with these different activities varied across a range of different factors, including type of qualification offer, size of organisation and organisation business model.
3. 3. Awarding organisations and centres encountered costs in the following areas (see section 3.1 ): direct delivery cost (for example developing a system for delivering centre assessment grades), people or staff cost (for example overtime and extended working hours), and opportunity cost (for example paused business improvement projects).
4. 4. Awarding organisations and centres particularly highlighted the impact of people or staff cost. Some activities required a specific level of knowledge and experience combined with delivery within short timescales. Therefore, awarding organisations and centres could not recruit and train new staff to support such activities. Instead they generally reprioritised their business activities for existing staff and staff worked extended hours.
5. 5. Some activities which awarding organisations and centres undertook in previous years did not take place in 2020, and they did not incur full costs for these (see section 3.2 ). Some awarding organisations and centres chose to incur a non-contractual cost by paying examiners and invigilators a proportion of their anticipated cost for 2020, as if examinations and other assessments had gone ahead as normal. Organisations made these payments to support workforce capacity planning for future exam series. Some awarding organisations furloughed a proportion of staff and did not incur the full associated salary costs.
6. 6. Some awarding organisations offered a rebate (a proportion of the qualification fee) to centres, to reflect the different cost structure for 2020 qualification delivery activities. The level of rebate, offered on the qualification fee, varied across the sector. For example, exam boards delivering general qualifications considered their overall savings from 2020 qualification delivery were greater than the cost of new activities and issued a rebate to centres (see table 1). Cost implications for new activities, versus activities which organisations did not undertake, also varied across different types of qualification. For awarding organisations delivering vocational and technical qualifications, some applied rebates whereas others considered they had overall increased costs and that it was not appropriate to offer a rebate to centres.

Table 1: Exam boards 2020 rebate to centres (GCSE, AS and A level)

Exam board	Qualification fee rebate percentage
AQA [footnote 4]	26%
OCR [footnote 5]	23%
Pearson [footnote 6]	23%
WJEC [footnote 7]	23%

Information in the table is from published exam board information [reviewed 4 January 2021].

1. 7. Centres had mixed views of the level of rebate received, some were positive, some were not. Indeed the sector representative body for colleges which we interviewed (Association of Colleges) expected a higher level of rebate. However, a sector representative body for awarding organisations (Federation of Awarding Bodies) reported that organisations had encountered an overall net cost increase and so were unable to offer any rebate for vocational and technical qualifications.
2. 8. Exam boards held a full exam series in autumn 2020 with a much lower entry than in a normal summer exam series. There was no economy of scale, while the qualification fee was the same price as for the anticipated summer series.
3. 9. The new activities and investments which organisations undertook in 2020 may lead to accelerated innovation for qualification delivery, and new ways of working, within the sector in the future. This may bring future efficiencies for qualification delivery (see section 3.4 ).
4. 10. The unpredictable external environment and impact of the coronavirus (COVID-19) pandemic provides ongoing uncertainty for the delivery of qualifications this year and beyond. This will continue to impact on awarding organisations and centres’ business planning and costs. As the arrangements for how qualifications will be delivered in 2021 are not yet finalised, awarding organisations and centres noted any potential future efficiencies, or additional costs, are dependent on final arrangements and associated timescales.

3. Summary from interviews

3.1 New activities and costs of qualification delivery

Awarding organisations and centres needed to undertake and develop new activities to deliver their qualifications. This section lists the activities which apply to each of the types of organisation we interviewed. We then include the activities which are specific for different organisation types: exam boards’, vocational and technical qualification awarding organisations’, and centres’.

Enabling staff to work effectively from home

In March 2020 all awarding organisations and centres needed to work remotely due to the pandemic. The level of costs involved varied depending on the size of the organisation and the level of its readiness for remote working. Remote working especially impacted on organisational ways of working and on IT department activity, as well as the development of new processes. Examples of additional costs include new hardware, software, licence costs, phones, data usage and home office furniture.

Keeping up to date with information and awareness about developing 2020 policy

All awarding organisations and centres needed to be aware of developing policy and its implications. This required significant time particularly from senior colleagues to interpret context, set direction and avoid miscommunication. Examples of these costs include time attending external stakeholder meetings, developing new communications, and cascading required messaging to internal colleagues.

Responding to regulatory consultations and data requests

Ofqual and other regulators published a number of regulatory consultations relating to arrangements for qualification delivery in 2020, with short timescales for response. Awarding organisations and centres therefore had the opportunity to prepare a response and shape the consultation decision. Ofqual also issued data requests to gather evidence from awarding organisations to inform decision making. This type of activity often required senior colleague involvement and especially impacted on employees with a compliance focus. Examples of this type of cost included gathering evidence and preparing data for consultation responses, agreeing consultation responses with internal colleagues, and following internal governance.

Developing new guidance and communications and responding to an increased number of customer queries relating to delivery of qualifications

Awarding organisations and centres needed to develop new customer communications along with new communications strategies. This took place alongside an increased number of customer queries. This especially impacted on communications and customer service teams, and senior staff for internal governance. Examples of this type of cost include developing communication materials for websites and emails, communicating with stakeholders, communicating directly with learners and other members of the public, and dealing with media enquiries.

Developing and implementing a new system for centre assessment grades

The 2020 data collection requirements were a substantial new cost with short lead in times for those awarding organisations and centres delivering qualifications involving centre assessment grades. This required awarding organisations to develop appropriate data collection systems, including deciding whether to implement a short term solution or include wider IT development, and centres to gather and submit the necessary information during a specific time period. Examples of this cost include communicating guidance to centres, centres developing new internal moderation processes, IT development and testing.

Quality checking data gathered and submitted

Awarding organisations needed to quality control centre data and centres needed to ensure accurate submission of centre assessment grades. This impacted on staff resource within awarding organisations and centres because the process was often manual and time intensive. It required staff with a specific level of knowledge and experience. Awarding organisations and centres could not easily increase capacity within the timescales involved. Examples of this cost include developing quality assurance processes and associated criteria, manually inputting data and communicating with centres for data clarifications.

Developing an appeal process and undertaking the appeals

Awarding organisations needed to adapt existing processes or develop and implement new fit-for-purpose appeal processes, covering general qualifications and vocational and technical qualifications, aligned with relevant policy criteria. Awarding organisations often needed to extend their timescales for the development of appeal processes due to ongoing policy revisions. This impacted on delivery staff and senior staff for governance. The appeal criteria meant each appeal could potentially be time consuming. Examples of this cost included gaining understanding of criteria, developing an appeal process with sufficient governance requirements, communicating the approach to centres and undertaking appeals.

Responding to changes in policy

Awarding organisations and centres needed to respond to changing policy during summer 2020 and this resulted in benefits from initial investment not always being realised. This especially impacted on delivery staff and senior colleagues. Examples of this type of cost are where organisations invested in remote invigilation before the policy move to using centre assessment grades, and where organisations needed to undertake new IT system changes to issue results based on centre assessment grades.

New activities and costs - exam boards

By the time the 2020 summer exam series was cancelled exam boards highlighted they had already realised costs from undertaking many activities in preparation of the series. In particular the content of exam papers is prepared approximately a year in advance. A high proportion of papers were already printed and papers were already packaged and ready for distribution (see ‘saved costs - exam boards’ for how exam boards could use the content of these papers for the autumn exam series). Exam boards also highlighted the following specific new activities and associated cost implications.

1. Developing IT systems with specific requirements within short timescales. For example, these system requirements included allowing for data collection from centres and programming to implement the standardisation model approach. Exam boards also needed to align new IT developments with existing systems and to update system programming under short timescales following policy decisions.
2. Developing and implementing a suitable process for awarding grades to private candidates where possible. This involved communication with centres and developing criteria and a governance process.
3. Running a full autumn exam series and incurring all the necessary fixed costs (including paper distribution, minimum number of assessors and moderation process) along with associated variable costs (including scanning and marking).
4. Providing hardware to employees for business continuity and remote working. The organisational size of exam boards meant this was a large cost for some organisations.

New activities and costs - awarding organisations offering vocational and technical qualifications

There is a large variety of vocational and technical qualifications, and new activities for awarding organisations varied depending on the types of qualification they delivered.

Below is a list of additional activities they may have needed to undertake due to the pandemic, incurring associated costs.

1. Developing, assessing and quality assuring centre assessment grades. Awarding organisations noted this activity was a significant cost. It involved developing a new process and communicating individually with centres to gather and potentially review evidence.
2. Negotiating with other stakeholders, including reviewing and agreeing interpretation of relevant government guidelines, to enable continuation of some qualification delivery. Awarding organisations noted they also needed to consult with other awarding organisations to ensure consistency around adaptations.
3. Implementing protocols to enable staff to enter premises safely to undertake operations that could only be done on site, for example printing certificates. This was in addition to more generic remote working costs. Some awarding organisations also incurred costs for providing required specialist software (for example, computer aided design software) to staff where needed.
4. Developing and delivering online training sessions to support centres with understanding guidance and requirements. Awarding organisations also needed to develop suitable processes to record the online sessions and distribute to centres.

New activities and costs - centres

In addition to activities captured in section 3.1, including those associated with producing centre assessment grades, centres also highlighted the following specific activities and costs associated with qualification delivery.

1. Ensuring a COVID-secure environment for delivering qualification assessments. This included providing personal protective equipment, relevant information signs and purchasing face masks. Centres needed to deliver this within short timescales to enable secure delivery of adapted assessments. For example, centres incurred additional costs for staff to work with students in smaller groups for practical assessments.
2. Undertaking an increased level of communication with learners and parents around the development of centre assessment grades and issuing of results. This required input from senior staff to ensure consistency and involved complex challenges to develop new messages when government policy was changing rapidly. Centres had an increase in the number of calls to results helplines and also held an increased number of conversations with university admissions after A level results day. The Association of Colleges also highlighted the resource burden associated with the level of complaints and legal challenges which some centres received in summer 2020.
3. Delivering the autumn exam series in a COVID-secure way, for example increased invigilation costs resulting from undertaking exams within year group bubbles. ^{[footnote 8]}

3.2 Saved costs for qualification delivery

While organisations needed to undertake new activities in light of the pandemic, other activities were no longer required. The following activities did not go ahead as normal in 2020 resulting in cost savings. As in section 3.1 some apply to each of the different types of organisation we interviewed while others are more specific: exam boards, vocational and technical qualification awarding organisations, and centres.

Exam paper distribution, invigilation, scanning, marking and moderation of exam scripts

This did not take place due to the cancelled summer exam series. This meant awarding organisations did not encounter costs associated with some temporary seasonal staff contracts.

Staff travel

Awarding organisation and centre employees did not undertake travel during summer 2020 and had a reduced cost for this activity. This includes external assessments and learner observations which normally require awarding organisation attendance. These were instead undertaken remotely during summer 2020.

In-person events

Awarding organisations and centres could not undertake all scheduled events which required in-person attendance, for example workshops or student exhibitions of work related to assessments. Organisations sometimes still had a level of committed spend on these activities, for example costs associated with venue hire.

Scheduled business improvements

Awarding organisations did not undertake all scheduled business improvement activities, for example pausing planned IT system improvement projects.

Other business-as-usual activities

Awarding organisations did not undertake all anticipated business as usual activities, for example sales and marketing, and associated graphic design activity.

Saved costs - exam boards

Although exam boards had an overall saving on the following activities, they did not always save the full costs. Exam boards had a level of committed spend for some activities which were no longer needed, for example minimum contracts with scanning companies. Whilst marking did not take place, generating a significant saving, some exam boards incurred a non-contractual cost by paying examiner retainer fees, especially for examiners at a more senior level. This decision was taken to address risks to the retention of this work force for future exam series, including the autumn series.

Some exam boards cancelled a number of temporary contracts for staff undertaking activities directly related to delivering an exam series, and saved on this activity. Some exam boards furloughed a proportion of employees where the government incurred some cost of the employee salary and the organisation incurred an optional top up cost.

For the autumn exam series, exam boards were able to use the content of exam papers prepared for the summer series, which prevented them incurring a large additional cost for the autumn series. Some costs were still incurred, for example papers were relabelled with a new exam series date.

Saved costs – vocational and technical qualification awarding organisations

External assessors did not undertake visits to centres for assessments during summer 2020, and awarding organisations did not incur costs for this activity. The organisational business model impacted on the level of savings, for example where assessors are directly employed by the organisation or contracted from a different company.

Some awarding organisations furloughed assessors while others paid a retainer fee to ensure future capacity. Others reprioritised assessors’ activities, for example having them review proposed qualification adaptations.

Awarding organisations did not incur some anticipated costs associated with activities related to specific qualifications, for example functional skills marking.

Saved costs - centres

Centres did not undertake certain activities associated with an exam series, for example there was no need for invigilation. Nonetheless, centres incurred costs associated with ensuring the training and safeguarding checks of invigilators were up to date. They also needed to develop communications to keep invigilators informed of the developing policy decisions and implications. Some centres, in light of future workforce planning, made payments to invigilators (aiming to retain capacity for the autumn 2020 exam series).

3.3 Impact across qualification type

The impact of activity related to qualification delivery in summer 2020 varied to a large extent across different types of qualification. In summer 2020, qualification delivery was subject to exceptional arrangements for exam grading and assessment and the extraordinary regulatory frameworks for both general qualifications and vocational and technical qualifications. The additional activities required for vocational and technical qualifications were also affected by whether qualification assessments were adapted, calculated or delayed. We capture the impact of each of these below.

Adapted assessments

Provision of adapted assessments to learners had a large impact on the range of new activities and therefore cost implications of awarding organisations and centres. Awarding organisations first needed to assess whether and how a qualification assessment could be adapted. In certain practical subjects this required assessing whether a learner had occupational competence, for example to fix brakes on a car. Some sectors such as childcare only allowed critical staff to visit premises. Learners and apprentices were sometimes not able to complete an assessment during that time. This all required awarding organisations and centres to undertake activity to consider the possibility, practicalities and means for delivering an adapted assessment, and then to compile guidance.

Awarding organisations also encountered costs related to investments in technology to support adapted assessments to learners within a COVID-secure environment. For example, some awarding organisations invested heavily in remote invigilation, both in purchasing the necessary software and initiating a pilot study to test the process. Centres also encountered costs with adapting an assessment to ensure a COVID-secure environment for delivering qualification assessments (see ‘new activities and costs - centres’).

The timescales involved in policy announcements during summer 2020 meant awarding organisations were not always able to issue finalised guidance to centres within their preferred timescale. Sometimes an awarding organisation issued initial guidance to a centre which was then iterated due to updated policy announcements. This had implications for the efficiency of activities for both awarding organisations and centres.

Centres needed to communicate with awarding organisations to gain approval for adapting qualification assessments, and this could involve detailed conversations. This sometimes required centres to reach out to awarding organisations on multiple occasions in order to gain approval.

Providing calculated results

For qualifications with provision of calculated results to learners, activities also varied across type of qualification. As well as the process of calculated results for GCSE, AS and A level, other qualifications required a new process for learners to obtain a calculated grade. General qualification grades were initially to be awarded using a standardisation model before a late change in policy to using centre assessment grades. For functional skills qualifications awarding organisations needed to develop a process for a calculated grade and collate learner evidence within a short 3-week timescale. This required intensive communication and resource from centres and awarding organisations to first gather the evidence and then assess whether the evidence was sufficient. At times this required extensive correspondence between the learner, centre and awarding organisation.

Delayed assessments

For qualifications with delayed assessments to learners this also had associated costs for organisations where learners could not continue with their assessments. Centres needed to provide practical ongoing support and keep the learner informed of changes in guidance and future timescales, including administrative costs for maintaining learner registration.

As a specific example, where a learner is unable to complete a functional skills qualification this can lead to a delay in progressing to their end-point assessment for an apprenticeship. The centre carries the risk of receiving the achievement payment associated with a learner completing a course. Therefore a delay in the learner assessment leads to a potential delay of the centre receiving the completion payment.

3.4 Implications for future qualification delivery

Awarding organisations and centres expressed concern about the arrangements for 2021 and the impact on future cost efficiencies. They noted the uncertainty about arrangements for 2021. This meant they could not yet determine the full impact of any potential efficiencies for the future. They anticipated increasing their contingency planning activity during 2021, due to continuing changes in the external environment.

A number of activities may lead to potential efficiencies or inefficiencies for future qualification delivery.

Staff or people costs

Ongoing people costs from the impact of summer 2020, including staff welfare due to the resource intensive activity and short delivery timescales. In many awarding organisations staff deferred annual leave which will impact on future resource levels. Some organisations made a good will gesture - allowing staff to take extra annual leave in 2021, due to the intense activity in 2020. On a more positive note, awarding organisations and centres required staff to undertake new and different activities across different areas of their operations to support delivery. This may have benefits in general up-skilling and lead to efficiencies in future. Awarding organisations also noted increased collaboration amongst staff in different business areas.

Opportunity costs from pausing business development and business improvement programmes

Awarding organisations found these costs difficult to quantify but they may have reduced future efficiencies. Examples include pausing development and design for new qualifications, pausing an organisational restructure and pausing IT improvement projects. There is also a potential loss of future efficiencies caused by senior staff focusing more on operationalising summer 2020 activity rather than future business planning.

More innovation within the sector

Awarding organisations and centres are now better set up for remote working and some have made accelerated investments in remote assessment and qualification delivery. A sector representative body (Federation of Awarding Bodies) noted its members implemented and trialled new systems for qualification delivery, whilst risking their investment not realising intended benefits. Examples of this include organisations undertaking pilot studies in remote invigilation, investigating e-certificates as a future development, developing more interactive and synchronous packages and learning material for qualification delivery, and improved standards and controls around using virtual platforms.

Potential impact on future income

Awarding organisations and centres noted some areas of loss of income as a consequence of 2020 delivery, for instance where assessment is delayed. This may impact on business cash flow and future income levels. Exam boards also noted reduced income from use of post-results services. Further, private candidates with insufficient evidence for a qualification grade in summer 2020 received a refund. Centres may use summer 2020 rebates to enter students for autumn series and 2021 exams, hence this is likely to result in reduced future income for exam boards.

Ways of working and adaptability

Some awarding organisations and centres consider the new processes, ways of working and more efficient decision making that were necessary to address the pandemic will have benefits for future delivery. Some note an improved staff work-life balance due to less travel. Awarding organisations and centres said they developed improved communications and processes, across many communication platforms, and considered this as a future benefit.

Increased collaboration, engagement and communication within the sector

Awarding organisations generally consider the sector – including other awarding organisations and stakeholders involved in qualification delivery – collaborated effectively during 2020. They also highlight increased communication with the regulator. This increase in sector engagement has strengthened relationships which may benefit future activities and provide efficiencies.

The coronavirus (COVID-19) pandemic had a significant effect across the qualifications industry in 2020. This report seeks to capture just one element – implications for costs of delivery for awarding organisations and centres. 2021 will also be atypical as the pandemic continues to affect teaching, learning and assessment. We will review the implications for delivery, costs and prices, whilst also monitoring price transparency. We intend to continue with our longitudinal price analysis and publish a 2021 qualification price index.

4. Authors

Dr Rachel O’Reilly (Senior Economist).

Dr Tom Barcham (Associate Director Risk and Markets).

4.1 With thanks to

Thanks to the awarding organisations and centres who generously offered their time and took part in interviews to discuss 2020 qualification delivery.

1. Ofqual (alongside CCEA Regulation and Qualifications Wales) now requires all regulated awarding organisations to publish fee information about their qualifications in a standard format that is easily accessible to potential purchasers. The consultation took place between August and October 2019 and the updated Condition F1 requirement was confirmed in February 2020 and came into effect on 18 January 2021. ↩

2. We reviewed responses from the following consultations: General Qualifications exceptional arrangements; Vocational and Technical Qualifications Extraordinary Regulatory Framework (ERF); Vocational and Technical Qualification Extended ERF Part 1; Vocational and Technical Qualifications Extended ERF Part 2; General Qualifications autumn series; General Qualifications summer 2021 series; Vocational and Technical Qualifications summer 2021 series. We also review other sources including awarding organisation websites, and other correspondence. ↩

3. Association of Colleges and Federation of Awarding Bodies. ↩

4. AQA’s published fees information ↩

5. We note OCR also offered a centre rebate for the qualification fee for Cambridge Nationals (20%) and Cambridge Technicals (20%). ↩

6. We note Pearson also offered a centre rebate for the qualification fee for BTEC level 2 and level 3 qualifications (6%). ↩

7. WJEC’s published fees information for the 2020 exams series. ↩

8. The Department for Education offered an exam support service to support centres with costs associated with the autumn exam series: fees, sites and invigilators. ↩