Research and analysis

Learning during the pandemic: review of international research

Published 12 July 2021

Applies to England

Authors

• Neil Stringer
• Ellie Keys

With thanks to

• Charlotte Lockyer
• Paul Newton

Executive summary

In March 2020, schools around the world began to close as part of a global effort to reduce the transmission of the coronavirus (COVID-19) pandemic. In early April 2020, at the peak of the closures, 1.6 billion students – over 90% of the world’s total – were thought to be affected. By the summer (northern hemisphere), when schools in many countries break for holidays, globally, the average student had missed almost 50 school days – or a quarter of a school year. To minimise the learning lost while schools remained closed, governments and schools around the world responded by putting in place remote learning. In England, this largely took the form of online learning, up to and including teacher-led lessons.

Rapid evidence assessments published at the beginning of the school closures painted a grim picture of the impact these closures could have on students’ learning. Modelling based on data pertaining to ‘summer learning loss’ suggested that students might learn little to nothing while schools remain closed, and studies of the efficacy of remote – including online – learning suggested it may only partially offset the losses.

The purpose of this report is to review international research papers and reports that may help our understanding of the scale and nature of learning loss in England. Three scenarios for learning loss were identified from the literature.

In the first scenario, mean attainment falls; however, the overall distribution of attainment remains the same. This means that any losses would be distributed evenly across the student population, that is all students would be affected equally.

In the second scenario, mean attainment falls, but the distribution of losses is uneven across the student population, causing the standard deviation^{[footnote 1]} to increase. Some students may do better than they would otherwise have done, some may be relatively unaffected, but many will do worse: some of them quite substantially so.

The third scenario is much like the second, except that a significant proportion of students score zero – or very close to it – on their assessments, producing a second peak at the low end of the distribution curve. This represents, at best, no progress and, at worst, regression for the worst-affected students.

In the first of 2 main sections of this report, we review the early papers that modelled the impact of school closures on students’ attainment and more recent reports based on students’ performance in large-scale testing programmes following school closures. In addition to this, we review the diverse literature on the impact of other – non-COVID-19-related – events on students’ attainment. These events include normal variations in teaching time, sickness absence, weather-related school closures, teacher strikes, natural disasters, previous pandemics, and the aforementioned summer learning loss.

Early modelling of the impact on learning of school closures projected average learning losses, with greater losses in mathematics than reading, and an uneven distribution of losses according to socio-economic factors. Subsequent studies reporting pre- and post-school closure assessment data suggest losses were not as great as some of the most pessimistic projections, although the overall pattern predicted was correct.

School closures in the second quarter of 2020 put students typically 2 to 3 months behind the academic milestones their cohorts would be expected to reach. As anticipated, losses were frequently greater in mathematics (3 months) than in reading (one-and-a-half months). In most studies, disadvantaged populations experienced above-average losses. There is some evidence that younger students were more adversely affected than older students, though there is variation across research studies.

The impact of variation in teaching time on student attainment has been studied internationally in several different contexts (for example, the length of school days, student absences, and teacher strikes). In general, a reduction in the time spent in school is associated with a reduction in student test scores. Individual student absences have a greater impact on attainment than whole school closures do, probably because it is easier for teachers to facilitate whole classes than individual students in catching up. In all of the different contexts we reviewed, mathematics was more severely impacted by reductions in teaching time than literacy, conceivably because students can develop their literacy skills outside of school more easily. A common finding across countries and contexts is that students from lower socio-economic backgrounds are most impacted by changes to teaching time.

In the second main section, we review evidence of the efficacy of online remote learning – before and during the COVID-19 related disruption – and how that efficacy – and thus learning loss – may vary according to students’ characteristics and circumstances.

Online learning is generally associated with poorer outcomes than traditional classroom teaching, although its effectiveness varies depending on implementation. Teaching quality is fundamental to successful learning, and therefore generally more important than the medium of delivery. Some of the shortcomings of online learning during school closures may stem from hurried implementation on an unparalleled scale; however, the medium is currently better suited to some tasks and types of content than others. The limited opportunities for interactions with peers and teachers may affect motivation and learning, particularly during the pandemic where this extends to life beyond school.

Evidence suggests the effectiveness of online remote learning will vary according to students’ characteristics and circumstances. It is expected that younger students may adapt less easily than older students, and that boys and girls may be affected in different ways. For example, boys appear to lose more learning time, while girls are at greater risk of developing mathematics anxiety.

Access to the necessary equipment for remote learning, a home environment conducive to learning, and parental support are all key to successful online and home-based learning. These factors are all heterogeneous across student populations, and variation in them can lead to substantial differences in learning outcomes. Students with special educational needs and disabilities may find the tailored support they require is not available when learning from home or that the online platforms and devices they use do not accommodate their needs.

The evidence regarding the impact of school closures and online tuition on students’ non-cognitive abilities and wellbeing, including physical fitness, is more limited, though the evidence available points to declines. Finally, the school closures in the learning loss literature are typically not accompanied by restrictions on social interactions outside of school. Therefore, it remains an empirical question how this affects recovery from learning loss as schools reopen fully.

Of the 3 learning loss scenarios identified, the evidence we reviewed suggests that the effect of socio-economic status on assessment scores alone might be expected to produce the second scenario. In this scenario, the distribution of losses across the student population is uneven, causing a fall in mean attainment and an increase in the standard deviation. Regarding the worst-case scenario, in which a subpopulation of students has effectively ceased learning, there is evidence to suggest some students will have fallen badly behind during school closures. Nevertheless, the medium to longer-term outcomes for these students are probably not inevitable and will depend on re-engagement and support with continuing to learn.

The aim of this literature review was to understand how COVID-19 has affected student learning across the globe, to inform our understanding of the variety of ways in which it has affected students in England. Of course, countries have experienced and responded to the pandemic in very different ways, and although many insights from our literature review are likely to generalise to circumstances in England, others may not. Our review should be read with this caveat in mind.

Introduction

Background and scope

In March 2020, the World Health Organization declared COVID-19 a pandemic. As part of the effort to reduce transmission of the virus, many countries closed their schools.^{[footnote 2]} At the peak of school closures in early April 2020, over 90% of the world’s students – 1.6 billion students across 194 countries – were estimated to be affected (UNESCO, 2020b). In an international survey conducted by UNESCO, UNICEF, and The World Bank during the summer,^{[footnote 3]} 108 countries reported missing an average of 47 days^{[footnote 4]} of in-person teaching due to school closures. This is equivalent to approximately one-quarter of a regular school year (UNESCO et al., 2020).

UNICEF (2020) estimates that, as of November 2020, more than 570 million students – 33% of the enrolled students – globally were affected by country-wide school closures in 30 nations. School closures have continued or recurred in many countries through 2020 and early 2021 – including England from January to March 2021 – though the number of students affected globally at any one time has remained lower than the initial peak (UNESCO, 2020b). The OECD (2021) reported that, 1 year after the pandemic hit, primary and secondary schools were open for at least the vast majority of students in less than 40% of the 33 countries for which they had comparable data.

Governments and schools quickly introduced remote teaching and learning so that students could continue their education during school closures (UNESCO et al., 2020). It is important for our understanding of learning loss to gauge the effectiveness of remote schooling compared with face-to-face schooling. It is also vital to understand the differential impact of school closures on the learning of children, depending on their circumstances. Access to technology is not homogeneous and home environments are not all equally conducive to learning.

In England, the scale of the disruption and the rollout of remote tuition have no modern precedents; however, globally, events such as natural disasters and war have resulted in comparable levels of disruption. This report is a companion to our report covering research on learning loss from England.

The focus of this report is on understanding the possible scale of learning loss England’s students may face based on a reading of international research on school closures. We review international reports and papers on the loss of learning caused by such events, including the COVID-19 pandemic, as well as teacher strikes, summer learning loss, and absenteeism. We consider how effective alternative arrangements – primarily remote learning – have been, including any disparities in effectiveness between students from different backgrounds and circumstances. We have chosen to focus on the issue of learning loss and the use of remote learning to alleviate it; longer term strategies for remedying learning loss and the longer-term economic impacts of learning loss are out of scope.

Our approach to the review is thorough, though not systematic. For the sections on the COVID-19 pandemic specifically, we attempt to be comprehensive in our coverage of reports that model or report real data regarding learning loss at scale. We elected not to focus on small-scale studies, such as reports based on data from individual institutions, because the findings are less likely to generalise well from very specific contexts to schools in England, and would likely add diminishing returns alongside large-scale studies.

For the sections on non-COVID-19 learning loss, the body of research we report on overlaps with that included in various reports by international organisations. We are therefore confident we provide a thorough overview of the relevant research, including journal articles and reports, though it is not intended to be as comprehensive as the coverage of the COVID-19-related research.

The section on non-cognitive abilities and wellbeing is briefest, as less research is available and our focus was on these factors as they related to learning loss rather than in their own right. Data on the effect of the pandemic on these factors is likely to emerge more slowly than large-scale assessment data.

Technical and vocational learning is largely conspicuous by its absence in the international literature and consequently in this report. This is perhaps unsurprising, as many studies focus on attainment in reading and mathematics and students at stages of their education prior to selecting academic or vocational tracks. Finally, our review is almost exclusively restricted to publications in the English language.

Better to have learned and lost than never to have learned at all

The literature on learning loss includes scenarios in which students have learned everything they are supposed to, but scheduled breaks in education – such as the summer holidays – result in performance dips in the following autumn term. There are also scenarios in which students have not learned everything they are supposed to. This is an important distinction, as it is likely to have implications for the trajectory of learning regain. Summer learning loss may be best thought of as rustiness or a loss of form, requiring revision or practice for swift regain; in contrast, the absence of original learning may be considerably harder to regain (Education Endowment Foundation, 2020a).

A study by Kuhfeld and Soland (2020) shows that rates of growth are higher at the beginning of the school year and slow towards the end. A plausible interpretation of this is that part of the growth at the beginning of the school year is regain that is acquired more quickly than new learning (Education Endowment Foundation, 2020a). Kuhfeld and Soland also show that this nonlinear growth may mean estimates of summer learning loss that assume linear growth are overestimated as much as twofold.

Quality vs. quantity

Discussion of learning loss sometimes focuses on the timetabled learning hours lost compared with the time spent studying remotely. This implicitly assumes an hour learning in school is worth an hour learning remotely at home. This may not always be the case; time in school may be more or less productive than time at home, and this will depend to an extent on how the time is used by the teacher. Moreover, it is likely that the students’ home environments are a significant source of variation in productivity that is normally moderated by their presence in school. It is important to bear in mind that lost time does not necessarily equal lost learning – though it might – and that lost time almost certainly does not result in the same degree of lost learning for all students.

Learning loss scenarios

UNESCO et al. (2020) warn that even short interruptions to children’s schooling can have long-lasting negative impacts on their learning (Alban Conto et al., 2020). The OECD (2020b) uses the language of labour economics to compare the long-term impact of school closures on students’ outcomes with hysteresis:^{[footnote 5]} the long-term effect of unemployment on a worker’s ability to find a job. They argue the potential learning loss is determined by 2 concurring factors: (i) how much students have learnt during school closures – or the “intensive margin” – refers to the efficiency of education continuity solutions; (ii) how many students continued to learn during the school closures – or the “extensive margin” – refers to the proportion of students engaged in the education continuity solutions.

Schools face challenges integrating information and communication technologies into the classroom (OECD, 2018b, 2019) and there were signs early on during school closures that some students either could not be – or had not yet been – contacted by their teachers (Goldstein et al., 2020; Les Echos, 2020). Potential for hysteresis of the COVID-19 crisis in education (Saavedra, 2020) stems from various elements, often linked to students’ socio-economic status (SES), leading to a withdrawal from the school system that will produce a long-term impact on students’ outcomes. Given the critical role IT resources and parental involvement have played in ensuring education continuity during the crisis (Gouëdard et al., 2020), the hysteresis caused by school closures may be more prevalent among students from less privileged backgrounds (OECD, 2020b).

The World Bank (Iqbal et al., 2020) identifies 3 distinct learning loss scenarios (Figure 1).^{[footnote 6]}

Figure 1. Three possible scenarios of how the learning curve may evolve in the coming months: a lower average, a higher standard deviation, or a sharp increase in low learning at the bottom.

They describe them as:

1. The most straightforward transformation is a reduction in average learning levels across the distribution (the blue curve, left). This scenario is very likely, despite the best efforts of school systems to provide distance learning. Variation in teaching time is associated with learning loss (Cattaneo et al., 2016) and shocks, like floods, affect learning outcomes across grade levels (Thamtanajit, 2020).
2. The curve may flatten or skew due to highly unequal effects of the crisis (the purple curve, centre). In this scenario, children at the top will make gains, while students at the bottom fall further behind. This divide may run along lines of SES. Wealthier families are better placed to mitigate the effects of school closures, being more likely to have: comfortable homes; good internet connections; the means to provide private tuition; and well-educated parents, better placed to home school their children. The bottom of the income distribution may also see a sharp increase in poverty from unemployment (Worley, 2020). For example, the 2008 to 2009 recession negatively affected learning, particularly in districts with higher proportions of disadvantaged and minority children (Shores & Steinberg, 2018).
3. The curve may change due to chronic absenteeism or dropouts (the green population, now permanently out of school, right). Iqbal et al. take a global perspective, noting that the 1997-98 Asian financial crisis (Cameron, 2009) and the 1916 polio pandemic (Meyers & Thomasson, 2017) showed that school enrolment can fall sharply due to both demand and supply side effects. On the demand side, income shock leads families to ask their children (particularly girls) to leave school to work. On the supply side, increased numbers of schools may close permanently, particularly in countries that have expanded schooling through low-fee private schools (Yousafzai, 2020). These kinds of effects will be less pronounced in developed countries like England, where education is compulsory, but they may have an impact on post-compulsory education, and on particular groups of students.

The World Bank also estimated that 5-month school closures could result in 25% more students falling below a baseline level of proficiency needed to participate effectively and productively in society, and in future learn¬ing (Azevedo et al., 2020). The modelling was at a coarse geographical level (for example, Europe and Central Asia, Latin America and the Caribbean, and the Middle East and North Africa). The percentage point changes by region were broadly consistent, though regions comprising less-developed countries started from a much higher baseline than those comprising more-developed countries.

David et al. (2020) expected examination-oriented progress to slow down for all students. They argued that examination-oriented syllabuses become unfit for purpose when tests and examinations have been put on hold, and that much of the curriculum cannot be adapted quickly to online teaching (Graham & Sahlberg, 2020).

Structure of the report

The body of the report is divided into 3 main sections. The first section considers the international evidence on learning loss and is organised into subsections on: (i) the COVID-19 pandemic, looking at initial modelling of potential learning loss and the real impact using subsequent assessment data; and (ii) evidence from non-COVID-related factors, including variations in teaching time, school summer holidays, and unplanned disruption.

The second section considers the use of remote learning and is organised into subsections on: (i) evidence regarding the efficacy of online learning as an alternative to traditional schooling; and (ii) the evidence on how the efficacy of online learning may be mediated by factors including students’ age, gender, ethnicity, SES, prior attainment, and special educational needs and disabilities.

The third section briefly considers the impact of school closures and online learning on: (i) students’ non-cognitive abilities – such as learning skills and social and communication skills; and (ii) wellbeing – including their physical and mental health –with particular regard to how these underpin learning.

International evidence on learning loss

Evidence from largescale testing programmes following COVID-19 school closures

Evidence summary

Some of the most pertinent evidence available regarding learning loss comes from testing programmes that have measured student progress before and after school closures. The performance of these students can be compared with the progress of previous cohorts that were unaffected by school closures. Prior to this a number of studies used data from testing programmes to model the likely learning loss based on the known impact of other events, in particular the school summer holidays and absenteeism.

Early modelling of the impact on learning of school closures anticipated average learning losses, with greater losses in mathematics than reading, and an uneven distribution of losses according to socio-economic factors. Studies of pre- and post-school closure assessment data from several countries suggest school shutdowns in the second quarter of 2020 put students typically 2 to 3 months behind the academic milestones their cohorts would be expected to reach.

Losses were frequently greater in mathematics (3 months) than in reading (1½ months), and in most studies disadvantaged populations experienced above average losses. There is some evidence that younger students were adversely affected more than older ones, though this appears less consistent across studies than the other effects. There is little evidence available on gender effects.

Modelling

Kuhfeld, Soland, et al. (2020) produced a series of projections of COVID-19-related learning loss based on estimates from the absenteeism literature and analyses of summer learning patterns of 5 million U.S. students in grades 3-8 (ages 8 to 14) who took MAP® Growth™ assessments in the 2017-18 and 2018-19 school years. Based on school closures prior to the summer holidays, they expected students to begin the autumn term with approximately 63% to 68% of the learning gains in reading and 37% to 50% of the learning gains in mathematics relative to a typical school year.

Learning loss was not expected to be universal, however, with the top third of students potentially making gains in reading. Unlike mathematics, for which the standard deviations^{[footnote 1]} of expected scores were like a typical year, for reading, not only was average attainment expected to fall, but the standard deviation of expected scores increased by up to 20%.

Some consolation was to be found in the prediction that students who would lose the most while out of school should gain the most the following year. However, these models are based on typical summer loss conditions, and it is uncertain how factors such as financial uncertainty, health issues related to the virus, and psychological stresses may affect subsequent academic growth.

The authors acknowledge that their analyses could not address the impact on ethnic groups but suggest it may be disproportionately great. For example, African Americans have experienced higher than average rates of COVID-19 infections and deaths (Bouie, 2020), and the economic downturn in the United States has been particularly damaging for Black and Hispanic parents, who are typically less able than other ethnic groups to work from home (Cerullo, 2020; Krogstad et al., 2019).

Modelling the relationship between schools’ student attendance rates and student proficiency levels, EmpowerK12 (2020a) estimated that metro Atlanta^{[footnote 7]} students losing 9 weeks of regular teaching (due to the COVID-mandated move to distance learning in mid-March) would result in the percentage of students demonstrating proficiency to drop 3.6 points in English language arts and 4.9 points in mathematics compared with the previous year. Economically disadvantaged students and students of colour, who were already behind their peers, were expected to fall further behind, reversing recent gains.

Kaffenberger (2020) used a calibrated model with a “pedagogical production function” (Kaffenberger & Pritchett, 2020) to estimate the potential long-term losses to children’s learning from the temporary shock of school closures. The model showed that without mitigation, students could lose more than a year’s worth of learning from a 3-month school closure, as short-term losses continue to compound after schools reopen.

Real data

The authors of several of the following studies acknowledge possible issues with the validity and/or representativeness of the data collected in 2020. In some instances, tests were taken online at home instead of in school, which resulted in some students performing better than anticipated. Curriculum Associates (2020a), for example, excluded data from tests taken at home. Another effect of home test taking and/or limited school opening was that the samples tended to be incomplete, with the missing students typically being from backgrounds most likely to be adversely affected by the school closures (for example Kuhfeld, Tarasawa, et al., 2020).

The result of the validity and sampling issues is that the effects may be under- or overestimated. The impact of school closures may be underestimated where students have received help or used prohibited resources during the test, or where the students most affected by the pandemic have not been assessed. Conversely, there may be students who have taken tests at home in suboptimal conditions (for example with noise, distractions, poor IT equipment) who would have performed better in schools. The learning lost by these candidates – both absolute and relative to their more advantaged peers – may be overestimated. For these reasons, it is probably best to treat the findings as an indication of what we might expect to see in England rather than a precise estimate.

Following up on their projections for learning loss, Kuhfeld, Tarasawa, et al. (2020) analysed data from MAP® Growth™ assessments taken by students in the autumn of 2020. They found students in grades 3 through 8 (ages 8 to 14) performed similarly in reading compared with same-grade students in autumn 2019, but about 5 to 10 percentile points lower in mathematics.

In nearly all grades, most students made learning gains in both reading and mathematics since the beginning of the COVID-19 pandemic. Nevertheless, gains in mathematics were lower on average than in previous years. Students scored better than projected in reading, while mathematics scores were in line with projections for grades 4 through 6 (ages 9 to 12), but slightly above projections in grades 7 and 8 (ages 12 to 14). There was some indication in the data that differences may be emerging along lines of race, and that student groups especially vulnerable to the impacts of the pandemic were most likely to be missing from the data.

A study published by McKinsey & Company (Dorn et al., 2020a) analysed assessment data from the Curriculum Associates i-Ready platform and reported that K-5 (ages 5 to 11) students in their sample learned only 67% of the mathematics and 87% of the reading that grade-level peers would typically have learned by the autumn. This translates into an average student losing the equivalent of 3 months of learning in mathematics and one-and-a-half months in reading. In schools with mainly students of colour, scores were 59% of the historical average in mathematics and 77% in reading.

Renaissance Learning (2020) analysed the results of over 5 million grade 1 to 8 students (ages 6 to 14) from across the United States who had taken Star Early Literacy, Star Reading, or Star Math assessments in both autumn 2019 and autumn 2020. They compared the progress of these students – who experienced school closures in 2020 – with historical norms based on the progress of previous, unaffected cohorts. They reported a small negative effect on reading achievement and growth, and a moderate negative effect on mathematics achievement and growth, with the impacts varying by grade.

In reading, students in many grades were performing close to expectations but in others they were as far as 7 weeks behind (such as 4 to 7 weeks behind for students in grades 4 through 7). For mathematics, students in all grades were performing behind expectations, with some grades 12 or more weeks behind (such as 8 to more than 12 weeks for students in grades 4 through 8).

A follow up study by Renaissance Learning (2021) reported that student growth during the first half of the 2020 to 2021 school year was approaching normal levels in both reading and mathematics, with the impact of COVID-19 on achievement beginning to shrink in many grades. Despite the encouraging headline figures, autumn-to-winter growth results varied by subgroup. Black, Hispanic, American Indian, English language learner (ELL) students, and students with disabilities grew less than the overall sample. Similarly, students attending urban or Title 1 Schools were more likely to experience below-typical rates of growth.

Engzell et al. (2020b, 2020a, 2021) estimated the learning loss that occurred when Dutch schools closed for 8 weeks by comparing data from national primary school examinations that took place just before and after schools closed with similar data from previous years. Students lost on average 3 percentile points or 0.08 standard deviations: a fifth of a year’s worth of learning – that is roughly the length of the school closures – with losses up to 60% greater among students from disadvantaged homes.^{[footnote 8]} Interestingly, learning loss did not vary according to past levels of student achievement (although see Grewenig et al., 2020).

Engzell et al. note that the Netherlands underwent only an 8-week lockdown, has an equitable system of school funding, and enjoys the world’s highest rate of broadband access. They argue that, in many respects, it represents a best-case scenario. However, Engzell et al. acknowledge that the circumstances were not necessarily ideal – the short duration of school closures allowed little time to adapt teaching and learning – and that these results may represent a temporary setback. It is also possible that some of what is lost in post-school closure assessments is examination technique and/or practice rather than knowledge, skills, or understanding in the domain being tested.

EmpowerK12 (2020b) studied the impact of school closures on students in the District of Columbia in the United States. Predictions of learning loss from the spring (EmpowerK12, 2020a), estimated in half to full numbers of years, did not come true. Students gained knowledge, though their gains were significantly less than during in-person schooling. Students lost 4 months of learning in mathematics and 1month of learning in reading, though students most at risk of academic failure^{[footnote 9]} lost more learning: 5 months in mathematics and 4 months in reading. It seems likely that one of the original assumptions of their model – that distance learning initiatives would have limited effect on average – was somewhat conservative.

There was evidence of small differential impact by student subgroup, with Black, Hispanic, and American Indian students experiencing slightly more negative impacts than the overall averages. Students attending schools serving low-income families were also more negatively impacted, as were students attending schools that were public as opposed to private, and schools in rural or town areas as opposed to urban or suburban areas.

Using data from i‑Ready Diagnostic for Reading and for Mathematics, Curriculum Associates (Curriculum Associates, 2020b; Huff, 2020) looked at the proportion of students performing below grade level overall by subject in autumn 2020 compared with prior academic school years. Although they found evidence of a “COVID slide”, they noted that it was not as severe as many observers had feared. The analysis showed that: (i) the proportion of students ready for grade-level teaching in autumn 2020 was smaller than the historical average; and (ii) the proportion of students performing below grade level was greater in mathematics than in reading.^{[footnote 10][footnote 11]}

An age breakdown of the data showed that, in reading, a higher proportion of elementary school students performed below grade level than students in middle school. The picture for Mathematics was less straightforward, with grades 2, 3, and 4 having higher proportions of students performing below grade level than students in grades 1, 5, 6, 7, and 8.

The data were also disaggregated by the proportion of Black, Indigenous, and people of colour (BIPOC) attending schools. In both reading and mathematics, schools with a higher proportion of BIPOC students saw their number of students below grade level increase more than schools with fewer BIPOC students. Likewise, when the data was disaggregated by each school’s median household income, lower-income schools saw their number of students below grade level increase more than higher-income schools in both reading and mathematics.

A study of schools in Flanders, the Flemish (Dutch)^{[footnote 12]} speaking part of Belgium, from 2015 to 2020 found significant learning loss for students in the 2020 cohort (Maldonado & De Witte, 2020). School closures appear to have led to a decrease in school averages of mathematics scores of 0.19 standard deviations and Dutch scores of 0.29 standard deviations compared with the previous cohort.

The Belgian study is unique in that language learning was more affected than mathematics. This may be because Belgium has 3 official languages, Flemish (Dutch), French, and German, which are spoken in different parts of the country. Although students attending schools in Flanders predominantly speak Flemish at home, in some areas closest to the French-speaking region, there may be more crossover in the students’ primary language.

These findings may extend to students in England who speak English as an additional language (EAL) (see also Armour et al., 2020). Additionally, variation within schools increased by 17% for mathematics and 20% for Dutch, while inequality between schools increased by 7% for mathematics and 18% for Dutch.

Bielinski et al. (2021) analysed data from FastBridge adaptive reading and mathematics assessments in the United States, comparing students’ growth from autumn 2019 to autumn 2020 with rates of growth derived from the 3 prior cohorts. The analysis shows COVID-19-related disruption negatively impacted K-8 (ages 5 to 14) students’ learning in mathematics and reading, with the greatest losses – as high as 3 to 4 months’ worth – in mathematics and in late elementary and middle school grades. The effects were observed uniformly across school strata (rural, urban, and suburban), school-level poverty rates, and race and/or ethnic composition.

Using data from a computer-based formative feedback system in Switzerland, Tomasik et al. (2020) compared learning gains in mathematics and language of 28,685 students during the 8 weeks of COVID-19-related school closures with learning gains in the 8 weeks prior to the school closures. Secondary school students were largely unaffected by the school closures in terms of learning gains; however, primary school students’ learning slowed down at the same time as interindividual variance in learning gains increased. The authors concluded that, although distance learning arrangements seemed an effective means to substitute for in-person learning (at least in an emergency), not all students benefit to the same degree.

In the Dallas Independent School District, Measure of Academic Progress (MAP) tests given to students in September and October 2020 – and compared with their midyear assessments in 2019 – indicated that half of all elementary and middle school students experienced learning loss in mathematics (by up to as much as 30 percentage points) and nearly a third slipped in reading during the COVID-19-related disruption. The slide in mathematics was attributed to students being unlikely to use mathematics concepts outside of school.

There were disparities between Black and Latino students – who make up more than 90% of the district’s student population – and the rest of the district, for example in mathematics, the percentages of Black students reaching the “meets grade level” projections ranged from 7.5% (in fourth grade) to 17.9% (in eighth grade) compared with a range of 41.8% to 61.2% for White students in the district (Carter, 2020; Dallas Independent School District, 2020; Smith, 2020). It is unclear though how these disparities compare with those in previous years, that is whether they are the same or greater.

A study published by Amplify (2021) analysed the effects of COVID-19 disruptions on literacy skills by comparing DIBELS 8th Edition (a battery of one minute fluency measures) benchmark data from the middle of the school year 2019-20 and 2020-21. The findings suggested that in 2019-2020, 28% of U.S. kindergarten students (age 5/6) were in the lowest category of early literacy skills; in 2020-21 it was 47%.

Learning losses due to COVID-19 disruptions were disproportionately concentrated among Black and Hispanic students – particularly the youngest ones (kindergarten and grade 1). In 2019-20, 27% of Black kindergarten students were in the lowest category of early literacy skills, compared with 54% in 2020-21. Similarly, 34% of Hispanic students were in this category in 2019-20 compared with 59% in 2020 to 2021.

Domingue et al. (2021) used data from an oral reading fluency (ORF) assessment administered in over 100 U.S. school districts to assess the effects of the pandemic on reading development. The findings suggested that in spring 2020 students showed limited or no growth in ORF gains relative to previous years; however, it is worth noting that, due to the impact of COVID-19, only a fraction (roughly 30%) of students were assessed in the spring.

In autumn 2020, there was a far more representative sample of students; in this case ORF growth approached levels observed in previous years, although autumn gains were insufficient to recoup spring losses. At this time, around 5% to 10% of students were missing from the sample: possibly those experiencing the most substantial impact to their learning. The biggest impact on ORF growth was observed for students in grades 2-3, as well as students in lower achieving districts. These students seemed to be falling further behind and, as gains appeared to be unequal across schools and districts, the effects of COVID-19 on oral reading fluency may be introducing new skills gaps across students.

Pier et al. (2021) studied 6 districts in California that administered the MAP assessment and 12 that administered the STAR assessment by comparing outcomes from autumn 2019 to autumn 2020 with the previous 3 years. The findings of this analysis suggested a significant learning loss in English Language Arts (ELA) and mathematics, with students in earlier grades most affected.

Students from a socio-economically disadvantaged background and English language learners (ELLs) were found to have significantly more learning loss compared with previous years. Again, the findings may exclude the worst-affected students, as only students who were present for assessments were measured.

Kogan and Lavertu (2021) found average achievement on the Ohio Third-Grade English Language Arts (ELA) assessment declined by approximately 0.23 standard deviations between autumn 2019 and autumn 2020. This is roughly equivalent to one-third of a year’s worth of learning. It was also found that Black students experienced test score declines that were nearly 50% larger than White students, and students from socio-economically disadvantaged backgrounds were also found to be more affected than those not identified as such.

It seemed the impact on student achievement was, to a large degree, tied to how significantly COVID-19 had affected unemployment in the area; the largest test score declines were in areas with the sharpest job losses. As well as this, districts that began the academic year using remote teaching saw more declines in achievement than those using in-person teaching or a combination of the 2. Most districts experienced declines to some extent, but just over 10% of districts experienced no test score declines during the pandemic, suggesting a decline resulting from COVID-19 disruption is not inevitable.

Evidence from other events: the impact of variation in school time on learning

Evidence summary

This section aims to estimate the impact that the pandemic might have on learning loss by considering what is known about the impact of changes to teaching time, in other circumstances, on student achievement. Variation in the time spent in school has been studied in several different contexts with consideration of the impact that this has on student skills and achievement. In general, a reduction in the time spent in school is associated with a reduction in student test scores.

Student absences seem to have more of an impact on this than school closures, perhaps because teachers face more challenges helping individual students catch up than the whole class. A consistent finding, demonstrated in each of the various contexts, is that reduced teaching time has more impact on mathematics than literacy, perhaps because students can develop their literacy skills outside of school more easily. Another common finding is that students from lower socio-economic backgrounds are most impacted by changes to teaching time.

Teaching Time

The notion of time as a resource for learning, and something required to achieve academically, is often considered to have first been conceptualised by Carroll (1963), whose model of school learning included time and opportunity to learn as a key contributor to the degree of learning achieved. Following this model, the relationship between teaching time and student achievement has been studied at length, most often using data on teaching time and students’ test scores. Several international comparisons have generally found a marginal increase in teaching time to have a positive impact on student achievement (Baker et al., 2004; Barro & Lee, 2001; Gromada & Shewbridge, 2016; OECD, 2011; Scheerens et al., 2013; Wößmann, 2003; Yeşil Dağlı, 2019).

Several countries have seen adjustments to the number of classroom hours required each day, or the number of days required in a school year, which have been used to assess how changes to teaching time impact academic achievement. Jensen (2013) studied the effects of a reform in Denmark, which led to changes in classroom hours by 2.2% to 2.3% in literacy and mathematics.

While there were no significant effects on student achievement in literacy, it was found that one additional hour of classroom time for mathematics per year increased test scores by 0.0021 standard deviations. Although this is a relatively small increase, it does suggest mathematics achievement is sensitive to even minor changes to teaching time. Marcotte (2007) suggested the reason mathematics is more affected by changes in teaching time, is that literacy teaching also takes place at home.

Lavy (2012) studied changes to teaching time in Israel, finding increased time spent on mathematics, science, and English positively impacted test scores in these subjects. There was a much smaller effect when increasing teaching time in Hebrew. As Hebrew is most people’s first language in Israel, the additional school time would be unlikely to make much difference to learning outcomes because, proportionally, it is very small.

Generally speaking, increased achievement in a given subject was the result of increased time spent on that particular subject alone; there did not seem to be crossover effects whereby increased time in one subject improved test scores in others. One exception was that there were modest increases in science scores from increased teaching time in mathematics. It appears that a reduction in mathematics teaching could have the biggest impact on student achievement, in mathematics and possibly other subjects with a mathematical basis.

In Germany, Mandel and Süssmuth (2011) used cross-state variation to study differences in classroom hours and found one additional hour of teaching time per week increased PISA^{[footnote 13]} test scores by around 0.12 standard deviations. Again, it was found that the effect was more prominent in mathematics but still sizeable in reading. In addition to this, they suggested that, while it seemed all students benefit from increased teaching time, it was most beneficial for lower-achieving students.

Cattaneo et al. (2016) also found that the effectiveness of teaching time varies substantially between students of different abilities; however, their findings suggested that higher ability students benefitted most from an additional hour of teaching. This study built on the research conducted by Lavy (2015), who analysed the effect of cross-national variation in hours of teaching on test scores, and found an additional hour of teaching time in a subject per week over the school year was associated with a gain in test scores of 0.06 standard deviations; the effects were even larger for children from lower socio-economic backgrounds.

It seems that the socio-economic background of students may be one of the most influential factors in determining the impact of teaching time on achievement. Several countries have introduced programmes of increased teaching time and seen a positive impact on student achievement, especially for students from low socio-economic backgrounds.

In South America, several countries have lengthened school days, from half-days to full-days, often finding that the effects have the most impact on the poorest students and those in rural areas (Bellei, 2009; Bonilla-Mejia, 2011; Cerdan-Infantes & Vermeersch, 2007; Garcia et al., 2013; Hincapie, 2016). Hincapie (2016) also found the positive effect of full school days was more beneficial for 9th grade than for 5th grade, suggesting age may also have an effect on the impact of teaching time.

Thompson (2021) studied schools in the United States that have changed from a traditional 5-day school week to a 4-day week with slightly longer hours. This led to a reduction in around 3 and a half hours of teaching time each week. The results suggested a negative impact on student achievement, with declines between 0.044 and 0.053 standard deviations in mathematics and 0.033 and 0.38 in reading.

In addition to this it was again found that low-income students were most affected, but also that boys were more impacted than girls. However, in the case of this study there may be other factors at play contributing to these declines, other than the reduction in teaching time. Longer days may have had negative effects on students’ ability to concentrate or there may have been more impact from how students spent their day off, which may also explain the differences between different groups of students.

Another group of students that may be more impacted by changes to teaching time are those from ethnic minority backgrounds. In the United States, Desimone and Long (2010) found the number of minutes spent on mathematics instruction was associated with achievement for Black students and students from lower socio-economic backgrounds. They suggested the importance of time was that it gave “opportunity to learn” and increased exposure to academic content.

It was estimated that increasing teaching time in mathematics could narrow, or close, the achievement gaps between Black and White students, as well as between students of different SES. However, Blank (2013) explored teaching time in science, considering family background and income, finding that teaching time alone could not account for achievement gaps.

Perhaps teaching time can reduce these gaps to some extent, not only through increasing test scores but also providing better opportunities for students. Garcia et al. (2013) found lengthening half days in Colombia to full days had a positive effect on rates of early dropout and grade repetition, which may be more likely to affect students from more disadvantaged backgrounds. Perhaps it is not just the time spent in a classroom that influences achievement, but being in the physical environment of the school.

Garcia et al. (2013) suggested increased time in school was beneficial for students, due to increased teaching time, but also due to the effects of adult supervision and socio-emotional support. Farbman and Kaplan (2005) suggested increased teaching time may also benefit achievement through increased interaction and positive relationships between teachers and students. This may imply that increased time in the classroom does not have a positive impact on achievement solely due to increased time learning.

Most studies of teaching time can only include measures of classroom hours, as teaching time is a complex concept to measure accurately due to the many factors influencing the productivity of teaching time. It may be engaged time, or actual learning time, that is the most influential resource on academic achievement (Gromada & Shewbridge, 2016; Scheerens et al., 2013).

Rivkin and Schiman (2015) found that teaching time did raise achievement; however, this varied depending on the quality of the classroom environment. Schools with low-quality classroom environments saw a much smaller benefit from teaching time. It seems engaged time and actual learning time are necessary to improve academic performance (Cotton & Savard, 1981), which may not necessarily be impacted just by increasing the number of teaching hours.

The teaching time that has taken place during the pandemic may have differed greatly between schools, classrooms, and individual students. It seems that the quality of teaching that has taken place, or will take place to recoup learning losses, is of great importance and perhaps of greater importance than the number of teaching hours.

Absences

The quality of teaching time may be determined mostly by the teacher. In instances of teacher absence, teaching time may be lost or reduced in quality. Clotfelter et al. (2009) found that having a teacher with 10 additional sick days per year was associated with a decline of 0.017 standard deviations in mathematics achievement and 0.009 standard deviations in reading. A far larger effect was found in low-achieving students, where 10 days of teacher absence saw a decline of 0.033 standard deviations, compared with 0.003 for above average achievers.

The impact of teacher absence may be complex, as a teacher being absent frequently may be associated with their lack of effort or skill. In addition to this, the levels of specialist knowledge or teaching experience of substitute teachers – and thereby their effectiveness – may vary.

Miller et al. (2007) suggested teacher absences may affect achievement through: (i) a reduction in teaching time; (ii) disruption of regular routines and procedures or consistent practices across classrooms throughout the school; and (iii) teachers losing out on collaborative time with other teachers, further impacting the achievement of their students, as well as perhaps other students taught by teachers’ colleagues.

Student absence is another way in which teaching time is reduced. Studies of student absences report higher estimates of learning loss and reduced academic achievement for a given reduction in teaching time.

Goodman (2014) showed that for every 10 days of student absence in Massachusetts, mathematics scores reduced by 0.05 standard deviations. Aucejo and Romano (2016) found that 10 days of absence due to flu in North Carolina led to a decline in mathematics scores of between 0.05 and 0.10 standard deviations, depending on grade, with the effects persisting to some extent in subsequent grades. Similarly, Cattan et al. (2017) used sibling differences in data from Sweden to isolate the effect of school absence. They found that 10 days of absence over a school year in elementary school led to a reduction in grade point average of 0.033 standard deviations: a moderate and persistent adverse effect, which faded out by mid-adulthood.

The studies using absence data at first seem to suggest that estimating the effect of learning loss from variation in teaching time may provide optimistic estimates. However, both Goodman’s and Aucejo and Romano’s studies shed further light on this.

Weather-related school closures

Goodman’s study, in addition to demonstrating the impact of student absences, found that school closures – in this case due to heavy snowfall – had no significant impact on test scores (Marcotte, 2007). Aucejo and Romano explicitly compared the effects of: (i) absences; and (ii) variation in the length of the school calendar on test score performance. They found that, while extending the school calendar by 10 days increased mathematics and reading test scores by 0.017 and 0.008 standard deviations, respectively, a similar reduction in absences led to increases of 0.055 standard deviations in mathematics and 0.029 standard deviations in reading.

Presumably, where teaching time varies, content coverage is still broadly complete – albeit it more or less thorough – because the reduced time is planned. Likewise, teachers can work and re-plan around emergency school closures to some degree. Individual absences, on the other hand, leave gaps in coverage. If these gaps are not addressed, they may widen where subsequent learning needs to build on the missing knowledge, skills, or understanding.

Lazear (2001) considers the effect of classroom disruption on student achievement, assuming that one student can reduce the ability of all other students in the class to learn through disruption to teaching. This disruption can take the form of asking questions when other students already know the answer; therefore, a student who has been absent and subsequently requires additional attention from the teacher may reduce the ability of all students to learn.

Presumably, the more students who are absent at one time, the more potential for disruption and the greater the challenge for teachers to coordinate teaching to meet the needs of all students. However, it is possible that instances of school closures where all students are absent at the same time present fewer negative impacts than instances of individual or multiple student absence. This is because all students are learning at the same rate and there is less disruption to teaching while students catch up on the lost teaching time.

There is nonetheless some evidence that unscheduled school closures can still negatively affect student performance. Marcotte and Hemelt (2008) found school closures due to snowfall negatively affected performance on mathematics and reading assessments. They observed the largest effect on students in lower grades, suggesting this was due to the cumulative teaching time that students in higher grades have had, including days that had been added on to the end of the school year to make up for snow days over the course of schooling.

Similarly, Hansen (2011) considered weather-related school closures and the practice of making up these days by adding them to the end of the year, after tests have been sat by students. It was found that more teaching time prior to test administration increased student performance.

In support of this, Carlsson et al. (2015) analysed the effect of the test date for a battery of cognitive tests^{[footnote 14]} on test scores and found that 10 days of additional learning time increased test scores of knowledge or crystallised intelligence by 0.01 standard deviations. The impact of increasing the number of days of teaching before test administration, rather than after, may explain the impact of weather-related school closures as time is not made up until after the tests have taken place.

Evidence comparing weather-related school closures and student absences suggests that, in the context of the pandemic, it will be challenging to fill the gaps in knowledge for all students, especially those with an increased number of individual absences. There will be great variation between schools, in some cases experiencing whole school closures, the absence of certain classes or year groups, individual absences, and multiple absences of students within the same class. Identifying knowledge gaps for individual students may be an important stage of mitigating lost learning, especially for those who have experienced the highest rates of individual absence.

Teacher strikes

A number of studies have addressed the impact on students’ learning of prolonged or recurrent school closures due to teacher strikes (Darmody et al., 2020). Teachers in the French community of Belgium were on strike between May and November in 1990. Belot and Webbink (2010) exploited the political division of Belgium – distinct French and Flemish communities, but with similar educational institutions – to estimate the long-term effects of the strikes. They concluded that the strikes reduced educational attainment, increased class repetition, and led to a significant reallocation of students to a lower level of higher education.

Similarly, Jaume & Willén (2019), investigating the effects of teacher strikes in Argentina on students’ long-term educational attainment and labour market outcomes, concluded negative labour market effects are driven, at least in part, by a reduction in educational attainment. Furthermore, the children of adults who were exposed to teacher strikes during primary school also experienced adverse educational attainment effects.

Baker (2013) studied 11 strikes taking place between 1989 to 2006 in Ontario and found a significant negative effect on test score growth between grades 3 and 6 of grade 6 teachers being on strike for in excess of 10 teaching days (teacher strikes in grades 2 or 3 had on average a small, negative, and statistically insignificant effect). The impact of a strike was primarily in the year in which it took place; a year later, there was no impact for most subjects other than mathematics.

Johnson (2009) also considered the teacher strikes in Ontario. He analysed the effect of a 10-day strike and found the proportion of students achieving the provincial standard on standardised tests reduced in mathematics. However, this effect was only observed in sixth grade students; when considering younger students, in third grade, there was no impact.

In Chile, during the 2011 to 2013 student protests, school occupations took place, ranging from a few days to almost 50 days of teaching time lost. The average effect for those missing 10 consecutive weeks – around 50 days – was a decrease of 6.97 percentile points in mathematics tests (0.24 standard deviations) and 2.48 percentile points (0.085 standard deviations) in language tests (Montebruno, 2020). While these may seem relatively high estimates, it is important to note that these protests lasted around 50 days and were demanding changes to the education system in Chile, perhaps indicating a reduction in effective teaching time prior to the school closures.

This study also found that mathematics was more affected than language skills, which may continue to develop without formal training. As well as this, the older cohorts (fourth to tenth grade) were more affected than the younger ones (fourth to eighth grade). In fact, it was shown there may have been no real effect on educational outcomes for younger students. These studies suggest that older students may be more affected by school closures, while younger students may not be affected at all.

Wills (2014) studied the 2007 public service strike in South Africa, finding that for students in more privileged schools, where strike participation was less common and duration of strikes were more limited, there was little to no effect on performance. However, in disadvantaged schools, where participation in strikes was widespread and long in duration, strike activity was detrimental to learning. The evidence suggested that the more marginalised students, in terms of both socio-economic background as well as academic ability, were most negatively affected by strikes.

In the United States, Zwerling (2008) studied strikes in Pennsylvania and found no relationship between teacher strikes with a 12-day average duration and performance on the Pennsylvania System of School Assessment test. It was suggested this may be due to the requirement for lost days to be made up, so that each school reaches the minimum number of teaching days within the academic year. This suggests the impact of school closures may differ between countries depending on their approach to making up for any time lost.

Evidence of the effects of teacher strikes on student learning is very much aligned with those in the summer learning loss and teaching time literature (Busso & Camacho Munoz, 2020; Warren & Wagner, 2020). Long strikes lead to declines in students’ attainment in mathematics, reading and writing, and have longer-term consequences for progress in education, employment, and earnings. The impact may be higher for older students, especially when schools do not replace the lost lessons.

However, the occurrence of teacher strikes suggests some level of dissatisfaction with the education system before school closures, possibly due to factors that already affect academic achievement. This may be a key factor that is not present in the context of the pandemic; therefore, these estimates may not give a precise picture of the impact on student achievement we should expect to see in the current context.

Natural disasters

Natural disasters may shed some light on the impact of absences or school closures that involve some form of catch up or alternative forms of teaching and learning that weather-related closures and school closures resulting from teacher strikes do not. In particular, there may be changes to everyday life and traumas experienced by students and teachers that may be more comparable to those experienced during the pandemic.

Students affected by the earthquake in Christchurch, New Zealand did not suffer much reduction to their end-of-year test scores (Beaglehole et al., 2017). In this instance, education was disrupted rather than schools closing completely, as students in the most affected areas co-located to less-damaged schools for a number of months.

A very different conclusion emerged following the earthquake in Pakistan in 2005; schools in some parts of the country closed for 14 weeks. Andrabi et al. (2020) report that, 4 years after the events in Pakistan, students who were directly affected by the earthquake were 1.5 years behind peers living in unaffected regions. This represents an approximately fourfold expansion of the immediate learning loss from school closures.

The worst losses were incurred by students from the most deprived backgrounds. The students whose mothers had the most education – the 16% of students whose mothers had completed primary school – had caught up with their unaffected peers within 4 years. This demonstrates that lost learning can be restored, that if it is not, students may fall behind further over time, and that the most vulnerable students may require the greatest help.

A study assessing the neurodevelopmental effects on resident children of the Great East Japan Earthquake found that verbal IQ (including information, arithmetic, and vocabulary sub-scores of the Wechsler Intelligence Scale for Children) at 7 years of age was significantly lower in the post-disaster group than in the pre-disaster group. There were no significant differences in performance IQ, full-scale IQ, or autonomic nervous indicators (assessed using electrocardiography) between the 2 groups (Tatsuta et al., 2015).

Sacerdote (2012) examined the academic performance of public school students affected by Hurricanes Katrina and Rita in the United States. Students who were forced to switch schools due to the hurricanes experienced sharp declines in test scores in the first year following the hurricane; after a median loss of 5 weeks’ schooling, evacuees’ scores dropped between 0.07 and 0.20 standard deviations relative to other Louisiana students.

By the second and third years after the disaster, Katrina evacuees displaced from the worst hit areas (which included the worst-performing schools) appeared to benefit from the displacement, experiencing a 0.18 standard deviation improvement in scores. This has been attributed in part to students being relocated to much better schools, which is supported by the finding that evacuees from less-deprived areas did not to make up for the decline in the subsequent 2 years.

Ward et al. (2008) also considered the impact of hurricane Katrina, finding the children affected experienced stress reactions, increased disruptive behaviour, anxiety and adjustment, all of which are a typical response to trauma. These reactions can impact cognitive functioning and can lead to attention deficits that may affect student achievement beyond the initial event.

Similarly, Ward and Shelley (2008) also suggested children affected by disasters experience behavioural problems, which can disrupt the educational environment. This might explain why some students caught up over time, while others did not. If some students are badly affected in a behavioural sense this may lead to reduced achievement some time after the disaster.

De Pietro (2018) examined the effect of the L’Aquila earthquake on the academic achievement of students at the local university. This caused significant disruption to daily life and caused psychological problems for many. In terms of academic achievement, it was found students were less likely to graduate on time and had an increased probability of dropping out during the academic year.

Alban Conto et al. (2020) suggested the impact of natural disasters, or pandemics, may not only affect students through school closures, but also through the accompanying economic disruption. This may lead to further inequalities between students from different socio-economic backgrounds, as those at the lower end are most affected.

The evidence from these natural disasters suggests that, while there may be some more immediate effects on academic achievement, students may be able to catch up on the learning that was lost. Considering these findings in the context of the pandemic, those from lower socio-economic backgrounds and those more psychologically impacted may be the most vulnerable and perhaps experience longer-term consequences for their academic achievement.

Previous pandemics

As with natural disasters, previous pandemics may fit more closely students’ and teachers’ experiences of the current pandemic, particularly regarding changes to everyday life and health emergencies and bereavement. Selbervik, (2020) considered the 2013 to 2014 Ebola outbreak in West Africa, finding many students did not return to school following school closures. This was mainly due to the inability to pay school fees or the need to generate income (Hoogeveen & Pape, 2020).

Similarly, Meyers and Thomasson (2017) studied the 1916 polio pandemic in the United States and found those living in areas harder hit by the outbreak experienced decreases in educational attainment depending on age. Those aged 14 to 17 were found to have fewer years of schooling due to the alternatives to education available to those aged 14 and older. When schools closed, students of working age often chose to not return to education, as they were able to start working instead.

Although it is important to note that these findings are reflective of the context of very different countries or the distant past; in both scenarios, the ability to drop out of education to pursue a career was a viable alternative to schooling. In countries such as England, where there is a legal requirement to attend school until the age of 16 and all students have the option to attend school for free, these findings may not apply to students before the end of compulsory education.

Goulas and Megalokonomou (2016) studied the effects of a policy in Greece during the swine flu pandemic. The policy relaxed rules around student absence, increasing the absence allowance by 30% without penalty and removing the need for students to provide a doctor’s note. Interestingly, the ablest students took more absences and subsequently performed better at the end of year examinations. This suggests students who are able to learn outside of the classroom may benefit from relaxed school attendance, and may be negatively affected by strict attendance policies. However, those at the lower end of the ability distribution saw decreases of 0.02 standard deviations when they missed an additional 10 hours of class.

This suggests the amount of teaching time different students need in school may vary. As well as this, it suggests that schools play an important role in equalizing opportunities between students. Time spent in school may pull down some higher achieving students who are able to more effectively learn outside of school, but time spent being taught also brings up students with lower ability, who require this time to achieve academically. This suggests students of lower ability may be the most vulnerable in the current pandemic and that they may require (the most) additional support.

García & Weiss (2020) cite further evidence of the relationship between teaching time and student attainment. They highlight the risk to students whose pandemic-related school closures may lead to chronic absenteeism, which is associated with falling behind in school, achieving lower grades and test scores, exhibiting behavioural issues, and, ultimately, dropping out (Balfanz, 2016; Gottfried & Ehrlich, 2018; U.S. Department of Education, 2016). In the context of the current pandemic, it may be important to focus support on students with the highest rates of absenteeism to avoid these longer-term consequences.

Summer learning loss

Teaching time is interrupted every year during the summer, often resulting in what is referred to as ‘summer learning loss’ or ‘summer slide’. Cooper et al. (1996) undertook a comprehensive systematic review of estimates of this. They drew on evidence from 39 studies (mainly from the United States) and reported a meta-analysis of the results from 13 of them. Their findings suggested that the summer holidays led to an average loss of learning equivalent to 1 month – or 0.10 standard deviations relative to spring test scores.

They identified subject differences, with mathematics being more likely to decline than reading, and computation and spelling being particularly affected. They also found that the effects depended on a student’s SES, with middle class students likely to gain on reading recognition tests, while lower-class students lost on them.

Similarly, Alexander et al. (2001) found comparable school-year verbal and quantitative achievement gains for upper SES status and lower SES children; however, the summer gains – when children are out of school – showed large disparities. While upper SES children’s skills continued to advance during the summer holidays (albeit more slowly than during the school year), lower SES children’s average gains were flat. Alexander et al. suggested that the increase in achievement gap across social lines during the primary grades can be attributed mainly to the out-of-school environment, that is influences situated in home and community.

A meta-analysis of studies of learning loss over summer holidays for primary-aged children by the Education Endowment Foundation (2020a) concluded that school closures will widen the attainment gap between disadvantaged children and their peers at an average rate of 0.022 standard deviations per month, likely reversing progress made to narrow the gap since 2011. The median estimate indicates that by September 2020 the gap would widen by 36%, although the range of plausible estimates – 11% to 75% – was large. The study also caveats that the projections might be an overestimate for those students who have returned to school or who will be returning to schools earlier than September, and for those students in schools where remote schooling has worked well.

Recent criticism of – and debate over – the methodological rigor of the aforementioned studies has raised questions over the evidence for summer learning loss (Alexander, 2020; Education Endowment Foundation, 2020a; von Hippel, 2019a, 2019b; von Hippel & Hamrock, 2019). More recent studies using nationally representative data showed smaller declines (2 weeks of learning) – or even small gains – during the summers following kindergarten and first grade, which has been called ‘summer slowdown’ (Kuhfeld & Tarasawa, 2020; von Hippel et al., 2018).

Taking account of the measurement issues, Atteberry and McEachin (2020) documented the huge diversity in learning loss, finding that a relatively small group of students tend to account for a lot of the loss. They found that, depending on grade, the average student loses between 17% and 28% of school-year language gains over the summer. In mathematics, the average student loses between 25% and 34%.

Although students vary in how much they learn during the school year, most students do make learning gains during this time. This differs in the summer; while some students maintain their school-year learning rate, others lose nearly all of their school-year progress. It seems these losses accumulate for the same students over time and add up to a sizeable impact on where students end up in the achievement distribution.

Kuhfeld (2019) analysed data from more than 3.4 million students in all 50 states who took NWEA MAP reading and mathematics assessments. The findings suggested 70% to 78% of students lost ground in mathematics during the summer and 62% to 73% lost ground in reading. These findings also indicate 22% to 38% of students show gain during the summer, meaning learning loss is not inevitable.

The most interesting finding of this study was that the main predictor of whether a student would experience summer learning loss was not income – which many previous studies indicate is the most influential factor – but the size of gain the student had made during the previous academic year. It seemed the more students had learned during the school year, the more likely they were to lose ground during the summer. In contrast, ethnicity only explained 1% of summer loss and, while higher-poverty schools showed significantly more loss at certain points, these differences were small.

Kuhfeld and Tarasawa (2020) note that, despite the controversy about the magnitude of summer learning loss, 3 trends are consistent across seasonal learning research findings (Quinn & Polikoff, 2017):

1. achievement typically slows or declines over the summer months
2. declines tend to be steeper for mathematics than for reading
3. the extent (proportionally) of loss increases in the upper grades

Despite these general trends, it seems the degree to which students lose ground during the summer can vary by data source, grade level, and subject (Kuhfeld et al., 2019; von Hippel & Hamrock, 2019).

Another aspect of summer learning loss to consider is the ability of students to regain the losses they may experience over the summer. Several studies have assessed how effective summer programmes and interventions have been in alleviating summer learning loss.

Allington et al. (2010) tested whether providing elementary school students from low-income families with a supply of books would offset summer reading setback. Outcomes on the state reading assessment showed a significant benefit of providing access to books for summer reading, the largest effects being observed for the most economically disadvantaged students.

McCombs et al. (2011) identified 13 experimental or quasi-experimental studies of 9 summer learning programmes published since 2000. In general, these studies suggested that summer learning programmes can mitigate summer learning losses and even lead to achievement gains.

Longitudinal studies conclude that the effects of summer learning programmes endure for at least 2 years after the student has engaged in the summer programme. This suggests that learning losses can be regained, which is encouraging in the context of the pandemic. However, it is also important to note that not all programmes yield positive results. To be effective, students need to attend regularly, and programmes must be high-quality and aligned with student needs.

It is also important to note, when considering summer learning loss, that the summer holiday is intended for students and teachers to take a break and recharge their batteries. This may explain differences in these findings compared with the effects of school closures, as these contexts are very different (Hattie, 2020). This is important to consider in applying estimates from other contexts to the pandemic. The amount of learning loss and impact on academic achievement is dependent on the expectations of students during school closures.

The early school closures in England may have been most similar to summer learning loss, as the arrangements for remote learning were not well established, and students and teachers may have seen the closures as a short-term interruption. The year groups whose formal summer examinations were cancelled (primarily years 11 and 13) are most likely to have disengaged during this period, as their summer term would normally be dedicated to examination revision.

Student dips in performance

Kuhfeld’s (2019) study of learning loss identified an additional factor that seemed to have a large impact on summer slide. She found that the summer between fifth and sixth grade, which corresponds to the transition between elementary and middle school for many students, represented the largest summer drop in mathematics, with 84% of students demonstrating summer slide.

These dips in attainment due to transition, as well as other factors affecting students that are not directly relevant to teaching time, may shed further light on the groups who would be most affected by learning loss. This may be especially relevant in the pandemic, as students transition from traditional classroom learning in school to remote learning at home.

An international review of attainment dips conducted by Whitby et al. (2006) found many countries reported a dip in student performance the first year after transferring from primary to secondary education, or between upper and lower secondary education, depending on the structure of the education system within the country. In England this is often referred to as a ‘key stage 3 dip’ (Pepper, 2007). This may be due to the required period of adjustment, which disrupts the learning and achievement of students.

Certain groups may be more impacted by times of transfer. In the United States, studies have found a negative impact on academic performance after transferring schools (Crockett et al., 1989; Rice, 2001; Simmons et al., 1987) with Earl et al. (1996) finding Black and Hispanic students were more impacted by the adjustment to secondary school than students from other ethnic groups.

The international analysis by Whitby et al (2006) also found that some countries suggested students’ ethnic and social group was related to potential dips. In addition to these groups, students from lower socio-economic backgrounds, foreign students who had recently arrived in the country, and those who were not native speakers were also found to be vulnerable to dips.

In Australia, Hill and Russell (1999) found evidence for a dip that had a greater and more persistent impact on boys than girls. This was also found in Whitby et al.’s (2006) study, with the majority of countries reporting that boys were more prone to dips in performance than girls. In Germany it was suggested dips were experienced by both boys and girls, but that they took place at different times, with girls being more affected at a slightly older age than boys.

In the review conducted by Whitby et al. (2006) it was also suggested that the more traditional, ‘academic’ subjects were found to be the most affected by times of transition; some countries also reported that language subjects were affected. However, it is important to note that these academic subjects are the most commonly assessed, with more data on student performance available. Therefore, this may explain why dips in these subjects are easier to identify than other subjects.

Considering more general reasons for attainment dips, Whitby et al (2006) also noted that this stage of early secondary school may affect performance due to changes to teaching and learning, school organisation, teenage maturation issues, other social circumstances, and perceived distance to the leaving examinations.

It is possible that personal characteristics are the most important factor in accounting for difficulties in adjustment following school transfer (West et al, 2008). Nisbet and Entwistle (1969) found a transfer process was particularly disruptive to academic performance for less socially mature students, who were more likely to suffer a setback in academic progress after transfer. This suggests poorer school transition experiences may predict lower attainment. It seems that times of transfer and transition can have disruptive effects on learning; perhaps dips in performance will be especially apparent for students who have found adjusting to changes challenging.

Summary of the international evidence on learning loss

The evidence available to date suggests students affected by school closures in the second quarter of 2020 may have lost 2 to 3 months compared with the academic milestones their cohorts would be expected to reach. Losses appear greater in mathematics than in reading, with disadvantaged populations potentially bearing the brunt of losses.

Some of the assessment data were collected straight after school closures or even while students were at home. This raises some issues to keep in mind. Firstly, summer learning loss research suggests that students may (re-) learn at the greatest rate in the autumn term, so earlier testing may have suggested greater learning loss than later testing would.

Secondly, some studies report that assessment data were collected while students were studying from home. This may have led to students cheating, including receiving help from others in the home. On the other hand, it may have reduced the effort that some students put into the assessment, or the suitability of the conditions under which they took the assessment; the same may be true of assessments taken in school under COVID-19 restrictions. A combination of these effects might lead to the progress of some students being overestimated and others underestimated, potentially exaggerating the differences between certain groups of students.

Findings from across a range of non-COVID-19-related contexts suggest that there is a positive relationship between time spent in school and student achievement. In contexts where time in school is reduced, instances of school closures or teacher absence, there appears to be some decline in student achievement.

The greatest decline from lost teaching time was observed in cases of student absence. It has been suggested this is due to these students either not catching up on their return, or disrupting normal teaching when they do catch up. On the other hand, the lower levels of achievement of those students absent most often may simply reflect their attitude and effort.

A clearer picture of the subjects most impacted was garnered. It has been found consistently, across all contexts, that the subject most affected by changes to teaching time is mathematics. English, or subjects where skills may develop at home, are less impacted.

In terms of the students most affected by changes to teaching time, it appears those from lower socio-economic backgrounds may be most sensitive. Schools act as an equalizer, giving students from all backgrounds the same opportunities, which may explain why these students are most impacted by changes to the amount of time spent in school. The age at which students are most affected is less clear-cut.

For every 10 days that a student is absent from school there is a decline of around 0.03 to 0.10 standard deviations in mathematics, with smaller declines in English or reading. Evidence from other events did not imply such dramatic estimates of decline.

Hattie (2020) considered several of the types of events mentioned in this section, suggesting that, while there may be effects of losing time spent in school, in general these effects are low and perhaps not something to worry about. He pointed out that countries with the longest school days, such as Australia and the United States, have more time in school, even with an entire term of 10 weeks removed, than many other countries who score better on the PISA test. Hattie (2020) argues that it is not the time in class, but what is done in that time, that matters most.

International evidence on learning remotely

In many countries around the world, when schools closed, technological solutions allowed students to continue their education. Depending on the prevalence of different technologies, this ranged from postal services, through radio and television programming, to teacher-led learning online (OECD, 2020f). In this section, we consider evidence from other countries relating to the effectiveness of online learning: the approach most applicable to remote learning in England.

Digital technology – commonly referred to as EdTech in this context – can make an education system more resilient to the kind of disruption caused by the COVID-19 pandemic (Haßler et al., 2020), (where available) enabling students to continue learning during school closures (David et al., 2020). We now focus on evidence of how the effectiveness of online learning may vary between groups of students according to characteristics such as age, race, socio-economic background, and special educational needs and disabilities.

Efficacy of online learning

Evidence summary

Online learning has generally been associated with poorer outcomes than traditional classroom teaching, although its effectiveness varies depending on implementation. Teaching quality, rather than the medium of delivery, is fundamental to successful learning, and some of the shortcomings of online learning are likely to be the result of hurried implementation on an unprecedented scale. The medium is currently – if not inherently – more or less suitable for certain tasks and types of content. Limited interactions with peers and teachers may affect motivation and learning, particularly during the pandemic where this extends to life beyond school.

Evidence regarding the efficacy of online learning

Recognising the potentially severe consequences of prolonged school closures, governments across the world put in place measures to enable the education of children and young people to continue, with various degrees of success (Dreesen et al., 2020). The main approach has been to move teacher-led learning online (Darmody et al., 2020). While, in general, online learning has been associated with poorer outcomes compared with traditional teaching (Shank, 2019), during the current school closures, it has provided a means of ensuring some continuity of learning (Galvis & McLean, 2020).

Little is known about the effectiveness of learning at home for the entire student population, although there are indications that teaching and learning for many students in the spring 2020 lockdowns was poor to non-existent (Hanushek & Wößmann, 2020). Hanushek & Wößmann cite the example of early tracking data from an online mathematics application used in some U.S. school districts prior to COVID-19, which suggest that the learning progress of students has suffered a strong decline during the crisis, especially in schools in low-income areas (Chetty et al., 2020).

More positively, Bubb and Jones (2020) reported the findings of a survey of students’, parents and/or carers’, and teachers’ views of remote learning in one Norwegian municipality, gathered in April 2020 (the peak of the first COVID-19 lockdown). Adaptation happened very quickly, and remote learning was well received by students and parents, with more creative learning, better progress, more useful feedback, and greater student independence. School leaders wanted to implement changes based on the experience of remote learning.

It is worth noting that Norway has one of the world’s highest incomes per capita and relatively low wealth inequality; the apparent ease of implementation may not generalise well to many countries. Furthermore, no hard performance measures were reported in the study, meaning there is no objective evidence that online learning was as successful as respondents subjectively felt it to be.

A national study from Stanford University’s Center for Research on Education Outcomes (CREDO) found that students in online^{[footnote 15]} charter schools learned significantly less on average than their peers taught in a classroom; however, the effectiveness of online tuition varied by state. Students who attended online charter schools in the 3 top-performing states learned as much as students in traditional classrooms; students who attended online charter schools in the lowest-performing half of states ended up further behind than when they had started the year (Woodworth et al., 2015). Online tuition is neither inherently effective nor ineffective; rather, it depends on implementation.

Since the initial school closures, internationally, numerous reviews of the existing evidence regarding the effectiveness of online learning have been published. The main findings of some key reports follow.

The OECD (2020d) summarise what is known about the impact of digital learning on students’ performance. Simply providing access to, or even using, digital technologies does not necessarily lead to better academic results (Escueta et al., 2017). Several studies of the impact of introducing technology for learning have reported disappointing effects, ranging from negative (Angrist & Lavy, 2002; Leuven et al., 2007), through neutral (Fairlie & Robinson, 2013), to negligible (Checchi et al., 2019).

Unsuccessful implementation has been associated with substituting technology for more effective traditional teaching (Bulman & Fairlie, 2016; Falck et al., 2018), whereas success has been associated with using technology to complement traditional teaching, for example extending study time and enhancing student motivation (Fleischer, 2012; Peterson et al., 2018). Co-operative and collaborative forms of learning are somewhat restricted, and this is especially true for hands-on learning in practical and performance-oriented subjects (Petrie et al., 2020).

A technical report by the Joint Research Centre (JRC), the European Commission’s science and knowledge service, concludes that distance learning is key to continuing education during school closures, but students are, on average, likely to experience learning loss during the lockdown. They identify 3 main causes:

• students learning remotely tend to spend less time learning than when schools are open
• many students confined to their home may feel stressed and anxious, negatively affecting their ability to concentrate on schoolwork
• physical school closure and the lack of in-person contact may make students less externally motivated to engage in learning

They estimate for France, Italy, and Germany that students will lose between 0.0082 and 0.023 standard deviations in test scores each week schools are closed (Di Pietro et al., 2020).

The Education Endowment Foundation’s rapid evidence assessment on remote learning (Education Endowment Foundation, 2020b) concluded that:

• teaching quality is more important than how lessons are delivered
• ensuring access to technology is key, particularly for disadvantaged students
• peer interactions can provide motivation and improve learning outcomes
• supporting students to work independently can improve learning outcomes
• different approaches to remote learning suit different tasks and types of content

A rapid research report by the Academy of the Social Sciences in Australia (Armour et al., 2020) concluded that remote learning arrangements had the potential to result in poorer educational outcomes for almost half of Australian primary and secondary students, if continued for an extended period (Brown et al., 2020). Students particularly at risk of poorer learning outcomes included those: from low socio-economic backgrounds; with English as an additional language; with special educational needs and disabilities; living in rural and remote areas (Fitzpatrick et al., 2020; Lamb et al., 2020; Reich & Ito, 2017; Watterston & O’Connell, 2019).

The factors that moderate the effectiveness of remote learning were identified as:

• access to digital technology and the internet (Australian Bureau of Statistics, 2018; Haßler et al., 2016; Rennie et al., 2018; Thomas et al., 2019)
• home learning environment and family support (Clinton, 2020; Cullinane & Montacute, 2020; El Asam & Katz, 2018; Hattie, 2020; Thomson, 2020)
• teacher and student readiness and capability (Fraillon et al., 2018; L. Jensen et al., 2020; Kirschner & De Bruyckere, 2017; Palloff & Pratt, 2013; Philipsen et al., 2019; Rodrigues et al., 2019; Wilson et al., 2020)

The high expectations of EdTech as a cure-all for the learning crisis have not yet materialised (Haßler et al., 2016; McBurnie & Haßler, 2020; Selwyn, 2016). As Petrie et al. (2020) argue, teachers have been forced, unprepared, into teaching using online systems; consequently, the scramble for emergency online home schooling finds many teachers far from using best practices for online learning (Guzdial, 2020; Lehmann, 2020; Milligan, 2020).

Differential impacts on students

Evidence summary

The effectiveness of online learning will vary according to students’ characteristics and circumstances. It is expected that younger students may adapt less easily than older students. Evidence suggests boys and girls may be affected in different ways, for example with boys losing more learning time, but girls being at greater risk of developing mathematics anxiety.

Access to the necessary equipment for remote learning, a home environment conducive to learning, and parental support are all key to successful online and home-based learning. These factors are all heterogeneous across student populations and variation in them can lead to substantial variations in learning outcomes.

Introduction

England used 3 main distance learning solutions during the school closures: online platforms; take-home packages; and television. Online platforms were used for at least one level of education (primary, lower secondary, or upper secondary) by all (32) countries surveyed by the OECD. Take-home packages and television programming were used by 84% of countries for at least 1 level of education.

England was among the one-third of countries^{[footnote 16]} that implemented 3 or fewer modes of delivery, the other main modes including mobile phones and radio (OECD, 2021). England’s main channels for delivering teaching are therefore comparable to those of many other countries, so findings relating to their experiences should generalise to our context.

Disruption to learning is likely to have long-term consequences for all students and their families; however, it may have a disproportionate impact on disadvantaged children (Burgess & Sievertsen, 2020), children in alternative care arrangements, care-leavers, children considered at risk due to their family situation, children experiencing poverty, children with disabilities, and migrant, asylum-seeking and refugee children (Larkins et al., 2020), who may find the alternative learning arrangements most challenging due to their more limited resources and support (Darmody et al., 2020).

Reporting on the responses to an OECD survey of countries’ contingency arrangements during COVID-19-related disruption to education, Reimers and Schleicher said there is a consensus that not all students have been able to engage consistently with their education under the contingency arrangements. Although most of the 59 countries responding to the survey were able to put alternative learning opportunities in place, respondents estimated that around half of the students were able to access all or most of the curriculum (Reimers & Schleicher, 2020b).

The evidence is clear that students from disadvantaged socio-economic backgrounds are the ones likely to suffer the most. Likewise, certain groups of students, such as those from particular ethnic groups or with special educational needs and disabilities, who are already marginalised will be highly impacted, which could be even worse if they are also from a low socio-economic background (Goldstein, 2020; OECD, 2020f).

Carretero et al. (2021) interviewed around 150 key stakeholders (students, parents, teachers & school leaders) from across 5 European Union Member States (Belgium, Estonia, Greece, Italy, and Poland ) that represented different degrees of readiness to use digital technologies in education. Participants were asked about their experiences and perceptions of online learning, including issues such as: unequal access to education; learning tools and availability of content; digital and social and emotional competences; the assessment and certification of students’ learning progress; and students’ psychological well-being.

Their findings suggest, with the current state of infrastructure and accessibility of equipment, full-time remote learning would exacerbate existing inequalities, especially for some groups of students who were prevented from attending online classes. Parents played key roles as motivators and facilitators of their children’s learning during remote schooling, particularly for students in primary education or those with special educational needs and disabilities, who required more parental support. However, respondents did not always perceive parents’ preparedness for supporting their children and the support received from schools as satisfactory.

Using survey data collected in April and early May 2020 in France and Italy, Champeaux et al. (2020) examined how lockdown and online tuition affected children’s emotional wellbeing and home learning processes. The negative impact of lockdown was stronger for boys, kindergarten students (in Italy) or secondary school students (in France), and for children whose parents had a lower education level.

Interactive distance learning – much more common in Italy than in France – significantly attenuated the negative impact of lockdown on learning. Champeaux et al. also report that increased screen time (YouTube, social media, etc.) is associated with worsening learning outcomes and emotional status, while the opposite is true for time spent reading books.

Age or grade

The impact of learning loss – through a combination of school closures and the move to online schooling – is likely to be greater for younger children, in the foundational years of education (DELVE Initiative, 2020; United Nations, 2020). Evidence suggests that investments in children’s learning tend to accumulate over time; investing in development in the early years makes it easier to learn more in the later years for a given level of investment (Cunha & Heckman, 2008).

Simulations on developing countries participating in PISA suggest that, without remediation, a loss of learning by one-third (equivalent to a 3-month school closure) during grade 3 might result in 72% of students falling so far behind that, by grade 10, they will have dropped out or will not be able to learn anything in school (Kaffenberger, 2020). The JRC report (Di Pietro et al., 2020) concludes that moving to online learning is likely to have a negative impact on primary and lower secondary students who experience difficulty adapting to new learning environments.

Grätz & Lipps (2021) used nationally representative, longitudinal data on 14 to 25-year-old Swiss students to analyse the effects of COVID-19 school closures on studying time. On average, students reduced their studying time from 35 to 23 hours per week. Secondary school students reduced their study hours more than students older than 18.

Gonzalez et al. (2020) analysed the effects of COVID-19 confinement on the performance of students in higher education. They compared assessment outcomes for students whose face-to-face study was interrupted by the pandemic (the 2019 to 2020 academic year) with students in previous cohorts whose studies were unaffected (2017 to 2018 and 2018 to 2019). They found a significant positive effect of the COVID-19 confinement on students’ performance.

An analysis of students’ learning strategies before confinement showed that students did not study on a continuous basis. Gonzalez et al. conclude that COVID-19 confinement changed students’ learning strategies to a more continuous habit, improving their efficiency.

Gender

There is likely to be a gender dimension to the impact of the pandemic on learning (UNESCO, 2020a), though it is unlikely to be straightforward. School closures and remote learning may affect girls and boys in different ways. For example, girls that are staying at home from school may be more commonly asked than boys to take care of household duties instead of learning; however, parents reportedly felt much more capable of supporting their daughters than their sons during school closures (Bol, 2020), and, on average, boys read for pleasure less than girls and score lower in reading in PISA. There is concern, and emerging evidence, that this reading gap may widen during school closures (OECD, 2020f) (see also Champeaux et al., 2020 above).

There are conflicting accounts of the impact of school closures and/or online learning on boys and girls. Grewenig et al. (2020) report that, prior to school closures, there was no gender gap, but during school closures boys spent on average half an hour less time learning each day than girls (3.4 versus 3.9 hours). Grätz & Lipps (2021) on the other hand report no variation between male and female students. The first study surveyed the parents of school-aged children (in Germany), whereas the second study includes older students (14 to 25, in Switzerland), so it may be that age and gender interact, particularly beyond the age of compulsory schooling.

The OECD (2020d) highlights the concern that gender differences in mathematics attitudes and achievements might be worsened during home schooling, when many children are supported in their schoolwork mainly by their mothers (Del Boca et al., 2020; Farré & González, 2020; Sevilla & Smith, 2020). Many women have high levels of mathematics anxiety and research suggests girls may be particularly sensitive to internalising mathematics anxiety when exposed to it from women (Beilock et al., 2010).

In terms of access to online learning, most OECD countries have a digital gender divide (OECD, 2018a), which may prevent girls engaging as fully as boys. Women worldwide have lower rates of access to the internet, including a gender gap of 5% in Europe (Iglesias, 2020). Gender may also interact with other characteristics, including SES, so it is important that learning loss analyses disaggregate the data accordingly.

Ethnicity & socio-economic factors

Evidence summary

Research on summer learning loss suggests that non-school factors are a principal source of inequalities in educational outcomes. Although COVID-19 related school closures differ from summer holidays insofar as learning is expected to continue online, the closures are likely to widen the learning gap between students from lower-income and higher-income families, because students from low-income households live in conditions that make home schooling difficult.

Evidence from the United States suggests Black and Hispanic students have been disproportionately affected by the pandemic, the school closures, and the move to online learning. It is not entirely clear what effect race has on students’ learning in addition to the effect of SES. Multivariate analyses will be important for understanding such effects.

Socio-economic status

Though online learning is recognised as being less effective than in-school learning, the growth of both cognitive and non-cognitive skills depends on more than schools alone. Peer interactions are vital, particularly for older children, and school closures and lockdown measures during the pandemic have affected children’s social interactions and peer environment.

Parents also play an important role, through their choice of parenting style, complementing education in school, and, during school closures, replacing some of the inputs usually provided by teachers. Parents’ ability to do these things interacts with their own exposure to the crisis, for example whether they lost their job or could work from home during lockdowns (Agostinelli et al., 2020).

Agostinelli et al. (2020) examined the interaction of these factors using a structural model of skill formation. They found that school closures have a large, persistent, and highly unequal effect on educational outcomes. High school students (grades 9 to 12) from poor neighbourhoods suffered a learning loss of 0.4 standard deviations, while children from wealthy neighbourhoods were unaffected. The channels operating through schools, peers, and parents all contributed to growing educational inequality during the pandemic.

Education in the home is affected by factors including having: a safe home environment; sensitive or responsive parenting (understood as family interactions in which parents are aware of their children’s emotional and physical needs and respond appropriately and consistently); and food security (Conti, 2020).

UNESCO et al. (2020) report that recent studies have found school closures from COVID-19 can lead to learning loss and widen the achievement gap even in high-income countries (Dorn et al., 2020b; Maldonado & De Witte, 2020). Researchers in Canada estimate that the socio-economic skills gap could increase by more than 30% due to the pandemic (Haeck & Lefebvre, 2020).

Darmody et al. (2020) cite the example of Ireland, where principals and teachers in DEIS schools^{[footnote 17]} have reported much lower levels of student engagement with remote learning than in non-DEIS schools during the school closures (Devitt et al., 2020; Mohan et al., 2020). They anticipate that the unprecedented scale of the COVID-19 related disruption will worsen existing educational inequalities in society.

Remote learning widened Australia’s achievement gaps at triple the pace of in-person learning, according to the Grattan Institute, with disadvantaged students set back one month during a 2-month lockdown (Sonnemann & Goss, 2020).

Finally, analysis from U.S. mathematics software provider Zearn suggests that student participation in mathematics coursework dropped by 16% for low-income students in autumn 2020, but just 2% for high-income students (Chetty et al., 2020).^{[footnote 18]}

Digital divide

The situation caused by COVID-19 has exposed the uneven distribution – across and within countries – of the technology that facilitates remote learning over a prolonged period (Dreesen et al., 2020). Consequently, the transition to online learning is expected to contribute to widening socio-economic gaps in student performance, as it disadvantages students with limited access to ICT at home, low exposure to ICT via their teachers, or whose teachers have limited proficiency with technology (OECD, 2020e, 2020c). The sudden move to online learning exposed digital inequities, including differences in familiarity with technology and teacher professional development, which are important enablers for the effective integration and use of technology in the classroom (Hennessy & London, 2013; OECD, 2010).

In almost all countries, students from low socio-economic backgrounds used digital technologies less frequently compared with their peers from high socio-economic backgrounds before the pandemic in 2018. Similar disparities were observed between students from public and private schools, with the latter making more frequent use of digital technologies for schoolwork (OECD, n.d. forthcoming).

In Scotland, a survey of students found that 48.2% of primary school-aged students suggested that, at times, they had to share a digital device with someone else at home, compared with 21.9% of secondary school-aged students. This suggests there may be an interaction between socio-economic background and age, perhaps with older siblings being given priority access to devices within households (Education Scotland, 2021a).

In the United States, a report showed that, prior to the pandemic, between 15 and 16 million out of a total 50 million K-12 students lived in households that lacked either internet access, a digital device, or both (Chandra et al., 2020; Lieberman, 2020). Of these students, 55 to 60% lacked both an internet connection and a digital device. The technology divide applies not only to students. Between 300,000 and 400,000 public school teachers (a little under 10% nationwide) lacked adequate access to the internet or computing devices at home.

The proportion of households with at least one K-12 student with inadequate access to IT varied according to race and location. Geographically, the proportions of students with inadequate access were: urban, 21%; suburban, 25%; rural, 37%. By race and/or ethnicity: White, 18%; Latin, 26%; Black, 30%; and Native American 35%. (Chandra et al., 2020; Lorente et al., 2020).

Stelitano et al. (2020) examined the relationship between teachers’ reports of their students’ internet access and their interaction with students and families during COVID-19-related school closures. Their data suggest that existing inequities for students in rural and high-poverty schools might be exacerbated by students’ limited access to the internet and communication with teachers where online teaching is in place. For example, teachers were more likely to report that their students were completing assignments and that they had contact with students’ families if their students had access to the internet at home.

The JRC report (Di Pietro et al., 2020) concludes that moving to online learning is likely to worsen existing educational inequalities because students from less advantaged backgrounds:

• are less likely to have access to the necessary resources, such as computers and broadband internet
• are less likely to have a suitable home learning environment, including a quiet space or their own desk
• may not receive as much (direct or indirect) support from their parents as more advantaged students do
• parents are less likely to be able to work from home and are less able to afford private online tuition
• schools may have less developed ICT-based infrastructure and teachers with lower levels of digital skills than the schools attended by more advantaged students

Most, if not all, of these factors could reasonably be expected to apply to the context in England.

Race

In the United States, the communities that were typically the hardest hit by the COVID-19 pandemic (Artiga et al., 2020; Centers for Disease Control and Prevention, 2020) are those that are historically underserved by the U.S. education system: where students live in poverty, and are affected by systemic racism. School closures were not only expected to cause an unprecedented slide in student learning (Chetty et al., 2020; Hobbs, 2020), but also increase the disparities in student access to high-quality teaching pervasive in the United States, further disadvantaging students living in poverty, who tend to be Black and Latinx (Dorn et al., 2020b; Mandery, 2020; Sternberg, 2020) (Curriculum Associates, 2020a). In the first weeks of the lockdown, over onethird of students in the United States were completely excluded from online learning, particularly in schools with large shares of low-income students, while elite private schools experienced almost full attendance (Khazan, 2020; The Economist, 2020).

Dorn et al. (2020a) report a 20 percentage-point difference between the proportion of Black or Hispanic and White students learning remotely. Among households with K-12 students (ages 5 to 18), 91% had regular access to devices and the internet; however, Black and Hispanic households were 3 to 4 percentage points less likely to have access than White households. Black (15%) and Hispanic (16%) students were also twice as likely as White students (8%) to have no real-time access to teachers in autumn 2020.

The students most likely to be learning remotely were those least likely to have suitable access to learning technology and teachers. Dorn et al. estimate that, while on average students could lose 5 to 9 months of learning by the end of June 2021, the distribution of this loss could be such that students of colour lose 6 to 12 months of learning compared with 4 to 8 months for White students.

Family support

Irrespective of socio-economic background, children develop better when their parents read to them almost every day (in the case of the younger ones), ensure there are many children’s books or e-books in the home, and have back-and-forth conversations with them (OECD, 2020g). That said, there are family-level factors that will affect students’ progress during school closures and online tuition.

It is possible that lower SES children may be more adversely affected by COVID-19 compared with their higher SES peers, particularly if their schools provide fewer resources and/or their parents engage in less active home schooling. Higher SES children may actually benefit from home schooling if their parents have higher levels of education and more motivation to invest in their child compared with teachers. A study by McDonnell and Doyle (2019) examining the relationship between maternal employment and childhood obesity, found that children from higher SES backgrounds were more adversely affected by maternal employment than lower SES families, possibly as the quality of alternative care is lower than parental care (Doyle, 2020).

Using high frequency internet search data, Bacher-Hicks et al. (2020) studied how U.S. households searched for online learning resources as schools closed due to the COVID-19 pandemic. By April 2020, nationwide search intensity for both school- and parent-centred online learning resources had approximately doubled relative to baseline. Areas of the United States with higher income, better internet access, and fewer rural schools saw substantially larger increases in search intensity. Bacher-Hicks et al. argue that the pandemic will probably widen achievement gaps along these dimensions due to schools’ and parents’ differing engagement with online resources.

Similarly, Bol (2020) reports findings from a survey of how parents school their children in primary and secondary education in the Netherlands. The results, gathered in April 2020, showed that, while all parents considered it important their children keep up with the schoolwork, children from advantaged backgrounds received much more parental support and had more resources (for example their own computer) to study from home.

The differences in parental support were driven by the ability to help, with more highly educated parents feeling more capable of helping their children with schoolwork than less-educated parents. Parents also reported that students in the academic track in secondary education received more extensive distance schooling than students in the pre-vocational track.

Bonal & González (2020) assessed the impact of COVID-19 school closures in Catalonia on the learning gap between students from different social backgrounds. An online survey of families with children aged between 3 and 18 was administered between 26 and 30 March 2020 and received 35,419 responses. The authors’ analysis showed learning opportunities varied significantly, with middle-class families able to maintain higher standards of education quality, while children from socially disadvantaged families had few learning opportunities both in terms of time and learning experiences.

They found results differed according to whether students attended public or private schools, the family’s economic, social and cultural capital, and living conditions. Given that the survey was online, it is possible that some of the worst-affected families were unable to respond to it.

Again, Grätz & Lipps (2021) study of 14- to 25-year-old students in Switzerland is somewhat at odds with the bulk of evidence. Their survey suggested that, in relative terms, reductions in students’ studying time did not vary by parental education and that, in fact, students from families with highly educated parents reduced their studying time more in absolute terms than students from families with low-educated parents. It is plausible that the older students they surveyed are more self-motivated or resilient than the average student at a younger age with a similar background, although Grewenig et al. (2020) similarly found no relationship between reduction in learning time and parents’ level of education across a wider range of student ages.

Prior attainment

Grewenig et al. (2020) argued that low-achieving students may be particularly affected by the lack of teacher support during school closures (although see Engzell et al., 2020b, 2020a, 2021). They surveyed 1,099 parents of students in Germany about the students’ time-use before and during the school closures. Students on average reduced their daily learning time of 7.4 hours by about half, but the reduction was significantly larger for low-achievers (4.1 hours) than for high-achievers (3.7 hours).

The low achievers disproportionately replaced learning time with activities such as TV, computer games, or consuming social media, instead of activities more conducive to learning, such as reading, playing music, creative work, or physical exercise. Parents and schools provided less support for low-achieving students.

Special educational needs and disabilities

School closures may disproportionately impact children with special educational needs and disabilities. Some are likely to be disadvantaged from the switch to online learning unless assistive technologies are promptly put in place and adapted to the online learning environment (Di Pietro et al., 2020). The suitability of remote learning can depend on children’s individual needs and their schools’ ability to provide tailored tuition.

Teachers may also find it more challenging to track progress and provide effective feedback to students with additional support needs, as this would usually be carried out through direct observation (Education Scotland, 2021b). Furthermore, evidence suggests that learning loss during scheduled school breaks can be higher for children with disabilities (Kerry & Davies, 1998). The presence of a sibling with a disability in the home may also affect parents’ abilities to meet the new demands of home schooling for other children and to manage family stress (OECD, 2020a).

Non-cognitive abilities and wellbeing

Reimers and Schleicher (2020b) argue that the continuity of academic learning has received more attention than the socio-emotional development of students. The knowledge, skills, and understanding learned in school are not only academic in nature. Students develop non-cognitive skills – including the ability to learn itself – that support learning and are important in their own right.

Mental health and wellbeing are similarly vital and underpin successful learning. Although evidence has emerged of the effect of COVID-19 on adult mental health, few studies around the world cover children (DEPRESSD Project, 2020); however, recent research from McKinsey’s Center for Societal Benefit through Healthcare, and elsewhere, points to a decline in students’ mental health and physical fitness (Chen et al., 2021).

Darmody et al. (2020) note that school closures could affect the health and wellbeing of young children, as schools provide food security in addition to learning. School lunches are associated with improvements in academic performance; food insecurity, including irregular and unhealthy diets, is associated with low educational attainment and can negatively impact the physical health and mental wellbeing of children (Van Lancker & Parolin, 2020).

School is important as a setting for social learning: the links between students and with teachers are vital for the personal development of each child (Institute of Medicine and National Research Council, 1997). Knowing how to express oneself, how to listen to others, how to respect diversity, and more abstractly how to respect the ‘social contract’ is not learned from books. Children confined to their homes suddenly find themselves unable to see their friends, teachers, and other members of their classes. Isolation could have a strong impact, especially if prolonged (Loades et al., 2020). In practice, there remains little evidence of school closures leading to delays in social learning, and online platforms, from games to social networks, will likely reduce any impact (Groupe URD, 2020).

Remote teaching and learning may cause greater inequality in students’ emotional well-being and motivation, as well as an unequal distribution of behavioural and psychological problems. Students from less advantaged backgrounds are more likely to be exposed to a stressful home environment, for example where they share limited space and digital devices with other family members. Additionally, parents in less affluent households, who are more likely to experience the pressure of financial and job insecurity issues during the COVID-19 pandemic, may not be well placed to support their children (Di Pietro et al., 2020). There is evidence that reduced economic resources and elevated stress affect children’s academic achievement and behaviour (Parolin, 2020; Stevens & Schaller, 2011).

Reimers and Schleicher (2020a) caution that, as well as worsening existing stresses and inequalities, extended school closures may raise a number of new issues during the closures and once schools re-open. One such issue is that, for some students, the habit of going to school and concentrating on learning will be broken. Intrinsically motivated students may find they are relatively unaffected in their progress without needing supervision; however, vulnerable groups and those who find themselves struggling to work independently will likely spend time being lost (Petrie et al., 2020). Effort may be required to rebuild students’ engagement and avoid increases in school drop-outs (OECD, 2020a).

The emotional well-being of teachers may also have been impacted as a result of the pandemic and the move to remote learning. A study of Scottish primary school teachers found the use of remote learning had negatively impacted teachers and created additional stress in adapting to these practices. Teachers’ well-being was also impacted by physical isolation and the psychological effect of feeling responsible for the well-being of their students (Beattie et al., 2021). This may have impacted the efficacy of remote teaching, but also highlights the importance of support for teachers, whose efforts will be key in making up for learning loss.

Summary of the international evidence on learning remotely

The literature tells us that online learning is often associated with poorer outcomes than traditional classroom teaching, although the varying degrees of success suggest implementation is critical. Teaching quality is essential to successful learning, and some of the shortcomings of online learning in the current context may stem from hurried implementation, where good practice has not translated into online provision. Existing online platforms are simply unsuitable for certain tasks and types of content, and the limitations on interactions with peers and teachers may affect motivation and learning.

Access to, and the ability to make effective use of, online remote learning will vary according to students’ characteristics and circumstances. Younger students may adapt less easily than older students, and boys and girls may be affected in different ways. Learning outcomes depend on students having access to learning technology and being able to engage with it. This includes having a home environment conducive to learning, including adult support generally. More specifically, students with special educational needs and disabilities may find the tailored support they require is not available when learning from home or that the online platforms and devices they use do not accommodate their needs. Independent study is a skill itself and one in which students’ proficiency will differ. Variation in these and other factors may widen inequalities in outcomes when students are not learning in school.

Conclusion

The pandemic led to worldwide school closures, which peaked in early April 2020 but continued more sporadically through the year and into 2021. The global scale of the disruption to teaching and learning was unprecedented, as was the level of disruption in England. Around the world, remote schooling was introduced, typically taking the form of online tuition where technology permitted.

This transition alone could be expected to set back learning, and the efficacy of online teaching and learning was uncertain and untested at such scale. The likely learning loss scenarios ranged from:

• a fall in mean learning outcomes; through
• a fall in mean learning outcomes with a greater spread, as students were differentially affected; to
• a fall in mean learning outcomes with a greater spread, plus a rump of students who stopped learning completely

The international literature on variations in teaching time and learning loss following events such as school summer holidays, snow closures, natural disasters, teacher strikes, and pandemics including COVID-19, were reviewed for evidence that might inform our understanding of the impact of COVID-19-related school closures in England.

Comparisons of the effects of whole school closures with individual absences suggest the former are typically less disruptive than the latter. School closures are easier for schools to plan around as they affect whole classes or cohorts, whereas individual absences can leave those students behind their peers and are therefore harder to cater for.

Persistent individual absence may also signify an underlying issue in that student’s life that is unfavourable for learning, such as chronic illness. Whilst this suggests that learning loss may be mitigated if disruption is managed well, it cautions us that individual students who struggle or fail to learn remotely will return to school out of step with their peers and may not benefit fully from whole-class remedial actions.

Early modelling of COVID-19 learning loss, typically based on historic summer learning loss data, suggested that U.S. students could start the 2020 autumn term with losses in the region of a third of a year for reading and over a half a year for mathematics compared with expected progress. Some reports anticipated the brunt of the learning loss being born by a small proportion of students, as disruption affected students differentially along lines of SES, race, having English as an additional language, and special educational needs and disabilities.

The literature tells us that physical schools are a great leveller. They provide many students with the time, space, peace and quiet, resources, and opportunity to learn that they would struggle to obtain outside of school. For some, school meets basic physical needs that underpin learning, including nourishment and warmth.

Schools cannot entirely offset the relative advantages or disadvantages of the home environment, as is borne out in the literature on summer learning loss, which shows that students from less advantaged backgrounds experience the lowest rates of learning when they are not in school. The closure of schools during the holidays undermines their levelling effects; it follows that the same may be true of the COVID-19 school closures, despite teaching continuing online.

When learning moves online, access to teachers and resources becomes mediated by technology, and access to technology is associated with factors such as geographical location, SES, and gender. Even assuming technical access to teacher-led online lessons, variation in the suitability of the learning environment increases greatly when schools are physically closed. While some students’ learning environments may see them make gains during school closures, others’ learning environments will prevent them from learning anything at all.

The effectiveness of online teaching depends on using it to implement good teaching practices. Where this happens, its effectiveness – inequalities in accessing it notwithstanding – can compete with face-to-face teaching. Some schools and teachers will have been more accustomed than others to using EdTech prior to the school closures, and there is evidence of division along lines of SES, including between private and state providers.

It is likely that teachers’ online practice has improved during the prolonged periods of remote learning. Combined with efforts to provide more students with the technology to access online learning from home, this may mean that learning losses later into school closures – or where some students are isolating despite the school being open – will be smaller and more evenly distributed than at the beginning.

As analyses of real learning (loss) data became available, it was clear that some of the modelling represented worst case scenarios, and that online learning had been reasonably effective. Reading appeared to be more resilient to disrupted learning than mathematics, probably for the obvious reason that it is used more on a day-to-day basis, and that more students are able to practise it on their own or with the adults they live with.

Unfortunately, many of the predictions regarding those students who would bear the brunt of learning loss appear to be playing out. More studies than not have reported widening disparities along lines of SES and race, and it is likely that students who speak English as an additional language, and those with special educational needs and disabilities, may be disproportionately affected.

The effect of school closures on students of different ages is somewhat mixed. Studies of the impact of teacher strikes, for example, suggest the youngest students were barely affected, while older students suffered learning loss. Findings from COVID-19 research, on the other hand, suggest that younger students may be more adversely affected than older students (although short term losses for older students nearing the end of Key Stages 4 and 5 have the potential for disproportionate impact on progression).

In the short run, younger students learn less independently than older students and so are more reliant on contact with their teachers and support from adults at home. More importantly perhaps, unrecovered losses have been shown to accumulate, so in the long run the youngest students have most to lose over the course of their school careers. There is some evidence that older students – particularly those in post-compulsory education – can adapt their study habits to learn effectively during periods of disruption.

The evidence regarding the impact of school closures and online tuition on students’ non-cognitive abilities and wellbeing, including physical fitness, is more limited, though the evidence available points to declines. The kinds of school closures captured in the learning loss literature are not typically accompanied by restrictions on social interactions outside of school, so how the impact of these factors affects recovery from learning loss as schools reopen fully remains to be seen.

Figure 1 set out 3 scenarios for learning loss. In the first, mean attainment falls, but the distribution remains the same, implying that losses would be distributed evenly across the student population. Based on the evidence we have reviewed, including the post-school closure assessment data we have seen to date, this scenario can be safely ruled out.

In the second scenario, mean attainment falls, but the distribution of losses is uneven across the student population, causing the standard deviation to increase. Some students may do better than they would have done, some may be relatively unaffected, but many will do worse: some of them quite substantially so. The evidence suggests this is a likely scenario; the observed effect of SES on assessment scores alone might be expected to produce this outcome.

The third scenario is much like the second, except that a significant proportion of students score zero – or very close to it – on their assessments, producing a second peak at the low end of the distribution curve. This represents, at best, no progress and, at worst, regression. Whilst there is evidence to suggest some students will have fallen badly behind during school closures, the medium to longer term outcomes for these students are probably not inevitable, and will depend on reengagement and support with continuing to learn.

References

Agostinelli, F., Doepke, M., Sorrenti, G., & Zilibotti, F. (2020). When the Great Equalizer Shuts Down: Schools, Peers, and Parents in Pandemic Times.

Alban Conto, C., Akseer, S., Dreesen, T., Kamei, A., Mizunoya, S., & Rigole, A. (2020). COVID-19 : Effects of School Closures on Foundational Skills and Promising Practices for Monitoring and Mitigating Learning Loss. In UNICEF - Innocenti Working Paper: Vol. WP 2020-13 (Issue October).

Alexander, K. L. (2020). Summer Learning Loss Sure Is Real. Education Next, 21(1), 1–4.

Alexander, K. L., Entwisle, D. R., & Olson, L. S. (2001). Schools, achievement, and inequality: A seasonal perspective. Educational Evaluation and Policy Analysis, 23(2), 171–191.

Allington, R. L., McGill-Franzen, A., Camilli, G., Williams, L., Graff, J., Zeig, J., Zmach, C., & Nowak, R. (2010). Addressing summer reading setback among economically disadvantaged elementary students. Reading Psychology, 31(5), 411–427.

Amplify. (2021). COVID-19 means more students not learning to read (Issue February).

Andrabi, T., Daniels, B., & Das, J. (2020). Human Capital Accumulation and Disasters : Evidence from the Pakistan Earthquake of 2005. RISE Working Paper, 20(039, May).

Angrist, J., & Lavy, V. (2002). New evidence on classroom computers and pupil learning. The Economic Journal, 112(October), 735–765.

Armour, D., Blackmore, H., Brown, N., Clinton, J., Geelan, D., Martin, A., Miller, J., Mulcahy, M., Prestridge, S., Selwyn, N., Taylor-Guy, P., Thomas, J., West, D., & Wilson, R. (2020). Learning outcomes for online versus in-class education.

Artiga, S., Corallo, B., & Pham, O. (2020). Racial Disparities in COVID-19: Key Findings from Available Data and Analysis. KFF.

Atteberry, A., & McEachin, A. (2020). School’s Out: The Role of Summers in Understanding Achievement Disparities. EdWorkingPaper, 19–82.

Aucejo, E. M., & Romano, T. F. (2016). Assessing the effect of school days and absences on test score performance. Economics of Education Review, 55, 70–87.

Australian Bureau of Statistics. (2018). Household use of information technology, 2016-17 financial year.

Azevedo, J. P., Hasan, A., Goldemberg, D., Iqbal, S. A., & Geven, K. (2020). Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates.

Bacher-Hicks, A., Goodman, J., & Mulhern, C. (2020). Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time. NBER Working Paper Series, 27555.

Baker, D. P., Fabrega, R., Galindo, C., & Mishook, J. (2004). Instructional time and national achievement: Cross-national evidence. Prospects, 34(3), 311–334.

Baker, M. (2013). Industrial actions in schools: Strikes and student achievement. Canadian Journal of Economics, 46(3), 1014–1036.

Balfanz, R. (2016). Absenteeism matters to schools and students. In Phi Delta Kappan.

Barro, R. J., & Lee, J.-W. (2001). Schooling quality. National Bureau of Economic Research.

Beaglehole, B., Bell, C., Frampton, C., & Moor, S. (2017). The impact of the Canterbury earthquakes on successful school leaving for adolescents. Australian and New Zealand Journal of Public Health, 41(1), 70–73.

Beattie, M., Wilson, C., & Hendry, G. (2021). Learning From Lockdown: Examining Scottish Primary Teachers’ Experiences of Emergency Remote Teaching. British Journal of Educational Studies, 00(00), 1–18.

Beilock, S. L., Gunderson, E. A., Ramirez, G., & Levine, S. C. (2010). Female teachers’ math anxiety affects girls’ math achievement. Proceedings of the National Academy of Sciences of the United States of America, 107(5), 1860–1863.

Bellei, C. (2009). Does lengthening the school day increase students’ academic achievement? Results from a natural experiment in Chile. Economics of Education Review, 28(5), 629–640.

Belot, M., & Webbink, D. (2010). Do teacher strikes harm educational attainment of students? Labour, 24(4), 391–406.

Bielinski, J., Brown, R., & Wagner, K. (2021). No Longer a Prediction: What New Data Tell Us About the Effects of 2020 Learning Disruptions.

Blank, R. K. (2013). Science instructional time is declining in elementary schools: What are the implications for student achievement and closing the gap? Science Education, 97(6), 830–847.

Bol, T. (2020). Inequality in homeschooling during the Corona crisis in the Netherlands. First results from the LISS Panel. SocArXiv.

Bonal, X., & González, S. (2020). The impact of lockdown on the learning gap: family and school divisions in times of crisis. International Review of Education, 66(5), 635–655.

Bonilla-Mejia. (2011). Doble jornada escolar y calidad de la educación in Colombia. Documentos de Trabajo Sobre Economía Regional, 143.

Bouie, J. (2020). [Opinion

Why Coronavirus Is Killing African-Americans More Than Others - The New York Times](https://www.nytimes.com/2020/04/14/opinion/sunday/coronavirus-racism-african-americans.html). New York Times.

Brown, N., te Riele, K., Shelley, B., & Woodroffe, J. (2020). Learning at home during COVID-19: Effects on vulnerable young Australians. Independent Rapid Response Report.

Bubb, S., & Jones, M.-A. (2020). Learning from the COVID-19 home-schooling experience: Listening to pupils, parents/carers and teachers. Improving Schools, 23(3), 209–222.

Bulman, G., & Fairlie, R. W. (2016). Technology and Education: Computers, Software, and the Internet. Handbook of the Economics of Education (Vol. 5, pp. 239–280). Elsevier.

Burgess, S., & Sievertsen, H. H. (2020). Schools, skills, and learning: The impact of COVID-19 on education. Voxeu.Org.

Busso, M., & Camacho Munoz, J. (2020). Pandemic and Inequality: How Much Human Capital Is Lost When Schools Close? Ideas Matter.

Cameron, L. (2009). Can a public scholarship program successfully reduce school drop-outs in a time of economic crisis? Evidence from Indonesia. Economics of Education Review, 28(3), 308–317.

Carlsson, M., Dahl, G. B., Öckert, B., & Rooth, D. O. (2015). The effect of schooling on cognitive skills. Review of Economics and Statistics, 97(3), 533–547.

Carretero, S., Napierała, J., Bessios, A., Mägi, E., Pugacewicz, A., Ranieri, M., Triquet, K., Lombaerts, K., Robledo-Bottcher, N., Montanari, M., & Gonzalez-Vazquez, I. (2021). What did we learn from schooling practices during the COVID-19 lockdown? EU Science Hub.

Carroll, J. (1963). A Model of School Learning. Teachers College Record, 64(8), 723–723.

Carter, S. (2020, November 9). Learning Loss is Rampant among Dallas ISD Students. Dallas Observer.

Cattan, S., Kamhöfer, D., Karlsson, M., & Nilsson, T. (2017). The Short- and Long-Term Effects of Student Absence: Evidence from Sweden. IZA Discussion Paper, 10995.

Cattaneo, M. A., Oggenfuss, C., & Wolter, S. C. (2016). The More, the Better? The Impact of Instructional Time on Student Performance.

Centers for Disease Control and Prevention. (2020). Health Equity Considerations and Racial and Ethnic Minority Groups. CDC.

Cerdan-Infantes, P., & Vermeersch, C. (2007). More time is better : an evaluation of the full time school program in Uruguay. World Bank - Impact Evaluation Series, 13, 24 p.

Cerullo, M. (2020). Black and Hispanic workers less able to work from home.

Champeaux, H., Mangiavacchi, L., Marchetta, F., & Piccoli, L. (2020). Learning at Home: Distance Learning Solutions and Child Development during the COVID-19 Lockdown (No. 13819; IZA Discussion Paper).

Chandra, S., Chang, A., Day, L., Fazlullah, A., Liu, J., Mcbride, L., Mudalige, T., & Weiss, D. (2020). Closing the K-12 Digital Divide in the Age of Distance Learning. Boston Consulting Group.

Checchi, D., Rettore, E., & Girardi, S. (2019). IC technology and learning: an impact evaluation of Cl@ssi 2.0. Education Economics, 27(3), 241–264.

Chen, L.-K., Dorn, E., Sarakatsannis, J., & Wiesinger, A. (2021). Teacher survey: Learning loss is global—and significant.

Chetty, R., Friedman, J. N., Hendren, N., Stepner, M., Autor, D., Chodorow-Reich, G., Farhi, E., Furman, J., Hamilton, S., Hurst, E., Jaravel, X., Katz, L., Saez, E., Straub, L., Yagan, D., Nichols, M., Sifain, S., Derek Doel, W., Thorpe, R., … Zhang, R. (2020). The Economic Impacts of COVID-19: Evidence from a New Public Database Built Using Private Sector Data.

Clinton, J. (2020). Supporting Vulnerable Children in the Face of a Pandemic: A paper prepared for the Australian Government Department of Education, Skills and Employment.

Clotfelter, C. T., Ladd, H. F., & Vigdor, J. L. (2009). Are Teacher Absences Worth Worrying About in the United States? Education Finance and Policy, 4(2), 115–149.

Conti, G. (2020). Supporting parents and children in the early years during (and after) the COVID-19 crisis. VoxEU CEPR Policy Portal.

Cooper, H., Nye, B., Charlton, K., Lindsay, J., & Greathouse, S. (1996). The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-Analytic Review. Review of Educational Research, 66(3), 227–268.

Cotton, K., & Savard, W. (1981). Time Factors in Learning. Research on School Effectiveness Project: Topic Summary Report. Alaska Department of Education Office of Planning And Research.

Crockett, L. J., Petersen, A. C., Graber, J. A., Schulenberg, J. E., & Ebata, A. (1989). School Transitions and Adjustment During Early Adolescence. The Journal of Early Adolescence, 9(3), 181–210.

Cullinane, C., & Montacute, R. (2020). COVID-19 and Social Mobility Impact Brief #1: School Closures.

Cunha, F., & Heckman, J. J. (2008). Formulating, identifying and estimating the technology of cognitive and noncognitive skill formation. Journal of Human Resources, 43(4), 738–782.

Curriculum Associates. (2020a). Overcoming the Digital Divide: Distance-learning successes during the pandemic. Curriculum Associates (No. 2020–46; Curriculum Associates Research Report).

Curriculum Associates. (2020b). Understanding Student Needs (October 2020; Curriculum Associates Research Brief).

Dallas Independent School District. (2020). Student Outcome Goals Update: Board Briefing.

Darmody, M., Smyth, E., & Russell, H. (2020). The implications of the COVID-19 pandemic for policy in relation to children and young people: a research review.

David, R., Pellini, A., Jordan, K., & Philips, T. (2020). Education during the COVID-19 crisis: opportunities and constraints of using EdTech in low-income countries.

Del Boca, D., Oggero, N., Profeta, P., & Rossi, M. C. (2020). Women’s Work, Housework and Childcare, before and during COVID-19. IZA - Institute of Labor Economics (No. 13409; IZA Discussion Paper Series).

DELVE Initiative. (2020). Balancing the Risks of Pupils Returning to Schools. DELVE Report No. 4, 1–47.

DEPRESSD Project. (2020). COVID-19 Mental Health. DEPRESSD Project.

Desimone, L. M., & Long, D. (2010). Teacher effects and the achievement gap: Do teacher and teaching quality influence the achievement gap between black and white and high- and low-SES students in the early grades? Teachers College Record, 112(12), 3024–3073.

Devitt, A., Banks, J., Bray, A., & Ni Chorcora, E. (2020). Teaching and Learning During School Closures: Lessons Learned. Irish Second-Level Teacher Perspectives.

Di Pietro, G. (2018). The academic impact of natural disasters: evidence from L’Aquila earthquake. Education Economics, 26(1), 62–77.

Di Pietro, G., Biagi, F., Costa, P., Karpiński, Z., & Mazza, J. (2020). The Likely Impact of COVID-19 on Education: Reflections based on the Existing Literature and Recent International Datasets. Publications Office of the European Union, Luxembourg: Vol. EUR 30275 (Issue JRC121071).

Domingue, B. W., Hough, H. J., Lang, D., & Yeatman, J. (2021). Changing Patterns of Growth in Oral Reading Fluency During the COVID-19 Pandemic.

Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2020a). COVID-19 and learning loss—disparities grow and students need help.

Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2020b). COVID-19 and student learning in the United States: The hurt could last a lifetime (Issue June).

Doyle, O. (2020). COVID-19: Exacerbating Educational Inequalities?

Dreesen, T., Akseer, S., Brossard, M., Dewan, P., Giraldo, J.-P., Kamei, A., Mizunoya, S., & Santiago Ortiz, J. (2020). Promising practices for equitable remote learning: Emerging lessons from COVID-19 education responses in 127 countries.

Earl, L., Hargreaves, A., & Ryan, J. (1996). Schooling for change: Reinventing education for early adolescents.

Education Endowment Foundation. (2020a). Impact of school closures on the attainment gap: Rapid Evidence Assessment.

Education Endowment Foundation. (2020b). Remote Learning: Rapid Evidence Assessment. www.educationendowmentfoundation.org.uk

Education Scotland. (2021a). National overview of practice in remote learning. Parents, carers and learners. 0–24.

Education Scotland. (2021b). National overview of practice in remote learning. Schools: a focus on meeting learning needs, including those with additional support needs.

El Asam, A., & Katz, A. (2018). Vulnerable Young People and Their Experience of Online Risks. Human-Computer Interaction, 33(4), 281–304.

EmpowerK12. (2020a). Quantifying the Impact of COVID-19 School Closures on Metro Atlanta Student Proficiency Introduction.

EmpowerK12. (2020b). COVID-19’s impact on student achievement and academic growth in DC.

Engzell, P., Frey, A., & Verhagen, M. (2020a). The collateral damage to children’s education during lockdown. VoxEU CEPR Policy Portal.

Engzell, P., Frey, A., & Verhagen, M. D. (2020b). Learning Inequality During the Covid-19 Pandemic.

Engzell, P., Frey, A., & Verhagen, M. D. (2021). Learning Loss Due to School Closures During the COVID-19 Pandemic.

Escueta, M., Quan, V., Nickow, A. J., Oreopoulos, P., Anzelone, C., Balu, R., Bergman, P., Bernatek, B., Castleman, B., Crowley, L., Duckworth, A., Guryan, J., Haslam, A., Ho, A., Jones, B., Kraft, M., Kroft, K., Laibson, D., Loeb, S., … Sweeten-Lopez, O. (2017). We are extremely grateful to Education Technology: An Evidence-Based Review (No. 23744; NBER Working Paper Series).

Fairlie, R. W., & Robinson, J. (2013). Experimental Evidence on the Effects of Home Computers on Academic Achievement among Schoolchildren. National Poverty Center Working Paper Series (No. 13–02; National Poverty Center Working Paper Series). National Poverty Center, University of Michigan.

Falck, O., Mang, C., & Wößmann, L. (2018). Virtually No Effect? Different Uses of Classroom Computers and their Effect on Student Achievement. Oxford Bulletin of Economics and Statistics, 80(1), 1–38.

Farbman, D., & Kaplan, C. (2005). Time for a change. Massachusetts Education Opportunity, 23(11).

Farré, L., & González, L. (2020). ¿Quién se encarga de las tareas domésticas durante el confinamiento? Covid-19, mercado de trabajo y uso del tiempo en el hogar. Nada Es Gratis.

Fitzpatrick, B. R., Berends, M., Ferrare, J. J., & Waddington, R. J. (2020). Virtual Illusion: Comparing Student Achievement and Teacher and Classroom Characteristics in Online and Brick-and-Mortar Charter Schools. Educational Researcher, 49(3), 161–175.

Fleischer, H. (2012). What is our current understanding of one-to-one computer projects: A systematic narrative research review. Educational Research Review (Vol. 7, Issue 2, pp. 107–122). Elsevier.

Fraillon, J., Gebhardt, E., Friedman, T., Duckworth, D., O’Malley, K., & Vernon, K. (2018). NAP Sample Assessment ICT Literacy: Years 6 and 10 (2017).

Galvis, M., & McLean, D. (2020). How prepared were primary teachers and pupils in England for the shift to online learning? Insights from TIMSS 2019 - NFER.

García, E., & Weiss, E. (2020). COVID-19 and student performance, equity, and U.S. education policy: Lessons from pre-pandemic research to inform relief, recovery, and rebuilding.

Garcia, S., Fernandez, C., & Weiss, C. C. (2013). Does Lengthening the School Day Reduce the Likelihood of Early School Dropout and Grade Repetition: Evidence from Colombia. SSRN Electronic Journal.

Goldstein, D. (2020, June 5). Research Shows Students Falling Months Behind During Virus Disruptions. The New York Times.

Goldstein, D., Popescu, A., & Hannah-Jones, N. (2020, April 6). As School Moves Online, Many Students Stay Logged Out. The New York Times.

Gonzalez, T., de la Rubia, M. A., Hincz, K. P., Comas-Lopez, M., Subirats, L., Fort, S., & Sacha, G. M. (2020). Influence of COVID-19 confinement on students’ performance in higher education. PLoS ONE, 15(10), e0239490.

Goodman, J. S. (2014). Flaking out: Student absences and snow days as disruptions of instruction time. NBER Working Paper Series, 20221, National Bureau of Economic Research.

Gottfried, M., & Ehrlich, S. B. (2018). Introduction to the Special Issue: Combating Chronic Absence. Journal of Education for Students Placed at Risk (Vol. 23, Issues 1–2, pp. 1–4). Routledge.

Gouëdard, P., Pont, B., & Viennet, R. (2020). Education responses to COVID-19: Implementing a way forward. (No. 224; OECD Education Working Papers). OECD iLibrary.

Goulas, S., & Megalokonomou, R. (2016). Swine Flu and The Effect of Compulsory Class Attendance on Academic Performance.

Graham, A., & Sahlberg, P. (2020, March 26). Schools are moving online, but not all children start out digitally equal. The Conversation.

Grätz, M., & Lipps, O. (2021). Large loss in studying time during the closure of schools in Switzerland in 2020. Research in Social Stratification and Mobility, 71, 100554.

Grewenig, E., Lergetporer, P., Werner, K., Wößmann, L., & Zierow, L. (2020). Covid-19 and Educational Inequality: How School Closures Affect Low- and High-Achieving Students. Publication. CESifo.

Gromada, A., & Shewbridge, C. (2016). Student Learning Time: A Literature Review. OECD Education Working Papers, No. 127, 127, 63–73.

Groupe URD. (2020). Covid-19: Youth and education: Impacts and options (No. 12; Briefing Note).

Guzdial, M. (2020). So much to learn about emergency remote teaching, but so little to claim about online learning. Computing Education Research Blog.

Haeck, C., & Lefebvre, P. (2020). Pandemic School Closures May Increase Inequality in Test Scores.

Hansen, B. (2011). School Year Length and Student Performance: Quasi-Experimental Evidence. SSRN Electronic Journal.

Hanushek, E. A., & Wößmann, L. (2020). The Economic Impacts of Learning Losses.

Haßler, B., Major, L., & Hennessy, S. (2016). Tablet use in schools: a critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32(2), 139–156.

Haßler, B., Nicolai, S., McBurnie, C., Jordan, K., Wilson, S., & Kreimeia, A. (2020). EdTech and COVID-19 response. EdTech Hub Report.

Hattie, J. (2020). Visible Learning Effect Sizes When Schools Are Closed: What Matters and What Does Not.

Hennessy, S., & London, L. (2013). Learning from International Experiences with Interactive Whiteboards: The Role of Professional Development in Integrating the Technology (No. 89; OECD Education Working Papers).

Hill, P. W., & Russell, V. J. (1999). Systemic, Whole-School Reform of the Middle Years of Schooling. Enhancing Educational Excellence, Equity and Efficiency (pp. 167–196). Springer Netherlands.

Hincapie, D. (2016). Do Longer School Days Improve Student Achievement? Evidence from Colombia. Inter- American Development Bank (IDB).

Hobbs, T. D. (2020, July 15). ‘Are They Setting My Children Up for Failure?’ Remote Learning Widens Education Gap. The Wall Street Journal.

Hoogeveen, J., & Pape, U. (2020). Data collection in fragile states: Innovations from Africa and beyond. Data Collection in Fragile States: Innovations from Africa and Beyond. Springer International Publishing.

Huff, K. (2020). National Data Quantifies Impact of COVID Learning Loss; Raises Questions About At-Home Testing.

Iglesias, C. (2020, March 10). The gender gap in internet access: using a women-centred method.

Institute of Medicine and National Research Council. (1997). Improving Schooling for Language-Minority Children: A Research Agenda (D. August & K. Hakuta (eds.)). National Academies Press.

Iqbal, S. A., Azevedo, J. P., Geven, K., Hasan, A., & Patrinos, H. A. (2020). We Should Avoid Flattening the Curve in Education - Possible Scenarios for Learning Loss during the School Lockdowns. World Bank Blog.

Jaume, D., & Willén, A. (2019). The long-run effects of teacher strikes: Evidence from Argentina. Journal of Labor Economics, 37(4), 1097–1139.

Jensen, L., Price, L., & Roxå, T. (2020). Seeing through the eyes of a teacher: differences in perceptions of HE teaching in face-to-face and digital contexts. Studies in Higher Education, 45(6), 1149–1159.

Jensen, V. M. (2013). Working longer makes students stronger? The effects of ninth grade classroom hours on ninth grade student performance. Educational Research, 55(2), 180–194.

Johnson, D. (2009). How do Work Stoppages and Work-to-Rule Campaigns Change Elementary School Assessment Results? Manuscript, Wilfred Laurier University.

Kaffenberger, M. (2020). Modeling the long-run learning impact of the Covid-19 learning shock: Actions to (more than) mitigate loss. RISE Insights.

Kaffenberger, M., & Pritchett, L. (2020). Failing to Plan? Estimating the Impact of Achieving Schooling Goals on Cohort Learning.

Kerry, T., & Davies, B. (1998). Summer Learning Loss: The Evidence and a Possible Solution. Support for Learning, 13(3), 118–122.

Khazan, O. (2020, August 17). Online School Is Harder Thanks to Unequal Internet Access. The Atlantic.

Kirschner, P. A., & De Bruyckere, P. (2017). The myths of the digital native and the multitasker. Teaching and Teacher Education (Vol. 67, pp. 135–142). Elsevier Ltd.

Kogan, V., & Lavertu, S. (2021). The COVID-19 Pandemic and Student Achievement on Ohio’s Third-Grade English Language Arts Assessment. Ohio Department of Education, 3–11.

Krogstad, J. M., Gonzalez-Barrera, A., & Noe-Bustamante, L. (2019). COVID-19 pay cuts, job losses hit Hispanics more than Americans overall. Pew Research Center.

Kuhfeld, M. (2019). Surprising new evidence on summer learning loss. Phi Delta Kappan, 101(1), 25–29.

Kuhfeld, M., Condron, D., & Downey, D. (2019). When Does Inequality Grow? A seasonal analysis of racial/ethnic disparities in learning in kindergarten through eighth grade (The Collaborative for Student Growth at NWEA Working Paper).

Kuhfeld, M., & Soland, J. (2020). The learning curve: Revisiting the assumption of linear growth across the school year. EdWorkingPaper, 20–214.

Kuhfeld, M., Soland, J., Tarasawa, B., Johnson, A., Ruzek, E., & Liu, J. (2020). Projecting the Potential Impact of COVID-19 School Closures on Academic Achievement. Educational Researcher, 49(8), 549–565.

Kuhfeld, M., & Tarasawa, B. (2020). The COVID-19 slide: What summer learning loss can tell us about the potential impact of school closures on student academic achievement.

Kuhfeld, M., Tarasawa, B., Johnson, A., Ruzek, E., & Lewis, K. (2020). Learning during COVID-19: Initial findings on students’ reading and math achievement and growth.

Lamb, S., Maire, Q., Doecke, E., Noble, K., Pilcher, S., & Macklin, S. (2020). Impact of learning from home on educational outcomes for disadvantaged children: Brief assessment to the Australian Government, Department of Education, Skills and Employment.

Larkins, C., Stoecklin, D., Milkova, R., Del Moral-Espin, L., Crowley, A., Mort, M., Easthope, L., Schuurman, M., Crook, D., & Fernandes, N. (2020). Building on Rainbows: Supporting Children’s Participation in Shaping Responses to COVID-19.

Lavy, V. (2012). Expanding School Resources and Increasing Time on Task: Effects of a Policy Experiment in Israel on Student Academic Achievement and Behavior. NBER Working Paper No. 18369. National Bureau of Economic Research, Volume 201(Number 95).

Lavy, V. (2015). Do Differences in Schools’ Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries. The Economic Journal, 125(588), F397–F424.

Lazear, E. (2001). Educational Production. National Bureau of Economic Research.

Lehmann, C. (2020). “Doing School” In The Time of Coronavirus – Practical Theory. Practical Theory.

Les Echos. (2020, March 31). Coronavirus : « Entre 5 et 8 % des élèves » sans continuité pédagogique depuis la fermeture des écoles. Les Echos. Les Echos.

Leuven, E., Lindahl, M., Oosterbeek, H., & Webbink, D. (2007). The Effect of Extra Funding for Disadvantaged Pupils on Achievement. The Review of Economics and Statistics, 89(4), 721–736.

Lieberman, M. (2020, June 29). A Third of K-12 Students Aren’t Adequately Connected for Remote Learning, Report Says. Education Week.

Loades, M. E., Chatburn, E., Higson-Sweeney, N., Reynolds, S., Shafran, R., Brigden, A., Linney, C., McManus, M. N., Borwick, C., & Crawley, E. (2020). Rapid Systematic Review: The Impact of Social Isolation and Loneliness on the Mental Health of Children and Adolescents in the Context of COVID-19. Journal of the American Academy of Child and Adolescent Psychiatry, 59(11), 1218-1239.e3.

Lorente, L. M. L., Arrabal, A. A., & Pulido-Montes, C. (2020). The Right to Education and ICT during COVID-19: An International Perspective. Sustainability, 12(21), 9091.

Maldonado, J. E., & De Witte, K. (2020). The effect of school closures on standardised student test outcomes. FEB Research Report Department of Economics (Issue September).

Mandel, P., & Süssmuth, B. (2011). Total instructional time exposure and student achievement: An extreme bounds analysis based on German state-level variation. Center for Economic Studies and Ifo Institute.

Mandery, E. (2020, May). We’re About to Have the Longest Summer Vacation Ever. That’s Going to Be a Problem. - POLITICO. Politico.

Marcotte, D. E. (2007). Schooling and test scores: A mother-natural experiment. Economics of Education Review, 26(5), 629–640.

Marcotte, D. E., & Hemelt, S. W. (2008). Unscheduled School Closings and Student Performance. Education Finance and Policy, 3(3), 316–338.

McBurnie, C., & Haßler, B. (2020). Is there learning continuity during the COVID-19 pandemic? Six lessons. Cambridge University Press. Cambridge University Press.

McCombs, J., Augustine, C., Schwartz, H., Bodilly, S., McInnis, B., Lichter, D., & Cross, A. (2011). Making Summer Count.

McDonnell, T., & Doyle, O. (2019). Maternal employment and childcare during infancy and childhood overweight. Social Science and Medicine, 243, 112639.

Meyers, K., & Thomasson, M. A. (2017). Paralyzed by Panic: Measuring the Effect of School Closures during the 1916 Polio Pandemic on Educational Attainment.

Miller, R. T., Murnane, R. J., & Willett, J. B. (2007). Do teacher absences impact student achievement? Longitudinal evidence from one urban school district. Educational Evaluation and Policy Analysis, 30(2), 181–200.

Milligan, I. (2020). Emergency Remote Teaching: A Post Secondary Reality Check – Active History. ActiveHistory.Ca.

Mohan, G., McCoy, S., Carroll, E., Mihut, G., Lyons, S., & Mac Domhnaill, C. (2020). Learning for all? Second-level education in Ireland during COVID-19.

Montebruno, P. (2020). CEP Discussion Paper No 1696 May 2020 Disrupted Schooling : Impacts on Achievement from the Chilean School Occupations Piero Montebruno. Centre for Economic Performance, 1696.

Nisbet, J. D., & Entwistle, N. J. (1969). The transition to secondary education. London: University of London, 59.

OECD. (n.d.). Skills Outlook 2021. OECD Publishing.

OECD. (2010). Inspired by Technology, Driven by Pedagogy: A Systemic Approach to Technology-Based School Innovations. OECD Publishing.

OECD. (2011). PISA - Quality Time for Students: Learning in and out of School. Organization of Economic Cooperation and Development.

OECD. (2018a). Bridging the digital gender divide: include, upskill, innovate.

OECD. (2018b). The Future of Education and Skills: Education 2030.

OECD. (2019). PISA 2018 Results (Volume II): Where All Students Can Succeed. OECD.

OECD. (2020a). Combatting COVID-19’s effect on children (OECD Policy Responses to Coronavirus (COVID-19)).

OECD. (2020b). Education and COVID-19: Focusing on the long-term impact of school closures.

OECD. (2020c). Learning remotely when schools close: How well are students and schools prepared? Insights from PISA.

OECD. (2020d). Strengthening online learning when schools are closed: The role of families and teachers in supporting students during the COVID-19 crisis.

OECD. (2020e). Teachers’ training and use of information and communications technology in the face of the COVID-19 crisis (No. 35; Teaching in Focus).

OECD. (2020f). The impact of COVID-19 on student equity and inclusion: Supporting vulnerable students during school closures and school re-openings.

OECD. (2020g). Early Learning and Child Well-being: A Study of Five-year-Olds in England, Estonia, and the United States. OECD Publishing.

OECD. (2021). The State of School Education: One Year into the COVID Pandemic.

Palloff, R. M., & Pratt, K. (2013). Lessons from the Virtual Classroom: The Realities of Online Teaching (2nd ed.). Jossey-Bass.

Parolin, Z. (2020). Unemployment and child health during COVID-19 in the USA. The Lancet Public Health, 5(10), e521–e522.

Pepper, D. (2007). The Key Stage 3 dip : myth or reality ? (Issue August).

Peterson, A., Dumont, H., Lafuente, M., & Law, N. (2018). Understanding innovative pedagogies: Key themes to analyse new approaches to teaching and learning (No. 172; OECD Education Working Papers).

Petrie, C., Aladin, K., Ranjan, P., Javangwe, R., Gilliland, D., Tuominen, S., & Lasse, L. (2020). Spotlight: Quality Education for all during Covid-19.

Philipsen, B., Tondeur, J., Pareja Roblin, N., Vanslambrouck, S., & Zhu, C. (2019). Improving teacher professional development for online and blended learning: a systematic meta-aggregative review. Educational Technology Research and Development, 67(5), 1145–1174.

Pier, L., Hough, H. J., Christian, M., Bookman, N., Wilkenfeld, B., & Miller, R. (2021). COVID-19 and the Educational Equity Crisis. Policy Analysis for California Education.

Quinn, D. M., & Polikoff, M. (2017). Summer learning loss: What is it, and what can we do about it?

Reich, J., & Ito, M. (2017). From Good Intentions to Real Outcomes: Equity by Design in Learning Technologies.

Reimers, F. M., & Schleicher, A. (2020a). A framework to guide an education response to the COVID-19 Pandemic of 2020.

Reimers, F. M., & Schleicher, A. (2020b). Schooling disrupted schooling rethought How the Covid-19 pandemic is changing education.

Renaissance Learning. (2020). How Kids Are Performing.

Renaissance Learning. (2021). How Kids Are Performing: Tracking the Midyear Impact of COVID-19 on Reading and Mathematics Achievement (Winter 2020–2021 Edition; Special Report Series).

Rennie, E., Yunkaporta, T., & Holcombe-James, I. (2018). Privacy Versus Relatedness: Managing Device Use in Australia’s Remote Aboriginal Communities. International Journal of Communication, 12.

Rice, J. K. (2001). Explaining the Negative Impact of the Transition from Middle to High School on Student Performance in Mathematics and Science. Educational Administration Quarterly, 37(3), 372–400.

Rivkin, S. G., & Schiman, J. C. (2015). Instruction Time, Classroom Quality, and Academic Achievement. Economic Journal, 125(588), F425–F448.

Rodrigues, H., Almeida, F., Figueiredo, V., & Lopes, S. L. (2019). Tracking e-learning through published papers: A systematic review. Computers and Education, 136, 87–98.

Saavedra, J. (2020, March 30). Educational challenges and opportunities of the Coronavirus (COVID-19) pandemic. World Bank Blogs.

Sacerdote, B. (2012). When the saints go marching out: Long-term outcomes for student evacuees from hurricanes Katrina and Rita. American Economic Journal: Applied Economics, 4(1), 109–135.

Scheerens, J., Luyten, H., & Glas, C. (2013). Productive Time in Education.

Selbervik, H. (2020). Impacts of school closures on children in developing countries: Can we learn something from the past?

Selwyn, N. (2016). Minding our language: why education and technology is full of bullshit … and what might be done about it. Learning, Media and Technology (Vol. 41, Issue 3, pp. 437–443). Routledge.

Sevilla, A., & Smith, S. (2020). Baby Steps: The Gender Division of Childcare during the COVID-19 Pandemic (No. 13302; IZA Discussion Paper Series).

Shank, S. (2019). Will technology transform education for the better?

Shores, K., & Steinberg, M. (2018). The Impact of the Great Recession on Student Achievement: Evidence from Population Data. SSRN Electronic Journal.

Simmons, R. G., Burgeson, R., Carlton-Ford, S., & Blyth, D. A. (1987). The Impact of Cumulative Change in Early Adolescence. Child Development, 58(5), 1220.

Smith, C. (2020, November 5). Dallas students suffered ‘horrifying’ learning loss during pandemic, new data suggests. The Dallas Morning News.

Sonnemann, J., & Goss, P. (2020). COVID catch-up: helping disadvantaged students close the equity gap.

Stelitano, L., Doan, S., Woo, A., Diliberti, M., Kaufman, J., & Henry, D. (2020). The Digital Divide and COVID-19: Teachers’ Perceptions of Inequities in Students’ Internet Access and Participation in Remote Learning.

Sternberg, M. (2020). Closing the Digital Divide.

Stevens, A. H., & Schaller, J. (2011). Short-run effects of parental job loss on children’s academic achievement. Economics of Education Review, 30(2), 289–299.

Tatsuta, N., Nakai, K., Satoh, H., & Murata, K. (2015). Impact of the Great East Japan Earthquake on Child’s IQ. Journal of Pediatrics, 167(3), 745–751.

Thamtanajit, K. (2020). The Impacts Of Natural Disaster On Student Achievement: Evidence From Severe Floods in Thailand. The Journal of Developing Areas, 54(4).

The Economist. (2020, April 30). No more pencils, no more books: Closing schools for covid-19 does lifelong harm and widens inequality. The Economist.

Thomas, J., Barraket, J., Wilson, C. K., Rennie, E., Cook, K., Louie, Y. M., Holcombe-James, I., Ewing, S., & MacDonald, T. (2019). Measuring Australia’s digital divide: The Australian digital inclusion index 2018.

Thompson, P. N. (2021). Effects of Four-Day School Weeks on School Finance and Achievement: Evidence From Oklahoma. Educational Researcher, 50(1), 30–40.

Thomson, S. (2020, April 20). What PISA tells us about our preparedness for remote learning. Teacher.

Tomasik, M. J., Helbling, L. A., & Moser, U. (2020). Educational gains of in-person vs. distance learning in primary and secondary schools: A natural experiment during the COVID-19 pandemic school closures in Switzerland. International Journal of Psychology.

U.S. Department of Education. (2016). Chronic Absenteeism in the Nation’s Schools: An Unprecedented Look at a Hidden Educational Crisis.

UNESCO. (2020a). COVID-19 school closures: Why girls are more at risk.

UNESCO. (2020b). School closures caused by Coronavirus (Covid-19).

UNESCO, UNICEF, & The World Bank. (2020). What have we learnt? Overview of findings from a survey of ministries of education on national responses to COVID-19. UNESCO, UNICEF, World Bank.

UNICEF. (2020). Averting a lost COVID generation A six-point plan to respond, recover and reimagine a post-pandemic world for every child.

United Nations. (2020). Policy Brief: Education during COVID-19 and beyond.

Van Lancker, W., & Parolin, Z. (2020). COVID-19, school closures, and child poverty: a social crisis in the making. The Lancet Public Health (Vol. 5, Issue 5, pp. e243–e244). Elsevier Ltd.

von Hippel, P. T. (2019a). Is summer learning loss real? Educationnext.Org, 19(4).

von Hippel, P. T. (2019b). Summer Learning: Key Findings Fail to Replicate, but Programs Still Have Promise. Education Next.

von Hippel, P. T., & Hamrock, C. (2019). Do test score gaps grow before, during, or between the school years? Measurement artifacts and what we can know in spite of them. Sociological Science, 6, 43–80.

von Hippel, P. T., Workman, J., & Downey, D. B. (2018). Inequality in Reading and Math Skills Forms Mainly before Kindergarten: A Replication, and Partial Correction, of “Are Schools the Great Equalizer?”. Sociology of Education, 91(4), 323–357.

Ward, M. E., & Shelley, K. (2008). Hurricane Katrina’s Impact on Students and Staff Members in the Schools of Mississippi. Journal of Education for Students Placed at Risk (JESPAR), 13(2–3), 335–353.

Ward, M. E., Shelley, K., Kaase, K., & Pane, J. F. (2008). Hurricane Katrina: A Longitudinal Study of the Achievement and Behavior of Displaced Students. Journal of Education for Students Placed at Risk (JESPAR), 13(2–3), 297–317.

Warren, H., & Wagner, E. (2020). Save Our Education: Protect every child’s right to learn in the COVID-19 response and recovery. Save the Children International.

Watterston, J., & O’Connell, M. (2019). Those Who Disappear: The Australian education problem nobody wants to talk about.

Whitby, K., Lord, P., O’Donnell, S., Grayson, H., & NFER. (2006). Dips in performance and motivation: a purely English perception? Thematic Probe. International Review of Curriculum and Assessment Frameworks Internet Archive, September, 1–55.

Wills, G. (2014). The Effects of Teacher Strike Activity on Student Learning in South African Primary Schools (No. 402; ERSA Working Paper).

Wilson, R., McGrath-Champ, S., & Mude, W. (2020). Preliminary results from a survey of remote learning arrangements during COVID-19.

Woodworth, J. L., Raymond, M. E., Chirbas, K., Gonzalez, M., Negassi, Y., Snow, W., & Van Donge, C. (2015). Online Charter School Study.

Worley, W. (2020, April 9). COVID-19 threatens decades of progress on global poverty, report says. Devex.

Wößmann, L. (2003). Schooling resources, educational institutions and student performance: The international evidence. In Oxford Bulletin of Economics and Statistics (Vol. 65, Issue 2, pp. 117–170). John Wiley & Sons, Ltd.

Wößmann, L. (2016). The importance of school systems: Evidence from international differences in student achievement. Journal of Economic Perspectives, 30(3), 3–31.

Yeşil Dağlı, Ü. (2019). Effect of increased instructional time on student achievement. Educational Review, 71(4), 501–517.

Yousafzai, A. (2020, April 1). Low-cost private schools may not be able to survive COVID-19 crisis. The News International.

Zwerling, H. (2008). Pennsylvania teachers’ strikes and academic performance. Journal of Collective Negotiations, 32(2), 151–172.

Endnotes

1. A quantity expressing by how much the members of a group differ from the mean value for the group. ↩ ↩²

2. To be precise, in England, for example, although schools were closed to the majority of students, most schools remained open to vulnerable students and the children of key workers. ↩

3. Answers were received from July 15, 2020 to October 15, 2020, with August 20, 2020 as average date. ↩

4. Countries where the academic year was still ongoing at the time of the survey reported more days of teaching lost (54 days) on average compared with those where the academic year had finished at the time of the survey (40 days). ↩

5. The term used to describe any system in which the response depends not only on its current state, but also on its historic states. ↩

6. ‘Three possible scenarios of how the learning curve may evolve in the coming months: a lower average, a higher standard deviation, or a sharp increase in low learning at the bottom’ by The World Bank is licensed under CC BY 4.0. ↩

7. In the United States, a metropolitan statistical area (MSA) is a geographical region with a relatively high population density at its core (typically a single large city) and close economic ties throughout the area. ↩

8. For reference, Wößmann (2016) suggested that, as a rule of thumb, learning gains during one year on most national and international tests are equal to between one-quarter and one-third of a standard deviation. ↩

9. Broadly defined as students whose characteristics or circumstances put them in a vulnerable position for having academic and/or social problems in school. ↩

10. The historical comparison sample is based on a rolling average of autumn test scores across 2017, 2018, and 2019, for any student in the same schools analysed in autumn 2020. ↩

11. Historically, 27% of students in Reading placed below grade level on the autumn Diagnostic. In autumn 2020, 28% of students placed below grade level. Historically, 23% of students in Mathematics placed below grade level on the autumn Diagnostic. In autumn 2020, 29% of students placed below grade level. ↩

12. The languages learned at school are officially labelled French and Dutch. ↩

13. Programme for International Student Assessment ↩

14. The tests were taken by almost all 18-year-old men in preparation for military service in Sweden. ↩

15. An online school is a school which provides the majority of classes (everything except PE, band, or a similar elective) to full-time students through a computer via the internet (Woodworth et al., 2015). ↩

16. These included the Czech Republic, Denmark, Estonia, Hungary, Italy, Lithuania, the Netherlands, Norway, the Slovak republic, and Sweden. ↩

17. Delivering Equality of Opportunity in Schools (DEIS) Plan 2017 is the Department of Education and Skills’ policy instrument to address educational disadvantage. ↩

18. For updated charts, see the Opportunity Insights Economic Tracker. Data accessed November 15, 2020. ↩