Pandemic Closures Crushed Kids
We're seeing our lowest test scores in decades among a key cohort.
The NYT’s Dana Goldstein analyzes “What the New, Low Test Scores for 13-Year-Olds Say About U.S. Education Now.”
The math and reading performance of 13-year-olds in the United States has hit the lowest level in decades, according to test scores released today from the National Assessment of Educational Progress, the gold-standard federal exam. The last time math performance was this low for 13-year-olds was in 1990. In reading, 2004.
That’s a pretty shocking drop given that the trend on standardized tests is upward over time (the “Flynn Effect”).
Performance has fallen significantly since the 2019-2020 school year, when the coronavirus pandemic wrought havoc on the nation’s education system. But the downward trends reported today began years before the health crisis, raising questions about a decade of disappointing results for American students.
The federal standardized test, known as NAEP, was given last fall, and focused on basic skills. The 13-year-olds scored an average of 256 out of 500 in reading, and 271 out of 500 in math, down from average scores of 260 in reading and 280 in math three years ago.
Achievement declined across lines of race, class and geography. But in math, especially, vulnerable children — including Black, Native American and low-income students — experienced bigger drops.
A large body of research shows that most American children experienced academic struggles during the pandemic. It has also been clear that low-income students of color were most heavily affected by school closures and remote learning, which in some districts lasted more than a year.
The latest NAEP results are the federal government’s final major release of data on pandemic learning loss. The scores add to educators’ understanding of the challenges that lie ahead for children of different ages and demographic groups.
The 13-year-olds who took this version of the NAEP exam last fall were 10 years old — and in fourth or fifth grade — when the pandemic began. Many were old enough to participate in remote learning without minute-to-minute adult assistance, as younger children often needed.
But the ages of 10 to 13 are also a crucial period for mastering foundational skills, from multiplication to recognizing a character’s feelings in a short narrative passage.
“The bottom line — these results show that there are troubling gaps in the basic skills of these students,” said Peggy Carr, commissioner of the National Center for Education Statistics, which gives the NAEP exam. “This is a huge-scale challenge that faces the nation.”
That essentially skipping a year of school and then continuing pandemic practices like putting a floor of 50% on grading even for assignments that were never turned in had a negative impact on student learning is not the least bit surprising. Nor is the disparate impact on those whose parents were least in a position to backstop the failures of our schools.
This, by the way, is true regardless of whether closing schools and keeping them closed as long as we did was a good idea. I favored closing them at the outset, when public health experts were recommending that we do so and the rationale was that we could “flatten the curve” by sucking it up for two or three weeks. When the evidence began to mount that it was safe for kids to go back to school, I was frustrated with teachers unions and others who resisted the call to do so. Regardless, there were no great options in a once-in-a-century crisis.
The results, though, speak for themselves. And, again, NAEP is the gold standard:
In the highly decentralized American education system, NAEP is one of the few consistent tests given across states lines over many years, making the results easily comparable.
Scores on the exam do not result in any rewards or punishments for students, teachers or schools, making them especially useful for research purposes, since there are fewer incentives to cheat or teach to the test.
Still, there are critics who downplay all of this:
Still, some education experts believe there is too much focus on NAEP. They point out that the content of the exams, in many cases, has little overlap with the material that is actually taught in classrooms across the country.
My experience is anecdotal and limited to Northern Virginia (where my kids go to school) and the Philly suburbs (where two of my three stepkids graduated high school) but I would be surprised if any significant school district in the country wasn’t teaching math and reading. And, if they’re not, they really ought to start rather than us adjusting the test to account for that fact.
A student survey given alongside the test turned up other interesting results that will keep educators buzzing. The percentage of 13-year-olds enrolled in algebra has declined to 24 percent from 34 percent in 2012. In some districts and states, notably California, there has been a push to equalize math education by placing fewer eighth graders into advanced math.
The percentage of 13-year-olds who reported reading for fun has also declined. Last fall, 31 percent said they “never or hardly ever” read for fun, compared to 22 percent in 2012.
Again, anecdotally, I didn’t take algebra until 9th grade (when I was 14 and 15) and my rising 9th grader (who’s 14) took pre-algebra last year, so I’m surprised that so many are taking it at 13. But there’s been a longstanding push to get rid of algebra as a graduation requirement because it’s a huge obstacle for some students and widely considered unnecessary for those not going on the major in STEM fields in college. And, seemingly in contradiction to that realization, there’s been a very recent push to pretend that every child is equally talented in math.
I’m curious, and maybe someone will know, what was the effect of the Blitz on London’s school children’s education?
@Michael Reynolds:
A fair number were evacuated. The bombings were at night, so I’m guessing they attended what school were left standing.
My wife is an elementary school counselor at a Title I school. This past year seems to have been the toughest yet. Kids with home problems, teachers with job and home problems, aggressive (in a bad way) parents, demoralized administrations fighting the good fight. The days were long and exhausting with the school police and state child protective services being called on a regular basis.
Next year is shaping up to be tough also. A lot of good teachers have indicated they are not coming back. Expect more backsliding.
There’s a new contagious disease. Kids get well. Some number of exposed teachers will die of it. More will have long-term health problems. The disease can be contagious but asymptomatic, so teachers’ families may also be at risk. Nevertheless, teachers should just suck it up and take their chances.
@Michael Cain: We asked people doing jobs much less ‘essential’ than teaching our children to return to work while masked. We did it at our institution, teaching mid-career officers, in July 2020. Most of Europe did so around the same time with schoolchildren.
To continue MR’s Blitz analogy and thoughts on James’s earlier post on disaster culture wars. It seemed Covid didn’t ask for sacrifice and getting on with what was important but competing among ourselves about health concerns vs. “ain’t nobody going to tell me what to do.” We asked essential workers to risk themselves but didn’t consider educators that essential although grocery workers and liquor retailers were.
@James Joyner:
My understanding — quite possibly wrong, I’m as prone to selectively remember reading material as anyone — is that in practice the large European countries ended up with significant missed time (40-50% of classroom hours), and similar loss-of-learning problems. What we’ve learned is that at least for most of the littles, distance learning doesn’t work and hybrid distance and in class learning isn’t much better. That’s not really surprising, since one of the things we’re trying to teach them is how to learn.
Full disclosure: For some time over the period 25-30 years ago I did research on providing real-time multi-person multi-media communication over internet protocols. It’s a hard problem, and a 25-person classroom is probably the hardest variant. So far as I can tell, much of what we did learn has been ignored.
@James Joyner:
“We left poor people who had no union to fight for their interests no choice but to risk their lives for slave wages.”
But since — finally (and conveniently) — we recognize that educators are essential and not just glorified babysitters, maybe we should start treating them like professionals and pay them as such. One can dream.
It’s also nice to see us suddenly in love with how Europe does things. Will we therefore start advocating for universal health care, robust mass transit via train, guaranteed housing, and free college?
You can’t manage a pandemic like Europe does when you start with population baselines that are fatter, dumber, and unhealthier.
Maybe let’s not blame teachers and their union for prioritizing safety (physical and emotionally) for a trend that started pre covid. And if we want to make a dent in repairing the damage, ubi for families and living wages for teachers and classroom staff is lowest hanging fruit.
This is a thought-provoking post. But the failure is not in American schools, the failure is in the American people.
Test scores have been in decline for a decade. School shootings have risen. Our response has been to give too much power to a Republican Party that seeks to defund schools and block gun control.
The United States is not serious about education and does not care about teachers or students. If we were, Democrats would have total control.
James- I would not assume that we know all of the reasons why we had the drop in scores. The obvious thing to jump to is the lockdowns. However, you can see the raw scores at my link fo individual states. We know that in some states the kids were mostly back in school the fall or spring after the pandemic started. We know that another states they waited much longer. I went through by hand the math scores, 8th grade, for all of the states and offhand I dont see a pattern favoring states that had shorter lockdowns. Just looking at California, Florida and Texas, large states with different approaches we see California having a 6 point drop, Florida and Texas 7 point drops. Someone with he time and energy can do numbers for all of the states and track it by lockdown length but just looking at my initial scan I dont expect to see a large difference. (The WSJ made a half hearted attempt but they compared already poorly performing districts in poor cities with wealthier areas in red states, though even their cherry picked areas had drops in scores.)
There are many other factors that could cause drops. Lots of kids have grandparents as caregivers, If those died or had long term illnesses that could affect scores. A parent who had severe covid and had long term sequelae. If teachers were out sick that could affect scores. Some parents kept kids at home even when schools opened. How many teachers retired or left the profession due to the pandemic and its fallout? How disrupted was home life if parents lost job due to pandemic?
This is just a quick off the cuff list. My gut feeling is that longer lockdowns probably did have an effect but we dont know how much and in many cases it wont be the major factor but we need some studies on this and not just make assumptions.
https://www.nationsreportcard.gov
Steve
@CSK: And yet, both of you bring up a good point. This was a (hopefully) once in a lifetime event, and we did lack the resolve (and probably the ability) to offset the damage that it does. We could (and probably should) recalibrate whether we have a cohort who will no longer be able to “graduate” (or attendance certificate, in some districts) by age 18 and simply admit that the obligation will expand for that cohort. But we won’t. Much easier and more entertaining to whinge about it.
ETA: @Mr. Prosser: “We asked essential workers to risk themselves but didn’t consider educators that essential although grocery workers and liquor retailers were.”
True, but that’s a comment on our priorities and the degree to which “essential” equaled “expendable.”
There hasn’t been a correlation shown between the school closure policies and the outcomes on the test scores. Scores are down across all states, at rates that show no obvious relationship to closures.
People (like our esteemed host) are reading their own preconceived biases into it.
We need a more detailed, county or district comparison, charting against days of school closure, probably broken down by income levels of the families and districts (does a wealthy white district have more resources for remote learning that will make it more effective? Probably), but at the moment, we don’t know.
I’ll offer another possible explanation: 70% of children have had covid, and since we know covid causes brain fog in adults, it might just be making kids growing brains dumber.
There’s no more evidence for or against that than there is for or against the school closure hypothesis. But, it has a more universal aspect that at fist glance matches the data better.
There are lots of other possible causes, including plain, simple trauma.
A hell of a lot more research is needed before we can start making policy judgments. Let’s hope we aren’t faced with another pandemic before that happens.
The actual reason is likely to be a mix of trauma, infection, closure and a dozen other things. Jumping straight to the closures means other causes that can be remedied will be ignored.
And then there’s this:
I doubt that covid had no effect, but it may be masking an entirely unrelated problem that we should be identifying and fixing.
@Mr. Prosser:
Liquor store workers as essential sounds absurd, and people make fun of it. But, sudden detox for alcoholics is a bitch, and was going to send a lot of people to ERs at a time when ERs were already overloaded.
It was good policy, despite sounding stupid.
No evidence mounted for the safety of kids in schools, just some hopeful claims based on nothing.
Current studies showed that kids did what we would naively expect them to do — bring home disease and infect their families. All the claims that kids can’t spread covid were just bullshit.
I’m not saying that we should have closed schools for longer — I’m just saying that decisions shouldn’t be made on the basis of bullshit claims.
This is another frustrating, US-centric report. Are we unique and how do we know? What is the comparison to kids in Europe, Asia?
@Sleeping Dog:
No, we’re not unique. People are doing cross-sectional work that includes many different countries.
https://blogs.worldbank.org/education/100-weeks-pandemic-importance-keeping-schools-open-and-investing-learning-recovery
It is unfortunate that so many kids under the thumb of the government schools have lost learning. But look on the brightside of the government forcing school abandonment of students. A good number of parents were made to see just how miserable their child was in school and they blossomed under alternative learning. Parents started seeing how weak the instruction and how alarming the material was their child was receiving. Of course, many parents who have started fighting the government schooling commissars for changes are being hounded by the DOJ, but others are creating alternative learning paths for their and other children.
Thousands of children have been rescued from “school helplessness”
and are now relearning to take the initiative in their learning, even if they are still subject to the experience of uncontrolled classrooms.
We are 3 years out from the inflection give it another 8 years or so to really see the impact. Likely, grossly negative for those still trapped, but so many kids have escaped.
@Gustopher:
One of my hypotheses is that somewhere down the road there will be PhD dissertations written about effective rather than reported contact rates in the US. Basically, that it is possible to work backwards from known hospitalization rates and how contagious those variants were to determine what the contact rate must have been. Also that nominal lockdown policies in the US had relatively little impact on transmission because of hidden contacts. For example, we already know that the NYC metro area had “hidden” mass contact events for weddings and funerals despite their official lockdown policies.
@James Joyner: James — could it be that it was not closures but the goddamn pandemic as other evidence (NAEP) showed no systemic pattern of differential test scores among states and counties that had long closures and states and counties that had short closures.
A massive shock that disrupts everything in society, kills a lot of grandparents and parents and puts even more into hospitals while disrupting routines not having effects on kids would be utterly bizarre.
@Michael Cain: Those papers are already being written in Epi and Public Health departments… I reviewed one in March (massive revision as there is fuzziness in their identification strategy that gave me the heebee jeebies (highly technical term there)). And if I am reviewing this flavor of paper at that particular journal, then there are a lot more of those papers out there at various stages.
@JKB:
Wow! That “hounded by the DOJ” stuff is really potentially damning. I wish Ben Shapiro had done an episode on that instead of having the incel guy on bloviating stuff about feminism in Math class that I know isn’t happening in my district. 🙁
ETA: And as a retired teacher, I’d be personally interested in hard data about the “creating alternative learning paths” part given that the ones that I’ve seen seem pretty unimpressive. (And I already know that homeschooling outperforms classroom learning at the early stages simply because of the intensified attention factors of face-to-face instruction in direct instruction situations and topics.)
@Michael Cain:
Case timing, rather than total cases, may have been affected. And, given that hospitals were on the brink of collapse in many areas (and past that in several), delaying cases may have saved lives.*
Plus, we figured out how to treat Covid better.
I don’t think we can say “lockdowns didn’t work” as much as we can say “lockdowns didn’t work based on metrics A, B and D, but did work on C, E, and G, and inconclusive on H, and F just makes no sense in retrospect.”
All of which will be important to know if bird flu makes the jump from sea lions to other mammals, and we are faced with similar decisions.
——
*: “may have saved lives” because I can see a possibility that care in a nearly overwhelmed hospital decreases, and that by holding hospitals at the nearly overwhelmed state longer that it could result in more deaths than just letting a lot of people die quickly and having the hospital system bounce back towards more normal faster.
Not the most likely scenario, but plausible. Complex systems are complex and there are all sorts of feedback loops.
So some learning loss was inevitable, but continuing to close schools after the vaccine was widely available and prioritized for teachers was a bad policy IMO that caused a variety of bad outcomes, especially among the most vulnerable populations.
Obviously, in a situation like this, the relative risks and costs must be weighed, and different people can make reasonably different judgments, which is what individual states and school districts did.
However, if education is an essential government-provided public service, then governments have a responsibility to ensure adequate education services are provided.
Here’s what I wrote to my own district’s school board in July 2000 when they were debating options for the fall of 2020, which I think has aged well:
Our district overall had good policies – offering the option of 100% remote, hybrid, or synchronous learning in the fall of 2020.
@Gustopher:
There have actually been several studies that show in-person students have better test scores compared to kids who were all-remote. And this makes sense – everyone understands that remote learning is inferior to in-person learning for most people, especially kids.