Data Collection/Analysis and Covid-19
Critics of the modeling and of the data analysis are being too simplistic.
Let me start this post by violating a cardinal rule of the internet these days and starting that I am not an expert on epidemiology nor am I qualified to offer a detailed critique of the models being used to project the effects of Covid-19. I note this to be upfront, but also to underscore that it seems a lot of people out there quite confident in their expertise. And, not surprisingly, their political preferences seem to shape the findings of their newfound ability to assess data related to the epidemic. Although, in fairness, like Peter Navarro, I am a social scientist with a Ph.D.*
A focus of these conversations is the University of Washington’s Institute for Health Metrics and Evaluation (IHME) model. This has been exemplified by Alex Bereson (see, via FNC: Meet the former NYT reporter who is challenging the coronavirus narrative). A related discussion is that model’s downward estimation of the death toll of the disease (from the 100k-240k range to 60k).
Ultimately, it is about the debate over whether shutting down the economy to try and mitigate the health effects of the virus was worth the economic costs. This is a fair question. A real answer is going to take time. We do not yet know the actual toll of this disease nor do we know what the actual economic damage will be. Further, it is all but impossible to know which choice was optimal, as there are clear downsides to both. At a minimum, I think it is worth noting that there was going to be an economic impact regardless because even without stay-at-home orders, the pandemic was going to create economic damage (and it is even possible that health-related economic damage has been mitigated since higher death tolls and greater stress to the healthcare system would have had effects as well).
I will confess that my bias leans towards the pro-mitigation via staying at home side because death is harder to recover from than is unemployment. I say that firmly understanding that unemployment is a real hardship.
The part of this that I find frustrating is the fact that people are attempting to draw very firm conclusions (and influence public opinion) based on incomplete data.
This conversation is inherently partisan insofar as Republicans are seeking in many cases to lessen to the amount of blame President Trump is receiving in regards to his response to the pandemic. After all, if the effects are not as bad as some said it would be, that means that either a) he did a great job responding (which is what he is going to say no matter what the death toll is) or b) that it was never all that bad to begin with. I will admit, that even if the death toll is “only” 60,000 (or if, by a miracle, it stops at 30,000) that is a far cry from the claims Trump was making just over a month ago (i.e., talking like it was all going to go away and that the federal government was doing a “great job”).
Moreover, since Trump was first more concerned about the stock market than he was about the virus and has been consistently arguing to “reopen the country” the public health v. economic health issue has also been partisan. The ways in which governors in the heavily Republican southeast have underscored this (see, e.g., Alabama, Georgia, Florida, and Mississippi).
And, without a doubt, media consumption affects one’s views of these matters.
While I am not qualified to tear down the models, I can comment on the basic problems of data collection and the coding of said data. Also, there is the fundamental problem of trying to reach firm conclusions about a complex situation in the middle of the event unfolding. We have to remind ourselves that we are not anywhere near done with this crisis, so the notion that we can make definitive statements about policy choices is just plain foolish and almost certainly are going to be heavily influenced by motivated thinking.
For example, any critique of the models (or praise thereof) has to deal with a fundamental problem that we have had from the beginning: the lack of widespread testing. This is true about the living and the deceased. Without that information, we cannot really know how well the models have predicted things like spread or morbidity. It also makes it harder to know about either the need for social distancing nor about its efficacy.
There is also the problem of coding deaths (i.e., how are deaths identified in terms of building datasets of who died from what?). Some (as noted in the comments to James Joyner’s earlier post) think that we are underestimating deaths due to lack of testing (and just lack of data/confusion over the issue across the country).
Others think we are over-counting. One person who has been touting skepticism has been Brit Hume, both on his Twitter feed and on Fox News Channel.** For example, the following from Tucker Carlson’s show:
But there’s another side to this. Dr. Birx said tonight during the briefing at the White House that all deaths from anyone who died with coronavirus is counted as if the person died from coronavirus.
Now, we all know that isn’t true. I remember my own doctor telling me at one point when I was discussing prostate issues, he said about prostate cancer — I didn’t have it, as it happened, but he said, “You know, a lot more people die with it than die from it.”
That’s a real possibility, that people who have this disease, particularly they’re — lots of people are asymptomatic, who may have other terrible diseases. And if everybody is being automatically classified, if they’re found to have covid-19, as a covid-19 death, we’re going to get a very large number of deaths that way and we’re probably not going to have an accurate count of what the — of what the real death total is.
The problem is, this doesn’t make any sense. For one thing, a person who dies with no Covid-linked symptoms isn’t going to be tested. For example, a person who dies from a brain aneurysm but who is asymptomatic for Covid-19, but who is infected, is not going to be tested nor are they going to be categorized as a Covid-19 death.
Indeed, it is almost certainly the case that deaths coded as being from Covid-19 are going to be from people who have respiratory symptoms. Indeed, one would think that the process is not unlike how the CDC assesses influenza-related deaths:
Seasonal influenza-related deaths are deaths that occur in people for whom influenza infection was likely a contributor to the cause of death, but not necessarily the primary cause of death.
Since skeptics seem to want to liken all of this to the seasonal flu, perhaps they should think through how those deaths are accounted for.
In all honesty, especially given the lack of testing capacity, it seems far more likely that we are underestimating deaths rather than overestimating them.
Fundamentally it is this lack of information that has us in the conundrum we are in. That is: we have closed down social interaction to the level we have because we simply don’t know how prevalent the virus is in a given community. Further, we do not how many people are asymptomatic, nor how many have had the disease and have recovered. We don’t even know for sure if the recovered are now immune.
Further, we simply don’t know if we have been more successful than predicted with social distancing (and that explains the models being off) or we didn’t need as much social distancing as we have deployed (and that is what explains the models being off). There may, in fact, be other explanations for the seeming problems with the models.
And the reality is: modeling this kind of this is hard.
It seems quite reasonable to say we don’t know for sure if we have done the right thing in terms of the stay-at-home orders? It may well be that we have over-reacted. It is also true that we do not have the testing necessary to determine exact numbers at this time.
(It’s as if preparing for testing on a mass scale several months ago would have been a really good idea.)
I understand that there is a huge economic cost to our current approach, but my frustration with the critics is based in the fact that they are being far too confident that the data they have are enough to draw conclusions about the models. Ther are assuming that the current infection and death toll figures are accurate (or even that the death tolls are inflated). Since we lack testing capacity (and since there is a lag in results for the tests we have) it is problematic, to put it mildly, to say that we really know what the situation really is. It is not even clear that we are getting clear results in general (see this story about nursing home deaths).
Ultimately it is easy to sit at home from a safe distance and be skeptical about the situation. And, it can be easy to see the economic impact and be rightly concerned. But I am skeptical of persons who are neither epidemiologists nor directly involved in data collection and analysis proclaiming that they know for sure what the best course of action is, especially when they are suggesting the riskier scenario in terms of public health.
One thing I know for sure: it will take years to fully understand what the effects of this disease were as well as to fully evaluate the human costs both in terms of health and the economy. Another thing I am certain of is that the policy choice at the moment is far more complex than “open up the country” or not.
*If my snark is too subtle here, I thought that Navarro was being an arrogant ass by suggesting that his social science skills meant that he should be taken seriously as an expert source in regards to whether hydroxychloroquine was an efficacious drug for use to treat Covid-19. I would further note that his training should have made him skeptical about a study with such a small N (although medical studies often have Ns that make social scientists very uncomfortable), the sample was not random, and especially the fact that several of the participants who took the drug were excluded from the final study because they had to go to the ICU, one of whom died.
**It is worth noting the Hume largely endorsed Texas Lt. Gov Dan Patrick’s call to reopen the economy knowing it would put the elderly at risk, so his thinking on this topic is suspect, IMHO.