[ad_1]
Frustrated statisticians and epidemiologists flocked to social media this week to report substantial flaws in two widely publicized studies trying to estimate the true spread of COVID-19 in two California counties, Santa Clara and Los Angeles.
The studies suggested that many more people in each of the counties have been infected with the new coronavirus than previously thought, that is, they estimated that true case counts in the two counties are up to 85 times and 55 times the number of currently confirmed cases in counties, respectively. Consequently, this suggests that COVID-19 is much less deadly than previously thought. The large number of cases relative to the number of deaths unchanged places the COVID-19 death rate in the same range as seasonal flu.
How dangerous is this?
We dig into the details of the studies below, but it’s important to note that none of them have been published in a scientific journal, nor have they gone through a standard peer review for scientific research. Instead, they were published online in draft form (a common occurrence in the midst of a rapidly evolving pandemic that tilts researchers to have quick access to data, albeit uncertain).
The findings seemed to support minority arguments that COVID-19 may be no worse than seasonal flu (a leading cause of death in the US) and that restrictive mitigation efforts currently strangling the economy may be unnecessary. . In fact, three researchers who co-authored the new studies have publicly made those exact arguments.
In a controversial opinion piece in the biomedical STAT media outlet, population health researcher John Ioannidis at Stanford argued in mid-March that the COVID-19 death rate may be much lower than expected, which could make current locks “totally irrational”. “Health policy researchers Eran Bendavid and Jay Bhattacharya, also both at Stanford, made a similar argument in The Wall Street Journal in late March. They called the current COVID-19 mortality estimates, in the range of 2 per one hundred to four percent, “deeply flawed”.
Ioannidis is a co-author of the Santa Clara County study, and Bendavid and Bhattacharya were lead investigators in both studies, which appeared online this month.
The new studies appear to support the researchers’ earlier arguments. But a chorus of his companions is far from convinced. In fact, criticism of the two studies has woven a damning tapestry of Twitter threads and blog posts pointing to flaws in the studies, from basic mathematical errors to alleged statistical oversights and sample biases.
In a blog review of the Santa Clara County study, Columbia University statistician Andrew Gelman detailed several troubling aspects of the statistical analysis. He concluded:
I think the authors of the article linked above owe us all an apology. We wasted time and effort discussing this document whose main selling point were some numbers that were essentially the product of statistical error.
I am serious about the apology. We all make mistakes. I don’t think they[sic] the authors need to apologize only because they were wrong. I think they need to apologize because these were avoidable mistakes
A Twitter account from the laboratory of Erik van Nimwegen, a computer systems biologist at the University of Basel, responded to the study by tweeting the joke “Big sobs reported from under the grave of Reverend Bayes.” The tweet refers to Thomas Bayes, an 18th-Reverend English statistician of the century who presented a fundamental theorem about probability.
Pleuni Pennings, an evolutionary biologist at San Francisco State University, noted in a blog about the Santa Clara study that “in research, we like to say that ‘extraordinary claims require extraordinary evidence.’ Here the claim is extraordinary but the evidence It is not. Also, we learn that even if a study comes from a large university, this does not guarantee that the study is good. “
Harvard epidemiologist Marc Lipsitch stated on Twitter that he coincided with similar statistical criticism done online Added a “compliment” to the authors for conducting the study and “providing an interpretation of the study (which supports their view” is exaggerated “)”.
So what does all these weapons investigators have?
The objective of the studies.
Both studies had the primary objective of estimating how many people in each of the two counties had ever been infected with SARS-CoV-2. This is an extremely important effort because it can tell us the true extent of the infection, help guide efforts to stop transmission, and better assess the full spectrum of COVID-19 disease severity and death rate.
Because diagnostic tests have been so limited in the United States. USA And with many cases of COVID-19 appearing to present with mild or even no symptoms, the researchers expect the actual number of people infected to be much higher than we know, based on confirmed cases. There is no debate about that. But how much higher is the subject of considerable debate.
The researchers conducted their studies by recruiting small groups of residents and testing their blood for antibodies to SARS-CoV-2. Antibodies are Y-shaped proteins that the immune system produces to attack specific molecular enemies, such as viruses. If a person has antibodies that recognize SARS-CoV-2 or its components, that suggests that the person was previously infected.
Santa Clara
In the Santa Clara County study, the researchers recruited volunteers using Facebook and brought them to one of three test drive sites. They ended up analyzing the blood of 3,330 adults and children for antibodies. They found 50 blood samples, or 1.5 percent, were positive for antibodies to SARS-CoV-2.
They then adjusted their numbers to try to estimate what positive tests they would have obtained if their group of volunteers matched the county’s demographics better. The volunteer group leaned toward certain zip codes in the county and grew rich for women and whites relative to the actual makeup of the county. The researchers’ adjustment ended up nearly doubling the prevalence of positives, taking them from 1.5 percent to an estimate of 2.8 percent.
They then adjusted the data again to account for the inaccuracies of the antibody test. Here are two precision metrics: sensitivity and specificity. Sensitivity relates to how good the test is to correctly identify all true positives. Specificity relates to how good the test is to correctly identify all true negatives, in other words, avoid false positives.
According to the authors of the Santa Clara study, the sensitivity and specificity data on their antibody test led them to estimate that the true prevalence of SARS-CoV-2 infections ranged from 2.49 percent to 4.16 percent.
Based on the county population, that would suggest that between 48,000 and 81,000 people in the county had been infected. The confirmed case count at the time of publication was only 956. That puts his estimate of infection 50 to 85 times higher than confirmed cases.
The researchers then estimated an infection death rate (IFR) with that large number of infections estimated and an estimate of just 100 cumulative deaths (even from infections at the time. Deaths lag behind initial infections, potentially during weeks). They calculated an IFR of 0.12 percent to 0.2 percent. This falls in the seasonal flu stage, which has an estimated death rate of about 0.1 percent.
the Angels
Less data is available from the Los Angeles study. In an unusual move, even by today’s pandemic standards, the findings were initially announced in a press release from the county public health department, which provided few statistical and methodological details. A short draft of the study (PDF found here) has also been circulated online, but still has less information about the methods than the Santa Clara study. Additionally, the draft has even higher prevalence estimates than the press release. It is not clear why the estimates differ, but we will focus primarily on the conclusions formally published by the health department.
Overall, for the study, the researchers used data from a market research firm to randomly select residents and invite them to be tested at one of six test sites. The researchers set quotas for the participants by age, gender, race, and ethnicity to match the characteristics of the county’s population. Their goal was to recruit 1,000 participants.
They tested 863 adults using the same antibody test used in the Santa Clara study by Premier Biotech of Minneapolis, Minnesota. Of the tests performed, 35 (or 4.1 percent) were positive. According to the press release, the adjusted data suggested that 2.8 percent to 5.6 percent of the county’s population had been infected with the new coronavirus.
Given the county’s population, that suggests that 221,000 to 442,000 adults in the county had been infected. That estimate is 28 to 55 times greater than the confirmed case count of 7,994 at the time. Similar to the Santa Clara study, that puts the IFR in the range of 0.3 percent to 0.13 percent, closer to the IFR for seasonal flu.
[ad_2]
