Explanation: Shot in the Dark – COVID-19 Vaccine Early Efficacy Explained

ZURICH (Reuters) – Moderna Inc MRNA.O delivered the latest good news from the developers of the COVID-19 vaccine, saying that their experimental vaccine is 94.5% effective in preventing the disease based on interim data from a late-stage trial.

Pfizer Inc. PFE.N and BioNTech SE 22UAy.DE On November 9, they announced the first data indicating that their candidate mRNA is more than 90% effective.

Then a Russian project touted an efficiency of 92% for the candidate Sputnik V, named for the Soviet-era satellite, based on a smaller data set.

HOW DO MANUFACTURERS REACH EFFICIENCY NUMBERS?

In Pfizer’s case, it waited until 94 volunteers in its late-stage clinical trial of more than 43,500 people (half received the vaccine, the other half received a placebo) tested positive after developing symptoms.

For an efficacy greater than 90%, no more than eight people among those who tested positive had received the vaccine, and the rest had received the placebo.

“Broadly speaking, there are likely eight to 86 cases in the treated and placebo groups,” David Spiegelhalter, a Cambridge risk professor and statistician, told Reuters. “It doesn’t take a lot of sophisticated statistical analysis to show that this is profoundly impressive. It just hits you between the eyes. “

Moderna’s interim analysis was based on 95 infections among trial participants who received a placebo or the vaccine. Of those, only five infections occurred in participants who received the vaccine, which is given in two injections 28 days apart.

In Russia, the Gamaleya Institute, developer of Sputnik V, reached its preliminary efficacy figure of 92% based on 20 diseases in 16,000 volunteers as the trial progresses in the last stage. It aims to reach 40,000 people.

Of the 16,000 people, about a quarter received the placebo.

“It suggests that there is some effect, but it is insufficient to estimate the magnitude of it,” Spiegelhalter said.

HOW MANY PEOPLE SHOULD GET SICK ON LARGE VACCINE TESTS?

Some experts say that ideally 150 to 160 people in a trial of tens of thousands of participants must become ill before making a reliable evaluation of the efficacy of a vaccine. However, it is an empirical rule open to interpretation.

“There is no such regulatory standard that requires X number of events to make a reliable decision,” said the government-funded Swiss Organization for Clinical Trials. “The amount of (infections) must be seen in relation to the disease and its risk profile. It’s more of a case-by-case assessment. “

Regulators generally strive to be at least 95% certain that the test reading is not the result of random variations with nothing to do with the compound tested.

For trial sponsors, there is certainty in the numbers, as a trial large enough can ensure that the 95% reliability hurdle is overcome, but the greater the underlying clinical benefit, the fewer trial participants will be needed to create that clarity.

In the Pfizer and BioNTech trial, they planned a final analysis when 164 people became ill, with multiple interim tests pre-planned along the way. They skipped an analysis of 32 patients, and once they were ready to take a look at the 62-person mark, 94 fell ill.

Moderna said it expects to have logged 151 infections when it seeks regulatory approval, meaning the test will have reached its end point.

Details of the Russian trial are unclear, without access to its protocol.

HOW DO THESE RESULTS COMPARE WITH OTHER DRUGS OR VACCINES FOR OTHER DISEASES?

In normal drug trials, for conditions such as terminal cancer, the benefits of new drugs may be less obvious, with survival benefits of just a few months, sometimes revolutionary for patients who are near death.

However, for vaccines, marginal protection is inadequate, and the World Health Organization ideally wants to see at least 70% efficacy in trials, while the US Food and Drug Administration wants at least the fifty%.

The efficacy rate reported in all three trials exceeds that of typical flu vaccines, which the U.S. Centers for Disease Control and Prevention (CDC) estimate reduces the risk of illness by 40% and 60%.

For other injections, the CDC estimates the effectiveness of a two-shot measles vaccine at 97% and a two-dose chickenpox vaccine at 90%. Two doses of the polio vaccine are 90% effective, and a third is almost 100% effective.

CAN WE EXPECT THE EFFICIENCY RATES TO BE MAINTAINED AS THE TESTS ADVANCE?

Pfizer recognized that its final percentage of vaccine efficacy may vary. Still, Spiegelhalter said it seems likely that the study design will stick overall, based on the 94 sick participants.

“In this case, the effect is so large, even if there is a little pullback, if the effects get a little bit smaller over time, it’s very unlikely to be significant.”

WHAT ABOUT EFFECTIVENESS IN THE REAL WORLD, MUST VACCINES BE APPROVED?

The interim data is promising as it appears to show that a vaccine can be effective in preventing COVID-19.

The jump to mass vaccines, however, presents new hurdles, particularly for an mRNA vaccine like Pfizer and BioNTech, which must be stored and shipped at minus 70 degrees Celsius (-94 ° F).

All three vaccines so far require two doses. In the case of Pfizer-BioNTech, ideally it is 21 days apart and Modern 28 days. If people don’t stick to the schedule, it can affect how well the vaccine works.

Protection against mumps, for example, drops from nearly 90% to 78% if people do not receive a follow-up vaccine.

Swiss epidemiologist Marcel Tanner, president of the Swiss Academies of Arts and Sciences and one of the government’s top COVID-19 scientific advisers, expects possible variations in efficacy among older people, whose immune systems weaken over time, or those with immune disorders.

“Efficacy says, ‘Does it work?’ Efficacy says: ‘Can it be applied? Can you bring effectiveness to people? ‘”Tanner said. “But there is no doubt: 90% efficiency, at that stage, is a pretty good result.”