DJ Covid-19 Vaccine Efficacy: What Do the Numbers Really Mean?

By Jared S. Hopkins and Sarah Toy

The spread of the Delta variant, an increase in vaccinated people testing positive for Covid-19, and the U.S. government's call for booster shots have raised new questions about the power of the coronavirus vaccines. Here's what we know about how authorized vaccines perform against the variant, and their ability to prevent infection and serious illness.

What does it mean to say that a vaccine is effective?

Health experts and scientists use a variety of terms and measures to describe how well a vaccine works. Vaccines can be evaluated based on their ability to prevent initial infection, symptomatic disease or severe illness that can lead to hospitalization or death.

When researchers use the term efficacy, they are describing how a vaccine performs under ideal, tightly controlled conditions such as clinical trials. Effectiveness refers to how the vaccine performs in the real world, when people are living their normal lives without the same controls in place.

The vaccines authorized in the U.S. were shown in clinical trials to protect against symptomatic Covid-19. Researchers looked at whether the vaccines prevented people from both testing positive for Covid-19 and showing at least one symptom. The studies didn't measure if the vaccines protect against asymptomatic infection, which means simply testing positive for Covid-19.

For instance, in large late-stage clinical trials conducted by vaccine makers, the vaccine from Pfizer Inc. and partner BioNTech SE was found last year to be 95% effective at preventing symptomatic Covid-19, while the Moderna Inc. vaccine was 94.1% effective. In the approximately 44,000-person study of the Pfizer vaccine, 170 developed Covid-19 with at least one symptom. Of those, just eight had been vaccinated, while 162 had received a placebo. The 95% efficacy rate is derived from that ratio.

The Food and Drug Administration's full approval of the Pfizer vaccine in August was based on six months of clinical-trial data that showed efficacy was 91.1%, a rate drawn from 77 vaccinated people who became sick, compared with 833 sick subjects who received a placebo. The lower efficacy rate was because the companies had more time to collect data than for the efficacy rate calculated from the late-stage trial last year, according to Pfizer.

Severe Covid-19 is defined by the FDA as including outcomes such as pneumonia, respiratory failure, organ failure and death.

Do Covid-19 vaccines prevent infection? Has Delta changed that?

The vaccines' manufacturers didn't study prevention of infection in their clinical trials -- they evaluated symptomatic Covid-19 -- but say they are exploring the issue.

Several studies conducted by other researchers before Delta began to spread widely showed that Covid-19 vaccines were highly effective at preventing infection. A study published in July in the Annals of Internal Medicine that analyzed Covid-19 testing records of more than 54,000 U.S. veterans between December and March showed the Pfizer vaccine was 96% effective at preventing infection and the Moderna vaccine 98% effective. An April study published in the Lancet of about 23,000 healthcare workers in the U.K., most of whom had received the Pfizer shot, concluded it was 85% effective against infection. Real-world evidence in Israel also demonstrated that the Pfizer vaccine was highly effective at preventing infection.

Some researchers, however, have found a drop in effectiveness against infection with the Delta variant.

A study by the Mayo Clinic Health System, posted in August on a preprint server that publishes papers before peer review, found that the Pfizer vaccine was 42% effective against infection in July, when the Delta variant became dominant in the U.S., compared with 76% before that month. The effectiveness of Moderna's shot fell to 76% from 86%. Another preprint by a research group in Qatar found that two doses of the Pfizer vaccine were 53.5% effective at stopping infection from Delta, while Moderna was 85% effective.

A study by New York state government researchers found that the effectiveness of the Pfizer, Moderna and Johnson & Johnson vaccines collectively to prevent infections dropped to about 80% in late July, when the Delta variant was spreading, from about 91% in early May.

In a U.S. government study of weekly case counts at about 3,800 nursing homes released in August, researchers found effectiveness of the Pfizer and Moderna vaccines against infection decreased over time from 75% in March to 53% on Aug. 1. A study published in the Lancet in June also showed declining effectiveness against infection in people in Scotland who received either Pfizer or AstraZeneca PLC vaccines. The AstraZeneca vaccine isn't authorized for use in the U.S.

John Moore, an immunologist at Weill Cornell Medical College in New York, said studies don't find the same rates of prevention because researchers are evaluating under slightly varying conditions such as time periods or geographies. "It's hard to know where the percentage of efficacy drop is because they vary in studies, and have different circumstances, and putting a single number on them is tricky," he said.

Jennifer Nuzzo, an epidemiologist at Johns Hopkins University, said infections in fully vaccinated people aren't surprising because the vaccines aren't entirely protective and vaccinated people have interacted with other people more frequently, making exposure to the virus more likely. "What we're seeing is more infections among vaccinated people," she said. "We always expected that."

The growing number of breakthrough cases reflects the greater contagiousness of the Delta variant combined with the likelihood that immunity from vaccines is declining over time, Dr. Moore said.

Symptoms in most of the breakthrough cases, though, tend to be mild, he said. "There's this perception that the vaccines are failing when they're really not," he said.

Do vaccines still prevent symptomatic illness in people infected with the Delta variant?

Studies have shown some declines in vaccine effectiveness against symptomatic illness from Delta.

A study published in the New England Journal of Medicine in July showed that the effectiveness of two doses of the Pfizer vaccine against symptomatic illness was lower among people in the U.K. with the Delta variant (88%) compared with those with the Alpha variant (93.7%), the strain previously dominant in the country.

More recent studies also show drops in effectiveness against symptomatic illness, including a preprint study from August by researchers at the University of Oxford that found that the Pfizer and AstraZeneca vaccines' effectiveness for both infections and symptomatic illness fell over time.

More symptomatic infections and declining vaccine effectiveness are likely a reflection of the vaccine's strength dropping as well as the high rate of spread of the Delta variant, said Saad Omer, a virologist and director of the Yale Institute for Global Health. "It's hard to disentangle the effects of the Delta variant itself and the waning immunity. I think it's both," he said.

Federal regulators will likely approve a Covid-19 booster shot for vaccinated adults starting at least six months after the previous dose amid data on waning immunity, The Wall Street Journal reported. Still, some public-health experts have said boosters aren't necessary based on studies that show continuing protection against hospitalizations and death from the initial round of shots. Pfizer and BioNTech asked regulators to authorize boosters of their Covid-19 vaccine and submitted additional data showing a third dose improves protection against the virus.

Do Covid-19 vaccines still prevent severe disease and hospitalization?

Despite mounting evidence of reduced effectiveness at preventing infection, studies show the vaccines continue to prevent severe illness and death.

A recent federal government study of more than 1,100 patients who received either the Pfizer or Moderna vaccine found effectiveness against hospitalization stayed steady over 24 weeks from early March to mid-July.

A study by New York state government researchers found that collectively the Pfizer, Moderna and Johnson & Johnson vaccine effectiveness rate of more than 95% at preventing hospitalizations held up from early May to late July, when the Delta variant was spreading.

What should I know about what happened in Provincetown, Mass.?

When the the Centers for Disease Control and Prevention recently said vaccinated people should wear masks again indoors in areas where Delta is spreading rapidly, it pointed to data from Provincetown, Mass., where it said large gatherings in July at bars, nightclubs and house parties led to at least 430 Covid-19 infections.

The CDC said that nearly three-quarters of infected people were fully vaccinated and that samples showed the amount of virus infected people carried was similar between vaccinated and unvaccinated people.

Some scientists say Provincetown data isn't enough to be the primary driver of a public-health policy change. Provincetown has a high vaccination rate, and missing from the data is the total number of people who were at risk of being infected, which public health experts say is needed to actually assess vaccine effectiveness. Provincetown's population grows to 60,000 from 3,000 during the summer, according to the government.

Some experts also say the data was too small of a sample and the circumstances too unique -- as there were unusually large gatherings at bars, nightclubs and house parties -- for it to be applied to other parts of the country.

Does data from Israel indicate vaccines don't protect well?

(MORE TO FOLLOW) Dow Jones Newswires

August 26, 2021 13:32 ET (17:32 GMT)

DJ Covid-19 Vaccine Efficacy: What Do the Numbers -2-

Data from Israel's government showed the Pfizer vaccine was 39% effective at reducing the risk of infection and 40% effective at reducing the risk of symptomatic disease between June 20 and July 27 , a period of when Delta was the dominant strain there. The vaccine was 91% effective at preventing severe illness in the same period. Researchers haven't published a full manuscript, given a sample size for the effectiveness or provided many details on the vaccinated subjects.

The same study showed that breakthrough cases in Israel counted between July 11 and July 17 were more likely for people whose second shots were administered in January and February. These individuals were among the earliest people to become vaccinated, and therefore the furthest out from the second shot.

Other researchers say the Israeli data was too limited to draw firm conclusions and are waiting for a peer-reviewed manuscript. They say that while the government researchers considered people's ages and sex, factors such as occupation, location and whether they live in congregate settings weren't accounted for.

Which studies should I trust? How can I know if a study is reliable?

Angela Rasmussen, a virologist at the Vaccine and Infectious Disease Organization at the University of Saskatchewan in Canada, recommends reading studies published in the New England Journal of Medicine, the Journal of the American Medical Association and the Lancet. For journals specific to infectious disease, she recommends Clinical Infectious Diseases and the Journal of Infectious Diseases. She also recommends people read Science, Nature, Cell and the Proceedings of the National Academy of Sciences.

Studies published in such peer-reviewed journals are evaluated by experts in that field to make sure the data presented is sound. That makes them generally more reliable than, say, a self-published paper or preprint, a paper posted online before peer review.

Preprints have attracted more public attention during the pandemic as scientists have raced to make their findings on Covid-19 public sooner. Although some preprint servers have a vetting process, the papers aren't peer reviewed the way they would be when submitted for publication in a journal, Dr. Rasmussen said.

"People should read any conclusions from a preprint with a grain of salt, " she said.

(END) Dow Jones Newswires

August 26, 2021 13:32 ET (17:32 GMT)

Copyright (c) 2021 Dow Jones & Company, Inc.