Fact check: Post misleads on effectiveness of J&J COVID-19 vaccine in older adults

·9 min read

The claim: Johnson & Johnson vaccine not effective in older adults

Johnson & Johnson's COVID-19 vaccine made headlines in early April after six women developed extremely rare blood clots following vaccination.

Use of the one-dose shot was paused for several weeks but lifted on April 23 after both the U.S. Food and Drug Administration and the U.S. Centers for Disease Control found the vaccine was overall safe and effective.

Despite that, doubts about the J&J vaccine, particularly in older adults, continue to linger on social media.

"The clinical trial indicates that in subjects 65 years and older, the (J&J) vaccine may be only 39% effective, and in subjects 75 years or older, the age group that comprises about 60% of all COVID-19 deaths, the vaccine may be 0% effective," claims a May 18 Facebook post by a California-based anti-vaccine group called Physicians for Informed Consent, or PIC.

The post, which includes a chart detailing these and other alleged statistics, also claims the vaccine "may not be effective" in people 60 years and older with severe risk factors for COVID-19.

Fact check: Peer-reviewed studies have shown safety, efficacy of COVID-19 vaccines

PIC general counsel Greg Glaser told USA TODAY the numbers in the post and its accompanying table were largely based on the U.S. Food and Drug Administration's emergency authorization memorandum for J&J's vaccine and PIC's own compiled references.

But in reality, the numbers PIC puts forth each overstate or oversimplify data contained in the FDA's official findings.

Vaccine efficacy versus vaccine effectiveness

First, it's important to clarify that while PIC refers to vaccine effectiveness in its post, the data it cites from the FDA's memorandum actually refers to vaccine efficacy, a slightly different but related measure.

Efficacy is a value obtained from randomized clinical trials and represents a best estimate of how well the product being tested (a vaccine in this case) works in an idealized condition, said Natalie Dean, assistant professor of biostatistics at the University of Florida.

In the setting of a vaccine trial, this is done by comparing the number of disease or infection cases occurring among the vaccinated group to that in the unvaccinated, or placebo, group, according to the U.S. Centers for Disease Control and Prevention.

If vaccine efficacy is 0%, this means the vaccine doesn't considerably reduce the risk of a disease or infection for the group who received it compared to those who received a placebo. A vaccine efficacy nearing or at 100% means it most likely does.

Effectiveness, on the other hand, gives an idea of how well a drug or other treatment works outside of a clinical trial under real-world settings, though it relies on a similar calculation.

Fact check: No, the CDC did not release data showing 7 in 10 Americans are declining COVID-19 vaccine

Dean said it's likely vaccine effectiveness will match vaccine efficacy, but there are a variety of reasons why they might not: age of the vaccine recipient, presence of underlying chronic illnesses and use of medications, as well as how a vaccine is stored and administered or if the targeted virus mutates.

39% is from a confidence interval

Glaser, the PIC attorney, said the group drew its figures from Table 12 of the FDA's EUA memorandum. The table presents vaccine efficacy, or VE, as percentages observed at 14 and 28 days after vaccination.

The values listed for the 65 years and older show J&J's COVID-19 vaccine is quite efficacious – 76.5% after 14 days and 68.6% after 28 days. That's actually better than the numbers for the 18 to 64 age group, which were 64.7% and 65.1%, respectively.

Because these numbers are based on a sampling of the general population – around 40,000 participants across the United States, South Africa and Latin America – 76.5% and 68.8% represent only estimates of vaccine efficacy, specifically what is called point estimate, said Dr. Angela Branche, co-director of the Vaccine and Treatment Evaluation Unit at the University of Rochester Medical Center.

Point estimates are typically accompanied by a range of values called a confidence interval, which is where PIC gets its "39%" effectiveness. The 95% confidence interval reported by J&J means there is a 95% likelihood the actual value is between 39% and 85%. PIC references only the low end of that range and ignores the actual estimate value of 68.6%.

"What (PIC) is reporting here is extremely misleading because you cannot interpret confidence intervals as efficacy. Efficacy is what it is – it's a point estimate," Branche told USA TODAY.

Confidence intervals themselves only express the precision scientists are able to assign to that number but aren't used to represent vaccine efficacy, said Dr. Matthew Laurens, an infectious disease specialist and vaccine researcher at the University of Maryland School of Medicine.

Laurens said the range of the confidence interval is a product of how many people are enrolled in the study.

"If you look at the confidence interval for the 18 to 64, it's much tighter because the numbers (of participants within that age group) are much higher. The more people we include, the more precision we have with the vaccine efficacy," he said.

Despite there being not as many older participants as younger ones, Laurens said what was especially telling about J&J's vaccine is that it performed well against other COVID-19 variants.

"Of the three vaccines that have received emergency authorization in the United States, (J&J's vaccine) is the only one that has been tested in clinical trials in South Africa (and Brazil)," he said. "Those variants were tested by this vaccine, still protected and still showed efficacy no matter the region whether it was North America, southern Africa or Latin America."

Fact check: Fauci didn't say half of CDC, FDA employees refused COVID-19 vaccine

This consistency across geographies, age and race is far more crucial and spoke to the fact J&J's vaccine efficacy resided in the upper 60% range than the lower bounds of a confidence interval, Laurens emphasized.

Vaccine efficacious for 75 years and older

PIC's claim that J&J's COVID-19 vaccine "may be 0% effective" for people 75 and older also mishandles the FDA data.

Table 12 actually shows that at the 14-day endpoint, VE is listed pretty high at 89.7%. VE at the 28-day endpoint, however, is left blank on the chart.

PIC refers to this as indicating it may be 0% effective, but the opposite is actually true, said Branche.

"Efficacy only goes up as you move forward in time," she said. "You're least likely to be protected early on in that 14-day period. As you move off toward 28 days and so forth, then you really have your best protection at that point."

Fact check: Fauci didn't say half of CDC, FDA employees refused COVID-19 vaccine

The VE was likely left blank at the 28-day point because J&J's trial didn't have enough time to collect all the data as the FDA's emergency use authorization was already underway, Branche said.

So the efficacy for that period could only be 89.7% or higher. Not as low as 0%, as the post claims.

Data does show J&J's vaccine works for older adults with COVID-19 risk factors

A lower VE estimate was observed for those 60 years and older with pre-existing conditions, particularly at the 28-day endpoint (65% versus 42%). However, a reason for this could be because of "imprecision owing to fewer cases and shorter follow-up in this subgroup," a Janssen spokesperson said in an email to USA TODAY.

"Furthermore, Kaplan-Meier curves (a statistical tool for estimating when there's incomplete data) indicated that the cumulative incidence of cases among vaccine recipients 60 years of age or older with coexisting conditions was similar to that in the overall trial population, which suggests a similar vaccine efficacy," the company said.

The FDA's EUA memorandum supported this point. It said no COVID-19 deaths or cases requiring medical intervention took place 28 days or more after vaccination among that high-risk subgroup.

Our rating: False

Based on our research, we rate FALSE the claim the Johnson & Johnson COVID-19 vaccine is not effective in older adults. The statistics provided by Physicians for Informed Consent in its post and accompanying table were taken from the FDA's emergency use authorization for J&J's vaccine but are misrepresented.

The 39% figure is the lower bound of a confidence interval for vaccine efficacy 28 days post-vaccination in the 65 and older age group. It's doesn't represent the actual efficacy. Confidence intervals are meant to give information about the precision of a value like vaccine efficacy, not to represent vaccine efficacy itself. The claim the vaccine could be "0% effective" among those 75 and older is flat wrong. The report actually shows the efficacy in that group is at least 89.7%. And while vaccine efficacy was lower for those 60 years and older with pre-existing conditions, that's because there were fewer COVID-19 cases and shorter follow-ups in this group. No COVID-19 related deaths or cases requiring medical intervention were observed in these individuals, according to the FDA.

Our fact-check sources:

Thank you for supporting our journalism. You can subscribe to our print edition, ad-free app or electronic newspaper replica here.

Our fact check work is supported in part by a grant from Facebook.

This article originally appeared on USA TODAY: Fact check: False claim about effectiveness of J&J COVID-19 vaccine