In mid-May, the Food and Drug Administration issued a rare public warning about an Abbott Laboratories COVID-19 test that for weeks had received high praise from the White House because of its speed: Test results could be wrong.
The agency at that point had received 15 “adverse event reports” about Abbott’s ID NOW rapid COVID test suggesting that infected patients were wrongly told they did not have the coronavirus, which had led to the deaths of tens of thousands of Americans.
The warning followed multiple academic studies showing higher “false negative” rates from the Abbott device, including one from New York University researchers who found it missed close to half of the positive samples detected by a rival company’s test.
But then, in a move that confounded lab officials and other public health experts, a senior FDA official later that month said coronavirus tests provided outside lab settings would be considered useful in fighting the pandemic even if they miss 1 in 5 positive cases — a worrisome failure rate.
The FDA has now received a total of 106 reports of adverse events for the Abbott test. The agency has not received a single adverse event report for any other point-of-care tests meant to diagnose COVID-19, an agency spokesperson said.
In a statement, Abbott Laboratories said the NYU research was “flawed” and “an outlier,” citing studies with higher accuracy rates.
Though the Abbott rapid test is one of over 100 COVID-19 diagnostic tests to receive FDA emergency use authorization during the pandemic, President Donald Trump has featured the product in the White House Rose Garden and the Health and Human Services Department’s preparedness and response division has issued more than $205 million worth of contracts to buy the test, according to federal contract records.
In its own COVID-19 testing policy for labs and commercial manufacturers, the FDA says a diagnostic test should correctly identify at least 95% of positive samples.
But medical professionals are split over the lower 80% threshold for the Abbott and other point-of-care tests’ “sensitivity” — a metric showing how often a test correctly generates a positive result. They are debating whether it’s sufficient, given the risks that an infected person unwittingly spreads COVID-19 after receiving a negative result.
False negatives increase the risk that patients will not self-isolate or exercise other precautions — such as wearing a mask — and make more people sick than if they had had an accurate diagnosis. Evaluations of the Abbott test have been among the most mixed, with some researchers finding that the test has bigger accuracy problems, but others saying it isn’t likely to miss sicker patients.
“There’s no way I would be comfortable missing 2 out of 10 patients,” said Susan Whittier, director of clinical microbiology at NewYork-Presbyterian/Columbia University Medical Center. Whittier and co-authors found that the Abbott test correctly identified 74% of positive samples compared with a rival test from Roche, another diagnostics giant.
A point-of-care test from Cepheid, a rival company, correctly identified 99% of positives.
An FDA official cited the 80% accuracy minimum for point-of-care tests in late May even after two White House aides tested positive for the virus. The Executive Office of the President has spent roughly $140,000 on Abbott test kits, according to contract records.
In a statement, Abbott said when its test is used as intended it “is delivering reliable results and is helping to reduce the spread of infection in society by detecting more positive results than would otherwise be found.” Studies from University Hospitals Cleveland Medical Center and OhioHealth found that its test detected at least 91% of positives.
In March, HHS officials announced that Cepheid would receive approximately $3.7 million through its Biomedical Advanced Research and Development Authority for coronavirus diagnostic development work; the Strategic National Stockpile also made a one-time $2.3 million purchase of Cepheid’s point-of-care tests, according to an agency spokesperson.
“Knowing the true performance of such a point-of-care test and knowing that it may be less sensitive than a central lab molecular test is important, but also can play a role in triaging patients who are suspected of having COVID-19,” Dr. Timothy Stenzel, director of the FDA’s Office of In Vitro Diagnostics and Radiological Health, said on the call when discussing the Abbott warning. “If we are able to determine that sensitivity of the assay in controlled trial circumstances is at least 80%, we feel like that test has a valuable place going forward in this pandemic.”
Until the FDA can complete post-market studies to verify performance, he said, negative results with the Abbott rapid test will be treated as “presumptive” negatives.
No test is perfect, whether it’s for a common illness like the flu or for COVID-19, which has killed nearly 120,000 Americans. Federal officials contend that the trade-off with point-of-care tests — especially ones as fast as Abbott’s, which can turn around a positive result in as little as five minutes and a negative one in 13 minutes — is that the tests can be used in spots where traditional lab tests aren’t as accessible. There’s also a greater risk of operator error when administering the test in the real world given the way patient specimens are collected and handled.
An FDA spokesperson said officials’ “general expectation” is that companies’ test validation data indicate a sensitivity of at least 95%; however, “based on the available information, FDA has issued EUAs [emergency use authorizations] to some tests that presented data indicating a sensitivity below 95%.”
“Rapid and reliable detection of positive patients can be important for public health,” the spokesperson said.
Cepheid would not comment on FDA’s 80% standard but pointed to a Northwell Health Laboratories study finding its test was 98% accurate in detecting positives. The company has shipped approximately 6 million tests since getting FDA authorization in March.
“Experience has shown that in COVID testing, pre-analytical variables such as the site of sample collection (nasal, nasopharyngeal, throat, saliva) and the quality of the sample collected can have a large impact on test performance,” Dr. David Persing, Cepheid’s chief medical and technology officer, said in a statement. “Tests with higher sensitivity have a natural advantage in this setting.”
As far as Abbott’s test, “there are certainly some elements of it that could be improved, but I think it’s a great assay,” said Michael Mina, an assistant professor of epidemiology at the Harvard T.H. Chan School of Public Health, who said the NYU study was “flawed” in part because of the number of patients researchers included with such low viral loads that they wouldn’t be infectious. An NYU spokesperson has defended the research.
Mina and others also took issue with the FDA’s threshold.
“For them to just say 80%, it lets people game the system,” he said. Companies presented with a single figure for sensitivity could manipulate which patients they test to exceed the minimum — for example, by including only very sick patients, which most tests would have an easier time detecting as a positive.
“They really need to fix this issue,” Mina said.