A review of studies has found major weaknesses in the evidence base for diagnostic accuracy of COVID-19 antibody tests, particularly for point-of-care tests performed directly with a patient, outside a laboratory, and does not support their continued use.
Serological tests to detect antibodies against COVID-19 could improve diagnosis and may be useful tools for monitoring levels of infection in a population, but it is important to formally evaluate whether there is sufficient evidence that they are accurate, the researchers said.
The study, published ín The BMJ, set out to determine the diagnostic accuracy of antibody tests for COVID-19.
The researchers, including those from Harvard Medical School in the US and University of British Columbia, Canada, searched medical databases and preprint servers from January 1 to April 30, for studies measuring sensitivity and specificity of a COVID-19 antibody test compared with a control test.
Sensitivity measures the percentage of people who are correctly identified as having a disease, while specificity measures the percentage of people who are correctly identified as not having a disease, they said.
Of 40 eligible studies, most (70 percent) were from China and the rest were from the UK, US, Denmark, Spain, Sweden, Japan, and Germany.
The researchers noted that half of the studies were not peer-reviewed and most were found to have a high or unclear risk of bias -- problems in study design that can influence results.
Only four studies included outpatients and only two evaluated tests at the point of care, they said.
When sensitivity results for each study were pooled together, they ranged from 66 per cent to 97.8 per cent depending on the type of test method used, meaning that between 2.2 per cent and 34 per cent of patients with COVID-19 would be missed, according to the researchers.
Pooled specificities ranged from 96.6 to 99.7 per cent, depending on the test method used, meaning that between 3.4 per cent and 0.3 per cent of patients would be wrongly identified as having COVID-19, they said.
The study found that pooled sensitivities were consistently lower for the lateral flow immunoassay (LFIA) test compared with other test methods.
The LFIA test is the potential point-of-care method that is being considered for ''immunity passports.''
The researchers explained that, if an LFIA test is applied to a population with a COVID-19 prevalence of 10 per cent, for every 1,000 people tested, 31 who never had COVID-19 will be incorrectly told they are immune, and 34 people who had the disease will be incorrectly told that they were never infected.
Pooled sensitivities were also lower with commercial test kits (65 per cent) compared with non-commercial kits (88.2 per cent) and in the first and second week after symptom onset compared with after the second week, they said.
The researchers point to some limitations, such as differences in study populations and the potential for missing studies.
However, study strengths include thorough search strategies and assessment of bias, they said.
"These observations indicate important weaknesses in the evidence on COVID-19 serological tests, particularly those being marketed as point-of-care tests," the researchers said.
"While the scientific community should be lauded for the pace at which novel serological tests have been developed, this review underscores the need for high-quality clinical studies to evaluate these tools," they added.
"This is a detailed and thorough assessment of antibody tests that were developed and published in the first few months of the pandemic," said Sanjeev Krishna, Professor at St George''s, University of London.
"Like most diagnostic tests that need to be developed with urgency, the first products may lack robustness through poor evaluation methodologies or poor test performance, but many tests can be improved with time," said Krishna, who was not involved in the study.
He noted that it is important not to dismiss them from the start, as antibody tests will take their place in the suite of diagnostic tests that are needed to understand and manage the COVID-19 pandemic.
Alexander Edwards, Associate Professor at the University of Reading said some important points are highlighted in the study about the way that antibody tests are validated or evaluated.
For example, he said, the accuracy that any study publishes will depend a lot on the population used.
"Many studies are incomplete because they select only very specific groups of samples. But it takes time and a lot of resources to collect all different types of sample needed to fully define the performance of an antibody test," Edwards added.
Serological tests to detect antibodies against COVID-19 could improve diagnosis and may be useful tools for monitoring levels of infection in a population, but it is important to formally evaluate whether there is sufficient evidence that they are accurate, the researchers said.
The study, published ín The BMJ, set out to determine the diagnostic accuracy of antibody tests for COVID-19.
The researchers, including those from Harvard Medical School in the US and University of British Columbia, Canada, searched medical databases and preprint servers from January 1 to April 30, for studies measuring sensitivity and specificity of a COVID-19 antibody test compared with a control test.
Sensitivity measures the percentage of people who are correctly identified as having a disease, while specificity measures the percentage of people who are correctly identified as not having a disease, they said.
Of 40 eligible studies, most (70 percent) were from China and the rest were from the UK, US, Denmark, Spain, Sweden, Japan, and Germany.
The researchers noted that half of the studies were not peer-reviewed and most were found to have a high or unclear risk of bias -- problems in study design that can influence results.
Only four studies included outpatients and only two evaluated tests at the point of care, they said.
When sensitivity results for each study were pooled together, they ranged from 66 per cent to 97.8 per cent depending on the type of test method used, meaning that between 2.2 per cent and 34 per cent of patients with COVID-19 would be missed, according to the researchers.
Pooled specificities ranged from 96.6 to 99.7 per cent, depending on the test method used, meaning that between 3.4 per cent and 0.3 per cent of patients would be wrongly identified as having COVID-19, they said.
The study found that pooled sensitivities were consistently lower for the lateral flow immunoassay (LFIA) test compared with other test methods.
The LFIA test is the potential point-of-care method that is being considered for ''immunity passports.''
The researchers explained that, if an LFIA test is applied to a population with a COVID-19 prevalence of 10 per cent, for every 1,000 people tested, 31 who never had COVID-19 will be incorrectly told they are immune, and 34 people who had the disease will be incorrectly told that they were never infected.
Pooled sensitivities were also lower with commercial test kits (65 per cent) compared with non-commercial kits (88.2 per cent) and in the first and second week after symptom onset compared with after the second week, they said.
The researchers point to some limitations, such as differences in study populations and the potential for missing studies.
However, study strengths include thorough search strategies and assessment of bias, they said.
"These observations indicate important weaknesses in the evidence on COVID-19 serological tests, particularly those being marketed as point-of-care tests," the researchers said.
"While the scientific community should be lauded for the pace at which novel serological tests have been developed, this review underscores the need for high-quality clinical studies to evaluate these tools," they added.
"This is a detailed and thorough assessment of antibody tests that were developed and published in the first few months of the pandemic," said Sanjeev Krishna, Professor at St George''s, University of London.
"Like most diagnostic tests that need to be developed with urgency, the first products may lack robustness through poor evaluation methodologies or poor test performance, but many tests can be improved with time," said Krishna, who was not involved in the study.
He noted that it is important not to dismiss them from the start, as antibody tests will take their place in the suite of diagnostic tests that are needed to understand and manage the COVID-19 pandemic.
Alexander Edwards, Associate Professor at the University of Reading said some important points are highlighted in the study about the way that antibody tests are validated or evaluated.
For example, he said, the accuracy that any study publishes will depend a lot on the population used.
"Many studies are incomplete because they select only very specific groups of samples. But it takes time and a lot of resources to collect all different types of sample needed to fully define the performance of an antibody test," Edwards added.