New research shows that, in a head-to-head comparison of five tests used to detect SARS-CoV-2 antibodies, an assay developed by an academic partnership led by the University of Oxford and one manufactured by Siemens had the most accurate results.
The study comparing these ‘immunoassays’ has been published in The Lancet Infectious Diseases, as part of a special session of a Conference on Coronavirus Disease organised by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID).
Testing for SARS-CoV-2 antibodies can help us understanding how many people have been infected with SARS-CoV-2, and how people respond to vaccines that are being evaluated in research studies.
The presence of antibodies may also correlate with protective immunity from SARS-CoV-2 re-infection, although this remains to be clearly demonstrated.
Scale-up
Several manufacturers have developed SARS-CoV-2 antibody immunoassays compatible with global laboratory infrastructures, enabling widespread testing of hundreds to thousands of samples per day. Understanding the performance of these tests is highly relevant to optimising their usage.
The scale-up required for regular population-wide testing might exceed the capacity of currently available commercial platforms, and additional, accurate, high-throughput tests would be of value.
To date, few thorough, direct assessments of immunoassay performance on large sample sets have been done, and governments, regulators, and clinical laboratories have had to balance the urgent need to facilitate the demand for serological testing with the few data available on assay performance. This has led to a relaxation of typical assessment criteria in the regulation and approval of tests on the market.
This study, carried out by The National SARS-CoV-2 Serology Assay Evaluation Group, a team of researchers and scientists collaborating across several UK institutions including Public Health England (Porton Down), involved a head-to-head assessment of four widely available commercial assays:
- the SARS-CoV-2 IgG assay (Abbott, Chicago, IL, USA),
- LIAISON SARS-CoV-2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy),
- Elecsys Anti-SARS-CoV-2 assay (Roche, Basel, Switzerland),
- SARS-CoV-2 Total assay (Siemens, Munich, Germany);
- a novel 384-well assay (the Oxford immunoassay)
The Oxford study was developed at the John Radcliffe Hospital with support from the NIHR Oxford Biomedical Research Centre (BRC).
Sensitivity and specificity
The study calculated the sensitivity (the ability of a test to correctly identify those with SARS-CoV-2 antibodies or ‘true positive’ rate) and the specificity (the ability of the test to correctly identify those without SARS-CoV-2 antibodies or ‘true negative’ rate).
Sensitivity and specificity were calculated by testing 976 pre-pandemic blood samples (collected several years before the SARS-CoV-2 pandemic started, and therefore known to be negative for SARS-CoV-2 antibodies) and 536 blood samples from patients with laboratory-confirmed SARS-CoV-2 infection (by RT-PCR), collected at least 20-days post symptom onset. This was in line with the UK Medicines and Healthcare products Regulatory Agency (MHRA) guidance on how these tests should be evaluated.
Using the tests exactly as specified by the manufacturers, the best results were delivered by the Siemens assay (sensitivity 98·1% / specificity 99·9%) and the Oxford immunoassay (sensitivity 99·1% / specificity 99·0%).
For the Abbott assay, sensitivity was 92·7% and specificity was 99·9%; for the DiaSorin assay sensitivity was 95·0% and specificity was 98·7%; for the Roche assay sensitivity was 97·2% and specificity was 99·8%.
The researchers also found that changing the assay thresholds (i.e. the test value distinguishing between a ‘positive’ and a ‘negative’ test result) and using them on samples taken 30 days or more post-symptom onset (i.e. allowing more time for antibody responses to develop in affected individuals) could result in improved test performance.
Limitations
“By running all the assays on the same large panel of blood samples, we showed that the Siemens assay and the Oxford immunoassay both achieved sensitivity and specificity of at least 98% on samples taken at least 20 days post symptom onset, in line with the current MHRA guidance for the regulatory approval of these tests.
“However, all assays could potentially achieve these specifications through threshold adjustment, or by assessing samples collected at least 30 days post symptom onset, consistent with the time-dependent nature of antibody responses,” explain the authors, who include Dr Nicole Stoesser, a clinician-scientist from the Nuffield Department of Medicine at the University of Oxford.
She adds: “There is no such thing as a ‘perfect test’, but accurately evaluating how these tests perform can help us understand their limitations and improve how they are used. Importantly, consideration needs to be given to how many false-positive and false-negative results might occur with any given test; this depends on both the test performance, and how many people in the population being tested genuinely have SARS-CoV-2 antibodies.
“Overall however, our study supports the fact that global serology testing needs can be met using different assays, mitigating against the risk of shortages, and allowing deployment in laboratories with different analysers already installed for other testing purposes.”
However, Dr Stoesser cautions: “Although all these assays can effectively detect SARS-CoV-2 antibodies, the nature and durability of any immunity conferred by these antibodies remain unclear.”
She concludes: “This study represents a benchmark for future assessments of serological tests. New tests should be similarly rigorously evaluated. Such assays will be an important part of the clinical and research landscape in guiding public health policy, with effects to be delivered at the individual level and population level.”