Across the world there is a clamour for covid-19 testing, with Tedros Adhanom Ghebreyesus, director general of the World Health Organization, encouraging countries to “test, test, test.”1 The availability of the complete genome of covid-19 early in the epidemic facilitated development of tests to detect viral RNA.2 Multiple assays with different gene targets have been developed using reverse transcriptase polymerase chain reaction (RT-PCR).3 These viral RNA tests use samples usually obtained from the respiratory tract by nasopharyngeal swab, to detect current infections. Serology blood tests to detect antibodies indicating past infection are being developed; these will not be considered in depth in this article.
Testing for covid-19 enables infected individuals to be identified and isolated to reduce spread,4 allows contact tracing for exposed individuals,5 and provides knowledge of regional and national rates of infection to inform public health interventions. However, questions remain on how to apply test results to make optimal decisions about individual patients.
This article was produced at speed to address an urgent need to address uncertainties in testing for covid-19. We searched Pubmed using the terms “covid”, “SARS-CoV-2”, “sensitivity”, “specificity”, “diagnosis”, “test”, and “PCR”, and KSR evidence using terms for covid and test. This was supplemented by discussion with colleagues undertaking formal systematic reviews into covid-19 diagnosis.
No test gives a 100% accurate result; tests need to be evaluated to determine their sensitivity and specificity, ideally by comparison with a “gold standard.” The lack of such a clear-cut “gold-standard” for covid-19 testing makes evaluation of test accuracy challenging.
A systematic review of the accuracy of covid-19 tests reported false negative rates of between 2% and 29% (equating to sensitivity of 71-98%), based on negative RT-PCR tests which were positive on repeat testing.6 The use of repeat RT-PCR testing as gold standard is likely to underestimate the true rate of false negatives, as not all patients in the included studies received repeat testing and those with clinically diagnosed covid-19 were not considered as actually having covid-19.6
Accuracy of viral RNA swabs in clinical practice varies depending on the site and quality of sampling. In one study, sensitivity of RT-PCR in 205 patients varied, at 93% for broncho-alveolar lavage, 72% for sputum, 63% for nasal swabs, and only 32% for throat swabs.7 Accuracy is also likely to vary depending on stage of disease8 and degree of viral multiplication or clearance.9 Higher sensitivities are reported depending on which gene targets are used, and whether multiple gene tests are used in combination.310 Reported accuracies are much higher for in vitro studies, which measure performance of primers using coronavirus cell culture in carefully controlled conditions.2
The lack of a clear-cut “gold-standard” is a challenge for evaluating covid-19 tests; pragmatically, clinical adjudication may be the best available “gold standard,” based on repeat swabs, history, and contact with patients known to have covid-19, chest radiographs, and computed tomography scans. Inevitably this introduces some incorporation bias, where the test being evaluated forms part of the reference standard, and this would tend to inflate the measured sensitivity of these tests.11 Disease prevalence can also affect estimates of accuracy: tests developed and evaluated in populations with high prevalence (eg, secondary care) may have lower sensitivity when applied in a lower prevalence setting (eg, primary care).11
One community based study of 4653 close contacts of patients with covid-19 tested RT-PCR throat swabs every 48 hours during a 14 day quarantine period. Of 129 eventually diagnosed with covid-19 by RT-PCR, 92 (71.3%) had a positive test on the first throat swab, equating to a sensitivity of 71% in this lower prevalence, community setting.12
Further evidence and independent validation of covid-19 tests are needed.13 As current studies show marked variation and are likely to overestimate sensitivity, we will use the lower end of current estimates from systematic reviews,6 with the approximate numbers of 70% for sensitivity and 95% for specificity for illustrative purposes.
Sensitivity and specificity can be confusing terms that may be misunderstood14 (see supplementary file ‘Definitions and formulae for calculating measures of test accuracy’). Sensitivity is the proportion of patients with disease who have a positive test, or the true positive rate. Specificity is the proportion of patients without disease who have a negative test, or true negative rate. These terms describe the operating characteristics of a test and can be used to gauge the credibility of a test result. They can be combined to calculate likelihood ratios, which are dimensionless numbers that indicate the strength of a positive or negative test result.15 For calculating probabilities, a likelihood ratio can be used as a multiplier to convert pre-test odds to post-test odds. Positive likelihood ratios greater than 1 are progressively stronger, with 10 representing a very strong positive test result. Negative likelihood ratios less than 1 are also progressively stronger, with 0.1 representing a very strong negative test result. In the case of the nasopharyngeal swab RNA test for covid-19, the positive likelihood ratio is about 14, which is excellent.6 A positive covid-19 test result should be very compelling. The negative likelihood ratio is 0.3, which is a moderate result, but not nearly as compelling as a positive result because of the moderate sensitivity (about 70%) of the covid-19 test.
Interpretation of a test result depends not only on the characteristics of the test itself but also on the pre-test probability of disease. Clinicians use a heuristic (a learned mental short cut) called anchoring and adjusting to settle on a pre-test probability (called the anchor). They then adjust this probability based on additional information. This heuristic is a useful short cut but comes with the potential for bias. When people fail to estimate the pre-test probability and only respond to a piece of new information, they commit a fallacy called base-rate neglect. Another fallacy called anchoring is failing adequately to adjust one’s probability estimate, given the strength of new information. Likelihood ratios can give a clinician an idea of how much to adjust their probability estimates. Clinicians intuitively use anchoring and adjusting thoughtfully to estimate pre- and post-test probabilities unconsciously in everyday clinical practice. However, faced with a new and unfamiliar disease such as covid-19, mental short cuts can be uncertain and unreliable and public narrative about the definitive nature of testing can skew perceptions.
Figure 1 shows how a clinician’s thinking about a patient’s probability should shift, based on either a positive or negative test result for covid-19. First, the clinician should estimate a pre-test probability, using knowledge of local rates of covid-19 infection from national16 and regional17 data and patients’ symptoms and signs,18 likelihood of alternative diagnoses, and history of exposure to covid-19. After choosing a pre-test probability on the x axis, one should then trace up to either the upper curve for a positive test result or the lower curve for a negative test result, then trace over to the y axis to read the estimate for post-test probability. The figure shows that the shift in the probability is asymmetric, with a positive test result having a greater impact than a negative test result, owing to the modest sensitivity and negative likelihood ratio of the RNA test.
Leaf plot for covid-19 RT-PCR tests based on a sensitivity of 70% and specificity of 95%. The x axis gives the estimated pre-test probability of covid-19 based on the clinical details. The post-test probability is obtained by tracing up and across to the y axis from the lower curve for a negative test, or to the upper curve for a positive test result. The dashed lines illustrate pre-test probability of 90% (clinical case 1) and 50% (clinical case 2)
The infographic (fig 2) shows the outcomes when 100 people with a pre-test probability of 80% are tested for covid-19 using natural frequencies, which are generally easier to understand. Online calculators are available which allow clinicians to adjust pre-test probability, sensitivity, and specificity to estimate post-test probability19
Infographic showing outcomes of 100 people who are tested for covid-19
A 52 year old general practitioner in London develops a cough, intermittent fever, and malaise. On day 2 of his illness he receives a nasopharyngeal swab test for covid-19, which is reported as negative. His cough and low-grade fever persist but he feels systemically well enough to return to work. What should he do?
Pre-test probability is high in someone with typical symptoms of covid-19, an occupational risk of exposure, and working in a high prevalence region, and negative test results can therefore be misleading. Table 1 shows that for a pre-test probability of 90%, someone with a negative test has a 74% chance of having covid-19; with two negative tests this risk is still around 47%. If this doctor were to return to work and subsequently the test was confirmed as a false negative, then the decision to work would potentially have significant consequences for his patients, colleagues, and everyone with whom he came into contact. It is therefore safest for this GP with strongly suggestive symptoms to self-isolate in line with guidelines for covid-19, even though his test results are negative. This case illustrates the fallacy of base-rate neglect; it can be tempting to trust the results of an “objective” test more than one’s own “subjective” clinical judgement. In general, during this pandemic, pre-test probabilities of covid-19 will be high, particularly in high prevalence secondary care settings.
Pre- and post- test probabilities for covid-19 RT-PCR tests, calculations based on a sensitivity of 70% and specificity of 95%
A 73 year old woman with severe chronic obstructive pulmonary disease (COPD) and a chronic cough develops acute shortness of breath and slight worsening of her non-productive cough. She reports no fever, has no known exposure to covid-19, and no recent travel. She presents to an emergency department where she is acutely short of breath. A chest radiograph shows possible infiltrates in the right upper and middle lung fields. She is admitted and placed in isolation on droplet precautions. She requires intubation for worsening respiratory distress. Initial nasopharyngeal covid-19 testing is negative. Should she remain in isolation on droplet precautions?
This patient has an alternative possible diagnosis: community-acquired pneumonia. Given her lack of other risk factors or clinical symptoms, and chest radiography findings we therefore estimate her pre-test probability at about 50%. One negative test reduces this risk to 24%, the patient therefore has an additional independently sampled nasopharyngeal swab RNA test which was negative, giving a post-test probability after two negative tests of less than 10%. She is treated with antibiotics and continues to recover.
While positive tests for covid-19 are clinically useful, negative tests need to be interpreted with caution, taking into account the pre-test probability of disease. This has important implications for clinicians interpreting tests and policymakers designing diagnostic algorithms for covid-19. The Chinese handbook of covid-19 prevention and treatment states “if the nucleic acid test is negative at the beginning, samples should continue to be collected and tested on subsequent days.”20 False negatives carry substantial risks; patients may be moved into non-covid-19 wards leading to spread of hospital acquired covid-19 infection,21 carers could spread infection to vulnerable dependents, and healthcare workers risk spreading covid-19 to multiple vulnerable individuals. Clear evidence-based guidelines on repeat testing are needed, to reduce the risk of false negatives.
Clinicians should ensure that patients are counselled about the limitations of tests (box 1). Patients with a single negative test but strongly suggestive symptoms of covid-19 should be advised to self-isolate in keeping with guidelines for suspected covid-19.
Serology tests, which detect immunoglobulins including IgG and IgM, are under development,222324 with the aim of detecting individuals who have had previous infection and therefore theoretically developed immunity. The time course and accuracy of serology tests are still under investigation, but the same principles of incorporating the test result with the clinical impression applies. False positive serology tests could cause false reassurance, behaviour change, and disease spread. If suitable accuracy can be established, the benefits of these antibody tests include establishing when healthcare workers are immune, helping to inform decisions about the lifting of lockdowns, and allowing the population to return to work.25
The WHO message “test, test, test”1 is important from a population perspective; low sensitivity can be accounted for when assessing burden of disease. However RT-PCR tests have limitations when used to guide decision making for individual patients. Positive tests can be useful to “rule-in” covid-19, a negative swab test cannot be considered definitive for “ruling out.”
Patients with covid-19 or possible covid-19 were not involved in the writing of this paper for practical reasons
This article is made freely available for use in accordance with BMJ's website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.
