Understanding Sensitivity, Specificity & Accuracy – What is the Difference?
INTRODUCTION
When it comes to any form of testing, especially SARS-CoV-2 rapid antigen tests, the terms sensitivity, specificity, and accuracy are the three key exponents used to portray the test efficacy. These terms can easily become confused, misunderstood, or mixed up. It is the combination of the features of sensitivity and specificity that determine the overall accuracy of the test, another concept that can easily be misaligned.
This article aims to provide a theoretical explanation of the three terms to understand how each score is derived and how these scores can be influenced. Using this theoretical understanding helps inform practical choice, procurement and implementation of test options that will offer high test efficacy, by which we mean comparative consistency in correctly identifying the presence or absence of infection. Achieving high test efficacy is vital in developing a route out of lockdown that does not allow for the tragic consequences of missing individual sources spreading SARS-CoV-2 infections.
DEFINITIONS (IN THIS CONTEXT)
Positive: an individual who has the disease. False positive refers to an individual who truly is negative and incorrectly received a positive result by the test. A false positive individual does not risk spreading SARS-CoV-2.
Negative: an individual who does not have the disease. False negative refers to an individual who truly is positive and incorrectly received a negative result by the test. A false negative individual risks spreading SARS-CoV-2.
Sample pool: refers to a group of individuals tested which will normally contain both positives and negatives.
Sensitivity: how effective a test is at identifying individuals that truly are positive.
It is usually expressed as a percentage which tells you how many positives are picked up by the test out of the total number of positives in the sample pool. All negative individuals are ignored.
High sensitivity means less false negatives. With SARS-CoV-2, high sensitivity helps identify infectious people to control the spread. Consider this the most important feature.
Specificity: how effective a test is at identifying individuals that truly are negative.
It is usually expressed as a percentage which tells you how many negatives are picked up by the test out of the total number of negatives in the sample pool. All positive individuals are ignored.
High specificity means less false positives. With SARS-CoV-2, high specificity does not reduce the spread of SARS-CoV-2 but allows individuals to be confirmed as negative and be free rather than locked down. This is still an important feature. Specificity and sensitivity can trade-off: Higher specificity can often mean lower sensitivity, and vice versa.
Accuracy: how effective a test is at identifying the correct result in a given sample pool, between both positives and negatives, also often called ‘clinical precision.’
It is usually expressed as a percentage which tells how many positives and negatives were correctly identified out of the entire sample pool.
Accuracy can help judge overall effectiveness; however, it can easily be skewed by the number of positives and the number of negatives in the sample pool and not accurately reflect performance comparative to another test. We will demonstrate this further on.
EXAMPLE SAMPLE POOL
As a mathematical example, a ‘new test’ (e.g. rapid antigen test) was carried out on the following sample pool totaling 24 individuals tested. To be able to judge this new test with any validity, an existing testing method called a ‘comparator’ needs to be used which is as accurate as possible to determine the true positives and negatives within the sample pool. For SARS-CoV-2 the comparator testing method will likely be ‘RT-PCR’ as this is the most accurate form of SARS-CoV-2 testing currently. It is very important to take note of the comparator method, as different swab/sampling types will affect the accuracy of the comparator itself. For example, a nasal swab compared with a nasal RT-PCR test will appear higher in clinical precision than when compared with a nasopharyngeal RT-PCR test, as the comparator itself is less accurate and aligns more with the precision of the antigen test. All things are relative.
Symbols:
An individual who is truly negative on RT-PCR and new test result was negative. True negative. See below:
An individual who is truly negative on RT-PCR and new test result was positive. False positive. See below:
An individual who is truly positive on RT-PCR and new test result was negative. False negative. See below:
An individual who is truly positive on RT-PCR and new test result was positive. True positive. See below:
Calculating Sensitivity
Ignoring all the negative individuals, here are all the positive individuals in the sample pool:
- The new test identified 4 positives out of the 5 positives
- The new test sensitivity is 80% (for this sample pool
Calculating Specificity
Ignore all the positive individuals, here are all the negative individuals in the sample pool:
- The new test identified 18 negatives out of the 19 negatives
- The new test specificity is 94.7% (for this sample pool)
Calculating Accuracy
Taking the entire sample pool, the 2 symbols noted below are individuals incorrectly identified by the test.
An individual who is truly negative on RT-PCR and new test result was positive. False positive. See below:
An individual who is truly positive on RT-PCR and new test result was negative. False negative. See below:
- The new test identified 2 incorrectly out of 24 individuals, so 22 out of 24 correct.
- The new test accuracy is 91.7% (for this sample pool)
HOW CAN ACCURACY BE SKEWED?
In the example above, the new test has higher specificity than sensitivity. This is nearly always the case with rapid antigen testing. The accuracy can be skewed towards a higher score by increasing the number of negatives in the sample pool, when conducting clinical studies.
For example, what happens if we add another 19 negatives to the sample pool, with 1 of those newly added a false positive? This looks like the below:
A quick calculation will confirm that the percentage scores for sensitivity and specificity are identical to the previous example.
- Sensitivity: 4 out of 5 positives correct = 80% (for this sample pool)
- Specificity: 36 out of 38 negatives correct = 94.7% (for this sample pool)
However, accuracy improves because there are more negatives, which the new test is better at identifying than positives.
- Accuracy: 3 incorrect out of 43 individuals, so 40 out of 43 correct = 93.0% (for this sample pool)
Beware of the many rapid antigen tests that have a large disparity with overall accuracy far higher than sensitivity! Yes, they’re highly correct when the sample pool contains a vast majority of negatives, but that doesn’t reduce the spread of infections, high sensitivity is required to identify & isolate the all too critical positives that inevitably will be present at some point.
RELIABILITY
Another influential factor is how many samples are tested, generally the more the better.
A quick example, let’s take from our original sample pool and imagine the sample pool size was just 12, consisting of 4 positives and 8 negatives, this could look like below:
Following the same calculations, our sensitivity, specificity, and accuracy for the new test all become 100%! Simply because this sample pool was so small it could quite feasibly miss containing any incorrectly classified results.
This is an extreme example, but it remains relevant when comparing percentage figures, which is equivalent to a score out of 100. If there are many less than 100 samples available in either the positives or negatives, how reliable are the sensitivity and specificity scores? The larger the sample pool in both positives and negatives, the more reliable both sensitivity and specificity become. But accuracy will continue to be skewed depending on the split up of negatives and positives in the sample pool.
CONCLUSION
When choosing a rapid antigen test, review sensitivity to establish how effective the test is at preventing the spread of infections, and specificity to see how effective the test is at allowing non-infectious individuals to be free.
Look at sensitivity first and foremost, but don’t leave specificity out of the picture; a lower specificity will mean increased false positive cases which causes unnecessary burden and impact. Instead of taking the overall accuracy of the study, consider the proportions of negatives and positives likely to be encountered in the testing scenario in which rapid antigen will be used, then calculate an overall accuracy with real-life relevance.
Look at the size of the sample pool studied, is it satisfactorily large enough to be trusted in the context of containing the spread of infections that can cause severe disease and death?
Remember that manufacturer’s claims and theoretical information is imperfect. Track actual results and look for any independent determination and validation available to make well-founded decisions.
