This document discusses key concepts for evaluating diagnostic tests, including accuracy, precision, sensitivity, specificity, and predictive values. It defines a diagnostic test as one that provides evidence for or against a pathology. Key factors for evaluating tests are described as accuracy, precision, sensitivity, specificity, and predictive values. Accuracy refers to how close a test value is to the gold standard, while precision refers to reproducibility of values. Sensitivity and specificity measure a test's ability to correctly identify diseases and non-diseases. Predictive values measure the probability that a positive or negative test result correctly identifies a disease or non-disease. Examples are provided to illustrate these concepts.
2. What is a diagnostic test?
•
A test or an instrument that provides some evidence for the presence or
an absence of a pathology
•
Aids evidence-based management of pathologies in medical practice
•
Retinoscopy is a diagnostic test – it provides evidence for the presence or
absence of refractive error
•
Goldmann Applanation tonometer is also a diagnostic test – it provides a
quantitative estimate of the eye’s IOP
3. Attributes of a diagnostic test
•
Accuracy
•
Precision
- Inter-subject and intra-subject variability
•
Sensitivity
•
Specificity
•
Positive and Negative Predictive values
•
Receiver Operating Characteristics (ROC) curves
•
Likelihood ratios
4. Accuracy & Precision
•
Accuracy: How close is the value measured by a test to the goldstandard value
•
Precision: How reproducible is the value when multiple measurements
are taken using this test
•
Accuracy and Precision need not be related to each other at all
5. The dreadful Gaussian distribution!!
•
Measures of accuracy and precision can be easily explained using the
Gaussian (aka Normal) distribution
•
x-axis of Gaussian distribution: All the measurements that you have taken
grouped into different bins (e.g. IOP)
•
y-axis of Gaussian distribution: Proportion of times a particular x-axis
value is featured in the population
6. The dreadful Gaussian distribution!!
•
•
Thinner and taller the Gaussian distribution: High precision
Fatter and flatter the Gaussian distribution: Low precision
•
Intra-subject (or intra-observer) variability: Precision determined by taking
multiple measurements within the same subject or observer
•
Inter-subject (or inter-observer) variability: Precision determined by taking
multiple measurements across different subjects or observers
7. Precision & Accuracy – A Brainy Application
Humans integrate visual and haptic information in a statistically optimal fashion
(Ernst & Banks, Nature, 2002)
Neural weighting of sensory information also depends on the accuracy and
precision of the cue
8. The four outcomes of a diagnostic test
Here, you are evaluating the diagnostic test against a gold-standard
1.
The test can turn out to be positive
•
True positive (TP)
•
False positive (FP) or False Alarm
2.
The test can turn out to be negative
•
True negative (TN)
•
False negative (FN)
Disease Present
Disease
Absent
Test Positive
TP
FP
Test
Negative
FN
TN
9. Example 1: World War II
•
A radio-engineer's job is to interpret signals coming over a transmitter
•
•
The signal could be from the enemy about an impending attack
The signal could just be unwanted noise from the transmitter
True Signal
Just Noise
Interpretation as
Signal
Great. Prepare for
counter-attack!
Nothing much to worry.
Just resources wasted
Interpretation as
Noise
You are totally
screwed dude!
Great. May peace reign
this world
10. Example 2: A magic-wand for Glaucoma
•
You have invented a magic-wand that is intended to put a clinician out of
business
•
You want to test the performance of this device against a gold-standard
test
Patients with
Glaucoma
Patients without
Glaucoma
Magic-wand
response positive
Great. The clinician is
in trouble!
Well… your clinic is
swamped with patients
Magic-wand
response negative
Don’t let glaucoma
darken your lives
Great. The clinician is in
trouble again!
11. Sensitivity & Specificity
Disease Present
Disease Absent
Test Positive
TP
FP
Test Negative
FN
TN
•
Sensitivity: How good is the test at identifying people with the disease in
a population
i.e. Sensitivity = TP / (TP + FN)
•
Specificity: How good is the test at eliminating people without the
disease in a population
i.e. Specificity = TN / (FP + TN)
•
Ideally, both parameters should be 100%
12. Sensitivity & Specificity Example
Disease Present
Disease Absent
Test Positive
TP = 40
FP = 20
Test Negative
FN = 10
TN = 30
•
Sensitivity: How good is the test at identifying people with the disease in
a population
i.e. Sensitivity = 40 / (40 + 10) = 0.8 or 80%
•
Specificity: How good is the test at eliminating people without the
disease in a population
i.e. Specificity = 30 / (20 + 30) = 0.6 or 60%
•
This test has good sensitivity but poor specificity
13. Diabetic Retinopathy Example
•
Role of frequency doubling technology perimetry in screening of
diabetic retinopathy (Parikh et al., IJO 2006).
•
Sensitivity and specificity calculated for seven different FDT criteria
for detecting diabetic retinopathy
•
Criterion 1: One abnormal point anywhere in the field, depressed to
any level.
•
Criterion 2: Two abnormal nonadjacent points anywhere in the field,
depressed to any level.
•
Gold standard was the slit-lamp findings by an Ophthalmologist
15. Diabetic Retinopathy Example
Criterion 3: Very few false negatives and false positives
Criterion 6: Very large false negatives but only few false positives
16. Positive & Negative Predictive Values
Disease Present
Test Positive
TP
FP
Test Negative
•
Disease Absent
FN
TN
Positive predictive value (PPV): If a person tests positive, what is the
probability that he/she has the condition
i.e. PPV = TP / (TP + FP)
•
Negative predictive value (NPV): If a person tests negative, what is the
probability that he/she does not have the condition
i.e. NPV = TN / (FN + TN)
17. PPV & NPV Example
Disease Present
Test Positive
TP = 40
FP = 20
Test Negative
•
Disease Absent
FN = 10
TN = 30
Positive predictive value (PPV): If a person tests positive, what is the
probability that he/she has the condition
i.e. PPV = 40 / (40 + 20) = 0.66 or 66%
•
Negative predictive value (NPV): If a person tests negative, what is the
probability that he/she does not have the condition
i.e. NPV = 30 / (10 + 30) = 0.75 or 75%
