Hypothesis testing and p-value, www.eyenirvaan.com

1. Shrikant R. Bharadwaj
BSopt., PhD
HYPOTHESIS TESTING
AND P – VALUE

2. Hypothesis Testing & p-value

3. This presentation will cover …
•
Hypothesis testing
•
Attributes of a sampling distribution
•
p-value
•
Type-I and Type-II errors in hypothesis testing

4. What is a Hypothesis?
•
Hypothesis is a proposed explanation of a phenomenon that can be
scientifically tested
•
Hypothesis is a tentative statement about the relationship between two
or more variables that is specific and testable
•
Evolution Vs. Creation controversy
•
Organisms evolve from one form to another is a testable hypothesis
proposed by Sir Charles Darwin
•
Organisms were created by a supernatural force (aka God) is a belief
and not a testable hypothesis
•
Tammy Kitzmiller, et al Vs. Dover Area Public School, et al (2005)
To view more presentations and articles, visit www.eyenirvaan.com

5. What is a Hypothesis?
•
Hypothesis is a proposed explanation of a phenomenon that can be
scientifically tested
•
Hypothesis is a tentative statement about the relationship between two
or more variables that is specific and testable
•
Drugs that lower IOP reduce retinal ganglion cell loss
•
Using 3 doses of Avastin injection reduces retinal angiogenesis by 50%
•
The number of people entering Patodia hall for morning class is
maximum between 6:59:50AM and 7:00:00AM
To view more presentations and articles, visit www.eyenirvaan.com

6. Null Vs. Alternate Hypothesis
•
Science is all about testing a given hypothesis
•
Two contradictory hypotheses under consideration
- Null Hypothesis (H0)
- Alternate Hypothesis (Ha)
•
Null hypothesis is typically the claim that is initially assumed to the true
- It is the default choice
•
Alternate hypothesis is typically opposite of the Null hypothesis
To view more presentations and articles, visit www.eyenirvaan.com

7. Examples of Null & Alternate Hypothesis
•
One is considered innocent unless proven guilty
- Null hypothesis (H0): A person accused of murder is innocent
- Alternate hypothesis (Ha): This person is guilty of murder
•
What is the impact of an IOP lowering drug on retinal ganglion cell
loss?
- H0: Drug lowering IOP has no impact on retinal ganglion cell loss
- Ha: Drug lowering IOP has some impact on retinal ganglion cell
loss
•
What is the impact of Avastin on retinal angiogenesis?
- H0: Avastin has no impact on angiogensis
- Ha: Avastin has some impact on angiogensis
•
The alternate hypothesis is typically bi-directional (aka two-tailed)
To view more presentations and articles, visit www.eyenirvaan.com

8. Null Vs. Alternate Hypothesis
What is the impact of beta
blockers on IOP?
Null hypothesis (H0)
The IOP in a placebo and
beta-blocker treated cohort
are not different from each
other
Alternate hypothesis (Ha)
The IOP in the beta-blocker
treated cohort is different from
the IOP in the placebo cohort
Mean of treatment group is lower than the placebo group
Lower-tail of the Placebo cohort’s Gaussian distribution

9. Null Vs. Alternate Hypothesis
•
The purpose of a study is to provide evidence for or against the null
hypothesis
•
Based on the evidence gathered by the study, you either support or reject
the null hypothesis
•
Only as a corollary, you reject or support the alternate hypothesis
Terminology clarification
•
You cannot PROVE the null hypothesis; you can only DISPROVE it
•
Science and hypothesis testing are based on the logic of falsification
•
http://www.statisticalmisconceptions.com/sample2.html
To view more presentations and articles, visit www.eyenirvaan.com

10. Proving Vs. Disproving
•
Null hypothesis: All crows in this world are black
•
To PROVE the null hypothesis, you need to get the color of every single
crow in this world
•
To DISPROVE the null hypothesis, you just need to show one white
crow

11. Proving Vs. Providing Evidence
•
PROVE is a dangerous word – it leaves no room for error!!
•
(a + b)2 = a2 + b2 + 2ab --------- this can be PROVED mathematically
•
What is the impact of beta
blockers on IOP?
•
You are NOT PROVING that beta
blockers reduce IOP
•
You are only PROVIDING
EVIDENCE that beta blockers can
reduce IOP
To view more presentations and articles, visit www.eyenirvaan.com

12. Proving Vs. Providing Evidence
Reasons why biological research
cannot PROVE anything
1.Humans react differently to a given
treatment
2.Measurement error
3.Data is not obtained from every
human being on Earth
Biological research can only
determine how likely or unlikely a
given result is
To view more presentations and articles, visit www.eyenirvaan.com

13. Sampling a population
•
Data cannot be obtained from every human being on Earth
•
A representative cohort is sampled and results from this cohort are
extrapolated to the entire population
Fully deterministic distribution
with no standard deviation
Realistic biological distribution
with standard deviation

14. Properties of a Sampling Distribution
μ = Mean of Gaussian distribution; σ = Standard deviation of Gaussian distribution
Data from 68.2% of the population falls within +/-1σ
Data from 95.4% of the population falls within +/-2σ
Data from 99.6% of the population falls within +/-3σ

15. Standard Deviation & Confidence Intervals
Standard deviation describes variability of measurements in your sample
Confidence intervals describe the interval over which the mean will fall when the
experiment is repeated multiple times
95% Confidence interval = +/-1.96 SD
99% Confidence interval = +/-2.58 SD
To view more presentations and articles, visit www.eyenirvaan.com

16. Z-scores
Z-score is a unitless quantity that describes how many standard deviations
away from the mean is your sample value
Z = (x – μ) / σ
1Z-score = 1SD; 2Z-scores = 2SD; 3Z-scores = 3SD

17. p-value
Biological research aims at determining the likelihood of the null hypothesis
being rejected
What is the likelihood that a lowered IOP was really due to the treatment and
not by chance?
p-value (or “probability” value) gives us this likelihood
p-value ranges from 0 to 1 or 0% to 100%
There can be 0% probability to 100% probability of rejecting the null
hypothesis
To view more presentations and articles, visit www.eyenirvaan.com

18. p-value
p-value estimates the false positive
rate (Type 1 error) that we are willing
to accept
Typically, we accept a false-positive
rate of <=5% (p <= 0.05)
95% confidence that the IOP value
came from the treated distribution
95% confidence that null hypothesis
can be rejected
5% (or 1 in 20 times) our results can
be incorrect

19. p-value
p = 0.01
•99% confidence that null hypothesis
can be rejected
•1% (or 1 in 100 times) our results
can be incorrect
p = 0.1
•90% confidence that null hypothesis
can be rejected
•10% (or 1 in 10 times) our results
can be incorrect

20. Determinants of the p-value
p-value is lower when…
1.Mean difference is large
2.Small variance in each distribution
Example 1:
Non diseased Mean + 1SD: 20 + 5mmHg
Diseased Mean + 1SD: 25 + 5mmHg
Example 2:
Non diseased Mean + 1SD: 20 + 5mmHg
Diseased Mean + 1SD: 35 + 5mmHg
Example 3:
Non diseased Mean + 1SD: 20 + 2mmHg
Diseased Mean + 1SD: 35 + 2mmHg
p-value of Eg 3 < Eg 2 < Eg 1

21. Determinants of the p-value
p-value is lower when…
1.Mean difference is large
2.Small variance in each distribution
Example 1:
Non diseased Mean + 1SD: 20 + 5mmHg
Diseased Mean + 1SD: 25 + 5mmHg
Example 2:
Non diseased Mean + 1SD: 20 + 5mmHg
Diseased Mean + 1SD: 35 + 5mmHg
Example 3:
Non diseased Mean + 1SD: 20 + 2mmHg
Diseased Mean + 1SD: 35 + 2mmHg
p-value of Eg 3 < Eg 2 < Eg 1

22. Use of p-value in a Student’s t-test
To view more presentations and articles, visit www.eyenirvaan.com

23. Use of p-value in a Student’s t-test
•
Two kinds of t-tests
•
Paired t-test: The two datasets are obtained from the same cohort (e.g.
IOP before and after treatment with beta-blockers)
•
Unpaired t-test: The two datasets are obtained on different cohorts (e.g.
Body weight of 30 – 40yr old males and females)
Practical demo of T-test in MS Excel
Large mean
difference
Small mean
difference
Large variance
p = 0.0325
p = 0.2906
Small variance
p < 0.0001
p = 0.1011

24. Type-I and Type-II Errors
•
Type-I error (α error): When the Null hypothesis is true, but it is rejected by
the test
• Type-I error is equivalent to generating False Positives
•
Type-II error (β error): When the Null hypothesis is false, but it is
erroneously accepted as true
• Type-II error is equivalent to generating False Negatives
Null hypothesis is true
Reject Null
hypothesis
Null hypothesis is false
Type-I error / FP
Correct decision / TP
ReferAccept Nullin my first presentation for/ its equivalent in diagnostic tests
to slide #8
Correct rejection TN
Type-II error / FN
hypothesis
To view more presentations and articles, visit www.eyenirvaan.com

25. Example of Type-I and Type-II Errors
The radio engineer during WW II receives a crackling sound over his
transmitter. Is this signal from the enemy or is it unwanted noise?
Null hypothesis: The sound received over the transmitter is just noise
Alternate hypothesis: The sound received over the transmitter is not noise
True Signal
Just Noise
Interpretation
as Signal
Correctly rejecting null
hypothesis
False rejecting null
hypothesis / Type-I error
Interpretation
as Noise
Falsely accepting null
hypothesis / Type-II
error
Correctly accepting the
null hypothesis

26. Example for Type-II error
What is the difference in macular thickness of eyes with AMD compared
to normals, as detected using OCT imaging?
•Null hypothesis: There is no difference in macular thickness between normal
eyes and eyes with AMD
•Alternate hypothesis: The macula in AMD patients is >50μ thicker than it is in
normal eyes
•Mean + 1SD macular thickness in normal eyes: 200μ + 50μ
•Mean + 1SD macular thickness in AMD eyes: 230μ + 55μ
•Based on these results, you have accepted the null hypothesis
To view more presentations and articles, visit www.eyenirvaan.com

27. Example for Type-II error
•
Repeatability of the OCT: 80μ
•
The test that you have used does not have the resolution to determine the
difference you are expecting
•
The mean difference in macular thickness between normal and AMD eyes
is 110μ using a gold-standard test
•
An Type-II error is therefore made, in erroneously accepting the Null
hypothesis
To view more presentations and articles, visit www.eyenirvaan.com

28. Thank You!
To view more presentations and articles, visit www.eyenirvaan.com