Hypothesis Testing

1. Hypothesis Testing

2. The logic of statistical hypothesis testing follows the
logic of judicial decision making.

3. A jury is asked to decide whether a defendant is guilty
or not guilty.

4. A jury is asked to decide whether a defendant is guilty
or not guilty. It is a dicho-tomous
decision, guilty or
not guilty.

5. A jury is asked to decide whether a defendant is guilty
or not guilty. It is a dicho-tomous
decision, guilty or
not guilty. There is no in-between
or partial decision.

6. A jury is asked to decide whether a defendant is guilty
or not guilty. It is a dicho-tomous
decision, guilty or
not guilty. There is no in-between
or partial decision.
The jury does not begin its
decision-making process in
a neutral position.

7. A jury is asked to decide whether a defendant is guilty
or not guilty. It is a dicho-tomous
decision, guilty or
not guilty. There is no in-between
or partial decision.
The jury does not begin its
decision-making process in
a neutral position.
The default position is “not guilty.”

8. A jury is asked to decide whether a defendant is guilty
or not guilty. It is a dicho-tomous
decision, guilty or
not guilty. There is no in-between
or partial decision.
The jury does not begin its
decision-making process in
a neutral position.
The default position is “not guilty.”
The prosecution must mount enough evidence to
convince the jury to move from its default position of
not guilty to a verdict of guilty.

9. The jury will make a decision which may or may not
coincide with reality.

10. When the jury decides “not guilty” and the defendant
is, in reality, not guilty,
It is true because the not guilty (negative) decision
aligns with the not guilty (negative) reality.

11. When the jury decides “not guilty” and the defendant
is, in reality, not guilty, they have made a correct
decision called a “true negative decision.”
It is true because the not guilty (negative) decision
aligns with the not guilty (negative) reality.

12. When the jury decides “not guilty” and the defendant
is, in reality, not guilty, they have made a correct
decision called a “true negative decision.”
It is true because the not guilty (negative) decision
aligns with the not guilty (negative) reality.
not guilty
and I really
wasn’t guilty!
true negative

13. When the jury decides “guilty” and the defendant is, in
reality, guilty,
It is true because the guilty (positive) decision aligns
with the guilty (positive) reality.

14. When the jury decides “guilty” and the defendant is, in
reality, guilty, they have made a correct decision called
a “true positive” decision.
It is true because the guilty (positive) decision aligns
with the guilty (positive) reality.

15. When the jury decides “guilty” and the defendant is, in
reality, guilty, they have made a correct decision called
a “true positive” decision.
It is true because the guilty (positive) decision aligns
with the guilty (positive) reality.
guilty
and I really
WAS guilty!
true positive

16. When the jury decides “not guilty” and the defendant
is, in reality, guilty,

17. When the jury decides “not guilty” and the defendant
is, in reality, guilty, they have made an incorrect
decision called a “false negative error” which is also
called a Type II or beta error.

18. When the jury decides “not guilty” and the defendant
is, in reality, guilty, they have made an incorrect
decision called a “false negative error” which is also
called a Type II or beta error.
It is false because the “not guilty” (negative) decision
does not align with the guilty (positive) reality.
not guilty
and I really
WAS guilty!
false negative

19. When a jury decides “guilty” and the defendant is, in
reality, not guilty,

20. When a jury decides “guilty” and the defendant is, in
reality, not guilty, they have made an incorrect decision
called a “false positive error”

21. When a jury decides “guilty” and the defendant is, in
reality, not guilty, they have made an incorrect decision
called a “false positive error” which is also called a Type
I or alpha error.

22. When a jury decides “guilty” and the defendant is, in
reality, not guilty, they have made an incorrect decision
called a “false positive error” which is also called a Type
I or alpha error.
It is false because the “guilty” (positive) decision is not
aligned with the not guilty (negative) reality.
guilty
but I really
WASN’T guilty!
false positive

23. Although we prefer correct decisions, if we cannot be
correct, we prefer the false negative error over the false
positive error.
In other words you’d rather render a “NOT GUILTY”
verdict when there is GUILT.
Than a “GUILTY” verdict where there is NO GUILT.

24. Although we prefer correct decisions, if we cannot be
correct, we prefer the false negative error over the false
positive error.
In other words you’d rather render a “NOT GUILTY”
verdict when there is GUILT.
not guilty
and I really
WAS guilty!
Than a “GUILTY” verdict where there is NO GUILT.

25. Although we prefer correct decisions, if we cannot be
correct, we prefer the false negative error over the false
positive error.
In other words you’d rather render a “NOT GUILTY”
verdict when there is GUILT.
not guilty
and I really
WAS guilty!
Than a “GUILTY” verdict where there is NO GUILT.
guilty
but I really
WASN’T guilty!

26. In judicial decisions we would rather let a guilty
defendant go free . . .
than convict and imprison an innocent defendant.
Our default position of “not guilty” supports this
preference and protects against the least favorable
condition.

27. In judicial decisions we would rather let a guilty
defendant go free . . .
than convict and imprison an innocent defendant.
Our default position of “not guilty” supports this
preference and protects against the least favorable
condition.

28. In judicial decisions we would rather let a guilty
defendant go free . . .
than convict and imprison an innocent defendant.
Our default position of “not guilty” supports this
preference and protects against the least favorable
condition.

29. Review the following slide and answer the questions
that follow:

30. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a guilty (+)
verdict is
rendered and
the person is
guilty (+)?

31. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a guilty (+)
verdict is
rendered and
the person is
guilty (+)?

32. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a not guilty (-)
verdict is
rendered and
the person is
not guilty (-)?

33. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a not guilty (-)
verdict is
rendered and
the person is
not guilty (-)?

34. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a guilty (+)
verdict is
rendered and
the person is
not guilty (-)?

35. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a guilty (+)
verdict is
rendered and
the person is
not guilty (-)?

36. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a not guilty (-)
verdict is
rendered and
the person is
guilty (+) ?

37. Review the following slide and answer the questions
that follow:
What type of
decision is
made when
a not guilty (-)
verdict is
rendered and
the person is
guilty (+) ?

38. Each conviction protects against Type I error at a
different stringency according to the gravity of the
punishment to be imposed.

39. The haunting reality is that we really never know the
reality of the guilt or innocence of defendants.
We make our best decisions knowing that there is a
probability that we have made an error.

40. The haunting reality is that we really never know the
reality of the guilt or innocence of defendants.
We make our best decisions knowing that there is a
probability that we have made an error.

41. Statistical hypothesis testing and decision-making are
directly analogous to judicial decision making.
Judicial Decisions
Statistical Decisions

42. Let’s consider an example:
A statistician is asked to decide whether a difference
exists between two groups of people in terms of some
attribute (e.g., excitability).
It is a dichotomous decision (meaning only two
options), different or not different.
There is no in-between or partial decision.

43. Let’s consider an example:
A statistician is asked to decide whether a difference
exists between two groups of people in terms of some
attribute (e.g., excitability).
It is a dichotomous decision (meaning only two
options), different or not different.
There is no in-between or partial decision.

44. Let’s consider an example:
A statistician is asked to decide whether a difference
exists between two groups of people in terms of some
attribute (e.g., excitability).
It is a dichotomous decision (meaning only two
options), different or not different.
There is no in-between or partial decision. x

45. The statistician does not begin her decision-making in a
neutral position.
The default position is “not different.”
This is also called the “null hypothesis.”

46. The statistician does not begin her decision-making in a
neutral position.
The default position is “not different.”
This is also called the “null hypothesis.”

47. The statistician does not begin her decision-making in a
neutral position.
The default position is “not different.”
This is also called the “null hypothesis.”

48. The research findings must present sufficient evidence
to convince the statistician to move from her default
position of no difference to a conclusion that the
groups are different in terms of the attribute.

49. The statistician will make a decision which may or may
not coincide with reality.
The apparent differences may be due to chance or may
be real.

50. The statistician will make a decision which may or may
not coincide with reality.
The apparent differences may be due to chance or may
be real.
OR Something
that is really
happening

51. When the statistician decides “not different” (fails to
reject the null hypothesis, maintains the default
position) and the groups are, in reality, not different,
she has made a correct decision called a “true negative
decision.”
true negative

52. It is true because the “no difference” (negative)
decision aligns with “no difference” reality.
not guilty
and I really
WASN’T guilty!
true negative

53. When the statistician decides that there is a difference
(rejects the null hypothesis, moves off of the default
position) and the groups are, in reality, different, she
has made a correct decision called a true positive
decision.
true positive

54. It is true because the “different” (positive) decision
aligns with the “different” (positive) reality.
guilty
and I really
WAS guilty!
true positive

55. When the statistician decides “not different” (fails to
reject the null hypothesis, maintains the default
position) and the group are, in reality different, she has
made a false negative error.
false negative

56. It is false because the decision of no difference
(negative) does not align with difference (positive)
reality.
not guilty
false negative
Ha ha! and I
really
WAS guilty!

57. Although we prefer correct decisions, if we cannot be
correct, we then prefer false negative error over the
alternative error.

58. When a statistician decides that there is a difference
(positive) between the groups and rejects the null
hypothesis of no difference and, in reality, there is no
difference, she has made a false positive error (also
called Type I error or alpha error.)
false positive

59. It is false because the “difference” (positive) decision
does not align with the “no difference” (negative)
reality.
guilty
but I really
WASN’T guilty!
false positive

60. Our hypothesis testing conventions protect against
false positive, Type I error by holding a default position
of the null hypothesis.
α
Beware of
Type I Error

61. We set a standard of evidence that is required before
rejecting the default null hypothesis.

62. The standard of evidence is based on the probability
density of the sampling distribution.

63. Using probability density we can estimate the
probability of Type I error.
If the mean of the
sample is here, then we
have a .0001 or .01%
chance that we made a
Type I error.

64. Or in other words, we have a
.01% chance of rejecting the
null hypothesis that the
group scores come from two
different populations
(claiming guilty) and being
wrong when both groups
were really part of the same
population (not guilty)

65. When the probability of Type I error is at a low enough
level, we reject the default, null hypothesis.
Like in our previous example.

66. The conventional level of tolerable Type I error is .05.
95%
.05 or 5% chance that
we selected a sample
from this population
and claimed it was a
sample from another
population = false
positive

67. This means that out of 100 similar decisions based on
these data …
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
06 07 08 09 10 11 12 13 14 15
05 06 07 08 09 10 11 12 13 14 15 16
5 10 15

68. … we will be wrong (make a Type I error) less than 5
times.
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
06 07 08 09 10 11 12 13 14 15
05 06 07 08 09 10 11 12 13 14 15 16
5 10 15

69. One advantage that statisticians have over juries is that
we can estimate the probability of Type I error while
they cannot.
I can estimate
the probability
of being right or
wrong
Not sure of the
probability of
being right or
wrong

70. (Or, at least it is easier for us to do so than for them.
There is some recent research in rape cases that has
estimated how frequently juries make Type I errors in
such cases.)

71. Even so, we do not get to make the similar decision 100
times.

72. We tend to make the decision once. The haunting
reality is that we never know in this one decision
whether it is one of the probably occurring Type I
errors.

73. In other words, we take a sample of 30 persons and get
a score of 7.
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
06 07 08 09 10 11 12 13 14
15
05 06 07 08 09 10 11 12 13 14
15 16
5 10 15

74. In other words, we take a sample of 30 persons and get
a score of 7. And then another sample and get a score
of 12, and another with a score of 11, and so on and so
on until the distribution below emerges.
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
06 07 08 09 10 11 12 13 14
15
05 06 07 08 09 10 11 12 13 14
15 16
5 10 15

75. But since, in real life, we usually only take one sample
of 30 for our research purposes,
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
05 06 07 08 09 10 11 12 13 15 16
5 10 15

76. But since, in real life, we usually only take one sample
of 30 for our research purposes,
10
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
05 06 07 08 09 10 11 12 13 15 16
5 10 15

77. But since, in real life, we usually only take one sample
of 30 for our research purposes, we don’t know if the
sample was selected from the far left of the distribution
below
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
06
05 06 07 08 09 10 11 12 13 15 16
5 10 15

78. But since, in real life, we usually only take one sample
of 30 for our research purposes, we don’t know if the
sample was selected from the far left of the distribution
below or the far right
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
05 06 07 08 09 10 11 12 13 15 16
5 10 15
16

79. But since, in real life, we usually only take one sample
of 30 for our research purposes, we don’t know if the
sample was selected from the far left of the distribution
below or the far right or the middle.
10 11
11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
05 06 07 08 09 10 11 12 13 15 16
5 10 15

80. But since, in real life, we usually only take one sample
of 30 for our research purposes, we don’t know if the
sample was selected from the far left of the distribution
below or the far right or the middle. So, we examine
the probability that the sample did or did not come
from the far left or the far right.
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
05 06 07 08 09 10 11 12 13 15 16
5 10 15

81. But since, in real life, we usually only take one sample
of 30 for our research purposes, we don’t know if the
sample was selected from the far left of the distribution
below or the far right or the middle. So, we examine
the probability that the sample did or did not come
from the far left or the far right.
10 11
09 10 11 12
08 09 10 11 12 13
07 08 09 10 11 12 13 14
14
14
06 07 08 09 10 11 12 13 15
05 06 07 08 09 10 11 12 13 15 16
5 10 15
Hmm. . .
What are
the chances
the sample
came from
the far
right or left
of the
Distri-bution?

82. So let’s say we want to know if the students who go to
a college party are more excited to be there than little
girls at a birthday party.

83. Here are the sampling distributions of the excitability
of young girls at a birthday party.

84. Let’s say we don’t have the same kind of distribution
for college student excitability at a party.
?

85. We want to know if there is a statistical difference
between the girls at the birthday party and the
excitability of college students at a Friday night party.

86. We randomly select a group of college students at a
party and measure their levels of excitability.

87. Our random selection is “13”.
13

88. Our random selection is “13”. Since this number does
not lie in the extreme ends we would reject the null
hypothesis or render a judgment of “not guilty”.
13

89. Our random selection is “13”. Since this number does
not lie in the extreme ends we would reject the null
hypothesis or render a judgment of “not guilty”.
College Students and little girls show no difference.
13

90. However, what if we randomly selected a college
student sample with an average excitability value of
“05”.
05

91. However, what if we randomly selected a college
student sample with an average excitability value of
“05”. Wow! This is a rare occurrence.
05

92. Because the chance of that happening is so rare we
would reject the null hypothesis.
05

93. Because the chance of that happening is so rare we
would reject the null hypothesis. We would say
“guilty!”
05

94. Because the chance of that happening is so rare we
would reject the null hypothesis. We would say
“guilty!” But if in reality there is no difference,
05

95. Because the chance of that happening is so rare we
would reject the null hypothesis. We would say
“guilty!” But if in reality there is no difference, then we
have made a type I error.
05

96. Because the chance of that happening is so rare we
would reject the null hypothesis. We would say
“guilty!” But if in reality there is no difference, then we
have made a type I error.
05
Researchers are
willing to take that
chance.

97. In conclusion, hypothesis testing, is a way of
determining the probability of our default position
(not guilty or no difference) being correct or incorrect.

98. In conclusion, hypothesis testing, is a way of
determining the probability of our default position
(not guilty or no difference) being correct or incorrect.
We determine the likelihood of being right or wrong
based on the results.

99. In conclusion, hypothesis testing, is a way of
determining the probability of our default position
(not guilty or no difference) being correct or incorrect.
We determine the likelihood of being right or wrong
based on the results. Then we decide if we are willing
to maintain our default position (no difference) or go
out on a limb and change our default position (yes
there is a difference).

100. What follows are exercises to help you
check your understanding.

101. Go as far as you feel you need to until you
have a good feel for what you know.

102. First Set of Questions

103. 1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?

104. 1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?
A. “Guilty as charged”
B. “Not guilty until proven innocent”
C. “Pleading no contest”

105. 1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?
A. “Guilty as charged”
B. “Not guilty until proven innocent”
C. “Pleading no contest”

106. 1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?
A. “Guilty as charged”
B. “Not guilty until proven innocent”
C. “Pleading no contest”
2. What is another way to say “Null-hypothesis”?

107. 1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?
A. “Guilty as charged”
B. “Not guilty until proven innocent”
C. “Pleading no contest”
2. What is another way to say “Null-hypothesis”?
A. Not clear
B. Not different
C. Not important

108. 1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?
A. “Guilty as charged”
B. “Not guilty until proven innocent”
C. “Pleading no contest”
2. What is another way to say “Null-hypothesis”?
A. Not clear
B. Not different
C. Not important

109. With hypothesis testing we are attempting to set up a
default position of not guilty. We stay in that position
unless we have enough evidence to overturn it.

110. With hypothesis testing we are attempting to set up a
default position of not guilty. We stay in that position
unless we have enough evidence to overturn it.
Let’s say our null-hypothesis is the following:

111. With hypothesis testing we are attempting to set up a
default position of not guilty. We stay in that position
unless we have enough evidence to overturn it.
Let’s say our null-hypothesis is the following:
There is no difference in IQ between children who are
exposed to classical music between the ages of 0 and 3
and those who were not.

112. With hypothesis testing we are attempting to set up a
default position of not guilty. We stay in that position
unless we have enough evidence to overturn it.
Let’s say our null-hypothesis is the following:
There is no difference in IQ between children who are
exposed to classical music between the ages of 0 and 3
and those who were not.
This is our default position. We are not neutral, we are
claiming at the outset that there is no difference.

113. But then along comes some evidence that over turns that
position. So we reject the null hypothesis and claim there
is a probable difference.

114. But then along comes some evidence that over turns that
position. So we reject the null hypothesis and claim there
is a probable difference.
Notice how we don’t say “there is a difference”. We say
there is a probable or statistical difference. This just
means that with statistics we are never 100% certain. We
just say that the probability that we are wrong is a certain
percent. Usually that percent needs to be pretty low.

115. If we have estimated that there is a 60% chance that we
are wrong, that is a risk not worth taking. If you were told
that you had a 60% chance of losing a lot of money and a
40% chance of making a lot of money, would you take that
chance?
Probably not. But if you were told that you had only a 5%
chance of losing a lot of money and a 95% of earning a lot,
that might be a chance you would be willing to take. The
same holds true with hypothesis testing.

116. If we have estimated that there is a 60% chance that we
are wrong, that is a risk not worth taking. If you were told
that you had a 60% chance of losing a lot of money and a
40% chance of making a lot of money, would you take that
chance?
Probably not. But if you were told that you had only a 5%
chance of losing a lot of money and a 95% of earning a lot,
that might be a chance you would be willing to take. The
same holds true with hypothesis testing.

117. Based on that instruction, consider your answer to these
questions again and explain the correct answer in your
own words.

118. Based on that instruction, consider your answer to these
questions again and explain the correct answer in your
own words.
1. Which expression below from the world of judicial
decision-making best describes the “Null-hypothesis”?
A. “Guilty as charged”
B. “Not guilty until proven innocent”
C. “Pleading no contest”
2. What is another way to say “Null-hypothesis”?
A. Not clear
B. Not different
C. Not important

119. Second Set of Questions – see if you can answer these
questions, if not go to the instruction that follows and
you’ll be given an opportunity to respond to the
questions armed with the instruction.

120. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

121. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

122. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

123. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

124. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.
4. When the jury decides “guilty” and the defendant
actually was “not guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

125. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.
4. When the jury decides “guilty” and the defendant
actually was “not guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

126. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.
4. When the jury decides “guilty” and the defendant
actually was “not guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

127. 3. When the jury decides “not guilty” and the defendant
really is “not guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.
4. When the jury decides “guilty” and the defendant
actually was “not guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

128. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

129. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

130. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

131. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.

132. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.
6. When the jury decides “guilty” and the defendant really
is “guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

133. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.
6. When the jury decides “guilty” and the defendant really
is “guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

134. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.
6. When the jury decides “guilty” and the defendant really
is “guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

135. 5. When the jury decides “not guilty” and the defendant
actually was “guilty”, in statistics that is the same as
saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were wrong to do so.
B. REJECT the null hypothesis and it turns out - - - you
were wrong to do so.
6. When the jury decides “guilty” and the defendant really
is “guilty”, in statistics that is the same as saying:
A. ACCEPT the null hypothesis and it turns out - - - you
were right to do so.
B. REJECT the null hypothesis and it turns out - - - you
were right to do so.

136. Accepting the null-hypothesis is essentially like saying “not
guilty” or that we accept the default position of innocence
or no difference.
Rejecting the null-hypothesis is essentially like saying
“guilty” or that we reject the default position of innocence
or there is enough evidence to suggest there is a
difference.

137. Here is a visual:

138. Here is a visual:
Null-hypothesis
ACCEPTED!

139. Here is a visual:
Null-hypothesis
ACCEPTED!
I was found
NOT
GUILTY!

140. Here is a visual:
Null-hypothesis
ACCEPTED!
I was found
NOT
GUILTY!
Na, na, . . . nanana! There is
NOT enough statistical
evidence to convict or reject
the null-hypothesis!

141. Here is a visual:
Null-hypothesis
ACCEPTED!
I was found
NOT
GUILTY!
Na, na, . . . nanana! There is
NOT enough statistical
evidence to convict or reject
the null-hypothesis!
Not Guilty = Accept the Null

142. Here is a visual:
Null-hypothesis
REJECTED!

143. Here is a visual:
Null-hypothesis
REJECTED!
I was
found
GUILTY!

144. Here is a visual:
Null-hypothesis
REJECTED!
I was
found
GUILTY!
Wa, Wa! There IS enough
statistical evidence to
convict or reject the null-
Hypothesis!

145. Here is a visual:
Null-hypothesis
REJECTED!
I was
found
GUILTY!
Wa, Wa! There IS enough
statistical evidence to
convict or reject the null-
Hypothesis!
Guilty = Reject the Null

146. Third Set of Questions - see if you can answer these
questions, if not go to the instruction that follows and
you’ll be given an opportunity to respond to the questions
armed with the instruction.

147. 7. When the jury decides “guilty” (reject the null) and the
defendant actually was “not guilty” (shouldn’t have
rejected the null), what type of error has been committed?
A. Type I error
B. Type II error

148. 7. When the jury decides “guilty” (reject the null) and the
defendant actually was “not guilty” (shouldn’t have
rejected the null), what type of error has been committed?
A. Type I error
B. Type II error
8. When the jury decides “not guilty” (accept the null) and
the defendant actually was “guilty” (reject the null), what
type of error has been committed?
A. Type I error
B. Type II error

149. 9. Which type of error is preferable?
A. Type I error
B. Type II error

150. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
have committed an error.
are correct in our hypothesis.

151. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
have committed an error.
are correct in our hypothesis.

152. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
A. have committed an error.
B. are correct in our hypothesis.

153. Let’s consider each type of error

154. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence,
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis,
4. You accept the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is a difference between
men and women sports-car color preference and you
should have rejected the null.
6. This is a type I error

155. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis,
4. You accept the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is a difference between
men and women sports-car color preference and you
should have rejected the null.
6. This is a type I error

156. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You accept the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is a difference between
men and women sports-car color preference and you
should have rejected the null.
6. This is a type I error

157. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You accept the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is a difference between
men and women sports-car color preference and you
should have rejected the null.
6. This is a type I error

158. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You accept the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is a difference between
men and women sports-car color preference and you
should have rejected the null.
6. This is a type I error

159. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You accept the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is a difference between
men and women sports-car color preference and you
should have rejected the null.
This is a type I error

160. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You reject the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is NO difference between
men and women sports-car color preference and you
should have accepted the null
This is a type II error

161. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You reject the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is NO difference between
men and women sports-car color preference and you
should have accepted the null
This is a type II error

162. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You reject the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is NO difference between
men and women sports-car color preference and you
should have accepted the null
This is a type II error

163. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You reject the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is NO difference between
men and women sports-car color preference and you
should have accepted the null
This is a type II error

164. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You reject the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is NO difference between
men and women sports-car color preference and you
should have accepted the null
This is a type II error

165. 1. State your null-hypothesis;
There is no significant difference between females and
males in terms of their preference of certain sports-car
colors
2. Collect your evidence
3. Determine if the evidence merits accepting or rejecting
the null-hypothesis
4. You reject the null
5. In reality (and you could never know this for sure) you
were wrong. In actuality there is NO difference between
men and women sports-car color preference and you
should have accepted the null
This is a type II error

166. You’ll never know if you
committed a type I or II error.
You can only estimate the
probability that you did!

167. That’s because with
statistics we deal in
probability, not
certainty.

168. Based on the instruction you just received, respond to
these questions again. Explain your reasoning for selecting
the options you did.

169. 7. When the jury decides “guilty” (reject the null) and the
defendant actually was “not guilty” (shouldn’t have
rejected the null), what type of error has been committed?
A. Type I error
B. Type II error

170. 7. When the jury decides “guilty” (reject the null) and the
defendant actually was “not guilty” (shouldn’t have
rejected the null), what type of error has been committed?
A. Type I error
B. Type II error

171. 7. When the jury decides “guilty” (reject the null) and the
defendant actually was “not guilty” (shouldn’t have
rejected the null), what type of error has been committed?
A. Type I error
B. Type II error
8. When the jury decides “not guilty” (accept the null) and
the defendant actually was “guilty” (reject the null), what
type of error has been committed?
A. Type I error
B. Type II error

172. 7. When the jury decides “guilty” (reject the null) and the
defendant actually was “not guilty” (shouldn’t have
rejected the null), what type of error has been committed?
A. Type I error
B. Type II error
8. When the jury decides “not guilty” (accept the null) and
the defendant actually was “guilty” (reject the null), what
type of error has been committed?
A. Type I error
B. Type II error

173. 9. Which type of error is preferable?
A. Type I error
B. Type II error

174. 9. Which type of error is preferable?
A. Type I error
B. Type II error

175. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
have committed an error.
are correct in our hypothesis.

176. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
have committed an error.
are correct in our hypothesis.

177. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
A. have committed an error.
B. are correct in our hypothesis.
Answers: 7-A, 8-B, 9-B, 10-A

178. 9. Which type of error is preferable?
A. Type I error
B. Type II error
10. Question: What is the haunting reality?
Answer: We actually never know for sure if we have
committed a type I or II error. All we are doing is
determining the probability that we . . .
A. have committed an error.
B. are correct in our hypothesis.
Answers: 7-A, 8-B, 9-B, 10-A

179. Fourth Set of Questions - see if you can answer these
questions, if not go to the instruction that follows and
you’ll be given an opportunity to respond to the questions
armed with the instruction.

180. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong

181. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong

182. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong
12. Question: What does a .05 rejection level mean?
Answer: If we were to take the same small sample 100
times from a population, we would be willing to
_____________________ .05 or 5% of the time
a. . . . take the chance of being wrong . . .
b. . . . reject the null hypothesis . . .

183. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong
12. Question: What does a .05 rejection level mean?
Answer: If we were to take the same small sample 100
times from a population, we would be willing to
_____________________ .05 or 5% of the time
a. . . . take the chance of being wrong . . .
b. . . . reject the null hypothesis . . .

184. In statistics we generally ask ourselves, “What is the
probability that we have made a type I error?”

185. In statistics we generally ask ourselves, “What is the
probability that we have made a Type I Error?”
Type I errors are considered a bigger issue because if we
are wrong, than we might waste a lot of money or impact
people negatively (e.g., spend millions of dollars on a new
drug that doesn’t work).

186. In statistics we generally ask ourselves, “What is the
probability that we have made a Type I Error?”
Type I errors are considered a bigger issue because if we
are wrong, than we might waste a lot of money or impact
people negatively (e.g., spend millions of dollars on a new
drug that doesn’t work).
Type II errors are considered less of an issue because if we
are wrong, than we may stop or continue researching.

187. We have to have determine a cut-off point as to when we
will reject the null-hypothesis. No matter what cut-off
point we could have chosen, the decision would always
have been somewhat arbitrary.

188. We have to have determine a cut-off point as to when we
will reject the null-hypothesis. No matter what cut-off
point we could have chosen, the decision would always
have been somewhat arbitrary.
Would we be satisfied with a 75% chance of committing a
type I error? Probably not. That means out of 100
experiments we would live with being wrong about our
conclusions 75 times.

189. Would we be satisfied with a .01% chance of committing a
type I error? Probably not. That means out of 10,000
experiments we would live with being wrong about our
conclusions only once. If that were the case, then almost
no null-hypothesis could ever be rejected.

190. Would we be satisfied with a .01% chance of committing a
type I error? Probably not. That means out of 10,000
experiments we would live with being wrong about our
conclusions only once. If that were the case, then almost
no null-hypothesis could ever be rejected.
In the discipline of statistics .05 or 5% of a chance of
committing a type I error has been deemed an acceptable
arbitrary cut-off point. This means that out of 100
experiments we will live with being wrong five times.

191. Based on the instruction you just received, respond to
these questions again. Explain your reasoning for selecting
the options you did.

192. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong

193. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong

194. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong

195. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong
12. Question: What does a .05 rejection level mean?
Answer: If we were to take the same small sample 100
times from a population, we would be willing to
_____________________ .05 or 5% of the time
a. . . . take the chance of being wrong . . .
b. . . . reject the null hypothesis . . .

196. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong
12. Question: What does a .05 rejection level mean?
Answer: If we were to take the same small sample 100
times from a population, we would be willing to
_____________________ .05 or 5% of the time
a. . . . take the chance of being wrong . . .
b. . . . reject the null hypothesis . . . Answers: 11-B, 12-A

197. 11. Question: How do we decide how much evidence is
required before we will reject the null hypothesis?
Answer: We estimate the probability of being ______ a
certain percent of the time (e.g., .05 or 5% of the time).
a. right
b. wrong
12. Question: What does a .05 rejection level mean?
Answer: If we were to take the same small sample 100
times from a population, we would be willing to
_____________________ .05 or 5% of the time
a. . . . take the chance of being wrong . . .
b. . . . reject the null hypothesis . . . Answers: 11-B, 12-A