1. Two Phases of establishing cutoff score
• (1) Obtaining estimated cutoff score using
expert judgments.
• (2) establishing the operational cutoff score
Three Components of the estimation phase
• (1) selection of an estimation method
• (2) data collection
• (3) analysis of data
1

2. Standard-setting Method
• Normative Method
• Absolute Method
– Contrasting Group
– Borderline group
• Absolute methods based on evaluation of the
test:
– Jaeger method
– Angoff method
– Nedelsky method
– Ebel Method
2

3. Normative method
• The passing score is based on an individual’s
ranking within a group.
• Standard is established that passes a given
percentage of a reference group of examines.
• Ex. 10% of the seniors taking the test will be
eligible for admission in Stanford University.
• Advantage: The number of people who will pass
the test is known.
• Disadvantage: It does not guarantee that all
individuals who are competent pass the test.
3

4. Absolute methods based on the
Evaluation of Examinees
• Contrasting groups
– Experts classify each examinee into one or two groups, for
example, master or nonmaster.
– Other criterion can be used to classify them as
maters/nonmasters, experts/novice (grades).
– Assess the number of classification errors: qualified
individuals who failed, unqualified individuals who passed.
– Cutoff the scores in the distribution if the difference in the
two groups is significant.
– Disadvantage: the selection may not match the scores; A
small number of high scores in the nonmaster group can
raise the cutoff score, low scores in the master group can
lower it. 4

5. Absolute methods based on the
Evaluation of Examinees
• Borderline group
– Scores of examinees who are classified as
borderline are used to set the cutoff score.
– Borderline (neither masters nor nonmasters)
– The cutoff score is placed at the mean or median
of the borderline group.
5

6. Absolute methods based on Evaluation of
the Test
• Jaeger method
– Each item is judged about its importance for the decision
to be made.
– “Should every high school graduate be able to answer the
item correctly?”
– Cutoff scores are established by counting the number of
items for which a correct response is judged to be critical.
– Advantage: No conceptualization of minimal competent
student.
– Disadvantage: Passing item could be denied due to the
answer of a single item.
6

7. Absolute methods based on
Evaluation of the Test
• Angoff method
– Judges must estimate the difficulty of the item for
a hypothetical group of minimally competent
examinees.
– The estimated cutoff score for a judged is
calculated by summing the item difficulty
estimates.
– Disadvantage: Difficulty rating is just looking at
the item stem.
7

8. Absolute methods based on
Evaluation of the Test
• Nedelsky Method
– Judges evaluate the attractiveness of each distractor on
the basis of whether or not the minimally competent
examinee would be able to identify the distractor as
incorrect.
– Item difficulty is then estimated by assuming borderline
examinees will guess randomly among the remaining
options.
– Advantage: The responses are evaluated aside from the
stem.
– Disadvantage: method can only be used for multiple
choice.
8

9. Absolute methods based on
Evaluation of the Test
• Ebel Method
– Gives two judgement for each item: its difficulty
(easy, moderate, difficult) and its relevance
(essential, important, acceptable, questionable).
– Items are classified in a 3 X 4 matrix:
Easy Moderate Difficult
Essential
Important
Acceptable
Questionable
9

10. Absolute methods based on
Evaluation of the Test
• Ebel method (cont.)
– A jusdgement of the percentage of items in each cell of
the matrix that will be answered correctly by the
minimally competent student is then used to weight the
number of items assigned to each cell.
– The cutoff scores is calculated using cell weights and the
number of items assigned to each cell.
– Advantage: items is evaluated not only on difficulty but on
relevance
– Disadvantage: Accuracy of judges
10

11. Compromise methods
• Hofstee method
– Identify limits of the absolute cutoff score and the limits of
the relative failing rates.
– Judges make judgments about the item:
– (1) what is the lowest acceptable cutoff score, even if every
examinee were to fail the examination if this cutoff score
was accepted?
– (2) What is the highest acceptable cutoff score?
– (3) What is the minimum acceptable failing rate?
– (4) What is the maximum acceptable failing rate?
– 2 points are plotted: The minimum acceptable cutoff score,
maximum acceptable failing rate and the maximum
acceptable failing rate , maximum acceptable failing rate.
– The intersection in the ogive curve is the cutoff point. 11

12. Compromise methods
• Beuk method
– A point defined by the average absolute cutoff
score and the average passing rate is plotted.
– The intersection of this point to the ogive curve is
the compromise cutoff score.
12

13. Compromise methods
• De Gruijter method
– Incorporation of uncertainty measures.
– Each judge provide estimates of his or her
uncertainty with respect to the true value of the
failing rate and the true value of the cutoff score.
– Each judge plots their points in the ogive curve.
– A family of ellipse is generated around the plots.
– The ellipse that touches the ogive curve is the
compromise cutoff score.
13