SlideShare a Scribd company logo
1 of 57
Developing a Teacher Made
           Test
       Dr. Carlo Magno




                            1
Advance Organizer
1   The Test Blueprint
    Outline of the Test Development Process
  Table of Specifications
2 Designing Selected-Response Items
    Binary-choice items
    Instructions in Writing Binary Type of Items
     Multiple-choice items
     Guidelines in Writing Multiple-choice Items
     Matching items
     Guidelines in Writing Multiple-choice Items
3   Designing Constructed-Response Items
    Short-answer items
    Guidelines in Writing Short Answer Items
    Essay items
4   Designing Interpretive Exercise
    Guidelines in Writing Intepretive Exercise
                                                   2
    Examples of Interpretive Exercise
Objectives
• 1. Explain the theories and concepts that
  rationalize the practice of assessment.
• 2. Make a table of specifications of the test
  items.
• 3. Design pen-and-paper tests that are aligned
  to the learning intents.
• 4. Justify the advantages and disadvantages of
  any pen-and-paper test.
• 5. Evaluate the test items according to the
  guidelines presented.
                                                   3
Outline of Test Development Process
•   1. Specify the ultimate goals of the education process
•   2. Derive from these the goals of the portion of the
    system under study
•   3. Specify these goals in terms of expected student
    behavior. If relevant, specify the acceptance level of
    successful learning.
•   4. Determine the relative emphasis or importance of
    various objectives, their content, and their behaviors.
•   5. Select or develop situations that will elicit the
    desired behavior in the appropriate context or
    environment, assuming the student has learned it.
•   6. Assemble a sample of such situations that together
    represent accurately the emphasis on content and
    behavior previously determined.

                                                              4
Outline of Test Development Process
•   7. Provide for the recording of responses in a form
    that will facilitate scoring but will not so distort the
    nature of the behavior elicited that it is no longer a true
    sample or index of the behavior desired.
•   8. Establish scoring criteria and guides to provide
    objective and unbiased judgment.
•   9. Try out the instrument in preliminary form.
•   10. Revise the sample of situations on the basis of
    tryout information.
•   11. Analyze reliability, validity, and score distribution
    in accordance with the projected use of scores.
•   12. Develop test norms and a manual, and reproduce
    and distribute the test.

                                                              5
Test Length
• The test must be of sufficient length to yield
  reliable scores
• The longer the test, the more the reliable the
  results
• The test should be valid if it is reliable
• For the grade school, one must consider the
  stamina and attention span of the pupils
• The test should be long enough to be
  adequately reliable and short enough to be
  administered
                                                   6
Test Instruction

• It is the function of the test instructions to
  furnish the learning experiences needed in
  order to enable each examinee to
  understand clearly what he is being asked
  to do?
• Instructions may be oral, a combination of
  written and oral instruction is probably
  desirable, except with very young children.
• Clear concise and specific.
                                               7
Test layout
• The arrangement of the test items influences the speed
  and accuracy of the examinee
• Utilize the space available while retaining readability.
• Items of the same type should be grouped together
• Arrange test items from easiest to most difficult as a
  means of reducing test anxiety.
• The test should be ordered first by type then by content
• Each item should be completed in the column and page
  in which it is started.
• If the reference material is needed, it should occur on the
  same page as the item
• If you are using numbers to identify items it is better to
  use letters for the options

                                                            8
Scoring the test
• Use separate answer sheets
• Punched key
• Overlay key
• Strip key
Plight of the student
• The teacher should discuss with the class the
  content areas and levels of the cognitive domain
  to be examined
• The discussion should utilize a vocabulary and a
  level of complexity appropriate to the
  development level of the student
• Types of test
• Examples of test type
                                                 9
Table of Specifications
                   One Grid TOS

          Content Outline          No. of items

1. Table of specifications             10
2. Test and Item characteristics       20
3. Test layout                          5
4. Test instructions                    5
5. Reproducing the test                 5
6. Test length                          5
7. Scoring the test                     5
TOTAL                                  55         10
Table of Specifications
Two Grid TOS
Weight                Content                      Knowledge         Comprehension         Application   No. of items
(Time                 Outline                        30%                 40%                 30%          by content
Frame)                                                                                                       area
 35%     1. Table of specifications                      1                   4                   4            9

 30%     2. Test and Item characteristics                2                   3                   3            8

 10%     3. Test layout                                  1                   1                   0            2

 5%      4. Test instructions                            0                   1                   0            1

 5%      5. Reproducing the test                         1                   0                   0            1

 5%      6. Test length                                  1                   0                   1            2

 10%     7. Scoring the test                             2                   1                   0            3

                                                         8                  10                   8           26


  The number of items in a cell is computed using the formula:
                                  Given time
                          items              X percentageof cognitive skill X total number of items                11
                                  Total time
Classification of test Items
•   Selected Response
    –   Binary Choices
    –   Multiple Choice
    –   Matching Type
•   Constructed Response “Supply Test”
    –   Short Form answers - identification
    –   Completion – fill in the blanks, cloz test
    –   Essay
•   Performance Type
    –   Paper and pencil type
    –   Identification type
    –   Simulation
                                                     12
Item Writing Commandments
•   Thou shall not produce opaque directions to students
    regarding how to respond to your instructions (opaque
    directions)
•   Thou shall not employ ambiguous statements in your
    assessment item (ambiguous statements)
•   Thou shall not unintentionally provide students with
    clues regarding appropriate response (unintended
    clues)
•   Thou shall not employ complex syntax in your
    assessment item (complex syntax)
•   Thou shall not use vocabulary that is more advanced
    than required (Difficult vocabulary)

                                                        13
SHORT ANSWER ITEMS
• 1. Word the item so that the answer is both brief
  and definite.
• 2. Do not take statements directly from books to
  use as a basis for short answer items.
• 3. A direct question is generally more
  acceptable than an incomplete statement.
• 4. The answer to be expressed in numerical
  units indicate the type of answer wanted.
• 5. Blanks for answers should be equal in length.
• 6. Do not use to many blanks.
                                                  14
Writing supply items
1. Require short, definite, clear-cut, and explicit answers
FAULTY: Earnest Hemingway wrote______
IMPROVED: The Old Man and the Sea was written by
   _______.
   Who wrote The Old man and the Sea?
2. Avoid multimutilated statements
FAULTY: _____ pointed out in ____ the freedom of thought
   in America was seriously hampered by ___, ____, & __.
IMPROVED: That freedom of thought in America was
   seriously hampered by social pressures toward
   conformity was pointed out in 1830 by ______.

                                                         15
Writing supply items
3. If several equal answers equal credit should be given to
     each one.
4. Specify and announce in advance whether scoring will
     take spelling into account.
5. In testing for comprehension of terms and knowledge
     of definition, it is often better to supply the term and
     require a definition than to provide a definition and
     require the term.
FAULTY: What is the general measurement term
     describing the consistency with which items in a test
     measure the same thing?
IMPROVED: Define “internal consistency reliability.”

                                                            16
Writing supply items
6. It is generally recommended that in completion items
   the blanks come at the end of the statement.
FAULTY: A (an) ________ is the index obtained by dividing
   a mental age score by chronological age and
   multiplying by 100.
IMPROVED: The index obtained by dividing a mental age
   score by chronological age and multiplying by 100 is
   called a (an) ________
7. Minimize the use of textbook expressions and
   stereotyped language.
FAULTY: The power to declare war is vested in ______
IMPROVED: Which national legislative body has the
   authority to declare war?

                                                       17
Writing supply items
8. Specify the terms in which the response is to be given.
FAULTY: Where does the Security Council of the United Nations
     hold its meeting?
IMPROVED: In what city of the United States does the Security
     Council of the United Nations hold its meeting?
FAULTY: If a circle has 4-inch diameter, its area is_____
IMPROVED: A circle has 4-inch diameter. Its area in square
     inches correct to two decimal places, is _____
9. In general, direct questions are preferable to incomplete
     declarative sentences.
FAULTY: Gold was discovered in California in the year ___
IMPROVED: In what year was gold discovered in California?

                                                           18
Writing supply items
10. Avoid extraneous clues to the correct answer
FAULTY: A fraction whose denominator is greater than its
    numerator is a _____
IMPROVED: Fractions whose denominator are greater
    than their numerators are called _____




                                                           19
ALTERNATIVE RESPONSE ITEM
• 1. Avoid broad general statements if they are to be
  judged true or false.
• 2. Avoid trivial statements.
• 3. Avoid the use of negative statements.
• 4. Avoid long complex sentences.
• 5. Avoid including two ideas in one statement unless
  cause and effect relationship are being measured.
• 6. If the opinion is being used, attribute it to some
  source unless the ability to identify opinion is being
  specifically measured.
• 7. True statements and false statements should be
  equal in length.
• 8. The number of true and false statements should be
  approximately equal.

                                                           20
Writing TRUE-FALSE Items
1. Avoid the use of “specific determiners”
FAULTY: No picture-no sound in a television set may
   indicate a bad 5U4G.
IMPROVED: A bad 5U4G tube in a television set will result
   in no picture sound.
2. Base true-false items upon statements that are
   absolutely true or false, without qualifications or
   exceptions.
FAULTY: World War II was fought in Europe and the Far
   East.
IMPROVED: The primary combat locations in terms of
   military personnel during World War II were Europe
   and the Far East.

                                                        21
Writing TRUE-FALSE Items

3. Avoid negative stated items when possible and
   eliminate all double negatives.
FAULTY: It is not frequently observed that copper turns
   green as a result of oxidation.
IMPROVED: Copper will turn green upon oxidizing.
4. Use quantitative and precise rather than qualitative
   language where possible.
FAULTY: Many people voted for Gloria Arroyo in the 2003
   Presidential election.
IMPROVED: Gloria Arroyo received more than 60 percent
   of the popular votes cast in the Presidential election of
   2003.

                                                           22
Writing TRUE-FALSE Items
5.  Avoid stereotypic and textbook statements.
FAULTY: From time to time efforts have been made to explode the
    notion that there may be a cause-and-effect relationship between
    arboreal life and primate anatomy.
IMPROVED: There is a known relationship between primate anatomy
    and arboreal life.
6.  Avoid making the true items consistently longer than the false
    items.
7.  Avoid the use of unfamiliar or esoteric language.
FAULTY: According to some peripatetic politicos, the raison d‟etre for
    capital punishment is retribution.
IMPROVED: According to some politicians, justification for the
    existence of capital punishment can be traced to the Biblical
    statement, “An eye for an eye.”


                                                                     23
Writing TRUE-FALSE Items
8. Avoid complex sentences with many dependent
   clauses.
FAULTY: Jane Austen, an American novelist born in 1790,
   was a prolific writer and is best known for her novel
   Pride and Prejudice, which was published in 1820.
IMPROVED: Jane Austen is best known for her novel Pride
   and prejudice.
9. It is suggested that the crucial elements of an item be
   placed at the end of the statement.
FAULTY: Oxygen reduction occurs more readily because
   carbon monoxide combines with hemoglobin faster
   than oxygen does.
IMPROVED: Carbon monoxide poisoning occurs because
   carbon monoxide dissolves delicate lung tissue.
                                                        24
Writing Matching Type Test
1.   Matching Exercises should be complete on a single
     page.
2.   Use response categories that are related but mutually
     exclusive.
3.   Keep the number of stimuli relatively small (10-15),
     and let the number of possible responses exceed the
     number of stimuli by two or three.
4.   The direction should clearly specify the basis for
     matching stimuli and responses.
5.   Keep the statements in the response column short and
     list them in some logical order


                                                        25
FAULTY: Match List A with List B. You will be given one
  point for each correct match.
  List A                   List B
  a. cotton gin            a. Eli Whitney
  b. reaper                b. Alexander Graham Bell
  c. wheel                 c. David Brinkley
  d. TU54G tube            d. Louisa May Alcott
  e. steamboat             e. None of these
•   Directions failed to specify the basis for matching
•   List are enumerated identically
•   Responses not listed logically
•   Lacks homogeneity
• Equal number of elements
• Use of “None of the above”
                                                          26
IMPROVED: Famous inventions are listed in the left-hand
  column and inventors in the right-hand column below.
  Place the letter corresponding to the inventor in the
  space next to the invention for which he is famous. Each
  match is worth 1 point, and “None of these” may be the
  correct answer. Inventors may be used more than once.
   Inventions                      Inventors
__ 1. steamboat            a. Alexander Graham-Bell
__ 2. cotton skin          b. Robert Fulton
__ 3. sewing machine       c. Elias Howe
__ 4. reaper               d. Cyrus McCormick
                           e. Eli Whitney
                           f. None of these

                                                         27
Writing Multiple Choice
1. It is recommended that the stem be a direct question.
2. The stem should pose a clear, define, explicit, and
   singular problem.
FAULTY: Salvador Dali is
   a. a famous Indian.
   b. important in international law.
   c. known for his surrealistic art.
   d. the author of many avant-garde plays.
IMPROVED: With which one of the fine arts is Salvador
   Dali associated?
   a. surrealistic painting
   b. avant-garde theatre
   c. polytonal symphonic music
   d. impressionistic poetry
                                                           28
Writing Multiple Choice
3. Include in the stem any words that might otherwise be
   repeated in each response.
FAULTY: Milk can be pasteurized at home by
   a. heating it to a temperature of 130o
   b. Heating it to a temperature of 145o
   c. Heating it to a temperature of 160o
   d. Heating it to a temperature of 175o
IMPROVED: The minimum temperature that can be used
   to pasteurize milk at home is:
   a. 130o
   b. 145o
   c. 160o
   d. 175o
                                                           29
Writing Multiple Choice
4. Items should be stated simply and understandably, excluding all
    nonfunctional words from stem and alternatives.
FAULTY: Although the experimental research, particularly that by
    Hansmocker must be considered equivocal and assumptions
    viewed as too restrictive, most testing experts would recommend as
    the easiest method of significantly improving paper-and-pencil
    achievement test reliability to
    a. increase the size of the group being tested.
    b. increase the differential weighting of items.
    c. increase the objective of scoring.
    d. increase the number of items.
    e. increase the amount of testing time.
IMPROVED: Assume a 10-item, 10-minute paper-and-pencil multiple
    choice achievement test has a reliability of .40. The easiest way of
    increasing the reliability to .80 would be to increased
    a. group size
    b. scoring objectivity
    c. differential item scoring weights
    d. the number of items
    e. testing time                                                      30
Writing Multiple Choice
5. Avoid interrelated items
6. Avoid negatively stated items
FAULTY: None of the following cities is a state capital
   except
   a. Bangor
   b. Los Angeles
   c. Denver
   d. New Haven
IMPROVED: Which of the following cities is a state capital?
   a. Bangor
   b. Los Angeles
   c. Denver
   d. New Haven
                                                          31
Writing Multiple Choice
7. Avoid making the correct alternative systematically different from
     other options
8.   If possible the alternatives should be presented in some logical,
     numerical, or systematic order.
9.   Response alternatives should be mutually exclusive.
FAULTY: Who wrote Harry Potter and the Goblet of Fire?
     a. J. K. Rowling
     b. Manny Paquiao
     c. Lea Salonga
     d. Mark Twain
IMPROVED: Who wrote Penrod?
     a. J. K. Rowling
     b. J. R. R. Tolkien
     c. V. Hugo
     d. L. Carrol

                                                                         32
Writing Multiple Choice
10. Make all responses plausible and attractive to the less
   knowledgeable and skillful student.
FAULTY: Which of the following statements makes clear the meaning
   of the word “electron”?
   a. An electronic tool
   b. Neutral particles
   c. Negative particles
   d. A voting machine
   e. The nuclei of atoms
IMPROVED: Which of the following phrases is a description of an
   “electron”?
   a. Neutral particle
   b. Negative particle
   c. Neutralized proton
   d. Radiated particle
   e. Atom nucleus


                                                                    33
Writing Multiple Choice
11. The response alternative “None of the above” should be used with
   caution, if at all.
FAULTY: What is the area of a right triangle whose sides adjacent to
   the right angle are 4 inches long respectively?
   a. 7
   b. 12
   c. 25
   d. None of the above
IMPROVED: What is the area of a right triangle whose sides adjacent
   to the right angle are 4 inches and 3 inches respectively?
   a. 6 sq. inches
   b. 7 sq. inches
   c. 12 sq. inches
   d. 25 sq. inches
   e. None of the above
                                                                       34
Writing Multiple Choice
12. Make options grammatically parallel to each other and consistent
   with the stem.
FAULTY: As compared with the American factory worker in the early
   part of the 19th century, the American factory worker at the close of
   the century
   a. was working long hours
   b. received greater social security benefits
   c. was to receive lower money wages
   d. was less likely to belong to a labor union.
   e. became less likely to have personal contact with employers
IMPROVED: As compared with the American factory worker in the
   early part of the century, the American factory worker at the close of
   the century
   a. worked longer hours.
   b. had more social security.
   c. received lower money wages.
   d. was less likely to belong to a labor union
   e. had less personal contact with his employer
                                                                        35
Writing Multiple Choice
13. Avoid such irrelevant cues as “common elements” and “pat verbal
   associations.”
FAULTY: The “standard error of estimate‟ refer to
   a. the objectivity of scoring.
   b. the percentage of reduced error variance.
   c. an absolute amount of possible error.
   d. the amount of error in estimating criterion scores.
IMPROVED: The “standard error of estimate” is most directly related to
   which of the following test characteristic?
   a. Objectivity
   b. Reliability
   c. Validity
   d. Usability
   e. Specificity
                                                                     36
Writing Multiple Choice
14. In testing for understanding of a term or concept, it is generally
   preferable to present the term in the stem and alternative definitions
   in the options.
FAULTY: What name is given to the group of complex organic
   compounds that occur in small quantities in natural foods that are
   essential to normal nutrition?
   a. Calorie
   b. Minerals
   c. Nutrients
   d. Vitamins
IMPROVED: Which of the following statements is the best description
   of a vitamin?
15. Use objective items – items‟ whose correct answers are agreed by
   experts


                                                                        37
Factual Knowledge
•    The Monroe Doctrine was announced about 10 years after the
     a. Revolutionary War
     b. War of 1812
     c. Civil War
     d. Spanish-American War
Conceptual Knowledge
2.   Which of the following statements of the relationship between
     market price and normal price is true?
     a. Over a short period of time, market price varies directly with
     changes in normal price.
     b. Over a long period of time, market price tends to equal normal
     price.
     c. Market price is usually lower than normal price.
     d. Over a long period of time, market price determines normal
     price.

                                                                     38
Translation from symbolic form to another form, or vice
     versa
  3. Which of the graphs below best represent the supply
     situation where a monopolist maintains a uniform price
     regardless of the amounts which people buy?

        A                        B                    C                      D
                               S
                                                             S                          S
                   S                      S
Price




                       Price




                                              Price




                                                                     Price
                                                                 S           S
        Quantity               Quantity               Quantity               Quantity




                                                                                            39
Application
In the following items (4-8) you are to judge the effects of a particular
      policy on the distribution of income. In each case assume that
      there are no other changes in policy that would counteract the
      effect of the policy described in the item. Mark the item:
A. If the policy described would tend to reduce the existing degree of
      inequality in the distribution of income,
B. If the policy described would tend to increase the existing degree
      of inequality in the distribution of income, or
C. If the policy described would have no effect, or an indeterminate
      effect, on the distribution of income.
__ 4. Increasingly progressive income taxes.
__ 5. Confiscation of rent on unimproved
__ 6. Introduction of a national sales tax
__ 7. Increasing the personal exemptions from income taxes
__ 8. Distributing a subsidy to sharecroppers on southern farms

                                                                       40
Analysis
9. An assumption basic to Lindsay‟s preference for
   voluntary associations rather than government order… is
   a belief
   a. that government is not organized to make the best use
   of experts
   b. that freedom of speech, freedom of meeting, freedom
   of association, and possible only under a system of
   voluntary associations.
   c. in the value of experiment and initiative as a means of
   attaining an ever improving society
   d. in the benefits of competition



                                                           41
Judgments in terms of external criteria
For items 14-16, assume that in doing research for a paper about the
      English language you find a statement by Otto Jespersen that
      contradicts one point of view in a language you have always
      accepted. Indicate which of the statements would be significant in
      determining the value of Jespersen‟s statement. For the purpose
      of these items, you may assume that these statements are
      accurate. Mark each item using the following key.
A. Significant positively – that is, might lead you to trust his
      statement and to revise your own opinion.
B. Significant negatively – that is, night lead you to distrust his
      statement
C. Has no significance
__ 14. Mr. Jesperson was professor of English at Copenhagen
      University
__ 15. The statement in question was taken from the very first article
      that Jespersen published
__ 16. Mr. Jespersen‟s books are frequently referred to in other works
      that you consult.


                                                                       42
Essay Questions
• 1. Ask questions or set tasks that will require the
  examinee to demonstrate a command of essential
  knowledge.
• 2. Ask questions that are determinate, in the sense that
  experts could agree that one answer is better than
  another.
• 3. Define the examinee‟s task as completely and
  specifically as possible without interfering with
  measurements of the achievement intended.
• 4. In general, give preference to more specific questions
  that can be more answered briefly.
• 5. Avoid giving the examinee a choice among optional
  questions unless special circumstances make such
  option necessary.
• 6. Test the questions by writing an ideal answer
                                                         43
Types of Essays:
• General – extensiveness of responses
• Restrictive Response – reliable scoring
Learning outcomes measured by Essay:
• Explain cause-effect relationship
• Describe applications of principles
• Present relevant arguments
• Formulate tangible hypothesis
• Formulate valid conclusions
• State necessary assumptions
• Describe the limitations of data
• Explain methods and procedures
• Produce, organize, and express ideas
• Integrate learnings in different areas
• Create original forms
• Evaluate the worth of ideas

                                            44
Understanding:
A. Comparison of two phenomena on a single designated basis:
    Compare the writers of the English Renaissance to those of the
    nineteenth century with respect to their ability tot describe nature
B. Comparison of two phenomena in general
    Compare the French and Russian Revolutions
C. Explanation of the use or exact meaning of a phrase or statement
    The book of John begins “In the beginning was the word…” From
    what philosophical system does this statement derive?
D. Summary of a text or some portion of it
    State the central theme of the Communist Manifesto
E. Statement of an artist’s purpose in the selection or organization of
    material
    Why did Hemingway describe in detail the episode in which
    Gordon, lying wounded, engage the oncoming enemy?
    What was Beethoven‟s purpose in deviating from the orthodox
    form of a symphony in Symphony No. 6?

                                                                      45
Application:
A. Causes or effects
     Why may too frequent reliance on penicillin for the treatment of
     minor ailments eventually result in its diminished effectiveness
     against major invasion of body tissues by infectious bacteria?
B. Analysis
     Why was Hamlet torn by conflicting desires?
C. Statement of relationship
     It is said that intelligence correlates with school achievement at
     about .65. Explain this relationship
D. Illustrations or examples of principles
     Name three examples of uses of the lever in typical American
     homes
E. Application of rules or principles
     Would you weigh more or less on the moon? On the sun?
     Explain.
F.   Reorganization of facts
     Some writers have said that the American Revolution was not
     merely a political revolution against England but also a social
     revolution, within the colonies, of the poor against the wealthy.
     Using the same evidence what other conclusion is possible?
                                                                          46
Judgment:
A. Decision for or against
    Should members of the Communist Party be allowed to teach in
    American colleges? Why or why not?
B. Discussion
    Discuss the likelihood that four-year private liberal arts colleges
    will gradually be replaced by junior colleges and state universities.
C. Criticism of the adequacy, correctness, or relevance of a
    statement
    The discovery of penicillin has often been called an accident.
    Comment on the adequacy of this explanation.
D. Formulation of new questions
    What should one find out in order to explain why some students of
    high intelligence fail in school?



                                                                       47
Designing Interpretive Exercise
• Guidelines in Writing Interpretive Exercise
• 1. Select an introductory that is in harmony with the
  objectives of the course.
   – Amount of emphasis of various interpretive skills is a factor.
   – Do not overload test takers with interpretive items in a particular
     area.
   – Selection of introductory should be guided by general emphasis
     to be given to the measurement of complex achievement.
• 2. Select introductory material that is appropriate to the
  curricular experience and reading ability of the
  examinees.



                                                                       48
Guidelines in Writing
          Interpretive Exercise
• 3. Select introductory material that is new to pupils.
• 4. Select introductory material that is brief but
  meaningful.
• 5. Revise introductory material for clarity, conciseness,
  and greater interpretive value.
• 6. Construct test items that require analysis and
  interpretation of introductory material.
• 7. Make the number of items roughly proportional to the
  length of the introductory material.
• 8. Observe all suggestions for constructing objective
  test items.
                                                          49
• Ability to Recognize the Relevance of
  Information




                                          50
• Ability to Recognize Warranted and
  Unwarranted Generalizations




                                       51
• Ability to Recognize Inferences




                                    52
• Ability to Interpret Experimental Findings




                                               53
• Ability to Apply Principles




                                54
• Ability to Recognize Assumptions




                                     55
Reading comprehension
•    Bem (1975) has argued that androgynous people
     are “better off” than their sex-typed counterparts     35. What is the independent variable in
     because they are not constrained by rigid sex-         the study?
     role concepts and are freer to respond to a wider
     variety of situations. Seeking to test this
     hypothesis, Bem exposed masculine, feminine,           a. Situations calling for independence and
     and androgynous men and women to situations
     that called for independence (a masculine                 nurturance
     attribute) or nurturance (a feminine attribute). The   b. Situation to make the sex type react
     test for masculine independence assessed the
     subject‟s willingness to resist social pressure by     c. Situations to make the androgynous be
     refusing to agree with peers who gave bogus               flexible
     judgments when rating cartoons for funniness (for
     example, several peers might say that a very           d. Situations like sex type, androgynous
     funny cartoon was hilarious). Nurturance or               and sex role concepts
     feminine expressiveness, was measured by
     observing the behavior of the subject when left
     alone for ten minutes with a 5-month old baby.         36. What are the levels of the IV?
     The result confirmed Bem‟s hypothesis. Both the
     masculine sex-typed and the androgynous
     subjects were more independent (less                   a. masculine attribute and feminine
     conforming) on the „independence” test than
     feminine sex-typed individuals. Furthermore, both         attribute
     the feminine and the androgynous subjects were         b. rating cartoons and taking care of a
     more “nurturant” than the masculine sex-typed
     individuals when interacting with the baby. Thus,         baby
     the androgynous subjects were quite flexible,          c. independence and nurturance
     they performed as masculine subjects did on the
     “feminine” task.                                       d. flexibility and rigidity
                                                                                                  56
Interpreting Diagrams
Instruction. Study the following illustrations and answer the following
   questions.
                                  101. Which group received the treatment?
Figure 1
                                  a. group A          b. group B
                                  b. c. none of the above

                                  102. Why did group B remain stable across the
                                  experiment?

                                  a. there is an EV
                                  b. there is no treatment
                                  c. there is the occurence of ceiling effect

                                  103. What is the problem at the start of the
                                  experiment?

    Pretest          Posttest     a. the groups are nonequivalent
                                  b. the groups are competing with each other
                                  c. the treatment took place immediately
                                                                                 57

More Related Content

What's hot

Performance assessment
Performance assessmentPerformance assessment
Performance assessmentKrisna Marcos
 
Assessment of Learning - Multiple Choice Test
Assessment of Learning - Multiple Choice TestAssessment of Learning - Multiple Choice Test
Assessment of Learning - Multiple Choice TestXiTian Miran
 
Developing Classroom-based Assessment Tools
Developing Classroom-based Assessment ToolsDeveloping Classroom-based Assessment Tools
Developing Classroom-based Assessment ToolsMary Grace Ortiz
 
Administering the test
Administering the testAdministering the test
Administering the testenylisac25
 
Multiple choice test items
Multiple choice test itemsMultiple choice test items
Multiple choice test itemslorne27
 
Types of Tests
Types of TestsTypes of Tests
Types of TestsIvy Autor
 
Process and product performane-based assessment
Process and product performane-based assessment Process and product performane-based assessment
Process and product performane-based assessment Dianopesidas
 
types of test items
types of test itemstypes of test items
types of test itemskavukavya
 
Testing, assessment, measurement and evaluation definition
Testing, assessment, measurement and evaluation definitionTesting, assessment, measurement and evaluation definition
Testing, assessment, measurement and evaluation definitionnorazmi danuri
 
Item and Distracter Analysis
Item and Distracter AnalysisItem and Distracter Analysis
Item and Distracter AnalysisSue Quirante
 
CONSTRUCTING PAPER-AND-PENCIL TESTS
CONSTRUCTING PAPER-AND-PENCIL TESTSCONSTRUCTING PAPER-AND-PENCIL TESTS
CONSTRUCTING PAPER-AND-PENCIL TESTSJhenq Campo
 
Completion type of test
Completion type of testCompletion type of test
Completion type of testubariel
 

What's hot (20)

Teacher made tests
Teacher made testsTeacher made tests
Teacher made tests
 
Performance assessment
Performance assessmentPerformance assessment
Performance assessment
 
Assessment of Learning - Multiple Choice Test
Assessment of Learning - Multiple Choice TestAssessment of Learning - Multiple Choice Test
Assessment of Learning - Multiple Choice Test
 
Developing Classroom-based Assessment Tools
Developing Classroom-based Assessment ToolsDeveloping Classroom-based Assessment Tools
Developing Classroom-based Assessment Tools
 
Administering the test
Administering the testAdministering the test
Administering the test
 
Types of test
Types of testTypes of test
Types of test
 
Multiple choice test items
Multiple choice test itemsMultiple choice test items
Multiple choice test items
 
Types of Tests
Types of TestsTypes of Tests
Types of Tests
 
Process and product performane-based assessment
Process and product performane-based assessment Process and product performane-based assessment
Process and product performane-based assessment
 
types of test items
types of test itemstypes of test items
types of test items
 
Table of specifications
Table of specificationsTable of specifications
Table of specifications
 
Testing, assessment, measurement and evaluation definition
Testing, assessment, measurement and evaluation definitionTesting, assessment, measurement and evaluation definition
Testing, assessment, measurement and evaluation definition
 
Essay Test
Essay TestEssay Test
Essay Test
 
Item and Distracter Analysis
Item and Distracter AnalysisItem and Distracter Analysis
Item and Distracter Analysis
 
CONSTRUCTING PAPER-AND-PENCIL TESTS
CONSTRUCTING PAPER-AND-PENCIL TESTSCONSTRUCTING PAPER-AND-PENCIL TESTS
CONSTRUCTING PAPER-AND-PENCIL TESTS
 
Constructing subjective test items
Constructing  subjective test itemsConstructing  subjective test items
Constructing subjective test items
 
Test Construction
Test ConstructionTest Construction
Test Construction
 
Test construction
Test constructionTest construction
Test construction
 
Type of Test
Type of TestType of Test
Type of Test
 
Completion type of test
Completion type of testCompletion type of test
Completion type of test
 

Similar to Developing Teacher-Made Tests

Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...
Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...
Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...Md Arman
 
Reading test specifications assignment-01-ppt
Reading test specifications assignment-01-pptReading test specifications assignment-01-ppt
Reading test specifications assignment-01-pptBilal Yaseen
 
QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...
QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...
QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...Shweta Sharma
 
Test Construction-Report.pdf
Test Construction-Report.pdfTest Construction-Report.pdf
Test Construction-Report.pdfRhagexRoble2
 
Types of test items and principles for constructing test items
Types of test  items and principles for constructing test items Types of test  items and principles for constructing test items
Types of test items and principles for constructing test items rkbioraj24
 
Be ch 7 assessing students progress
Be ch 7   assessing students  progressBe ch 7   assessing students  progress
Be ch 7 assessing students progressAbdelaziz Aittaleb
 
Testing a Test: Evaluating Our Assessment Tools
Testing a Test: Evaluating Our Assessment ToolsTesting a Test: Evaluating Our Assessment Tools
Testing a Test: Evaluating Our Assessment ToolsEddy White, Ph.D.
 
Assessment of learning
Assessment of learning Assessment of learning
Assessment of learning CherryDangoy
 
Preparation of Classroom Assessment (SLP-B @ BISCAST)
Preparation of  Classroom Assessment (SLP-B @ BISCAST)Preparation of  Classroom Assessment (SLP-B @ BISCAST)
Preparation of Classroom Assessment (SLP-B @ BISCAST)Ireno Alcala
 
stages of test construction
stages of test constructionstages of test construction
stages of test constructionirshad narejo
 
Assessment of learning
Assessment of learningAssessment of learning
Assessment of learningAhlamModiarat
 
Assessment oflearning ppt
Assessment oflearning pptAssessment oflearning ppt
Assessment oflearning pptJohnKinnoCasia
 

Similar to Developing Teacher-Made Tests (20)

Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...
Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...
Readingtestspecifications assignment-01-ppt-141130013903-conversion-gate01 - ...
 
Reading test specifications assignment-01-ppt
Reading test specifications assignment-01-pptReading test specifications assignment-01-ppt
Reading test specifications assignment-01-ppt
 
QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...
QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...
QUESTION BANK PREPARATION, VALIDATION, MODERATION BY PANEL, UTILIZATION AND D...
 
Assessment in Learning
Assessment in LearningAssessment in Learning
Assessment in Learning
 
Lesson-4-EDUC-5GROUP-3 (1).pptx
Lesson-4-EDUC-5GROUP-3 (1).pptxLesson-4-EDUC-5GROUP-3 (1).pptx
Lesson-4-EDUC-5GROUP-3 (1).pptx
 
La notes (5 10)
La notes (5 10)La notes (5 10)
La notes (5 10)
 
Test Construction-Report.pdf
Test Construction-Report.pdfTest Construction-Report.pdf
Test Construction-Report.pdf
 
Assessment of Learning
Assessment of LearningAssessment of Learning
Assessment of Learning
 
PPT ques bank.pptx
PPT ques bank.pptxPPT ques bank.pptx
PPT ques bank.pptx
 
Test construction
Test construction Test construction
Test construction
 
Types of test items and principles for constructing test items
Types of test  items and principles for constructing test items Types of test  items and principles for constructing test items
Types of test items and principles for constructing test items
 
Be ch 7 assessing students progress
Be ch 7   assessing students  progressBe ch 7   assessing students  progress
Be ch 7 assessing students progress
 
Testing a Test: Evaluating Our Assessment Tools
Testing a Test: Evaluating Our Assessment ToolsTesting a Test: Evaluating Our Assessment Tools
Testing a Test: Evaluating Our Assessment Tools
 
Assessment of learning
Assessment of learning Assessment of learning
Assessment of learning
 
Preparation of Classroom Assessment (SLP-B @ BISCAST)
Preparation of  Classroom Assessment (SLP-B @ BISCAST)Preparation of  Classroom Assessment (SLP-B @ BISCAST)
Preparation of Classroom Assessment (SLP-B @ BISCAST)
 
stages of test construction
stages of test constructionstages of test construction
stages of test construction
 
Assessment of learning
Assessment of learningAssessment of learning
Assessment of learning
 
Assessment oflearning ppt
Assessment oflearning pptAssessment oflearning ppt
Assessment oflearning ppt
 
Objective Tests
Objective TestsObjective Tests
Objective Tests
 
7 assessment and the cefr
7 assessment and the cefr 7 assessment and the cefr
7 assessment and the cefr
 

More from Carlo Magno

Assessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptxAssessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptxCarlo Magno
 
Social and Emotional Learning
Social and Emotional LearningSocial and Emotional Learning
Social and Emotional LearningCarlo Magno
 
Educational assessment in the 4 ir
Educational assessment in the 4 irEducational assessment in the 4 ir
Educational assessment in the 4 irCarlo Magno
 
The process of research mentoring
The process of research mentoringThe process of research mentoring
The process of research mentoringCarlo Magno
 
Quality management services sustainability training
Quality management services sustainability trainingQuality management services sustainability training
Quality management services sustainability trainingCarlo Magno
 
Managing technology integration in schools
Managing technology integration in schoolsManaging technology integration in schools
Managing technology integration in schoolsCarlo Magno
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teachingCarlo Magno
 
Empowering educators on technology integration
Empowering educators on technology integrationEmpowering educators on technology integration
Empowering educators on technology integrationCarlo Magno
 
Designing an online lesson
Designing an online lessonDesigning an online lesson
Designing an online lessonCarlo Magno
 
Curriculum integration
Curriculum integrationCurriculum integration
Curriculum integrationCarlo Magno
 
Accountability in Developing Student Learning
Accountability in Developing Student LearningAccountability in Developing Student Learning
Accountability in Developing Student LearningCarlo Magno
 
The Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School ImprovementThe Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School ImprovementCarlo Magno
 
Guiding your child on their career decision making
Guiding your child on their career decision makingGuiding your child on their career decision making
Guiding your child on their career decision makingCarlo Magno
 
Assessing Science Inquiry Skills
Assessing Science Inquiry SkillsAssessing Science Inquiry Skills
Assessing Science Inquiry SkillsCarlo Magno
 
Assessment in the Social Studies Curriculum
Assessment in the Social Studies CurriculumAssessment in the Social Studies Curriculum
Assessment in the Social Studies CurriculumCarlo Magno
 
Quantitative analysis in language research
Quantitative analysis in language researchQuantitative analysis in language research
Quantitative analysis in language researchCarlo Magno
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teachingCarlo Magno
 
Hallmarks of textbook
Hallmarks of textbookHallmarks of textbook
Hallmarks of textbookCarlo Magno
 
managing the learner centered-classroom
managing the learner centered-classroommanaging the learner centered-classroom
managing the learner centered-classroomCarlo Magno
 
Assessing learning objectives
Assessing learning objectivesAssessing learning objectives
Assessing learning objectivesCarlo Magno
 

More from Carlo Magno (20)

Assessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptxAssessment Using the SOLO Framework.pptx
Assessment Using the SOLO Framework.pptx
 
Social and Emotional Learning
Social and Emotional LearningSocial and Emotional Learning
Social and Emotional Learning
 
Educational assessment in the 4 ir
Educational assessment in the 4 irEducational assessment in the 4 ir
Educational assessment in the 4 ir
 
The process of research mentoring
The process of research mentoringThe process of research mentoring
The process of research mentoring
 
Quality management services sustainability training
Quality management services sustainability trainingQuality management services sustainability training
Quality management services sustainability training
 
Managing technology integration in schools
Managing technology integration in schoolsManaging technology integration in schools
Managing technology integration in schools
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teaching
 
Empowering educators on technology integration
Empowering educators on technology integrationEmpowering educators on technology integration
Empowering educators on technology integration
 
Designing an online lesson
Designing an online lessonDesigning an online lesson
Designing an online lesson
 
Curriculum integration
Curriculum integrationCurriculum integration
Curriculum integration
 
Accountability in Developing Student Learning
Accountability in Developing Student LearningAccountability in Developing Student Learning
Accountability in Developing Student Learning
 
The Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School ImprovementThe Instructional leader: TOwards School Improvement
The Instructional leader: TOwards School Improvement
 
Guiding your child on their career decision making
Guiding your child on their career decision makingGuiding your child on their career decision making
Guiding your child on their career decision making
 
Assessing Science Inquiry Skills
Assessing Science Inquiry SkillsAssessing Science Inquiry Skills
Assessing Science Inquiry Skills
 
Assessment in the Social Studies Curriculum
Assessment in the Social Studies CurriculumAssessment in the Social Studies Curriculum
Assessment in the Social Studies Curriculum
 
Quantitative analysis in language research
Quantitative analysis in language researchQuantitative analysis in language research
Quantitative analysis in language research
 
Integrating technology in teaching
Integrating technology in teachingIntegrating technology in teaching
Integrating technology in teaching
 
Hallmarks of textbook
Hallmarks of textbookHallmarks of textbook
Hallmarks of textbook
 
managing the learner centered-classroom
managing the learner centered-classroommanaging the learner centered-classroom
managing the learner centered-classroom
 
Assessing learning objectives
Assessing learning objectivesAssessing learning objectives
Assessing learning objectives
 

Developing Teacher-Made Tests

  • 1. Developing a Teacher Made Test Dr. Carlo Magno 1
  • 2. Advance Organizer 1 The Test Blueprint Outline of the Test Development Process Table of Specifications 2 Designing Selected-Response Items Binary-choice items Instructions in Writing Binary Type of Items Multiple-choice items Guidelines in Writing Multiple-choice Items Matching items Guidelines in Writing Multiple-choice Items 3 Designing Constructed-Response Items Short-answer items Guidelines in Writing Short Answer Items Essay items 4 Designing Interpretive Exercise Guidelines in Writing Intepretive Exercise 2 Examples of Interpretive Exercise
  • 3. Objectives • 1. Explain the theories and concepts that rationalize the practice of assessment. • 2. Make a table of specifications of the test items. • 3. Design pen-and-paper tests that are aligned to the learning intents. • 4. Justify the advantages and disadvantages of any pen-and-paper test. • 5. Evaluate the test items according to the guidelines presented. 3
  • 4. Outline of Test Development Process • 1. Specify the ultimate goals of the education process • 2. Derive from these the goals of the portion of the system under study • 3. Specify these goals in terms of expected student behavior. If relevant, specify the acceptance level of successful learning. • 4. Determine the relative emphasis or importance of various objectives, their content, and their behaviors. • 5. Select or develop situations that will elicit the desired behavior in the appropriate context or environment, assuming the student has learned it. • 6. Assemble a sample of such situations that together represent accurately the emphasis on content and behavior previously determined. 4
  • 5. Outline of Test Development Process • 7. Provide for the recording of responses in a form that will facilitate scoring but will not so distort the nature of the behavior elicited that it is no longer a true sample or index of the behavior desired. • 8. Establish scoring criteria and guides to provide objective and unbiased judgment. • 9. Try out the instrument in preliminary form. • 10. Revise the sample of situations on the basis of tryout information. • 11. Analyze reliability, validity, and score distribution in accordance with the projected use of scores. • 12. Develop test norms and a manual, and reproduce and distribute the test. 5
  • 6. Test Length • The test must be of sufficient length to yield reliable scores • The longer the test, the more the reliable the results • The test should be valid if it is reliable • For the grade school, one must consider the stamina and attention span of the pupils • The test should be long enough to be adequately reliable and short enough to be administered 6
  • 7. Test Instruction • It is the function of the test instructions to furnish the learning experiences needed in order to enable each examinee to understand clearly what he is being asked to do? • Instructions may be oral, a combination of written and oral instruction is probably desirable, except with very young children. • Clear concise and specific. 7
  • 8. Test layout • The arrangement of the test items influences the speed and accuracy of the examinee • Utilize the space available while retaining readability. • Items of the same type should be grouped together • Arrange test items from easiest to most difficult as a means of reducing test anxiety. • The test should be ordered first by type then by content • Each item should be completed in the column and page in which it is started. • If the reference material is needed, it should occur on the same page as the item • If you are using numbers to identify items it is better to use letters for the options 8
  • 9. Scoring the test • Use separate answer sheets • Punched key • Overlay key • Strip key Plight of the student • The teacher should discuss with the class the content areas and levels of the cognitive domain to be examined • The discussion should utilize a vocabulary and a level of complexity appropriate to the development level of the student • Types of test • Examples of test type 9
  • 10. Table of Specifications One Grid TOS Content Outline No. of items 1. Table of specifications 10 2. Test and Item characteristics 20 3. Test layout 5 4. Test instructions 5 5. Reproducing the test 5 6. Test length 5 7. Scoring the test 5 TOTAL 55 10
  • 11. Table of Specifications Two Grid TOS Weight Content Knowledge Comprehension Application No. of items (Time Outline 30% 40% 30% by content Frame) area 35% 1. Table of specifications 1 4 4 9 30% 2. Test and Item characteristics 2 3 3 8 10% 3. Test layout 1 1 0 2 5% 4. Test instructions 0 1 0 1 5% 5. Reproducing the test 1 0 0 1 5% 6. Test length 1 0 1 2 10% 7. Scoring the test 2 1 0 3 8 10 8 26 The number of items in a cell is computed using the formula: Given time items X percentageof cognitive skill X total number of items 11 Total time
  • 12. Classification of test Items • Selected Response – Binary Choices – Multiple Choice – Matching Type • Constructed Response “Supply Test” – Short Form answers - identification – Completion – fill in the blanks, cloz test – Essay • Performance Type – Paper and pencil type – Identification type – Simulation 12
  • 13. Item Writing Commandments • Thou shall not produce opaque directions to students regarding how to respond to your instructions (opaque directions) • Thou shall not employ ambiguous statements in your assessment item (ambiguous statements) • Thou shall not unintentionally provide students with clues regarding appropriate response (unintended clues) • Thou shall not employ complex syntax in your assessment item (complex syntax) • Thou shall not use vocabulary that is more advanced than required (Difficult vocabulary) 13
  • 14. SHORT ANSWER ITEMS • 1. Word the item so that the answer is both brief and definite. • 2. Do not take statements directly from books to use as a basis for short answer items. • 3. A direct question is generally more acceptable than an incomplete statement. • 4. The answer to be expressed in numerical units indicate the type of answer wanted. • 5. Blanks for answers should be equal in length. • 6. Do not use to many blanks. 14
  • 15. Writing supply items 1. Require short, definite, clear-cut, and explicit answers FAULTY: Earnest Hemingway wrote______ IMPROVED: The Old Man and the Sea was written by _______. Who wrote The Old man and the Sea? 2. Avoid multimutilated statements FAULTY: _____ pointed out in ____ the freedom of thought in America was seriously hampered by ___, ____, & __. IMPROVED: That freedom of thought in America was seriously hampered by social pressures toward conformity was pointed out in 1830 by ______. 15
  • 16. Writing supply items 3. If several equal answers equal credit should be given to each one. 4. Specify and announce in advance whether scoring will take spelling into account. 5. In testing for comprehension of terms and knowledge of definition, it is often better to supply the term and require a definition than to provide a definition and require the term. FAULTY: What is the general measurement term describing the consistency with which items in a test measure the same thing? IMPROVED: Define “internal consistency reliability.” 16
  • 17. Writing supply items 6. It is generally recommended that in completion items the blanks come at the end of the statement. FAULTY: A (an) ________ is the index obtained by dividing a mental age score by chronological age and multiplying by 100. IMPROVED: The index obtained by dividing a mental age score by chronological age and multiplying by 100 is called a (an) ________ 7. Minimize the use of textbook expressions and stereotyped language. FAULTY: The power to declare war is vested in ______ IMPROVED: Which national legislative body has the authority to declare war? 17
  • 18. Writing supply items 8. Specify the terms in which the response is to be given. FAULTY: Where does the Security Council of the United Nations hold its meeting? IMPROVED: In what city of the United States does the Security Council of the United Nations hold its meeting? FAULTY: If a circle has 4-inch diameter, its area is_____ IMPROVED: A circle has 4-inch diameter. Its area in square inches correct to two decimal places, is _____ 9. In general, direct questions are preferable to incomplete declarative sentences. FAULTY: Gold was discovered in California in the year ___ IMPROVED: In what year was gold discovered in California? 18
  • 19. Writing supply items 10. Avoid extraneous clues to the correct answer FAULTY: A fraction whose denominator is greater than its numerator is a _____ IMPROVED: Fractions whose denominator are greater than their numerators are called _____ 19
  • 20. ALTERNATIVE RESPONSE ITEM • 1. Avoid broad general statements if they are to be judged true or false. • 2. Avoid trivial statements. • 3. Avoid the use of negative statements. • 4. Avoid long complex sentences. • 5. Avoid including two ideas in one statement unless cause and effect relationship are being measured. • 6. If the opinion is being used, attribute it to some source unless the ability to identify opinion is being specifically measured. • 7. True statements and false statements should be equal in length. • 8. The number of true and false statements should be approximately equal. 20
  • 21. Writing TRUE-FALSE Items 1. Avoid the use of “specific determiners” FAULTY: No picture-no sound in a television set may indicate a bad 5U4G. IMPROVED: A bad 5U4G tube in a television set will result in no picture sound. 2. Base true-false items upon statements that are absolutely true or false, without qualifications or exceptions. FAULTY: World War II was fought in Europe and the Far East. IMPROVED: The primary combat locations in terms of military personnel during World War II were Europe and the Far East. 21
  • 22. Writing TRUE-FALSE Items 3. Avoid negative stated items when possible and eliminate all double negatives. FAULTY: It is not frequently observed that copper turns green as a result of oxidation. IMPROVED: Copper will turn green upon oxidizing. 4. Use quantitative and precise rather than qualitative language where possible. FAULTY: Many people voted for Gloria Arroyo in the 2003 Presidential election. IMPROVED: Gloria Arroyo received more than 60 percent of the popular votes cast in the Presidential election of 2003. 22
  • 23. Writing TRUE-FALSE Items 5. Avoid stereotypic and textbook statements. FAULTY: From time to time efforts have been made to explode the notion that there may be a cause-and-effect relationship between arboreal life and primate anatomy. IMPROVED: There is a known relationship between primate anatomy and arboreal life. 6. Avoid making the true items consistently longer than the false items. 7. Avoid the use of unfamiliar or esoteric language. FAULTY: According to some peripatetic politicos, the raison d‟etre for capital punishment is retribution. IMPROVED: According to some politicians, justification for the existence of capital punishment can be traced to the Biblical statement, “An eye for an eye.” 23
  • 24. Writing TRUE-FALSE Items 8. Avoid complex sentences with many dependent clauses. FAULTY: Jane Austen, an American novelist born in 1790, was a prolific writer and is best known for her novel Pride and Prejudice, which was published in 1820. IMPROVED: Jane Austen is best known for her novel Pride and prejudice. 9. It is suggested that the crucial elements of an item be placed at the end of the statement. FAULTY: Oxygen reduction occurs more readily because carbon monoxide combines with hemoglobin faster than oxygen does. IMPROVED: Carbon monoxide poisoning occurs because carbon monoxide dissolves delicate lung tissue. 24
  • 25. Writing Matching Type Test 1. Matching Exercises should be complete on a single page. 2. Use response categories that are related but mutually exclusive. 3. Keep the number of stimuli relatively small (10-15), and let the number of possible responses exceed the number of stimuli by two or three. 4. The direction should clearly specify the basis for matching stimuli and responses. 5. Keep the statements in the response column short and list them in some logical order 25
  • 26. FAULTY: Match List A with List B. You will be given one point for each correct match. List A List B a. cotton gin a. Eli Whitney b. reaper b. Alexander Graham Bell c. wheel c. David Brinkley d. TU54G tube d. Louisa May Alcott e. steamboat e. None of these • Directions failed to specify the basis for matching • List are enumerated identically • Responses not listed logically • Lacks homogeneity • Equal number of elements • Use of “None of the above” 26
  • 27. IMPROVED: Famous inventions are listed in the left-hand column and inventors in the right-hand column below. Place the letter corresponding to the inventor in the space next to the invention for which he is famous. Each match is worth 1 point, and “None of these” may be the correct answer. Inventors may be used more than once. Inventions Inventors __ 1. steamboat a. Alexander Graham-Bell __ 2. cotton skin b. Robert Fulton __ 3. sewing machine c. Elias Howe __ 4. reaper d. Cyrus McCormick e. Eli Whitney f. None of these 27
  • 28. Writing Multiple Choice 1. It is recommended that the stem be a direct question. 2. The stem should pose a clear, define, explicit, and singular problem. FAULTY: Salvador Dali is a. a famous Indian. b. important in international law. c. known for his surrealistic art. d. the author of many avant-garde plays. IMPROVED: With which one of the fine arts is Salvador Dali associated? a. surrealistic painting b. avant-garde theatre c. polytonal symphonic music d. impressionistic poetry 28
  • 29. Writing Multiple Choice 3. Include in the stem any words that might otherwise be repeated in each response. FAULTY: Milk can be pasteurized at home by a. heating it to a temperature of 130o b. Heating it to a temperature of 145o c. Heating it to a temperature of 160o d. Heating it to a temperature of 175o IMPROVED: The minimum temperature that can be used to pasteurize milk at home is: a. 130o b. 145o c. 160o d. 175o 29
  • 30. Writing Multiple Choice 4. Items should be stated simply and understandably, excluding all nonfunctional words from stem and alternatives. FAULTY: Although the experimental research, particularly that by Hansmocker must be considered equivocal and assumptions viewed as too restrictive, most testing experts would recommend as the easiest method of significantly improving paper-and-pencil achievement test reliability to a. increase the size of the group being tested. b. increase the differential weighting of items. c. increase the objective of scoring. d. increase the number of items. e. increase the amount of testing time. IMPROVED: Assume a 10-item, 10-minute paper-and-pencil multiple choice achievement test has a reliability of .40. The easiest way of increasing the reliability to .80 would be to increased a. group size b. scoring objectivity c. differential item scoring weights d. the number of items e. testing time 30
  • 31. Writing Multiple Choice 5. Avoid interrelated items 6. Avoid negatively stated items FAULTY: None of the following cities is a state capital except a. Bangor b. Los Angeles c. Denver d. New Haven IMPROVED: Which of the following cities is a state capital? a. Bangor b. Los Angeles c. Denver d. New Haven 31
  • 32. Writing Multiple Choice 7. Avoid making the correct alternative systematically different from other options 8. If possible the alternatives should be presented in some logical, numerical, or systematic order. 9. Response alternatives should be mutually exclusive. FAULTY: Who wrote Harry Potter and the Goblet of Fire? a. J. K. Rowling b. Manny Paquiao c. Lea Salonga d. Mark Twain IMPROVED: Who wrote Penrod? a. J. K. Rowling b. J. R. R. Tolkien c. V. Hugo d. L. Carrol 32
  • 33. Writing Multiple Choice 10. Make all responses plausible and attractive to the less knowledgeable and skillful student. FAULTY: Which of the following statements makes clear the meaning of the word “electron”? a. An electronic tool b. Neutral particles c. Negative particles d. A voting machine e. The nuclei of atoms IMPROVED: Which of the following phrases is a description of an “electron”? a. Neutral particle b. Negative particle c. Neutralized proton d. Radiated particle e. Atom nucleus 33
  • 34. Writing Multiple Choice 11. The response alternative “None of the above” should be used with caution, if at all. FAULTY: What is the area of a right triangle whose sides adjacent to the right angle are 4 inches long respectively? a. 7 b. 12 c. 25 d. None of the above IMPROVED: What is the area of a right triangle whose sides adjacent to the right angle are 4 inches and 3 inches respectively? a. 6 sq. inches b. 7 sq. inches c. 12 sq. inches d. 25 sq. inches e. None of the above 34
  • 35. Writing Multiple Choice 12. Make options grammatically parallel to each other and consistent with the stem. FAULTY: As compared with the American factory worker in the early part of the 19th century, the American factory worker at the close of the century a. was working long hours b. received greater social security benefits c. was to receive lower money wages d. was less likely to belong to a labor union. e. became less likely to have personal contact with employers IMPROVED: As compared with the American factory worker in the early part of the century, the American factory worker at the close of the century a. worked longer hours. b. had more social security. c. received lower money wages. d. was less likely to belong to a labor union e. had less personal contact with his employer 35
  • 36. Writing Multiple Choice 13. Avoid such irrelevant cues as “common elements” and “pat verbal associations.” FAULTY: The “standard error of estimate‟ refer to a. the objectivity of scoring. b. the percentage of reduced error variance. c. an absolute amount of possible error. d. the amount of error in estimating criterion scores. IMPROVED: The “standard error of estimate” is most directly related to which of the following test characteristic? a. Objectivity b. Reliability c. Validity d. Usability e. Specificity 36
  • 37. Writing Multiple Choice 14. In testing for understanding of a term or concept, it is generally preferable to present the term in the stem and alternative definitions in the options. FAULTY: What name is given to the group of complex organic compounds that occur in small quantities in natural foods that are essential to normal nutrition? a. Calorie b. Minerals c. Nutrients d. Vitamins IMPROVED: Which of the following statements is the best description of a vitamin? 15. Use objective items – items‟ whose correct answers are agreed by experts 37
  • 38. Factual Knowledge • The Monroe Doctrine was announced about 10 years after the a. Revolutionary War b. War of 1812 c. Civil War d. Spanish-American War Conceptual Knowledge 2. Which of the following statements of the relationship between market price and normal price is true? a. Over a short period of time, market price varies directly with changes in normal price. b. Over a long period of time, market price tends to equal normal price. c. Market price is usually lower than normal price. d. Over a long period of time, market price determines normal price. 38
  • 39. Translation from symbolic form to another form, or vice versa 3. Which of the graphs below best represent the supply situation where a monopolist maintains a uniform price regardless of the amounts which people buy? A B C D S S S S S Price Price Price Price S S Quantity Quantity Quantity Quantity 39
  • 40. Application In the following items (4-8) you are to judge the effects of a particular policy on the distribution of income. In each case assume that there are no other changes in policy that would counteract the effect of the policy described in the item. Mark the item: A. If the policy described would tend to reduce the existing degree of inequality in the distribution of income, B. If the policy described would tend to increase the existing degree of inequality in the distribution of income, or C. If the policy described would have no effect, or an indeterminate effect, on the distribution of income. __ 4. Increasingly progressive income taxes. __ 5. Confiscation of rent on unimproved __ 6. Introduction of a national sales tax __ 7. Increasing the personal exemptions from income taxes __ 8. Distributing a subsidy to sharecroppers on southern farms 40
  • 41. Analysis 9. An assumption basic to Lindsay‟s preference for voluntary associations rather than government order… is a belief a. that government is not organized to make the best use of experts b. that freedom of speech, freedom of meeting, freedom of association, and possible only under a system of voluntary associations. c. in the value of experiment and initiative as a means of attaining an ever improving society d. in the benefits of competition 41
  • 42. Judgments in terms of external criteria For items 14-16, assume that in doing research for a paper about the English language you find a statement by Otto Jespersen that contradicts one point of view in a language you have always accepted. Indicate which of the statements would be significant in determining the value of Jespersen‟s statement. For the purpose of these items, you may assume that these statements are accurate. Mark each item using the following key. A. Significant positively – that is, might lead you to trust his statement and to revise your own opinion. B. Significant negatively – that is, night lead you to distrust his statement C. Has no significance __ 14. Mr. Jesperson was professor of English at Copenhagen University __ 15. The statement in question was taken from the very first article that Jespersen published __ 16. Mr. Jespersen‟s books are frequently referred to in other works that you consult. 42
  • 43. Essay Questions • 1. Ask questions or set tasks that will require the examinee to demonstrate a command of essential knowledge. • 2. Ask questions that are determinate, in the sense that experts could agree that one answer is better than another. • 3. Define the examinee‟s task as completely and specifically as possible without interfering with measurements of the achievement intended. • 4. In general, give preference to more specific questions that can be more answered briefly. • 5. Avoid giving the examinee a choice among optional questions unless special circumstances make such option necessary. • 6. Test the questions by writing an ideal answer 43
  • 44. Types of Essays: • General – extensiveness of responses • Restrictive Response – reliable scoring Learning outcomes measured by Essay: • Explain cause-effect relationship • Describe applications of principles • Present relevant arguments • Formulate tangible hypothesis • Formulate valid conclusions • State necessary assumptions • Describe the limitations of data • Explain methods and procedures • Produce, organize, and express ideas • Integrate learnings in different areas • Create original forms • Evaluate the worth of ideas 44
  • 45. Understanding: A. Comparison of two phenomena on a single designated basis: Compare the writers of the English Renaissance to those of the nineteenth century with respect to their ability tot describe nature B. Comparison of two phenomena in general Compare the French and Russian Revolutions C. Explanation of the use or exact meaning of a phrase or statement The book of John begins “In the beginning was the word…” From what philosophical system does this statement derive? D. Summary of a text or some portion of it State the central theme of the Communist Manifesto E. Statement of an artist’s purpose in the selection or organization of material Why did Hemingway describe in detail the episode in which Gordon, lying wounded, engage the oncoming enemy? What was Beethoven‟s purpose in deviating from the orthodox form of a symphony in Symphony No. 6? 45
  • 46. Application: A. Causes or effects Why may too frequent reliance on penicillin for the treatment of minor ailments eventually result in its diminished effectiveness against major invasion of body tissues by infectious bacteria? B. Analysis Why was Hamlet torn by conflicting desires? C. Statement of relationship It is said that intelligence correlates with school achievement at about .65. Explain this relationship D. Illustrations or examples of principles Name three examples of uses of the lever in typical American homes E. Application of rules or principles Would you weigh more or less on the moon? On the sun? Explain. F. Reorganization of facts Some writers have said that the American Revolution was not merely a political revolution against England but also a social revolution, within the colonies, of the poor against the wealthy. Using the same evidence what other conclusion is possible? 46
  • 47. Judgment: A. Decision for or against Should members of the Communist Party be allowed to teach in American colleges? Why or why not? B. Discussion Discuss the likelihood that four-year private liberal arts colleges will gradually be replaced by junior colleges and state universities. C. Criticism of the adequacy, correctness, or relevance of a statement The discovery of penicillin has often been called an accident. Comment on the adequacy of this explanation. D. Formulation of new questions What should one find out in order to explain why some students of high intelligence fail in school? 47
  • 48. Designing Interpretive Exercise • Guidelines in Writing Interpretive Exercise • 1. Select an introductory that is in harmony with the objectives of the course. – Amount of emphasis of various interpretive skills is a factor. – Do not overload test takers with interpretive items in a particular area. – Selection of introductory should be guided by general emphasis to be given to the measurement of complex achievement. • 2. Select introductory material that is appropriate to the curricular experience and reading ability of the examinees. 48
  • 49. Guidelines in Writing Interpretive Exercise • 3. Select introductory material that is new to pupils. • 4. Select introductory material that is brief but meaningful. • 5. Revise introductory material for clarity, conciseness, and greater interpretive value. • 6. Construct test items that require analysis and interpretation of introductory material. • 7. Make the number of items roughly proportional to the length of the introductory material. • 8. Observe all suggestions for constructing objective test items. 49
  • 50. • Ability to Recognize the Relevance of Information 50
  • 51. • Ability to Recognize Warranted and Unwarranted Generalizations 51
  • 52. • Ability to Recognize Inferences 52
  • 53. • Ability to Interpret Experimental Findings 53
  • 54. • Ability to Apply Principles 54
  • 55. • Ability to Recognize Assumptions 55
  • 56. Reading comprehension • Bem (1975) has argued that androgynous people are “better off” than their sex-typed counterparts 35. What is the independent variable in because they are not constrained by rigid sex- the study? role concepts and are freer to respond to a wider variety of situations. Seeking to test this hypothesis, Bem exposed masculine, feminine, a. Situations calling for independence and and androgynous men and women to situations that called for independence (a masculine nurturance attribute) or nurturance (a feminine attribute). The b. Situation to make the sex type react test for masculine independence assessed the subject‟s willingness to resist social pressure by c. Situations to make the androgynous be refusing to agree with peers who gave bogus flexible judgments when rating cartoons for funniness (for example, several peers might say that a very d. Situations like sex type, androgynous funny cartoon was hilarious). Nurturance or and sex role concepts feminine expressiveness, was measured by observing the behavior of the subject when left alone for ten minutes with a 5-month old baby. 36. What are the levels of the IV? The result confirmed Bem‟s hypothesis. Both the masculine sex-typed and the androgynous subjects were more independent (less a. masculine attribute and feminine conforming) on the „independence” test than feminine sex-typed individuals. Furthermore, both attribute the feminine and the androgynous subjects were b. rating cartoons and taking care of a more “nurturant” than the masculine sex-typed individuals when interacting with the baby. Thus, baby the androgynous subjects were quite flexible, c. independence and nurturance they performed as masculine subjects did on the “feminine” task. d. flexibility and rigidity 56
  • 57. Interpreting Diagrams Instruction. Study the following illustrations and answer the following questions. 101. Which group received the treatment? Figure 1 a. group A b. group B b. c. none of the above 102. Why did group B remain stable across the experiment? a. there is an EV b. there is no treatment c. there is the occurence of ceiling effect 103. What is the problem at the start of the experiment? Pretest Posttest a. the groups are nonequivalent b. the groups are competing with each other c. the treatment took place immediately 57