How should we test reading skills? (Report from Hughes, A. (2003). Testing for language teachers. Testing Reading, 136-160)

1. L U C Í A R U B I O R U B I O
How should we test reading
skills?

2. 1. Is it easy to test reading skills? - Problems
2. Which skills the reader perform? - Operations
3. Which texts candidates are expected to be able to deal with? –
Parameters
4. Criterial level of performance
5. Setting tasks:
a. Selecting texts
b. Writing items
c. Possible techniques
d. Which language for items and responses?
e. Procedures for writing items – Practical advice
6. What should we test on a reading test?

3. 1. Is it easy to test reading skills?
• A good one?
• What do you want to measure?
Quickly
construction of a
reading test
• Which skills are involved?
• Measuring a particular skill – Successful item?
Manifest of
receptive skills
• Reading purpose
• The kind of text
Different ways of
reading depending
on…
• Diagnostic test
• Achievement test
• Placement test
• Profiency test
The purpose of the
test

4. 2. Which skills the reader perform? - Operations
Operations based on different
purposes
Expeditious
reading
Skimming
Search
reading
Scanning
Careful
reading
Identify
Interpret
Distinguish
Recognise
Outline
Make
inferences

5. 3. Which texts candidates are expected to be able
to deal with?
• Text types
• Text forms
• Graphic features
• Topics
• Style
• Intended readership
• Lenght
• Readability
• Range of vocabulary
• Range of grammar
• Authenticity
Parameters
We want the texts included in a
test to be representative of the
texts candidates should be able
to read successfully. Content
validity and backwash.

6. 4. Criterial level of performance
Traditional passmarks expressed in percentages: no direct
interpretation
To use the test tasks themselves to define
the level
When can we compare performance on the test using
scales: ACTFL or ILR?

7. 5. Setting tasks
• Advices:
• Representative sample: many texts and operations
• Appropriate length
• Content validity & reliability: Many passages (Fresh starts) VS.
Practicality
• Search reading: discrete information
• Scanning: Texts with the specified elements
• Structure ability: Clear structure
• Interest but not overexcite or disturb
• Avoid texts about general knowledge
• Avoid too culturally laden texts
• New texts for students
a. Selecting texts: Experience, judgement and common sense

8. • Measure the ability in which we are interested
• Elicit reliable behaviour from candidates
• Permit highly reliable scoring
b. Writing items that…
• Multiple choices
• Short answer
• Gap filling
• Information transfer
c. Possible techniques

9. Multiple choice
The man with a
dog was attacked
in the street by a
woman
True/False: by
chance?
-> Add another
distractor

10. Short answer
It is better with an unique correct response
 For testing the ability to identify referents
 For testing the ability to predict the meaning of a word in
context
 For testing the ability to make various distinctions
 For testing scanning
 For testing the ability to structure a text (Problems on
scoring of sequencing items)

15. Gap filling
When the required response is so complex and it may cause
writing and scoring problems
• To test if the candidate had grasped the main idea
• To test the ability to recognise detail presented support a
main idea
• Summary cloze

19. Information transfer
 Minimising demands on writing
ability
 Example: a table, following a
route, labelling a picture etc.

20. • Items: Less demanding than the text itself
• Responses: Minimal demands on writing ability
• Target language or native language?
d. Which language for items and responses?
• 1. Careful reading: operations in mind
• 2. What a competent reader should derive from the
text?
• 3. To take notes
• 4. To decide the tasks
• 5. Paragraph and line numbers (to reference)
• 6. Moderation by colleagues and modification
e. Procedures for writing items – Practical advice

21. Moderation
checklist

22. TIPS
Items in
order
Answers
found
without
reading
No
general
knowledg
e answers
Independient
items
Minor
changes –
Ask a
native

23. 6. What should we test on a reading test?
Grammar,
vocabulary,
spelling or
punctuation
To test receptive and productive skills at the same time
makes the measurement less valid

24. 7. References
 Hughes, A. (2003). Testing for language teachers. Testing
Reading, 136-160.