14. “Language use, embracing language learning, comprises the actions performed by persons who as individuals and as social agents develop a range of competences, both general and in particular communicative language competences. They draw on the competences at their disposal in various contexts under various conditions and under various constraints to engage in language activities involving language processes to produce and/or receive texts in relation to themes in specific domains, activating those strategies which seem most appropriate for carrying out the tasks to be accomplished.” (CEFR: 9)
15. “Language use, embracing language learning, comprises the actions performed by persons who as individuals and as social agents develop a range of competences, both general and in particular communicative language competences. They draw on the competences at their disposal in various contexts under various conditions and under various constraints to engage in language activities involving language processes to produce and/or receive texts in relation to themes in specific domains, activating those strategies which seem most appropriate for carrying out the tasks to be accomplished.” (CEFR: 9)
16. “Language use, embracing language learning, comprises the actions performed by persons who as individuals and as social agents develop a range of competences, both general and in particular communicative language competences. They draw on the competences at their disposal in various contexts under various conditions and under various constraints to engage in language activities involving language processes to produce and/or receive texts in relation to themes in specific domains, activating those strategies which seem most appropriate for carrying out the tasks to be accomplished.”
CEFR Chapter 4 Language use and the language user/learner
17. “Language use, embracing language learning, comprises the actions performed by persons who as individuals and as social agents develop a range of competences, both general and in particular communicative language competences. They draw on the competences at their disposal in various contexts under various conditions and under various constraints to engage in language activities involving language processes to produce and/or receive texts in relation to themes in specific domains, activating those strategies which seem most appropriate for carrying out the tasks to be accomplished.” (CEFR: 9)
CEFR Chapter 5
The competences of the user/learner
19. Ch4: Descriptors: Activities
Communicative
Strategies
Communicative
Language Competencies
Reception Production
Understanding
a native speaker
Conversation
Informal
Discussion
Formal
Discussion
Obtaining Goods
and Services
Interviewing &
being interviewed
Spoken Written
Interaction Mediation
Overall Language Proficiency
Communicative
Activities
20. Informal Discussion B2
Can take an active part in informal discussion in familiar contexts, commenting, putting point of view clearly, evaluating alternative proposals and making and responding to hypotheses.
Can with some effort catch much of what is said around him/her in discussion, but may find it difficult to participate effectively in discussion with several native speakers who do not modify their language in any way.
Ch4: Descriptors
22. Communicative
Strategies
General
Linguistic
Vocabulary
Range
Range
Grammatical
Accuracy
Phonological
Control
Vocabulary
Control
Orthographic
Control
Control
Linguistic Sociolinguistic Pragmatic
Communicative
Language Competencies
Communicative
Activities
Overall language Proficiency
Ch4: Descriptors: Quality
23. Vocabulary control B2
Lexical accuracy is generally high, though some confusion and incorrect word choice does occur without hindering communication.
Ch4: Descriptors
52. Core Inventory: Aim
Document the core of English/French taught at CEFR levels A1 to C1
•Functions and notions
•Discourse markers
•Grammar
•Lexis
•Socio-cultural (French only)
53. Core Inventory: Outcome
•Map of language problems across levels
•Map of text types across levels
•Core language points
•Exponents
•Prototype Scenarios
54.
55.
56.
57.
58.
59. Scenarios
•Starting from real world needs
•Fitting everything together, putting everything in context
•Going from theoretical to practical and vice-versa
•Summarising what language, strategies and enabling skills are needed for a task
61. LISTENING ASSESSMENT: SITE TOUR (HAMPTON COURT) OVERVIEW B1 DOMAIN CONTEXT TASKS ACTIVITIES TEXTS Personal / Educational Home Following a simple, factual TV/web documentary Following the commentary, accompanying the visuals Listening as a member of an audience Watching TV/web video clip A guided commentary on a place (historic building, resort, town etc.) LEVEL B1 CAN-DOS* I can catch the main points in TV programmes on familiar topics when the delivery is relatively slow and clear. I can follow a lecture or talk within my own field, provided the subject matter is familiar and the presentation straightforward and clearly structured. MICROACTIVITIES* RECOGNISE useful information DISTINGUISH main points from specific details aspects reported as facts from those reported as opinion UNDERSTAND an explicitly signalled line of argument main conclusions specific details TEXT FEATURES* Long but straightforward text Clear, standard, straightforward, relatively slow Clearly signposted/signalled with explicit markers Familiar topics regularly encountered in a school, work or leisure context * Content From CEFR scales and/or Swiss EAQUALS-ALTE Portfolio in normal print Elaborated content (e.g. Micro-activities chart) in italics COMPETENCES STRATEGIC Recognise the beginning of a significantly new and different part of the text Recognise where difficulty lies (subject/assumed knowledge, linguistic) Use context to deduce probable meaning of unknown words (repetition, visuals, gesture, what comes next) Use the beginning of a significantly new and different part of the text to intensify effort PRAGMATIC Functional Discourse Describing places Describing events Describing feelings, emotions, attitude Linkers: sequential – past time Connecting words expressing cause and effect, contrast etc. (e.g. on the other hand; however; despite) Summarising LINGUISTIC Grammatical Lexical Phonological Past time: Simple past, past continuous, used to, past perfect Passive (past) Reporting structures 3rd conditional / mixed conditionals Must/can’t/might have Intensifiers Comparatives / Superlatives Adjectives for places and people Time phrases (e.g. In the last century ; 50 years ago) Verbs describing construction, development (e.g. plan, construct, rebuild, renovate, demolish) Emphasis in sentence stress
62. LISTENING ASSESSMENT: SITE TOUR (HAMPTON COURT) IMPLEMENTATION B1 TASK 1 GENERAL DESCRIPTION Follow short introduction to a TV guided commentary on Hampton Court, answering 5 True / False / Not Stated questions while listening SOURCES “The Tudors: Behind Hampton Court”: http://www.youtube.com/watch?v=mX7ABmAlcAE Natalie Dormer of The Tudors celebrates the 500th anniversary of Henry 8's coronation by touring the Hampton Court palace Other possibilities might be: Simon Schama going round Versailles; extract from travel programme, tourist promotion) AUTHENTICITY Authentic LENGTH Ca. 7-10 minutes TEXT FEATURES VISUAL SUPPORT Yes – commentary should match unfolding film ITEM TYPE / NUMBER Introduction: True / False / Not stated: (5 questions) TASK RUBRIC Follow this introduction to a TV guided commentary on a historic place of interest “Hampton Court Palace,” near London and answer the 5 questions. Mark “T” if the statement is True, “F” if it is false and “NS” if the information is Not Stated in the commentary. TIME While playing MARK SCHEME 5 x 1 mark = 5 marks TASK 2 GENERAL DESCRIPTION Match 10 things that are mentioned (avoiding 3 distractors) to the parts of the palace they relate to. ITEM TYPE / NUMBER Matching: Match things mentioned in the commentary to the part of Hampton Court concerned (3 extra distractors). TASK RUBRIC Follow main body of the guided commentary. While you are listening match the 10 things in the list to the parts of the palace they relate to. Put a cross in the correct box for each point on the list. Note: 3 of the points on the list are not mentioned. TIME While playing MARK SCHEME 10 x 0.5 marks = 5 marks TASK 3 GENERAL DESCRIPTION Answer 5 open questions which are read before listening to the recording a second time. ITEM TYPE / NUMBER Open questions: 5 TASK RUBRIC After listening, answer the following 5 questions with information from the commentary. TIME 10 minutes MARK SCHEME 5 x 2 marks = 10 marks. 2: complete answer; 1: partial answer. Grammar and spelling not penalised.
63. LISTENING ASSESSMENT: SITE TOUR (HAMPTON COURT) TASK 1 GENERAL DESCRIPTION Follow short introduction to a TV guided commentary on Hampton Court, answering 5 True / False / Not Stated questions while listening SOURCES “The Tudors: Behind Hampton Court”: http://www.youtube.com/watch?v=mX7ABmAlcAE Natalie Dormer of The Tudors celebrates the 500th anniversary of Henry 8's coronation by touring the Hampton Court palace Other possibilities might be: Simon Schama going round Versailles; extract from travel programme, tourist promotion) Authentic Ca. 7-10 minutes TEXT FEATURES Yes – commentary should match unfolding film ITEM TYPE / NUMBER Introduction: True / False / Not stated: (5 questions) TASK RUBRIC Follow this introduction to a TV guided commentary on a historic place of interest “Hampton Court Palace,” near London and answer the 5 questions. Mark “T” if the statement is True, “F” if it is false and “NS” if the information is Not Stated in the commentary. TIME While playing MARK SCHEME 5 x 1 mark = 5 marks
64. LISTENING ASSESSMENT: SITE TOUR (HAMPTON COURT) TASK 2 GENERAL DESCRIPTION Match 10 things that are mentioned (avoiding 3 distractors) to the parts of the palace they relate to. ITEM TYPE / NUMBER Matching: Match things mentioned in the commentary to the part of Hampton Court concerned (3 extra distractors). TASK RUBRIC Follow main body of the guided commentary. While you are listening match the 10 things in the list to the parts of the palace they relate to. Put a cross in the correct box for each point on the list. Note: 3 of the points on the list are not mentioned. TIME While playing MARK SCHEME 10 x 0.5 marks = 5 marks
65. Questions Section A: 1. Henry VIII, King of England, died 500 years ago. F 2. Henry VIII was one of the richest kings in Europe. NS 3. Henry bought Hampton Court from Cardinal Wolsey. F 4. Hampton Court was one of the most luxurious and modern palaces of its time. T 5. Henry had other palaces too, all connected by the river. T Section B: Matching The Great Kitchen The Great Hall The Council Chamber The Haunted Gallery The Gardens The most magnificent room in the palace A Collection of gold Animals representing the different queens’ personal symbols A picture of the Pope Very big fireplaces Theatre and drama Discussion of state business Tapestries with gold thread A ghost (of Catherine Howard) Windows that make coloured light Weapons Entertainment Court ceremonies
66. Assessment ideas– B1
CEFR Descriptor Micro-activities Text features Task features Item types Example
B1 I can generally follow the main
points of extended discussion
around me, provided speech is
clearly articulated in standard
dialect.
Understand main/most
important information
Understand main
conclusions
Series of 3-4 extracts
Perhaps from same TV talk
show
Clear, standard, straight-forward,
relatively slow
Extremely short (c 1 min)
Hear once only (in
test)
One item per
extract
No tricky distracters
True / False / NS
Matching (really a
repeated MCQ with
5-6 alternatives)
New MCQ each
extract
Identify main point of
short conversation
extracts
B1 I can listen to a short narrative
and form hypotheses about what
will happen next.
Follow, though not
necessarily in detail
Understand an explicitly
signalled line of narrative
/ argument
Distinguish main point /
relevant facts and inform-ation
from specific details
– not central to story line
Narrative in linear order
Chain of events –
consequences
Clear, standard,
straightforward, relatively
slow
Hear once only
Guess what comes
next - recording
stops at question
point
Answer to each
question on
separate paper,
handed in
Series of MCQ –
only one alternative
makes sense
Follow a narrative and
answer 5 MCQs to
predict what comes
next when the audio
text stops
B1 I can understand the main points
of radio news bulletins and sim-pler
recorded material on topics
of personal interest delivered
relatively slowly and clearly.
Understand an explicitly
signalled line of narrative
/ argument
Understand main/most
important information
Topics in field of general
personal interest
Clear, standard,
straightforward, relatively
slow
Hear once only
Straight forward
transfer of info, in
order of text
T /F / NS
Matching
Information
transfer to table
Open questions
Follow radio news and
answer questions /
complete table about 4
or 5 of the stories (Total
c 10 questions)
B1 I can catch the main points in TV
programmes on familiar topics
when the delivery is relatively
slow and clear.
Follow, though not
necessarily in detail
Understand an explicitly
signalled line of narrative
/ argument
Understand main/most
important information
Distinguish conclusion
from preceding detail
Familiar topics regularly
encountered in a school,
work or leisure context
TV programmes:
(interviews) short lectures,
news reports
Clear, standard,
straightforward, relatively
slow
Could hear
twice/three times
Identifying main
information
Identifying when
conclusion is
starting
Catching main
conclusion
Information
transfer to table
Open ended
questions
Follow a simple, factual
TV/web documentary (c
5-10 mins), understand
the main points and
complete the table /
answer open questions
B1 I can catch the main points in TV
programmes on familiar topics
when the delivery is relatively
slow and clear.
I can follow a lecture or talk
within my own field, provided the
subject matter is familiar and the
presentation straightforward and
clearly structured.
Distinguish main points
from specific details
Understand an explicitly
signalled line of narrative
/ argument
Understand specific
details
TV programme with short
report/guide and
interview(s)
Familiar topics regularly
encountered in a school,
work or leisure context
Clear, standard,
straightforward, relatively
slow
Could hear
twice/three times
Recognising
sections (relevant
to topics of
questions)
Identifying
essential
information
T / F / NS
Matching
Information
transfer (table or
diagram)
Open ended
questions
Follow a TV guided
commentary on a place
(e.g. Tour of Hampton
Court, Versailles;
extract from travel
programme / tourism
promotion)
B1+ I can understand a large part of
many TV programmes on topics
of personal interest such as
interviews, short lectures, and
news reports when the delivery
is relatively slow and clear.
Understand main/most
important information
Understand main
conclusions
Daytime / local TV news –
perhaps a studio audience
Factual interview
Straightforward factual
interview questions
Descrip.s of events / plans
Descrip.s of feelings,
wishes
Identifying adv.s /
disadv.s of a plan
Identifying
consequences
Identifying the
opinions of the
main speakers (for
/ against: why?)
T / F / NS
Information
transfer (table)
Open ended
questions
Listen to a
straightforward, factual
interview from a current
affairs TV magazine
programme (or local
news) and understand
both main points and
specific details
67. CEFR Descriptor Micro- activities Text features Task features Item types Example I can catch the main points in TV programmes on familiar topics when the delivery is relatively slow and clear. I can follow a lecture or talk within my own field, provided the subject matter is familiar and the presentation straightforward and clearly structured. Distinguish main points from specific details Understand an explicitly signalled line of narrative / argument Understand specific details TV programme with short report/guide and interview(s) Familiar topics regularly encountered in a school, work or leisure context Clear, standard, straightforward, relatively slow Could hear 2/3 times Recognise sections (relevant to questions Identify essential info T / F / NS Matching Info transfer (table or diagram) Open ended questions Follow a TV guided commentary on a place (e.g. - Tour of Hampton Court - Tour of Versailles; - Extract from travel programme - Extract from tourism promotion)
68. CEFR Descriptor Micro- activities Text features Task features Item types Example I can catch the main points in TV programmes on familiar topics when the delivery is relatively slow and clear. I can follow a lecture or talk within my own field, provided the subject matter is familiar and the presentation straightforward and clearly structured. Distinguish main points from specific details Understand an explicitly signalled line of narrative / argument Understand specific details TV programme with short report/guide and interview(s) Clearly structured Familiar topics regularly encountered in a school, work or leisure context Clear, standard, straightforward, relatively slow Could hear 2/3 times Recog- nise sections (relev- ant to questions Identify essential info T / F / NS Matching Info transfer (table or diagram) Open ended questions Follow a TV guided commentary on a place (e.g. - Tour of Hampton Court - Tour of Versailles; - Extract from travel programme - Extract from tourism promotion)
69. CEFR Descriptor Micro- activities Text features Task features Item types Example I can catch the main points in TV programmes on familiar topics when the delivery is relatively slow and clear. I can follow a lecture or talk within my own field, provided the subject matter is familiar and the presentation straightforward and clearly structured. Distinguish main points from specific details Understand an explicitly signalled line of narrative / argument Understand specific details TV programme with short report/guide and interview(s) Clearly structured Familiar topics regularly encountered in a school, work or leisure context Clear, standard, straightforward, relatively slow Could hear 2/3 times Recog- nise sections (relev- ant to questions Identify essential info T / F / NS Matching Info transfer (table or diagram) Open ended questions Follow a TV guided commentary on a place (e.g. - Tour of Hampton Court - Tour of Versailles; - Extract from travel programme - Extract from tourism promotion)
70. Developing scenarios
1.Start with descriptors
2.Brain-storm ideas for example activities – right hand column
3.Select Micro-activities from selected descriptor(s) & micro-activities chart
4.Select text features from selected descriptors & CEFR Salient features chart
5.What would the learners need to do? Task features
6.What item types might be suitable for that? Item types
72. Assessing Speaking
Elicitation: Activities appropriate to an action-oriented curriculum
•Select the communicative activities – e.g. types of speaking activities
•Design tasks
Judgement: Qualitative criteria
•Consult CEFR Table 3 & descriptors
•Create marking grids
73. CEFR Table 3
RANGE
ACCURACY
FLUENCY
INTERACTION
COHERENCE
C2
Shows great flexibility reformulating ideas in differing linguistic forms to convey finer shades of meaning precisely, to give emphasis, to differentiate and to eliminate ambiguity. Also has a good command of idiomatic expressions and colloquialisms.
Maintains consistent grammatical control of complex language, even while attention is otherwise engaged (e.g. in forward planning, in monitoring others' reactions).
Can express him/herself spontaneously at length with a natural colloquial flow, avoiding or backtracking around any difficulty so smoothly that the interlocutor is hardly aware of it.
Can interact with ease and skill, picking up and using non-verbal and intonational cues apparently effortlessly. Can interweave his/her contribution into the joint discourse with fully natural turntaking, referencing, allusion making etc.
Can create coherent and cohesive discourse making full and appropriate use of a variety of organisational patterns and a wide range of connectors and other cohesive devices.
C1
Has a good command of a broad range of language allowing him/her to select a formulation to express him/ herself clearly in an appropriate style on a wide range of general, academic, professional or leisure topics without having to restrict what he/she wants to say.
Consistently maintains a high degree of grammatical accuracy; errors are rare, difficult to spot and generally corrected when they do occur.
Can express him/herself fluently and spontaneously, almost effortlessly. Only a conceptually difficult subject can hinder a natural, smooth flow of language.
Can select a suitable phrase from a readily available range of discourse functions to preface his remarks in order to get or to keep the floor and to relate his/her own contributions skilfully to those of other speakers.
Can produce clear, smoothly flowing, well-structured speech, showing controlled use of organisational patterns, connectors and cohesive devices.
B2+
B2
Has a sufficient range of language to be able to give clear descriptions, express viewpoints on most general topics, without much conspicuous searching for words, using some complex sentence forms to do so.
Shows a relatively high degree of grammatical control. Does not make errors which cause misun- derstanding, and can correct most of his/her mistakes.
Can produce stretches of language with a fairly even tempo; although he/she can be hesitant as he or she searches for patterns and expressions, there are few noticeably long pauses.
Can initiate discourse, take his/her turn when appropriate and end conversation when he/she needs to, though he/she may not always do this elegantly. Can help the discussion along on familiar ground confirming comprehension, inviting others in, etc.
Can use a limited number of cohesive devices to link his/her utterances into clear, coherent discourse, though there may be some "jumpiness" in a long con- tribution.
B1+
B1
Has enough language to get by, with sufficient vocabulary to express him/herself with some hesitation and circumlocutions on topics such as family, hobbies and interests, work, travel, and current events.
Uses reasonably accurately a repertoire of frequently used "routines" and patterns associated with more predictable situations.
Can keep going comprehensibly, even though pausing for grammatical and lexical planning and repair is very evident, especially in longer stretches of free production.
Can initiate, maintain and close simple face-to-face conversation on topics that are familiar or of personal interest. Can repeat back part of what someone has said to confirm mutual understanding.
Can link a series of shorter, discrete simple elements into a connected, linear sequence of points.
74. CEFR Table 3 - reduced
RANGE
ACCURACY
FLUENCY
B2+
5
•clear descriptions, views
•not much sign of having to restrict what wants to say
•variety of linking words used efficiently to mark clearly the relationships between ideas
•good grammatical control
•occasional rare "slips" often corrected
•communicate spontaneously
•often remarkable ease of expression in longer stretches
•use circumlocution and paraphrase to cover gaps
4
B2
3
•Clear, coherent descriptions and views
•little searching for words
•some complex sentence forms
But: limited variation
•no errors causing misunderstanding,
•correct most of his/her mistakes
But: mistakes and wrong vocab do often occur:
•stretches of language with fairly even tempo
•few noticeably long pauses_
But: can be hesitant searching for expressions
may be "jumpiness" in long contribution.
2
B1+
1
• express main points
• express own thoughts
• with reasonable precision
•reasonable accuracy
•But: noticeable mother tongue influences
•able to keep going effectively without help
•But: some problems resulting in pauses and "cul-de-sacs"
75. Criteria at B2?
RANGE
•clear descriptions, views
•little searching for words
•some complex sentence forms
1
2
3
4
5
ACCURACY
•no errors causing misunderstanding,
•correct most of his/her mistakes
1
2
3
4
5
FLUENCY
stretches of language with fairly even tempo
•few noticeably long pauses_
•But: can be hesitant searching for patterns and expressions
1
2
3
4
5
COHERENCE
coherent discourse,
•But: limited variation,
•may be "jumpiness" in long contribution.
1
2
3
4
5
GLOBAL
Overall impression mark
1
2
3
4
5
76. Standardising interpretation of levels
–Standardisation training with calibrated examples and common CEFR criteria
–Transfer to local examples (videos, scripts)
Standardisation
Standardising assessment practice
–Assessing the same thing
–Valid judgements, Practicality
77. Standardisation Training
1.Illustration with documented samples
2.Small group discussion of other documented samples
3.Individual rating of documented samples - followed by group discussion – consensus
4.Individual rating of local samples - followed by group discussion – consensus
78. Why do Moderation?
Assessment Errors:
•Using own, private concepts and criteria
•Unconscious lead criterion (accuracy / fluency)
•Severity / lenience
•Refusal to give top grade/mark
Training cannot change “hard cases”
79. Moderation Techniques
•Collectivity (e.g. second assessor)
•Other information (e.g. “anchor test”)
•Quality Control
Moderation removes extremes and makes all aware of the problem