Presentation given at the MBA Directors' Forum, Brisbane, 3 May 2012

1. www.jeremybwilliams.net
Chief Academic Officer
Knowledge Universe
Stamford Plaza Hotel, Brisbane
3 May 2012

2. Overview
1. The Five Minute University
2. The case for authentic assessment
3. Open-book, open-Web examinations
4. A sample OBOW exam
5. How to construct an OBOW exam
6. Summary and conclusions
2

3. 3

4. 4

5. 5

6. "Life is an open book exam."
• Learners need to be convinced of the
authenticity of the task if they are to fully engage
6
Professor Alan Blinder
Princeton University

7. • “... Engaging and worthy
problems or questions of
importance, in which students
must use knowledge to
fashion performances
effectively and .”
7
Authentic assessment defined

8. 8
Learning design for understanding

9. • multiple-choice tests
• fill-in-the-blanks
• true-false
• matching words
• … Students are passive
learners 
Authentic assessment is not:
9

10. 10

11. 11

12. Sound familiar?
12

13. Assessment of learning
Content
Assessment
Learning outcomes
13

14. Assessment for learning
Learning outcomes
Assessment
Content
14

15. Closed book, invigilated exams are more likely to foster
cramming/ data dumping than
15

16. 16

17. In brief …
• A semi-structured ‘mini-case (or ‘caselette’)
• Harnesses the power of ICTs to emphasise
currency and real world authenticity
• A summative assessment item …
… invites the student to draw on all that
they have learnt (determining what is
relevant).
17

18. Dull? Boring? Something to fear?
• Final assessment 
• Boredom and stress not conducive to deep
learning
• Important to catch the imagination and appeal
to the creativity of the learner
• Student satisfaction is influenced by positive
perceptions toward technology and an
autonomous learning mode
(Drennan, Kennedy & Pisarski 2005)
18

19. Key features
• Students play the role of decision-
maker, auditor, consultant or advisor
• They are presented with a
unstructured (open-ended) problem
that requires resolution (usually in the
form of a set of recommendations)
• No pre-exam night 'cramming'
19

20. The template
the setting in which the
problem/situation is identified and framed
the project and issues to resolve
the setting of
parameters and suggestions about
methods/concepts/models/tools to employ.
20

21. The ground rules
• To minimise the scope for unethical
behaviour …
1) Time period for the exam must be
sufficiently tight
1) Make clear (as a stated objective of the
subject) that
is the key to success
1) 'Text-book' impersonal responses will not
attract high grades.
21

22. 22

23. 23

24. You have to have
invigilated exams or
students will cheat
24

25. 1) Students cheat during
invigilated exams
1) In the adult learner
context, only a small
percentage will attempt
to cheat
2) These people will cheat
whatever the exam
instrument
Seldom observed points
25

26. 26

27. Getting started
• Keep a look out for material all the
time (not exam time!)
• e.g. Local newspaper, periodical
websites, magazines, television news
or current affairs programmes
27

28. What to look for
• A that learners can easily relate to
in lay terms
• Objective: to get them to
about an issue
• Student to act as ‘expert witness’ – an
effective mechanism for the validation of
their learning in their own minds
28

29. Creating a scenario
• Having settled on a theme,
gather together various media
that can bring the case to life
• The inclusion of hyperlinks,
photographs and/or streaming
media adds a human dimension

29

30. Lead characters
• No story is complete without lead
characters
• Using people with names, and
pictures and voices acts as a catalyst to
student engagement
• Fictional characters must give the
appearance of being real!
30

31. Setting
• Role play  the bridge between a
learner's education and their
professional practice
• Placing the learner in the role of the key
decision maker, the expert advisor, or
the auditor
• Revisit the stated learning outcomes
31

32. Defining the parameters
• The definition of the assessment
task might amount to no more than
a paragraph
• Ideally it should invite a wide of
variety of
32

33. Striking a balance
• Avoid 'spoon-feeding' but …
• … not so unstructured a student
is either struck by 'writers block'
or goes off in the wrong direction.
33

34. Expectations
• Before writing ,
it is helpful to develop an outline of the
kind of response one expects from the
learner and, importantly, …
• This process may also lead to
being refined
34

35. 35

36. OBOW exams …
• A form of assessment that fosters
… as opposed to a display of inert knowledge
• Test problem-solving skills not memory
• Equips learners with 21st Century skills
36

37. What OBOW exams deliver…
• An assessment instrument that is more relevant
to goals of the curriculum, greater authenticity,
where real-world problems take centre-stage
• Allow ICTs to be harnessed to encourage
interaction
• Student engagement with the assessment task 
induces
• Low cost solution for exam delivery in open and
distance learning
37

38. • Studies show stimulation with audio will
increase retention rate by 20%. If
stimulated with audiovisual, memory
retention climbs to 30%. If presented with
interactive multimedia involvement, the
retention rate can be as high as 60%.
38

39. References
• Williams, Jeremy B. (2009)The efficacy of the final examination: a
comparative study of closed-book, invigilated exams and open-book,
open-web exams (with Amy Wong), British Journal of Educational
Technology, 40 (2), 227-236).
• Williams, Jeremy B. (2007) E-xams: harnessing the power of ICTs to
enhance authenticity, (with Wing Lam and Alton Chua), Educational
Technology and Society, 10 (3), 209-221.
• Williams, Jeremy B. (2007) Using digital storytelling as an assessment
instrument: Preliminary findings at an online university, (with Kanishka
Bedi), Proceedings of the 11th CAA Conference, pp.433-447,
Loughborough, England, 10-11 July.
• Williams, Jeremy B. (2006) The place of the closed book, invigilated final
examination in a knowledge economy, Educational Media International,
43(2), 107-119.
39

40. Alan Blinder, http://halleinstitute.emory.edu/images/research/blinder_large
Grant Wiggins, http://myworldpearson.com/images/wiggins.jpg
MCQs, http://www1.istockphoto.com/file_thumbview_approve/2856522/2/istockphoto_2856522_multiple_choice_exam.jpg
Exam halls, http://i.telegraph.co.uk/multimedia/archive/01529/exam-hall_1529387c.jpg
~ http://www.admin.ox.ac.uk/schools/graphics/ewert2.JPG
~ http://picasaweb.google.com/lh/photo/SQORvbJYKhErQ67sYQgdVg
~ http://qixsaliva.blogspot.com/2007/04/final-destination-2.html
~ http://www.fotosearch.com/thumb/DGV/DGV078/200239868-001.jpg
Bruce Wellman http://pairadimes.davidtruss.com/question-everything/
Students will cheat http://i36.photobucket.com/albums/e47/priyankashis/cheating.jpg
Cheating http://i36.photobucket.com/albums/e47/priyankashis/cheating.jpg

41. authenticlearning.wordpress.com
@jeremybwilliams
http://sg.linkedin.com/in/jembwilliams
http://www.slideshare.net/jembwilliams
41