Education needs to face the challenges of a world changing at faster pace. It is no longer the case, that we need to educate enormous amounts of people for the very same job profiles. Instead, constant change, the ability to learn continuously flexibility, creativity are at the forefront of educational goals. Playing is one of the key principles allowing children to learn in safe environments. Games as structured forms of play transport playfulness into the adult world. With the advent of augmented reality and wearable technologies new forms of games appear, that take the context of the player into account and that enable players to transfer their games into their local environment. These new forms also enable new learning scenarios, which also require new pedagogical approaches to be covered. In this talk, several examples of mobile serious games and augmented reality are given.

1. Roland Klemke
IMTEL Innovation Day
1
07.05.19
Extend your playground:
Mixed-Reality Games for Learning
Prof. Dr. Roland Klemke
IMTEL Innovation Day
Immersive Tech and Serious Games
NTNU, Campus Dragvoll
Trontheim, Norway, 07.05.2019
1
Welten Institute, Open University of the Netherlands, http://www.ou.nl/
Cologne Game Lab, TH Köln, http://www.colognegamelab.de/
Humance AG, Cologne, Germany, http://www.humance.de/
Image: KMK/BEHRENDT&RAUSCH

2. Roland Klemke
IMTEL Innovation Day
2
07.05.19Image: Simon Blackley,
http://www.flickr.com/photos/sblackley/8130334415/sizes/h/in/photostream/
“Fun is the original educational technology.”
– Chris Crawford
Have you ever watched little children
learning? Or playing?
Discovered any difference?

3. Roland Klemke
IMTEL Innovation Day
3
07.05.19
Education
• Industrialized
• Built for Scalability
• Suited for the creation of „comparable” people
Image: John Hartnup,
http://www.flickr.com/photos/hartnupj/111438266/sizes/l/in/photostream/

4. Roland Klemke
IMTEL Innovation Day
4
07.05.19
Is this kind of education appropriate for
the skills and competencies
we need to teach today?
Image: Michael Hopkinsii, http://www.flickr.com/photos/hopkinsii/3888593489/sizes/l/in/photostream/
Creativity
Flexibility
Ability to reflect
Ability to learn

5. Roland Klemke
IMTEL Innovation Day
5
07.05.19
Image: Thomas Favre-Bulle, http://www.flickr.com/photos/lnx/6257130/sizes/o/in/photostream/
How can education be playful, individual and joyful?
• When students are grouped by age, not by abilities
or interests?
• When all students receive the same learning
materials?

6. Roland Klemke
IMTEL Innovation Day
6
07.05.19
TODO Überleitung Gaming
How to unite gaming
and learning again?
Image: Monobanda Fusionplay, http://www.flickr.com/photos/monobanda/8241276188/sizes/h/in/photostream/
How to place learning
and gaming in context?

7. Roland Klemke
IMTEL Innovation Day
7
07.05.197
Image: Krissy Venosdale: https://www.flickr.com/photos/venosdale/7051065819

8. Roland Klemke
IMTEL Innovation Day
8
07.05.19
LOOKING AT THE LEARNER
8
Image: chelsea (:, https://secure.flickr.com/photos/chelseasmile/3144504064

9. Roland Klemke
IMTEL Innovation Day
9
07.05.19
Learners are individual and mobile
How can we provide learning
resources with …
• ... different media formats
• ... different devices
• ... different target audiences?
How can we personalize
learning ...
• ... for the learner ...
• ... in the learning context?
Image: Steven Harris, https://secure.flickr.com/photos/srharris/6979776901

10. Roland Klemke
IMTEL Innovation Day
10
07.05.19
10

11. Roland Klemke
IMTEL Innovation Day
11
07.05.19
LogiAssist – Mobile Assistance in Logistics
11
Klemke, R. (2012, 26 March). Mobile Learning in Logistics. Workshop on Learning in Context LIC12, Brussels, Belgium. http://www.logiassist.de/
Scheffel, M., Kirschenmann, U., Taske, A., Adloff, K., Kiesel, M., Klemke, R., and Wolpers, M. (2013). Exploring LogiAssist – the mobile learning and
assistance platform for truck drivers. In D. Hernández-Leo et al. (Eds.), Scaling up Learning for Sustained Impact. Proceedings of European
Conference on Technology Enhanced Learning (EC-TEL), LNCS 8095 (pp. 357–370). Berlin Heidelberg, Germany: Springer-Verlag
You are exceeding the allowed driving time.
Please select a resting place.

12. Roland Klemke
IMTEL Innovation Day
12
07.05.19
GIVING ACCESS IS ONE THING ...
... MAKING IT FUN ANOTHER
12
Image: Ian Dick, http://www.flickr.com/photos/ian_d/30859899/sizes/o/in/photostream/

13. Roland Klemke
IMTEL Innovation Day
13
07.05.19
“Tell me, and I will forget.
Show me and I may remember.
Involve me, and I will understand.”
Confucius (probably)
Image: Länsmuseet Gävleborg, https://secure.flickr.com/photos/lansmuseetgavleborg/6263044945

14. Roland Klemke
IMTEL Innovation Day
14
07.05.19
!
Gaming, Learning, and Communities for IT Knowledge
Schmitz, B., Klemke, R., Totschnig, M., Czauderna, A., & Specht, M. (2011, 23 September). Transferring an
outcome-oriented learning architecture to an IT learning game. Presented at the 6th European conference on
Technology enhanced learning: towards ubiquitous learning (EC-TEL 2011), Palermo, Italy.

15. Roland Klemke
IMTEL Innovation Day
15
07.05.19
Is it possible to create a 3D learning game
with freely available technologies / contents
at low modelling cost?
Klemke, R. (2012). StreetLearn - Community-based immersive 3D-video learning games in Streetview.
Innovatie door Inspiratie, Maarssen, The Netherlands.

16. Roland Klemke
IMTEL Innovation Day
16
07.05.19
Use Google Streetview
and the real world as a
playground allowing
interaction with
virtual items in
location-based games
Klemke, R. (2012). StreetLearn - Community-based immersive 3D-video learning games in Streetview.
Innovatie door Inspiratie, Maarssen, The Netherlands.

17. Roland Klemke
IMTEL Innovation Day
17
07.05.19
THE WORLD AS PLAYGROUND …
… IS NOT A TOTALLY NEW IDEA
17
„Breakout“ is a silly game!
Image: Oliver Peters, Flickr, https://secure.flickr.com/photos/53127197@N06/5218440997

18. Roland Klemke
IMTEL Innovation Day
18
07.05.19
Scavenger Hunt is now called Geocaching
18
Image: cachemania, https://secure.flickr.com/photos/mycachemania/3988147164

19. Roland Klemke
IMTEL Innovation Day
19
07.05.19
MMMARG (Massive multiplayer mobile augmented reality game)
19
Bild: http://www.ingress.com

20. Roland Klemke
IMTEL Innovation Day
20
07.05.19
We don‘t collect stamps any more
20Images:
Brian Miller, https://www.flickr.com/photos/135306143@N02/30151138115
Su-May, https://www.flickr.com/photos/su-may/30151415712

21. Roland Klemke
IMTEL Innovation Day
21
07.05.19
Situation Awareness
Augmented Reality
Serious Gaming
21
MOBILE SERIOUS GAMES
Image: Rétrofuturs, https://secure.flickr.com/photos/hulk4598/4646721045/

22. Roland Klemke
IMTEL Innovation Day
22
07.05.19
What is a mobile serious game?
Image: Phil Rogers,
http://www.flickr.com/photos/erase/5572308872/sizes/l/in/photostream/
• Embedded in Context
• Exploration and
utilization of context
• Augmentation of reality

23. Roland Klemke
IMTEL Innovation Day
23
07.05.19
Gamification for LogiAssist –
The TEGA Project
A learning game that encourages to drive economic and
safe.
Klemke, R., Kravcik, M., and Bohuschke, F. (2013). Energy-efficient and safe driving using a situation-aware gamification
approach in logistics. Proceedings of the Games and Learning Alliance Conference (GALA 2013), 23-25 October 2013, Paris,
France.

24. Roland Klemke
IMTEL Innovation Day
24
07.05.19
TEGA App
Klemke, R., Kravcik, M., and Bohuschke, F. (2013). Energy-efficient and safe driving using a situation-aware gamification
approach in logistics. Proceedings of the Games and Learning Alliance Conference (GALA 2013), 23-25 October 2013, Paris,
France.

25. Roland Klemke
IMTEL Innovation Day
25
07.05.19
SALOMO
Decision training for disruption handling in
logistics
Image: hhla.de
Klemke, R., Kurapati, S., & Kolfschoten, G. (2013). Transferring an educational board game to a multi-user mobile
learning game to increase shared situational awareness. Presentation at the 3rd Irish Symposium on Game Based
Learning, Dublin, Ireland.

26. Roland Klemke
IMTEL Innovation Day
26
07.05.19
Mobile learning game based on team processes
simulate disruption situations
! !
Klemke, R., Kurapati, S., & Kolfschoten, G. (2013). Transferring an educational board game to a multi-user
mobile learning game to increase shared situational awareness. Presentation at the 3rd Irish Symposium on
Game Based Learning, Dublin, Ireland.
From board game to multi-user mobile learning game

27. Roland Klemke
IMTEL Innovation Day
27
07.05.19
EmUrgency: Mobile Game for Basic Life Support Training
Klemke, R., Ternier, S., Kalz, M., Schmitz, B., & Specht, M. (2013). Multi-stakeholder decision training games with
ARLearn. In D. Milosevic (Ed.), Proceedings of the fourth international conference on eLearning (eLearning 2013)
(pp. 1-9). Belgrade Metropolitan University, Belgrade, Serbia.
Chances for survival …
… are increased with the factor 3
through immediate cardiac
massage
Cardiac arrest one of the main
causes of death
Only 20% of affected people survive

28. Roland Klemke
IMTEL Innovation Day
28
07.05.19
EmUrgency: Mobile Game for Basic Life Support Training
But: bystanders …
… often don‘t know what to do or fear to do the wrong things
Klemke, R., Ternier, S., Kalz, M., Schmitz, B., & Specht, M. (2013). Multi-stakeholder decision training games with
ARLearn. In D. Milosevic (Ed.), Proceedings of the fourth international conference on eLearning (eLearning 2013)
(pp. 1-9). Belgrade Metropolitan University, Belgrade, Serbia.

29. Roland Klemke
IMTEL Innovation Day
29
07.05.19
EmUrgency: Mobile Multi-user Game for Basic Life Support
Schmitz, B., Ternier, S., Klemke, R., Kalz, M., & Specht, M. (2013). Designing a mobile learning game to investigate
the impact of role-playing on helping behavior. Proceedings of European Conference on Technology Enhanced
Learning (EC-TEL), LNCS 8095 (pp. 357–370). Berlin Heidelberg, Germany: Springer-Verlag.
Learn what to do, where to find resources, how to organise help

30. Roland Klemke
IMTEL Innovation Day
30
07.05.1930
AUGMENTATION?
IMMERSION?
Sensorama: http://www.augmented-reality.fr/wp-content/uploads/2014/08/sensorama.jpg

31. Roland Klemke
IMTEL Innovation Day
31
07.05.1931
Gameplay, Learning, and Player are united
in an immersive environment: the real world!
Image: Ars Electronica: https://secure.flickr.com/photos/arselectronica/7682581006

32. Roland Klemke
IMTEL Innovation Day
32
07.05.1932
Augmented Reality Glasses: Hands free interaction
Image: Ted Eytan, https://secure.flickr.com/photos/taedc/11966163406

33. Roland Klemke
IMTEL Innovation Day
33
07.05.1933
Image: StatesChronicle, http://stateschronicle.com/wp-content/uploads/2015/01/hololens.jpg
3D holographic glasses for full immersive interaction

34. Roland Klemke
IMTEL Innovation Day
34
07.05.1934
Image: Virtuix, http://www.virtuix.com/how-do-you-move-in-virtual-reality-with-a-treadmill-like-this-one-i-just-tried/
Full immersion gaming: VR glasses, VR motion system

35. Roland Klemke
IMTEL Innovation Day
35
07.05.19
Virtual Reality? Augmented Reality? Reality?
35
Images: Parrot

36. Roland Klemke
IMTEL Innovation Day
36
07.05.19
Out of body experiments
Two variants
• Projection of another body onto
one's own position
• Projection of one's own body to
another position
Outcomes
• Identification with virtual body even
if it's obviously artificial
• perceived feeling of touch or even
pain after successful initiation
• Identification with the virtual
location: participants return to the
place of the virtual body instead of
the place of observation
Slater, M., Pérez Marcos, D., Ehrsson, H., & Sanchez-Vives, M. V. (2009). Inducing illusory ownership of a virtual body.
Frontiers in neuroscience, 3, 29.
D. Cowie et al. The development of bodily self-consciousness: changing responses to the full body illusion in childhood.
Developmental Science. Published March 22, 2017. doi:10.1111/desc.12557.

37. Roland Klemke
IMTEL Innovation Day
37
07.05.19
PROJECT EXAMPLE: WEKIT
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING:
EXPLORING AUGMENTED REALITY AND WEARABLES FOR TRAINING
37

38. Roland Klemke
IMTEL Innovation Day
38
07.05.19
WEKIT - WEARABLE EXPERIENCE FOR
KNOWLEDGE-INTENSIVE TRAINING

39. Roland Klemke
IMTEL Innovation Day
39
07.05.19
Instructional Design Methods
Example: TM03 - Directed Focus
Description Capture
Method
Re-
Enactment
Method
Visual
pointer for
relevant
objects
outside the
visual area of
the trainee
Eye tracker
and video
recording
Task analysis
for pointing
to the next
location
Eye tracker
for formative
feedback
AR display for
feedback
The markers direct the trainee to the relevant areas.

40. Roland Klemke
IMTEL Innovation Day
40
07.05.19
Instructional Design Methods
Example: TM05 – Annotations
Description Capture Method Re-Enactment
Method
Allow a physical
object to be
annotated by the
expert during
task execution
(similar to sticky
notes but with
more modalities)
Methods to tag
media into
physical object
Manual
annotation or
done by expert
on the fly
AR display
mechanism to
read the
annotations
Mechanism for
unobtrusive
playback of
information
A physical object is annotated by the expert with
an image to allow the trainee to recognize the image.

41. Roland Klemke
IMTEL Innovation Day
41
07.05.19
Instructional Design Methods
Example: TM06 – Ghost Track
Description Capture Method Re-Enactment
Method
Allows visualization
of the movement of
the expert or the
earlier recording of
the trainees
themselves for
imitation and
reflection
Sensors to capture
the whole-body
movements
Recording of results
of the action
performed by the
expert
Visualization
mechanisms
Tracking of
current state
for feedback
and evaluation
An expert’s body and hand movement is
recorded and replayed as ghost track.

42. Roland Klemke
IMTEL Innovation Day
42
07.05.19
Instructional Design Methods
Example: TM08 – Cues & Clues
Description Capture Method Re-Enactment
Method
Cues and clues are
pivots that trigger
solution search.
They can be in the
form of image or
audio. They should
represent the
solution with a
single annotation
Task Analysis
Mechanism to
allow content
creation to be used
for clues and cues
AR displays the
clues anchored
to the physical
object
Additional help
when
requested
A clue is left by the expert for the trainee in form of a text.

43. Roland Klemke
IMTEL Innovation Day
43
07.05.1943

44. Roland Klemke
IMTEL Innovation Day
44
07.05.19
Using VR to lower barriers and risks for beginners
44
Scared of heights?
Not sure what to expect?
Still want to experience and learn how to fly?

45. Roland Klemke
IMTEL Innovation Day
45
07.05.1945

46. Roland Klemke
IMTEL Innovation Day
46
07.05.19
@rklemke
Roland.klemke@ou.nl
rk@colognegamelab.de
www.linkedin.com/in/rolandklemke
Image: Rockman of Zymurgy, http://www.flickr.com/photos/rockmanzym/4422834765/ Image: Véronique Debord-Lazaro, https://www.flickr.com/photos/debord/4932655275/