Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Wearable Experience: New Educational Media for Knowledge Intensive Training

This slides were presented at the invited speech at the World Conference on Educational Media and Technology (EdMedia) which was held in Vancouver, BC, Canada on June 28-30, 2016.
Abstract: Wearable computing and augmented reality are disruptive technologies. They fundamentally change the way we educate and train people to a master level of performance. With advanced sensors we can capture experience as it emerges. For example, a trainee can receive live guidance in the form of semi-transparent 3D hands that appear at the right place spatially and operated by a remote expert using sensor data. Captured guidance provides reference to scale, allowing repeated access to the information asynchronously at the right time and in the right place where it is most urgently needed. Expert guidance can be captured with wearable sensors and later re-enacted by trainees with augmented reality creating a believable illusion of a master-apprentice knowledge sharing. The captured experience therefore represents a new type of educational media that has properties of carrying both explicit and tacit knowledge. This new media helps to convert experience to knowledge and enable learning by bringing closer the theoretical knowledge and immediate experience, which are traditionally separated. Tailored content of captured experience can be presented with augmented reality using intuitive and immersive user interfaces. This can have a positive impact on mental processing and memorization, not only adding scaffolds for high performance, but also acting as a safety net preventing potential problems sensed in the environment. Learning how to master a complex task usually involves reflecting on your own performance, looking back at your behavior and comparing it to that of others. The goal of this new training methodology is to enable the full cycle of immersive experience observing an expert, training with and without guidance, and observing own performance.

  • Login to see the comments

Wearable Experience: New Educational Media for Knowledge Intensive Training

  1. 1. Wearable Experiences for Knowledge Intensive Training Mikhail Fominykh, PhD Europlan UK Molde University College 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 1 June 28-30, 2016 • Vancouver, BC, Canada Wearable Experience: New Educational Media for Knowledge Intensive Training Invited speech
  2. 2. 29/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 2 June 28-30, 2016 • Vancouver, BC, Canada #WEKIT #edmedia #edmediaconf
  3. 3. About me 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 3 Project manager in workplace training with Augmented Reality and Wearables Associate Professor and researcher in Emergency Management training Adjunct Professor and teacher in Virtual Reality Software developer in psychological treatment training Software developer in welfare personnel training Software developer in cooperation training for medical students
  4. 4. Background: Experiential learning 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 4 Mikhail Fominykh, Peter Leong, and Brenda Cartwright (2016/2017) A Study on Experiential Learning in a 3D Virtual Environment within a Professional Counseling Distance Course, in progress.
  5. 5. Background: Virtual Reality Recording 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 5 Real time class R0 R1 R1 R2 Recording Replay Recording Mikhail Morozov, Alexey Gerasimov, Mikhail Fominykh, and Andrey Smorkalov (2013) Asynchronous Immersive Classes in a 3D Virtual World: Extended Description of vAcademia, DOI: 10.1007/978-3-642-38803-3_5
  6. 6. Background: Experiential learning 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 6
  7. 7. Background: Experiential learning 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 7
  8. 8. Photo by Mikhail Fominykh Wearable Computing Augmented Reality Experience and Knowledge WEKIT methodology
  9. 9. Information communication 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 9 Experience- communication Images: http://www.unmuseum.org/, http://en.wikipedia.org/, http://digitalxtrememedia.com/; https://www.youtube.com/watch?v=7d59O6cfaM0, https://www.microsoft.com/en-us/research/project/holoportation/
  10. 10. https://www.youtube.com/watch?v=qAsu-RoVgWc AR?
  11. 11. https://www.youtube.com/watch?v=7d59O6cfaM0 AR!
  12. 12. Object- centric 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 12 Action- centric
  13. 13. Performance Augmentation with wearables sensor-sensory loop: super-real experience 010000100001000010001010001111110 The guest enacts experiences The stager produces narratives WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 13
  14. 14. Photo by Ravensbourne Wearable Computing Wearable computing pursues an interface ideal of a continuously worn, intelligent assistant that augments memory, intellect, creativity, communication, and physical senses and abilities*. * Starner (2011) The challenges of wearable computing: Part 1, IEEE Micro, Volume 21, Issue 4, DOI: 10.1109/40.946681
  15. 15. Why Wearable Computing? 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 15 Better access and management of information Portability during operation Enable hands free or hands-limited use Can run continuously; and attempt to sense the user’s current context (Rhodes, 2000) Focuses the user’s attention and presents information in an unobtrusive, context-dependent manner (Kortuem et al.)
  16. 16. Wearability 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 16 Head mounted (headsets, glasses, ear attachments) Hand, wrist and arm mounted (gloves, watches, rings, bands)Chest, neck back mounted (belts, bands, wests, shirts) Leg, feet mounted (bands, shoes)
  17. 17. Photo by Mikhail Fominykh Augmented Reality Augmented Reality provides an enriched view onto the physical world, adding layers with contextually useful information, delivered visually or by stimulating other senses.
  18. 18. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 18
  19. 19. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 19 +80 / +100%
  20. 20. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 20 https://www.youtube.com/watch?v=YJg02ivYzSs
  21. 21. Photo by Mikhail Fominykh Reality is more rough
  22. 22. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 22 Working example Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning
  23. 23. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 23 Kaj Helina, Timo Kuulaa, Jaakko Karjalainena, Iina Aaltonena, Fridolin Wildb, Juhani Viitaniemia, Antti Väätänena (2015) Usability of the ARgh! Augmented Reality system for on-the-job learning
  24. 24. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 24 Working example
  25. 25. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 25 Working example
  26. 26. What is WEKIT? 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 26 Funding: EU Horizon 2020, ICT-20 2015: Technologies for better human learning and teaching Budget: EUR 2.753.143,75 WEKIT Community https://wekit-community.org/ WEKIT project website http://wekit.eu/
  27. 27. Experience and knowledge Learning = converting experience to knowledge 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 27 Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.
  28. 28. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) separated 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 28
  29. 29. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 29
  30. 30. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 30
  31. 31. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 31
  32. 32. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 32
  33. 33. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 33
  34. 34. Experience and knowledge Learning = = converting experience to knowledge immediate experience (aka ‘practice’) information for a master level (aka ‘theory’) Experiencedlearner 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 34
  35. 35. Methodology What is Wearable Experience? 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 35 The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and enabling master-apprentice knowledge sharing. Capture Re-enact Evaluate
  36. 36. Methodology What is Wearable Experience? 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 36 The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing. Giuseppe Scavo, Fridolin Wild and Peter Scott(2015) The GhostHands UX: telementoring with hands-on augmented reality instruction, DOI: 10.3233/978-1-61499-530-2-236 Capture example
  37. 37. Methodology What is Wearable Experience? 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 37 The Wearable Experience training methodology aims to provide an innovative learning method that is based on the idea of capturing the experience of an expert and enabling trainees to wear it while re-enacting, thus giving the trainee access to the tacit knowledge of the expert and master-apprentice knowledge sharing. Wild, Perey, Helin, Davies and Ryan (2014) TELLME’s Learning Experience Model in action on the shop floor of a textile weaving mill Re-enact example
  38. 38. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 38 Jan Schneider, Dirk Börner, Peter van Rosmalen and Marcus Specht (2015) Presentation Trainer: Polishing your communication skills, DOI: 10.1007/978-3-319-24258-3_52 Evaluate example
  39. 39. Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298. Capture | Re-Enact WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016 39
  40. 40. Technology What is Wearable Experience? An experience that is facilitated by devices located on or in the body that can include: tracking user activity, biometric and environmental data matching and mapping user performance augmenting user’s activity with another user’s data adapting user’s activity and behavior 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 40
  41. 41. # 41 Maintenance workers in Aeronautics
  42. 42. # 42 Imaging and diagnostic workers in Healthcare IT
  43. 43. # 43 Sub-system integration, operation & maintenance in Space Photo by European Space Agency
  44. 44. WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING29/06/2016 44
  45. 45. WEKIT methodology Phase A Capturing
  46. 46. WEKIT methodology Phase B Re-enactment
  47. 47. WEKIT methodology Phase C Evaluation
  48. 48. Goals Capture of Expertise What are relevant aspects of expertise and how to capture them? Enable enactment of expertise How to actively support the transfer of expertise to a learner? Design of learning task How should learning activities be designed to support / complement expertise transfer? Requirement generation How should technology be designed to foster all of the above? 48WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING28/06/2016
  49. 49. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 49 Domain limited (Fernand and Herbert, 1996) Failure to recall surface features (Schmidt and Boshuizen, 1993) Inaccurate prediction judgement and advice (Hinds, 1999) Inflexibility and bias (Sternberg and Frensch, 1992) Qualities of Experts Superior Mental Knowledge (De Groot, 1978) Perceptual abilities (Lesgold et al., 1988) Qualitative analysis (Klein, 1998) Better strategies (Lemaire & Siegler, 1995) Cognitive effort (Patricia 2003) Shortcomings of Experts
  50. 50. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 50 Top-down approach Identify tangible and intangible aspects relevant in all tasks Find abstract key factors and rating scheme Assess tasks and mechanisms according to factors and schemes WEKIT methodology Bottom-up approach Start to analyse concrete tasks in industrial settings Classify them according to common scheme identifying aspects of expertise and learning Try to find mechanism suitable to support expertise transfer
  51. 51. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 51 Task Classifications Transfer mechanism Highprecision Highspeed Decisiontaking Collaborative tasks Highmemory load Perceptualtask Assertivetask Dualtask Performance Managing Uncertainty HighSpatial Task Slow motion Zoom 3 13 Think aloud protocol 1 8 Object enrichment 2 Contextualization 3, 9 3 Self awareness of physical state 14 Directed focus 15 Case identification 16 Spatial Simulation 18 In Situ Real-time Feedback 4 17 Mobile control Virtual Post its 5 5 Haptic hints 5, 12 Virtual/Tangible Manipulation 6 3, 10, 11 Remote symmetrical tele- assistance 7 7 Literature 1Schraagen, 1993 2Tang et al., 2003 3Quarles et al., 2008 4BMW AR system, 2010 5Sabine et al., 2011 6Lahanas V et al., 2014 7Chinthammit et al., 2014 8Lundgrén-Laine et al, 2010 9Curtis et al., 1998 10Seichter, 2004 11Wang, Chen, Gong, & Hsieh, 2007 12Khademi et al., 2012 13Mulloni, Dünser &Schmalstieg 2010 14Larkin, 1983 15Jarodzka et al., 2013 16Mulzoff, 1990 17Kotranza, Lind, Pugh & Lok 2009 18Martin-Gutierrez et al., 2010
  52. 52. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 52 Tasks Cognitive Perceptual Developmental Human Machine Environment Space Time Target Domains Dimensions Top-down approach
  53. 53. What is WEKIT? 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 53 TEL community – driven project
  54. 54. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 54
  55. 55. 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 55
  56. 56. jTEL summer schooler jTEL summer schooler jTEL summer schooler jTEL summer schooler jTEL summer schooler jTEL summer schooler jTEL summer schooler
  57. 57. 28/06/2016 https://wekit-community.org/ideas/
  58. 58. 28/06/2016 http://bit.ly/wekit-facebook http://bit.ly/wekit-twitter http://bit.ly/wekit-linkedin http://bit.ly/wekit-plus http://bit.ly/wekit-youtube WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 59
  59. 59. Disclaimer This project has received funding from the Europea n Union’s Horizon 2020 research and innovatio n programme under grant agreeme nt No 687669. http: //we kit. e u/ Q & A #WEKIT Presented by Mikhail Fominykh mikhail.fominykh@europlan-uk.eu http://www.mikhailfominykh.com/ http://www.linkedin.com/in/fominykh/ https://www.facebook.com/mikhail.fominykh/ http://slideshare.net/mfominykh/ 28/06/2016 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 60

×