Invited speech at IMTEL Innovation Day at the Norwegian University of Science and Technology on November 20, 2018. The WEKIT training methodology and the technological platform allow creating educational experience efficiently using the time of the expert, aimed for the areas where expertise is rare and experts are scarce. This approach is based on the idea of using wearable sensors to capture performance of an expert and then making it available for trainees using Augmented Reality.

1. Workplace training 4.0 for Industry 4.0
Experience Capturing and Re-enactment with
WEKIT
IMTELInnovationDay20.11.2018
Norwegian University of Science and Technology
Trondheim, Norway
Mikhail Fominykh
Project manager, Europlan UK ltd, United Kingdom

4. # 423/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING

5. # 523/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING

6. # 623/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING

7. Can Technology improve
hands-on learning at the
workplace?
? ???

9. Computerization
Freya and Osborne (2016) The future of employment: How susceptible are jobs to computerisation?
https://doi.org/10.1016/j.techfore.2016.08.019

10. Computerization

11. Industry 4.0

12. Augmented Reality for a
human in Industry 4.0

13. More and more
complex workplaces
have to be managed by
less and less
people

14. Workplace training
will become
more and more
frequent

15. Augmented Reality

16. https://vimeo.com/133828847
Giuseppe Scavo, Fridolin Wild and Peter Scott (2015) The GhostHands UX: telementoring with hands-on
augmented reality instruction, DOI: 10.3233/978-1-61499-530-2-236
Enriching the ‘real’ world with virtual visual overlays

17. vtt.fi
Enriching the ‘real’ world with virtual visual overlays
Kaj Helin, Timo Kuula, Jaakko Karjalainen, Iina Aaltonen, Fridolin Wild, Juhani Viitaniemi, Antti Väätänen
(2015) Usability of the ARgh! Augmented Reality system for on-the-job learning
See video at:
https://www.youtube.com/watch?v=Snoyt5-pZRY

18. AR and wearables
in Society
Page 18WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING23/11/2018
o Wearables
o Games
o Tablets
o AR glasses

19. AR and wearables
in the Industry
o Operations
o Maintenance
o Error prevention
o Training
Date Page 19

20. Error reduction in
the industry
The use of Augmented Reality when
performing a 46-step task ranging in
complexity from selecting the correct parts,
to properly aligning and fastening bolts
through multiple parts reduced time to task
completion by 30% and reduced errors (first
time quality results improved 90%).
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 20
RICHARDSON ET AL. (2014)
http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1090&context=imse_conf

21. Wearable Experience for
Knowledge Intensive Training -
WEKIT
Disclaimer
This project has received funding from the European Union’s
Horizon 2020 research and innovation programme under
grant agreement No 687669. http://wekit.eu/

22. WEKIT partners

23. Training with AR –
knowledge conversion

24. Experience and knowledge
Learning = converting
experience to knowledge
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 24
Mikhail Fominykh, Fridolin Wild, Carl Smith, Victor Alvarez and Mikhail Morozov: "An Overview of Capturing
Live Experience with Virtual and Augmented Reality”, DOI: 10.3233/978-1-61499-530-2-298.

25. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
separated
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 25

26. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 26

27. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 27

28. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 28

29. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 29

30. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 30

31. Experience and knowledge
Learning =
= converting experience to knowledge
immediate experience
(aka ‘practice’)
information for a master
level
(aka ‘theory’)
Experiencedlearner
23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 31

32. Why WEKIT.one
There are 26 million active enterprises with some 144
million persons employed in Europe alone. One third of
industrial enterprises in Europe offer continuing
vocational training.
Industries need high-quality specialized workplace
training if they want to stay competitive. Such training is
either ineffective or expensive. This makes expertise
transfer very difficult.
Holographic training and experience capturing enabled
by Wearables and Augmented Reality can address these
challenges by offering an effective training while
reducing the trainer’s workload.

33. Why WEKIT.one
There are 26 million active enterprises with some 144
million persons employed in Europe alone. One third of
industrial enterprises in Europe offer continuing
vocational training.
Industries need high-quality specialized workplace
training if they want to stay competitive. Such training is
either ineffective or expensive. This makes expertise
transfer very difficult.
Holographic training and experience capturing enabled
by Wearables and Augmented Reality can address these
challenges by offering an effective training while
reducing the trainer’s workload.

34. Why WEKIT.one
There are 26 million active enterprises with some 144
million persons employed in Europe alone. One third of
industrial enterprises in Europe offer continuing
vocational training.
Industries need high-quality specialized workplace
training if they want to stay competitive. Such training is
either ineffective or expensive. This makes expertise
transfer very difficult.
Holographic training and experience capturing enabled
by Wearables and Augmented Reality can address these
challenges by offering an effective training while
reducing the trainer’s workload.

35. WEKIT.one
Industrial Training Platform
WEKIT.one platform allows to create training scenarios in three steps
Capture
Experience
Select appropriate
Transfer Mechanisms
from the library and
demonstrate the tasks
and procedures by
performing them.
Re-enact
Experience
Perform the tasks and
procedures guided by
the captured
demonstrations and
assisted by formative
feedback.
Review and
Analyze
Review the recordings
in post-analysis,
compare performances,
discover parts where
improvement is
needed.

36. WEKIT methodology
Capturing

37. Capture Experience

38. Capture Experience
See video at:
https://www.youtube.com/watch?v=AiKt9wJ3chg

39. WEKIT methodology
Re-enactment

41. Re-enact Experience
See video at:
https://youtu.be/JRMLs9SYg6k

42. WEKIT methodology
Review

43. Review and Analyze
Experience

44. WEKIT methodology
Learning
Experience
Modeling

45. Compatible with the draft IEEE
ARLEM Standard
Augmented Reality Learning Experience Model
(ARLEM) is an integrated conceptual model and the
according data model specifications for representing
activities, learning context and environment.

46. 23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 46
Augmented
Reality
Learning
Experience
Model
P15891 working group of the
IEEE standards association
http://arlem.cct.brookes.ac.uk

47. WEKIT Software
Trainer features
• Virtually annotating objects in the physical space (text, image, video, audio, 3d objects)
• Capturing expert performance using a multimodal sensor recording
• Transforming recording into ARLEM

48. WEKIT Software
Trainee features
• Importing ARLEM recording
• Automatically generated task list and guide functions to all annotations
• Contextualized multimodalguidance and re-enactment of experience
• Capturing trainee performance

49. WEKIT Software
Analysis features
• Mapping data to ARLEM
• Visualization of performance and biological data
• Comparison of multiple performances

50. WEKIT Hardware

51. WEKIT Hardware

52. 23/11/2018 WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 52
Participatory design
See video at:
https://www.youtube.com/watch?v=ND913dF8iyc

53. Wearable
prototype 1
Wearable
prototype 2

55. Wearable
prototype 3

58. Training Methodology
Research-based
Specifically for industrial training at the workplace
Designed for Augmented Reality and Wearables

59. Training Methodology
Step 1. Prepare
Break down complex tasks to subtasks, identify properties
of subtasks and
Select Transfer Mechanisms

60. Training Methodology
Step 2. Capture
Demonstrate each subtask while wearing the WEKIT
wearable solution
Step 3. Re-enact
Perform the tasks and procedures guided by the captured
demonstrations and assisted by formative feedback.
Step 4. Review and Analyze
Review the recordings in post-analysis, compare
performances, discover parts where improvement is
needed.

61. Aircraft Maintenance
Use Case

62. Medical Training
Use Case

63. Astronaut Training
Use Case

64. Evaluation results
from trials 1 in 2017
Trials 2 are running now autumn-winter
2018

65. TAMARA
Technology Acceptance Model for AR/WT (TAMARA) questionnaire
contains some questions to check the technology acceptance of the
user toward the proposed prototype. The questionnaire has been
developed in Wild et al. (2017).
The first 19 items use a 7-point Likert scale ranging from strongly
disagree (1) to strongly agree (7), while the final item, usage frequency,
is rated on a 6-point scale. They are presented to the experiment
participants in the questionnaire in the following manner:
ATU4 I look forward to those aspects of my job that require me to use AR & WT.
65
strongly
disagree (1)
disagree
(2)
somewhat
disagree (3)
neither agree
or disagree
(4)
somewhat
agree (5)
Agree
(6)
strongly
agree (7)

66. 66

67. 67
I am addicted to
using AR & WT.
I like working with AR
& WT.
Learning curve for AR &
WT is too high
compared with the
value they would offer.

68. SUS
System Usability Scale (SUS) is a tool for measuring both usability and
learnability. The SUS scores calculated from individual questionnaires represent
the system usability. SUS yields a single number representing a composite
measure of the overall usability of the system being studied. Scores for
individual items are not meaningful on their own. SUS scores have a range of 0
to 100 (Brooke, 1996; 2013). According to validation studies, the acceptable SUS
score is about 70 (Brooke, 2013; Bangor et al., 2009).
SUS is based on John Brooke (2013): SUS: A Retrospective, In: Journal of
Usability Studies, 8(2):29-40
68

69. SUS
69

70. SSQ
The Simulator Sickness questionnaire (SSQ) was filled in by the
participants before and after the test. The questionnaire before the test
is used to make sure that the user does not have medical issues since
the test has to be done only with healthy subjects. The questionnaire
after the test is used to check the symptoms of the participants for
three main areas: Nausea, Oculomotor and Disorientation.
According to the NASA Johnson Space Centre Table, the score of the
SSQ questionnaire are interpreted as: no symptoms if score < 0,
negligible symptoms if score <5, minimal symptoms if score is between
5-10, significant symptoms if scores is between 10-15, concerning if
score is between 15-20 and problem simulator if >20.
70

71. SSQ
71

72. SGUS
Smart Glasses User Satisfaction (SGUS) questionnaire was created for the
WEKIT trials. SGUS measures subjective satisfaction focusing especially on test
participants’ experiences on the features that support learning. SGUS is based
on the evaluation criteria for web-based learning by Ssemugabi & de Villiers
(2007) and statements taken from Olsson, T. (2013) ‘Concepts and Subjective
Measures for Evaluating User Experience of Mobile Augmented Reality
Services’. SGUS consists of 11 items (statements) with a seven point Likert scale
(1-7).
For the analysis, the overall average from all items was calculated. Then this
overall average was compared with averages from individual items on a scale of
1-7.
72

73. SGUS
73

74. SGUS
74
I had a good
conception of what is
real and what is
augmented when
using AR-glasses.
Performing the task
with the help of AR-
glasses was natural
to me.
While using AR-glasses,
I was aware of the
phase of the task at all
times during the
execution of the task.

75. QUIS
The Questionnaire for User Interaction Satisfaction (QUIS) measures subjective
satisfaction with specific aspects of the interface, including usability and user
experience (Chin et al., 1988). It also measures the feeling about the AR glasses.
QUIS was modified, using only the relevant items from the viewpoint of this
study. Altogether 15 QUIS items with a scale mapped to numeric values of 1 to 7
were used.
For the analysis, the overall average from all items was calculated. Then this
overall average was compared with averages from individual items.
75

76. QUIS
76

77. TM
The Transfer mechanism (TM) Questionnaire was designed to evaluate
the adherence of the prototype to the Wekit framework.
Each Transfer Mechanism was formulated into one or several self
reflective statements
Participants had no pre-knowledge of Transfer Mechanisms and were
unaware that they were following the framework
Both experts and students were asked to rate statements that were
oriented to the successful implementation of each Transfer Mechanism
after being exposed to the prototype.
77

78. Lufttransport
Ebit ESAOTE
ALTEC (Thales, ESA)
78

79. General conclusions
• The organization and execution of the 3 trials was very successful
• The use of questionnaires and interviews to quickly collect feedback from the
users were a very good idea
• The second iteration will implement the results and feedback collected during
this first campaign of test
• Emphasis in enhancing the design and the usability of the WEKIT applications
• For both evaluation cycles we promised 600 people in total
• We have achieved around 50 per Industrial case during iteration 1
• We had almost 150 people for Iteration 1, which leaves 450 people for Iteration 2
WEARABLE EXPERIENCE FOR KNOWLEDGE-INTENSIVE TRAINING 7905/07/2017

80. Q & A
Slides: https://www.slideshare.net/mfominykh