Presentation on interoperability models for activities and unified reference spaces ('workplaces') given at the Augmented World Expo 2014 in Santa Clara, US.†

1. The Technology Enhanced Learning
Living Lab for Manufacturing Environments.
Fridolin Wild1), Peter Scott1), Jaakko Karjalainen2), Kaj Helin2),
Erik Isaksson3), Ambjoern Naeve3), Maurizio Megliola4), Gianluigi Di Vito4)
1) The Open University, UK 2) VTT, Finland,
3) Royal Institute of Technology (KTH), Sweden, 4) Piksel, Italy
ARgh! - kinesthetic learning.
Augmented World Expo, 27.-29.5., Santa Clara/CA

2. Agenda
• Context ‘Manufacturing’
• Process Model (‘Mix, Match, Optimize’)
• Activity Model
• Workplace Model
• xAPI Tracking
• (Analytics with cRunch)
2

3. Context ‘Manufacturing’
Workforce in EU: 225.6 M total employment
Manufacturing EU: > 28.4 M1) employees
> 12.6 %1)
of which in SMEs: 59.9 %2)
Source: Eurostat, NACE R2 (2009), LFSI EMP A (2009)
1) Based on incomplete data, actual numbers higher: e.g. France missing.
2) Based on incomplete data.
3

4. Workplace !=
Workplace
4
(pictures: d7.1, study: d3.3)

5. Job Performance Aids
5
- Independent, self-regulated learners
- As individuals or in collaboration
- Engage in problem solving and devise
new applications (‘prototyping’)
- Extreme levels of memorization training
and rehearsal (‘training to memory’)
- Prepared to perform in challenging
circumstances
- Scaffold people in carrying
out an unfamiliar task
Adaptivity
Adaptability
consuming
do-torials
creating
do-torials

6. BLUE COLLAR WORKER
EXPERIENCE
Enquire
Mix
Match
Optimise
?
Traces
Need,
Problem
Activity
XML incl.
Constraints
Report,
Analytics
Suggestion,
Recommendation
Classifiable?
Known?
Unknown?
• Navigational
positioning in
taxonomy
• Discovery support
• Selection of existing mixes
with ranked search
• Authoring of new or
modified mixes
• Personalised suggestions for
improvement of mix /
experience
tracking
Constraints:
• e.g. returned tool 15
• e.g. watched video A
• e.g. 14/15 in PT session
• e.g. 0 FOD problems
• e.g. energy < daily limit
activities:
• e.g. job cards
• e.g. tasks
• e.g. learning paths
• Queries
• (Reasoning)
Learning by Experience
(Wild et al., to appear; Wild et al., 2013)

7. BLUE COLLAR WORKER
EXPERIENCE
Enquire
Mix
Match
Optimise
?
Traces
Need,
Problem
Activity
XML incl.
Constraints
Report,
Analytics
Suggestion,
Recommendation
Classifiable?
Known?
Unknown?
• Navigational
positioning in
taxonomy
• Discovery support
• Selection of existing mixes
with ranked search
• Authoring of new or
modified mixes
• Personalised suggestions for
improvement of mix /
experience
tracking
Constraints:
• e.g. returned tool 15
• e.g. watched video A
• e.g. 14/15 in PT session
• e.g. 0 FOD problems
• e.g. energy < daily limit
activities:
• e.g. job cards
• e.g. tasks
• e.g. learning paths
• Queries
• (Reasoning)
Learning by Experience
(Wild et al., to appear; Wild et al., 2013)
Sourcing: help to find or devise
next-generation training solutions
for blue collar workers in
manufacturing
Analytics: allow comparative analysis
on such measures as time-to-
competence, long-term recall,
maximum productivity achievable, time
to spot and remedy mistakes, …
Transformate: help reduce
or eliminate effort needed to
upgrade training solutions

8. Scenarios: AR goes HTML5!
8
explicit tacit

9. Summer
/Winter cycle
Functional
fabrics
Set up of the weaving mill.Each topic may hav.)
Prevent potential defects
before they happen.
Failed step: waste of
resources.
SB: Improve manufacturing process
with attribute changes

10. The situation: as-is

11. PROFESSIONAL QUALIFICATION
(e.g. courses, titles)
MATERIALS
(e.g. yarn 76/2 710)
MACHINE
Each topic may have
several alternative learning
contents attached
(objects, actions, choices...)
Dornier
MTV38
Yarn
76/2
710
Machine parts Materials
OCCUPATIONAL HEALTH
AND SAFETY
3D MODEL
Schlag
baum
SET UP PARAMETERS

12. ARgh! Component Architecture
12

13. ARgh! UI Model
13TELL ME project27-May-14
‘Traditional’ AR layer
o markers
o 3D objects
o Tied to object

14. PAINT GUN USE
PROFESSIONAL QUALIFICATION
(e.g. courses, titles)
FURNITURE MODEL, 3D MODEL
MATERIALS
(e.g. Pinewood)
SOLVENTS,
CATALYSTS &
VARNISHES
PAINTS
Each topic may have
several alternative learning
contents attached
(objects, actions, choices...)
Tinting and Varnishing (all spraying
benefit from same technology)
OCCUPATIONAL HEALTH AND
SAFETY
LOCATION

15. Do-torial demo:
Varnishing for tropical climate
15

16. Interoperability
is a property that emerges, when
distinctive information systems (subsystems)
cooperatively exchange data
in such a way that
they facilitate the
successful accomplishment
of an overarching task.

17. The Activity Model
17

18. The Activity Model
18
“find the spray
gun nozzle size
13”
Messaging in the
real-time presence
channel and
tracking to xAPI
Chaining of
actions; modeling
of ‘reactions’:
onEnter /
onTrigger
Styling
(cascading) of
viewports and
UI elements
Constraint modeling:
specify validation
conditions and model
workflow branching
e.g. smart player;
e.g. request widget

19. Serialized Activity Model
<activity id="fodw109” name="do-torial: learn about …” start="SprayGun”>
<action id="SprayGun” viewport="actions" type="findtool” location="SprayBooth">
<message type=“trace" channel=“lrs">user %u% searches for tool spraygun</message>
<summons type="onClick" removeSelf="true">
<toggle viewport="actions" type="action">video15</toggle>
<toggle viewport="objects" type="object">SprayGun</toggle>
<message type=“exit" channel="rpc">user %u% interacted spraygun</message>
</summons>
<constraint channel="rpc">user %u% interacted spraygun</constraint>
<instruction><![CDATA[<h1>Find the Spray Gun</h1><p>bla…</p>]]></instruction>
</action>
<action id="video15" viewport="actions” type="widget” location="SprayBooth">
<widget type="smartplayer" id="15” />
<message type="invoke" channel="rpc">user %u% accesses video15</message>
<summons type="onClick" removeSelf="true">
<toggle viewport="actions" type="action">microass1</toggle>
<message type="exit" channel="rpc">user %u% watched video 15</message>
</summons>
<constraint channel="rpc">user %u% watched video15</constraint>
<instruction><![CDATA[<h1>Well done!</h1><p>…!</p> ]]></instruction>
</action>
</activity>
19

20. The Workplace Model
20

21. The Workplace Model
21
The ‘tangibles’:
Specific persons,
places, things
The ‘configurables’:
devices (styling),
apps+widgets
The ‘triggers’:
Markers trigger
Overlays; Overlays
trigger human action

22. Unified Reference Space: Workplace
<?xml version="1.0" encoding="utf-8"?>
<Workplace xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Resources>
<Tangibles>
<Place Name="SprayBooth" Category="Place" MarkerID="007">
<YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” />
<Labels IsEnabled="false">
<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” />
</Labels>
</Place>
<Thing Name="SprayGun" Category="Tool" MarkerID="011" Thumbnail=“pistol-blue-small.png">
<YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” />
<Labels IsEnabled="false">
<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” />
</Labels>
</Thing>
<Person Name="Jake" Category="Person" MarkerID="012">
<YesNoOverlay IsEnabled="true” X_Offset="0” Y_Offset="0” Z_Offset="0” />
<Labels IsEnabled="false">
<Label Name="Name“ X_Offset="0” Y_Offset="0” Z_Offset="0” />
</Labels>
</Person>
<Tangibles>
</Resources>
</Workplace>
22

23. Tracking with the Experience API
23

24. Analytics
24
= t(im) %*% im
plot(net, usearrows = TRUE, usecurve = T)

25. Technology Enhanced Learning Living Lab for Manufacturing
Environments
25