Overview of research into VR and learning, presented at the monthly Augment Reality New York Meetup. We cover such things as "presence," the "proteus effect" and others.

1. Immersive Education:
What Do We Know?
October 25th, 2016

2. Background
• Book Research for
“Immersive Education”
covering VR and AR
• Some early findings about
VR
• Invite me back in
December for AR!
@aquinastraining

3. Overview of What We Know
VR doesn’t solve everything
Things We Learn VR Will Help by VR Will Not Help
Because
Jury’s Out
Trade Skills:
e.g., Pipe fitting
Apprenticeship – like situated
learning.
Soft skills:
Management Training
Simulations ; intuitive recognition
of situation-types
Declarative Memory:
facts
VR doesn’t add
anything, per se
Executions of VR can add
virtual note-taking & other
memory-building aids
Sports: Pattern
recognition
Repeated exposure to situations
that can be recognized quickly
Sports: Physical skills Not using full brain to
muscle chain
Abstract concepts Visualization is a distraction
Novice Performance Helps to ‘scaffold’ new learners
with no existing knowledge base
Expert performance Visualization is a

4. Situated Learning
• Complex skills are best learned by doing them.
• Apprenticeships are the best example:
– You see and mentally model the skill
– You can do part of the task with teacher doing the rest
– You can mentally model what the teacher thinks is
important, and their general approach
• This is what simulations are for
• Absolutely critical to efficiency of Situated
Learning is reflection. Thinking about what you’re
learning.

5. VR as Situated Learning
Same situation, identically delivered across a
workforce.
Allows testing, uniform skill development, and faster
learning outcomes.
You can take voice notes & think about it later.

6. PRESENCE
First, what do we know about Doing VR?

7. REALLY ABOUT HCI:
HIGH-PRESENCE VR IS THE MOST HIGH-
BANDWIDTH HUMAN TO COMPUTER
COMMUNICATION CHANNEL

8. What & Why of Presence
• Presence is how we measure if you’re in VR,
vs. playing with a new video viewer.
• “Presence” is the sense of being there:
– How much is the illusion of VR really working?

9. What Happens When You’re “Present”
• You experience the artificial reality as if it were
real:
– The “Pit” – heart rates go up, GSR increases when
presented with a high elevation
– Subjects in an experience with high levels of perceived
presence more likely to be persuaded by health
messages
– Children report false memories
– Short people feel tall
– Etc.

10. Immersion is not the same as presence
• Presence is a psychological thing
– Am I committed to the experience?
• Immersion is a technological thing
– Is the outside world shut off, and the experience
compelling?

11. Mo-Cap is More Important Than
Massive Polygons
We care more about signs of sentience than
perfect rendering.
We’re used to recognizing people in bad light, or
with clothes on, or whatever. It’s not the face,
it’s the Gestalt.

12. Making You Feel Present:
Inducing Embodiment
Three top-level factors:
–the sense of self-location – ego centric view
is the baseline.
–the sense of agency – am I able to act?
–the sense of body ownership – do those
hands move where I tell them to?

13. Involvement Drives How Much VR
Tech Quality Matters
• When we care, we notice
quality
• The less we care, the less
the immersive environment
has to convince us for us to
just accept it.
Gaming is going to
be a high bar.
Communications,
co-presence often
less so.

14. Motion & Reactions as Key Cues
• We’re looking for signs
of sentience:
– Verbal response
– Natural-looking motion
– Emotional interaction
Think of Pixar:
Famous Voices lend
personality
Motion-capture
lends natural
reactions

15. High Presence, Low Recall?
• Some studies have used highly vivid
experiences, and found that recall was low
• Why?
• Just like overly stimulating advertising, the
experience can distract from the message.

16. Other People in There:
Co-Presence
• In addition to being there, you are there with others.
• What determines the sense that others are there?
• Not richness of tech, not really movement – mostly
what they say, how naturally they respond. Essentially
social cues.
– Several sessions have shown that users barely move their
avatars, preferring instead to focus on conversation.
• This ‘how they respond’ means the sense of being
perceived – that is a critical factor.

17. Avatars vs. Agents
• When thinking of instructional
tech – are we using a person or
a machine-generated
instructor?
• Avatar is representing a human
• Agent is representing a
machine
Blascovich theorizes that users
will respond as if the virtual
human is real no matter the
technical richness, as long as
they think there’s a real person
behind the avatar.
Studies show that:
1. Virtual Humans exert influence
if:
a. User believes the Virtual
Human is computer or
human controlled
b. Accuracy of behaviors
c. Realistic low-level
reactions
d. Situational relevance – do
they care?

18. Presence in “Classroom”
• We know that students who have the sole
attention of a teacher learn more.
• In VR, studies have shown that this works as
well – with the effect that everyone in the
class can have the sole attention of the
teacher.

19. Use of Space
• Several researchers have tested co-location as
a means to foster creativity.
• Space in VR doesn’t have to be ‘stateless’ we
can make notes in thin air, make personal
audio recordings. The whole environment can
be a toolkit.

20. Useful for both AR & VR:
Designing Space
• Three kinds of space around you:
– Peripersonal space – as far as your arms will go
– Extrapersonal space – as far as you’re likely to
walk/navigate
– Vista space – background
• Judging distances is hard. Several cues, like
shadows and brightness, don’t really work.
Perspective, and ‘motion parallax’ are most
effective for users.

21. PROTEUS EFFECT

22. Embodying a Different Body
What happens when you become:
• Taller = more confident
• Beautiful = more social
• Minority = more understanding, less biased
Proteus effect is robust – we take on, sometimes
for weeks, elements of the identities we inhabit
in VR

23. UNCANNY VALLEY

24. Hypothesis
As one makes artificial humans,
especially faces, more and more
real, there is a degree of ‘realness’
between completely real and
some midpoint, that users find
aversive.
Some research on the subject – VR
research I’ve found said that the
only uncanny valley effects found
were when features were
exaggerated or otherwise
disturbing on their own.

25. Avatars & Uncanny Valley
• Research on this will be
fraught, as a badly drawn
execution is just as aversive
as the ‘uncanniness’ of an
almost-human.
• Make avatars look as good
as we can, and make 100%
sure their reactions, and
movements, and voices all
sound human – as these
are not affected by the
‘twilight effect’.

26. Some Key Takeaways
• Presence is key to holding attention
• Creating presence is about embodiment – feeling you
“own” the thing in the virtual environment (VE)
• That comes from matching what your real body does
to what it sees in the virtual environment
• Richness of VE isn’t the key, richness of human
communication is
• Users will forgive all sorts of technical failings if the
voice at the other end sounds human, reacts
sentiently.

27. THANK YOU

28. LEARNING THEORY REVIEW

29. Cognitive Load
Intrinsic: Keep the tasks just beyond the
learner’s ability
Extraneous: Keep distraction to a minimum
Germane: Enable reflection on the material to
be learned

30. Extraneous Load
• Immersion, as described above, should enable
minimization of extraneous load
• Presence, as a psychological construct, should
further enable minimization of distractions &
extraneous load

31. Intrinsic Load
• More complex problems can be handled if
working memory tasks can be offloaded – by
storing thoughts in the environment.
• Notes, but multi-modal, sitting in thin air,
automatically stored or deleted.
• Relatedly, worked examples can be experienced.
• Faded examples can also be experienced vs. just
imagined.

32. Germane Load
• 100% environmental control allows for
pointers to appear, reminders to think about
the lesson to be learned vs. the task to be
completed.
• Exact duplicate of an experience can be
replicated, allowing learners to re-experience
a half-learned lesson.
• An immersive experience can be rewinded.

33. Expertise
• Multi-media learning studies have shown that
novices benefit from video & graphics,
whereas experts often find them distracting.
• VR, with it’s overwhelming experience of a
virtual world, can be expected to have an
enhanced version of this:
– Novices will learn faster when conceptual support
is helpful
– Experts will find it distracting

34. Multi-Media Learning Lessons Apply
• Don’t put text over graphics
• Use VO to add to graphic
• Use graphic instead of text whenever possible
• Avoid music
• Avoid accents
• Keep lessons short

35. Summary
• We know a fair bit about how to create good
VR Education.
• By this Sunday, I’ll have a 10-15 page
summary. Would love your input!
– I’ll post it on the meetup
• About 100+ papers went into this – they’ll all
be on a google drive I’ll share via meetup.

36. Thank you