A talk from the Creator Track at AWE USA 2019 - the World's #1 XR Conference & Expo in Santa Clara, California May 29-31, 2019. Hannah Nye (Design Interactive, Inc.): We are No Longer Confined to 2D User Interfaces – So Design Like it! Spatial AR has provided an entirely new canvas for designers. No longer encumbered by a 2 dimensional window, UX designers are free to explore a 3 dimensional space. Yet, designers seem confined to traditional UI elements, influenced by standards for web or mobile applications even when we know the potential for spatial AR to present information not just when we need it, but where. Inspired by the concept of embodied perception, Hannah Nye will teach attendee’s DI’s process for breaking old design paradigms. She will teach designers how to influence user action and improve performance within enterprise AR maintenance and medical AR training experiences. This tutorial will be approximately 50 minutes. https://awexr.com

1. XR DESIGN | DESIGN INTERACTIVE
We are No Longer Confined
to 2D User Interfaces – So
Design Like it!
Hannah Nye
SR XR UI/UX Designer
Design Interactive

2. XR DESIGN | DESIGN INTERACTIVE

3. XR DESIGN | DESIGN INTERACTIVE
About
Me

4. XR DESIGN | DESIGN INTERACTIVE
Utility
Usability
Desirability
Process

5. XR DESIGN | DESIGN INTERACTIVE
Utility
• Why XR?
• Contextualize
• Determine the Tech
Usability
• World Space
• Navigation
• Affordance & Symbolic
Language
Desirability
• Immersion & Presence
• Embodied Perception
• Natural Interaction
Process
• Audio
• Scripts
• Video Storyboards
• Spatial Layouts

6. XR DESIGN | DESIGN INTERACTIVE
• Why XR?
• Contextualize
• Determine the Tech
Utility

7. XR DESIGN | DESIGN INTERACTIVE
• What’s the goal?
• Who’s the user?
• Does it NEED to be in XR?
• Provide context
• AR: Integrate with real-world space
• VR: Determine use of virtual world
Why XR?
Utility / Usability / Desirability / Process

8. XR DESIGN | DESIGN INTERACTIVE
Contextualize
Utility / Usability / Desirability / Process

9. XR DESIGN | DESIGN INTERACTIVE
The system must be designed to support the human in performing specific
tasks in a particular context.
Utility / Usability / Desirability / Process

10. XR DESIGN | DESIGN INTERACTIVE
• What is the best XR tech?
• Does the user need to see
their body?
Determining
the Tech
Utility / Usability / Desirability / Process

11. XR DESIGN | DESIGN INTERACTIVE
• World Space
• Navigation
• Affordance & Symbolic
Language
Usability

12. XR DESIGN | DESIGN INTERACTIVE
• What makes it actually
usable for the user?Picture here
Beat Saber
Utility / Usability / Desirability / Process

13. XR DESIGN | DESIGN INTERACTIVE
• World Space
• User Space
Consider
the Space
Utility / Usability / Desirability / Process

14. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process

15. XR DESIGN | DESIGN INTERACTIVE
External Factors
World Space
Size
Utility / Usability / Desirability / Process
Dynamic +
Contextually
Aware System
User defined
boundaries

16. XR DESIGN | DESIGN INTERACTIVE
World Space
Size
Utility / Usability / Desirability / Process

17. XR DESIGN | DESIGN INTERACTIVE
World Space
External Factors
Utility / Usability / Desirability / Process

18. XR DESIGN | DESIGN INTERACTIVE
• Area within arm's reach
• Needs to be physically open
• Critical information likely
needs to be placed here
User Space
Personal
Utility / Usability / Desirability / Process

19. XR DESIGN | DESIGN INTERACTIVE
• Area of user’s public action
• The available walking space
needs to be parallel with the
available physical space
• Provide obvious area
boundaries
User Space
Action
Utility / Usability / Desirability / Process

20. XR DESIGN | DESIGN INTERACTIVE
• Area 30 meters out, limited
interaction zone
• Objects should look further
away.
• Difference in motion, height,
relative size difference to
closer objects
User Space
Vista
Utility / Usability / Desirability / Process

21. XR DESIGN | DESIGN INTERACTIVE
Navigation
Utility / Usability / Desirability / Process

22. XR DESIGN | DESIGN INTERACTIVE
Affordance
• Affordance
• How do I inherently
understand something?
Utility / Usability / Desirability / Process

23. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process

24. XR DESIGN | DESIGN INTERACTIVE
• Engagement
• Natural Interaction
• Perceived User ControlDesirability

25. XR DESIGN | DESIGN INTERACTIVE
Engagement Immersion Presence
Utility / Usability / Desirability / Process
Activity Loop
Meaningful
Activities
Symbolic, Novel,
Natural

26. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process

27. XR DESIGN | DESIGN INTERACTIVE
Point of Engagement / Re-
engagement
Engagement Disengagement
Activities Accomplishments Affect

28. XR DESIGN | DESIGN INTERACTIVE
Engagement
Point of Engagement / Re-
engagement
Disengagement
Intrinsic Motivation
Absence of meaningful information
(builds presence)
Activities Accomplishments Affect

29. XR DESIGN | DESIGN INTERACTIVE
Meaningful
Information
Utility / Usability / Desirability / Process

30. XR DESIGN | DESIGN INTERACTIVE
Natural
Interactions
• Embodied Perception
Leap Motion
Utility / Usability / Desirability / Process

31. XR DESIGN | DESIGN INTERACTIVE
Perceived User
Control
• Decision making
• Encourage flexibility
• Control over
inconsequential
Utility / Usability / Desirability / Process

32. XR DESIGN | DESIGN INTERACTIVE
• Audio
• Scripts
• Video Storyboards
• Spatial Layouts
Process

33. XR DESIGN | DESIGN INTERACTIVE
Audio is half the
experience
Utility / Usability / Desirability / Process

34. XR DESIGN | DESIGN INTERACTIVE
Audio is half the
experience
Process

35. XR DESIGN | DESIGN INTERACTIVE
"Sound is what truly convinces the mind is in a place, in other words
hearing is believing"
Jesse Schell, The Art of Game Design
"Films are 50 percent visual and 50 percent sound. Sometimes
sound even overplays the visual”.
David Lynch, Director
“The sound and music are 50% of the entertainment in a movie”.
George Lucas, Director
Utility / Usability / Desirability / Process

36. XR DESIGN | DESIGN INTERACTIVE
Audio
• How do we include it?
• Determine the mood that a
certain portion in the
system is trying to
represent
• Audio helps to create a
symbolic language
Utility / Usability / Desirability / Process

37. XR DESIGN | DESIGN INTERACTIVE
Scripts
• Start with your normal task
flows
• Create a use case
• Flesh it out
Utility / Usability / Desirability / Process

38. XR DESIGN | DESIGN INTERACTIVE
Scripts
Utility / Usability / Desirability / Process

39. XR DESIGN | DESIGN INTERACTIVE

40. XR DESIGN | DESIGN INTERACTIVE
Video
Storyboards
• What is the user seeing right
now?
• Are they actually looking there?
• How do I make them look there?
• First person view
• Third person view
• Spatial map
Utility / Usability / Desirability / Process

41. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process
Auggie: “Hi there, I’m Auggie. I’ll be here
to help you learn about TC3 – Tactical
Combat Casualty Care.”
Audio: Start up sound
Animation: Visible text of Auggie speech
Auggie: “To get started, press or say
“Start Scenario”.
Audio: Pleasant BG music
Animation: Start button appears
Interaction: User selects button or says
“Start Scenario”
Auggie: “In the theatre of war, tension
pneumothorax is the second leading
cause of preventable death”.
Audio: War, military music with drums
Animation: Video of military members in
the field.

42. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process
Auggie: “Accounting for 33% of all
primary causes of preventable death.”
Audio: War, military music with drums
Animation: Pie chart grows with 33% on
top.
Auggie: “If not identified and treated
promptly, ”.
Audio: War, military music with drums.
Heartbeat sound
Animation: Heart appears where the
heart would be on mannequin
Auggie: “- shock and cardiac arrest
may occur. ”.
Audio: War, military music with
drums. Heartbeat sound
Animation: Heart starts to glow red.

43. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process
Auggie: “I’ve uploaded a virtual
collapsed lung in front of you.”
Audio: Pleasant BG music
Animation: Lungs appear
Auggie: “Take a closer look at it by
rotating and enlarging it.”
Audio: Pleasant BG music
Animation: Auggie shows how to
rotate/enlarge
Auggie:
Audio: Pleasant BG music
Interaction: user rotates/enlarges
lung

44. XR DESIGN | DESIGN INTERACTIVE
Test Spatial
layout samples
Process

45. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process
Test
Spatial
Layout

46. XR DESIGN | DESIGN INTERACTIVE
Utility / Usability / Desirability / Process
Test
Spatial
Layout

47. XR DESIGN | DESIGN INTERACTIVE
Microsoft Marquette
WiarFrame
AugMentor

48. XR DESIGN | DESIGN INTERACTIVE
?

49. XR DESIGN | DESIGN INTERACTIVE
Questions?
Thank you!