The document discusses designing immersive experiences using mixed reality. It begins by defining mixed reality as bringing together physical and digital worlds. It then discusses environment perception with the HoloLens mixed reality headset, which uses spatial mapping and understanding. Example use cases by industry are provided, including real estate, retail, transportation, construction, and healthcare. The document emphasizes optimizing the user experience by considering factors like application type, environment, user space, lighting, and ambient sound. It provides guidelines for optimal hologram placement and user comfort zones. Finally, it outlines tools and a three step process for getting started developing mixed reality applications.
2. | Designing Immersive Experiences with Mixed Reality
What We’ll Cover
• What is Mixed Reality……………………………………………………………………….3
• Environment Perception
• Meet the HoloLens
• Types of Mixed Reality Applications………………………………………………………8
• Interaction Design
• Example Use Cases by Industry
• Optimizing the User Experience………………………………………………………..…13
• Important Questions to Ask
• Optimal Viewing / Comfort Zones
• Tools…………………………………………………………………………………………...17
• Tools To Help You Get Started
• Get Started in Three Simple Steps
| 2
4. | Designing Immersive Experiences with Mixed Reality
Human-Computer Interaction Hasn’t Changed Much
| 4
1980-2000 2000-Present 2008-Present
5. | Designing Immersive Experiences with Mixed Reality
| 5
Computer
Human
Environment
Mixed
Reality
Perception
Interaction
Conventional
Reality
Mixed reality brings people,
places, and objects from
your physical and digital
worlds together.
Advantages of Mixed Reality
9. | Designing Immersive Experiences with Mixed Reality
Enhanced vs. Blended Environment
| 9
10. | Designing Immersive Experiences with Mixed Reality
Example Use Cases by Industry
| 10
Real Estate
• Home and building tours
• Safety and inspections tools
• Interior design visualization
Retail
• Space planning
• Try-before-you-buy
• Installation / repair support
Transportation
• Heads-up display navigation
• Safety and maintenance tools
• Various conditions simulations
Construction
• Visualize & convey complex 3D data
• Collaborate with remote teams
• Perform assisted safety inspections
Healthcare
• Professional training simulations
• 3D Holographic Medical Data
• Helping to overcome phobias
Energy
• Safety and inspections tools
• Site Visualization
• Drilling and Exploration tools
11. | Designing Immersive Experiences with Mixed Reality
Immersive Experiences Consist Of
Spatial Sound / Voice Input
Spatial sound is an enhanced immersive
audio experience where sounds can flow
around you in three-dimensional virtual
space. Devices such as the HoloLens also
receive voice input allowing users to perform
tasks with simple verbal commands.
| 11
12. | Designing Immersive Experiences with Mixed Reality
Understanding Mixed Reality User Interactions
| 12
The Device’s… which… allows users to…
Camera
Detects Gestures Interact with Holograms, perform actions
Spatial maps your environment Place holograms around them that interact with real-world objects
Includes media capturing Take pictures and record video
Microphone
Interprets Voice Commands Perform tasks hands-free
Captures audio recordings Capture and store voice notes
Speaker
Relays audio Have contextual guidance within an app
Plays back sound Know when they’ve successfully or unsuccessfully completed a task
Provides Spatial Audio Have a enriched experience by 3D sounds with environment
Projection Projects holographic content Have holograms appear in their real-world environment
IMU - inertial measurement unit Tracks head movements / Gaze Targets Focus on objects and show intent
Gamepad Controls device via hand-held remote Have another option if they can’t operate the device through gestures
14. | Designing Immersive Experiences with Mixed Reality
Important Questions to Ask
1. Is this an enhanced or blended environment app (see slide 9)?
2. What environment will the app be used in?
3. How much space will the user have to operate in (both in width, depth and height)?
4. What light conditions will the device be subject to?
5. Will there be any ambient sound/noise that the app will have to compete with?
6. Will the user need to be hands-free or benefit from voice input/command?
| 14
15. | Designing Immersive Experiences with Mixed Reality
Optimal Viewing Zone - between 1.25m and 5m
5m 2m
Optimal Zone
Ideal Hologram Placement
1.25m
16. | Designing Immersive Experiences with Mixed Reality
Optimal Comfort Zone – Neck Flexion
Upper region
0 to 10 degrees above horizon
Optimal region
0 to 35 degrees above horizon
Lower region
35 to 60 degrees above horizon
Extension above the upper
region (looking up) beyond
the highlighted area is
uncomfortable and limited
due to eye and neck
morphology.
18. | Designing Immersive Experiences with Mixed Reality
Tools To Help You Get Started
| 18
Website
Website
Website
Website
19. | Designing Immersive Experiences with Mixed Reality
Get Started in Three Simple Steps
| 19
this check list from Microsoft VS and deploy
20. | Designing Immersive Experiences with Mixed Reality
About the Author
| 20
Adam Wisniewski is UX Designer and Founder of EX3 Labs, an
experience design firm based in Chicago. He specializes in
product strategy, user research, interaction design, rapid
prototyping, interface design, and development. Adam is
passionate about designing user-centered product experiences
that solve big problems for individuals, communities and
businesses.
Adam founded the Chicago HoloLens Meetup group with the
goal of educating his community around mixed reality. In 2016,
he launched the Immersive Learning Lab Mission, where he
committed to donating a percentage of EX3 Labs’ revenue to
bringing mixed and augmented reality devices and immersive
learning content to public schools. adamwiz
@adamwiz
21. | Designing Immersive Experiences with Mixed Reality
Citations
Windows Mixed Reality documentation
https://docs.microsoft.com/en-us/windows/mixed-reality/
Wikipedia Contributors
https://en.wikipedia.org/wiki/Microsoft_HoloLens
Tools Checklist
https://docs.microsoft.com/en-us/windows/mixed-reality/install-the-tools
Deploying your first HoloLens App
https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-100
| 21
