Lecture 1 of the VR/AR class taught by Mark Billinghurst and Bruce Thomas at the University of South Australia. This lecture provides an introduction to VR and was taught on July 26th 2016.
3. Lecturers
â˘âŻ Mark Billinghurst
â˘âŻ Director of the Empathic Computing Lab
â˘âŻ Expert in AR, 3D user interfaces
â˘âŻ mark.billinghurst@unisa.edu.au
â˘âŻ Bruce Thomas
â˘âŻ Director of the Wearable Computing Lab
â˘âŻ Expert in AR/VR, wearable computing
â˘âŻ bruce.thomas@unisa.edu.au
â˘âŻ Teaching Assistants
â˘âŻ Tham Piumsomboon (Unity expert)
â˘âŻ Bradley Herbert (AR/Unity)
4. Class Logistics
â˘âŻ Weekly lecture (2 hrs)
â˘âŻ Tuesday 11am â 1pm
â˘âŻ Room F1-16
â˘âŻ Weekly Lab (1 hr)
â˘âŻ Tuesday 2-3pm
â˘âŻ Assessment
â˘âŻ 3 projects @ 20%, 30%, 40%
â˘âŻ Paper reading/class presentation @ 10%
â˘âŻ What you will need
â˘âŻ iOS or Android phone/tablet
â˘âŻ Google Cardboard VR display
7. Lecture Schedule â 13 Lectures
â˘âŻ July 26th : 1. Introduction - (no lab)
â˘âŻ Aug 2nd : 2. VR Technology â Intro to Unity
â˘âŻ Aug 9th : 3. VR Content Creation â Intro to Cardboard SDK
â˘âŻ Aug 16th : 4. VR Systems â Building VR scenes in Unity
â˘âŻ Aug 23rd : 5. 3D User Interfaces â Scripting in Unity
â˘âŻ Aug 30th : 6. Interaction Design â Navigation in VR
â˘âŻ Sept 6th : 7. VR Applications â Interaction in VR
â˘âŻ Sept 13th : 8. Introduction to AR â Interaction in VR
â˘âŻ Oct 4th : 9. AR Tracking â Building Apps using ARToolKit
â˘âŻ Oct 11th : 10. AR Interface Design â AR Tracking
â˘âŻ Oct 18th : 11. Mobile AR â Handheld AR Interaction
â˘âŻ Oct 25th : 12. AR Applications - Handheld AR Interaction
â˘âŻ Nov 1st : 13. Research Directions - (no lab)
8. Projects
â˘âŻ Assignment One â VR Scene â August 23rd
â˘âŻ Create a VR scene with at least 10 objects in it
â˘âŻ Include complex 3D models, texture, skybox, lighting, animations
â˘âŻ Deploy to Google Cardboard
â˘âŻ Assignment Two â VR Interaction â September 13th
â˘âŻ Add interactivity to the VR scene
â˘âŻ Movement navigation
â˘âŻ Ability to select and manipulation objects
â˘âŻ Assignment Three â AR Application â November 1st
â˘âŻ Handheld AR application
â˘âŻ Using image based tracking
â˘âŻ Screen based interaction techniques
9. What You Will Learn
â˘âŻ What Augmented Reality/Virtual Reality is
â˘âŻ History of AR/VR
â˘âŻ Current AR/VR commercial market
â˘âŻ Different AR/VR applications
â˘âŻ Human perception side of VR/AR
â˘âŻ AR/VR technology
â˘âŻ 3D user interface guidelines
â˘âŻ How to design good AR/VR experiences
â˘âŻ How to build your own AR/VR applications
â˘âŻ Important directions for future research in AR/VR
13. The Ultimate Display
âThe ultimate display would, of course, be a room
within which the computer can control the
existence of matter. A chair displayed in such a
room would be good enough to sit in. Handcuffs
displayed in such a room would be confining, and
a bullet displayed in such a room would be fatalâ.
Ivan Sutherland, 1965
17. Trend Towards Invisible Interfaces
â˘âŻTrend from room scale to invisible computing
â˘âŻMaking Computers Invisible
â˘âŻhide the computer in the real world
â˘âŻ Ubiquitous Computing
â˘âŻput the user inside the computer
â˘âŻ Virtual Reality
18. Making Interfaces Invisible
Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented
interaction with real world environments. In Proceedings of the 8th Annual ACM Symposium on
User interface and Software Technology. UIST '95. ACM, New York, NY, 29-36.
26. Other Definitions
Virtual reality is..
a computer technology that replicates an environment, real
or imagined, and simulates a user's physical presence and
environment to allow for user interaction. (Wikipedia)
electronic simulations of environments experienced via
head mounted eye goggles and wired clothing enabling the
end user to interact in realistic three-dimensional situations.
(Coates, 1992)
an alternate world filled with computer-generated images
that respond to human movements. (Greenbaum, 1992)
an interactive, immersive experience generated by a
computer (Pimental 1995)
27. Key Characteristics for VR
â˘âŻVirtual Reality has three key characteristics
â˘âŻ3D stereoscopic display
â˘âŻWide field of view display
â˘âŻLow latency head tracking
â˘âŻWhen these three things are combined they
provide a compelling immersive experience
30. Defined in Terms of Presence
â˘âŻ Presence is the key to defining VR in terms of experience
â˘âŻ Presence is defined as the sense of being in an environment
â˘âŻ Telepresence is defined as the experience of presence in an
environment by means of a communication medium.
â˘âŻ A âvirtual realityâ is defined as a real or simulated environment
in which a perceiver experiences telepresence.
31. David Zeltzerâs AIP Cube
!âŻAutonomy â User can to
react to events and stimuli.
!âŻInteraction â User can
interact with objects and
environment.
!âŻPresence â User feels
immersed through sensory
input and output channels
Interaction
Autonomy
Presence
VR
Zeltzer, D. (1992). Autonomy, interaction, and presence. Presence: Teleoperators
& Virtual Environments, 1(1), 127-132.
32. Defined in Terms of User Experience
Inducing targeted behavior in an organism by using artificial
sensory stimulation, while the organism has little or no
awareness of the interference. (LaValle 2016)
â˘âŻ Targeted Behavior: Having an experience designed by
another person.
â˘âŻ Organism: Person or other life form
â˘âŻ Artificial Sensory Stimulation: One or more sensors
replaced by artificial means
â˘âŻ Awareness: Organism unaware of the interface
33. Virtuix Omni â Human on VR Treadmill
â˘âŻ https://www.youtube.com/watch?v=aTtfAQEeAJI
34. Fly on Treadmill â Insect VR
â˘âŻ Fruit fly on treadmill and VR display
â˘âŻ https://www.youtube.com/watch?v=1ezL8nGo--I
36. Augmented Reality Definition
â˘âŻDefining Characteristics [Azuma 97]
â˘âŻCombines Real andVirtual Images
â˘âŻBoth can be seen at the same time
â˘âŻInteractive in real-time
â˘âŻThe virtual content can be interacted with
â˘âŻRegistered in 3D
â˘âŻVirtual objects appear fixed in space
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
41. Making Interfaces Invisible
Rekimoto, J. and Nagao, K. 1995. The world through the computer: computer augmented
interaction with real world environments. In Proceedings of the 8th Annual ACM Symposium on
User interface and Software Technology. UIST '95. ACM, New York, NY, 29-36.
43. Milgramâs Reality-Virtuality continuum
Mixed Reality
Reality - Virtuality (RV) Continuum
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
"...anywhere between the extrema of the virtuality continuum."
P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays
IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
45. Other Context: The Metaverse
â˘âŻNeal Stephensonâs âSnowCrashâ
â˘âŻThe Metaverse is the convergence of:
â˘âŻ1) virtually enhanced physical reality
â˘âŻ2) physically persistent virtual space
â˘âŻMetaverse Roadmap
â˘âŻhttp://metaverseroadmap.org/
46. Metaverse Dimensions
⢠Augmentation technologies that layer information
onto our perception of the physical environment.
⢠Simulation refers to technologies that model
reality
⢠Intimate technologies are focused inwardly, on the
identity and actions of the individual or object;
⢠External technologies are focused outwardly,
towards the world at large;
49. MirrorWorlds
â˘âŻMirror worlds are informationally-enhanced virtual
models of the physical world.
â˘âŻ Google Earth, MS StreetView, Google Maps
53. Summary
â˘âŻVirtual Reality can be defined in a number of ways
â˘âŻIn terms of technology
â˘âŻFrom a Presence perspective
â˘âŻIn terms of User Experience
â˘âŻVR can also be classified with other technologies
â˘âŻInvisible Interfaces
â˘âŻMilgramâs Mixed Reality continuum
â˘âŻMetaVerse
56. When anything new comes along, everyone,
like a child discovering the world thinks that
theyâve invented it, but you scratch a little
and you find a caveman scratching on a wall
is creating virtual reality in a sense.
Morton Helig (Hammit 1993)
57. Early History (30,000 BC - )
The history of VR is rooted in humanâs first
attempts to reproduce the world around them
61. 3D Cinema Golden Era (1950-60s)
â˘âŻ Polarized 3D projection or anaglyph (red/blue)
62. 1900s â Interactive Experiences
â˘âŻEarly Simulators (<1960s)
â˘âŻ Flight simulation
â˘âŻ Sensorama (1955)
â˘âŻEarly HMDs (1960s)
â˘âŻ Philco, Ivan Sutherland
â˘âŻMilitary + University Research (1970-80s)
â˘âŻ US Airforce, NASA, MIT, UNC
â˘âŻFirst Commercial Wave (1980-90s)
â˘âŻ VPL, Virtual i-O, Division, Virtuality
â˘âŻ VR Arcades, Virtual Boy
63. Pygmalion's Spectacles (1935)
â˘âŻ Stanley Weinbaum Short Story
â˘âŻ Main character, Dan Burke, tries
a pair of goggles
The goggle provide: "a movie that gives one sight and
sound [...] taste, smell, and touch. [...] You are in the story,
you speak to the shadows (characters) and they reply,
and instead of being on a screen, the story is all about
you, and you are in it."
64. Link Trainer (1929 â 1950s)
â˘âŻ Flight Simulator Training
â˘âŻ Full six degree of freedom rotation
â˘âŻ Force feedback and motion control
â˘âŻ Simulated instruments
â˘âŻ Modeling common flight conditions
â˘âŻ Over 500,000 pilots trained
65. Link Trainer Video (1966)
â˘âŻ https://www.youtube.com/watch?v=MEKkVg9NqGM
69. First HMD (1961)
â˘âŻ Philco Headsightâ Remote Camera Viewing
â˘âŻ Components â HMD,closed circuit television
â˘âŻ Custom magnetic tracking for head orientation
72. Super Cockpit (1965-80âs)
â˘âŻ US Airforce Research Program
â˘âŻ Wright Patterson Air Force Base
â˘âŻ Tom Furness III
â˘âŻ Multisensory
â˘âŻ Visual, auditory, tactile
â˘âŻ Head, eye, speech, and hand input
â˘âŻ Addressing pilot information overload
â˘âŻ Flight controls and tasks too complicated
â˘âŻ Research only
â˘âŻ big system, not safe for ejecting
73.
74. UNC Haptic Systems (1967 â 80âs)
â˘âŻ Haptic/kinesthetic display system
â˘âŻ 6D force fields of molecular structures
â˘âŻ Progression
â˘âŻ Grope I, simple fields, particle feedback
â˘âŻ Grope II, 1978, childrenâs building blocks
â˘âŻ Grope III, late 80âs, Remote Manipulator
â˘âŻ Sarcos arm
76. Aspen Movie Map (1978)
â˘âŻ Andrew Lippman, MIT
â˘âŻ Car driven through Aspen
â˘âŻ 4 stop motion cameras
â˘âŻ Pictures every 10 feet
â˘âŻ Interactive playback
â˘âŻ Played from laserdisc, touch screen
â˘âŻ People able to navigation in any direction
â˘âŻ Map interface
â˘âŻ 3D graphics overlay
â˘âŻ Early version of Google StreetView
77. Aspen Movie Map Demo
â˘âŻ https://www.youtube.com/watch?v=X5Oj4S_x2Cc
78. VideoPlace (1975-80âs)
â˘âŻ Myron Krueger
â˘âŻ Graphics/gesture recognition
â˘âŻ Real time interaction
â˘âŻ VideoDesk
â˘âŻ camera captures gestures
â˘âŻ relays to remote collaborator
â˘âŻ gestures control graphics
â˘âŻ paint, draw, menu selection
â˘âŻ Wrote Book â Artificial Reality 2
80. LEEP Optics (1979)
â˘âŻ Large Expanse, Extra Perspective optics
â˘âŻ Developed by Eric Howlett
â˘âŻ Lens design for extremely wide field of view
â˘âŻ High resolution in centre, lower resolution in periphery
â˘âŻ 90o direct FOV, 140o corneal FOV
â˘âŻ Used as basis for most VR HMDs
88. VPL Research (1985 â 1999)
â˘âŻFirst Commercial VR Company
â˘âŻ Jaron Lanier, Jean-Jacques Grimaud
â˘âŻProvided complete systems
â˘âŻ Displays, software, gloves, etc
â˘âŻ DataGlove, EyePhone, AudioSphere
89. The University of North Carolina
at Chapel Hill (1980s-1990s)
8
9
Head-Mounted Displays
Tracking, Haptics, Applications
90. University of Washington (1989 - )
â˘âŻ Human Interface Technology Laboratory (HIT Lab)
â˘âŻ Founded by Tom Furness III
â˘âŻ Many AR/VR Innovations
â˘âŻ Virtual Retinal Display
â˘âŻ ARToolKit AR Tracking library
â˘âŻ GreenSpace shared VR experience
â˘âŻ VR and pain care
â˘âŻ VR and Education
91. TheVirtual Retinal Display (1991)
â˘âŻImage scanned onto retina
â˘âŻCommercialized through Microvision
â˘âŻ Nomad System - www.mvis.com
â˘âŻLead to MagicLeap technology
93. VR Learning in Lawnmower Man
â˘âŻ https://www.youtube.com/watch?v=zTrgHXNAs24
94. CAVE (1992)
â˘âŻ Projection VR system
â˘âŻ 3-6 wall stereo projection, viewpoint tracking
â˘âŻ Developed at EVL, University of Illinois Chicago
â˘âŻ Commercialized by Mechdyne Corporation(1996)
C. Cruz-Neira, D. J. Sandin, T. A. DeFanti, R. V. Kenyon and J. C. Hart. "The CAVE: Audio Visual
Experience Automatic Virtual Environment", Communications of the ACM, vol. 35(6), 1992, pp. 64â72.
98. MyVR Predictions (1996)
â˘âŻI knew everyone would useVR when:
â˘âŻHMDs were cheap (<$500)
â˘âŻComputers generate millions of polys/sec
â˘âŻTracking was inexpensive
â˘âŻGood 3D input devices
101. Overview of VR in the 1990âs
https://www.youtube.com/watch?v=tdAaU0CRHng&feature=youtu.be&t=1m51s
102.
103. â˘âŻApril 2007 Computer World
â˘âŻVRVoted 7th on list of 21 biggest technology flops
â˘âŻ MS Bob #1
104. Lessons Learned
â˘âŻDonât believe the hype
â˘âŻNot everything is better inVR
â˘âŻMany factors determine technology acceptance
â˘âŻHuman Centered Design/Design for users
â˘âŻNeed to move from Demo to Production
â˘âŻProfitable niche markets first
â˘âŻFollow the money
125. Why 2016 wonât be like 1996
â˘âŻItâs not just VR anymore
â˘âŻHuge amount of investment
â˘âŻInexpensive hardware platforms
â˘âŻEasy to use content creation tools
â˘âŻNew devices for input and output
â˘âŻProven use cases â no more Hype!
â˘âŻMost important: Focus on User Experience
126. Conclusion
â˘âŻ Virtual Reality has a long history
â˘âŻ > 50 years of HMDs, simulators
â˘âŻ Key elements for VR were in place by early 1990âs
â˘âŻ Displays, tracking, input, graphics
â˘âŻ Strong support from military, government, universities
â˘âŻ First commercial wave failed in late 1990âs
â˘âŻ Too expensive, bad user experience, poor technology, etc
â˘âŻ We are now in second commercial wave
â˘âŻ Better experience, Affordable hardware
â˘âŻ Large commercial investment, Significant installed user base
â˘âŻ Will Virtual Reality be a commercial success this time?