Creating Connection Using AR and Empathic Computing
•
9 likes•4,224 views
This document discusses how augmented and virtual reality technologies can be used to create more empathetic and collaborative experiences. It outlines trends in content capture, networking bandwidth, and natural interfaces that enable new types of shared experiences. Examples are presented of past and current AR/VR systems that allow remote users to share live video, 3D spaces, gestures, and physiological cues like gaze and expression. The document concludes that AR and VR are well-suited for developing empathetic computing applications by allowing users to understand, experience, and share perspectives and emotions.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (14)
Similar to Creating Connection Using AR and Empathic Computing
Similar to Creating Connection Using AR and Empathic Computing (20)
More from Mark Billinghurst
More from Mark Billinghurst (20)
Recently uploaded
Recently uploaded (20)
Creating Connection Using AR and Empathic Computing
- 1. FIFTY SHADES OF AUGMENTED REALITY: CREATING CONNECTION USING AR Mark Billinghurst mark.billinghurst@unisa.edu.au March 24th 2017 Laval Virtual
- 4. • jg
- 7. Evolution of Communication Tools
- 9. Modern Technology Trends 1. Improved Content Capture • Move from sharing faces to sharing places 2. Increased Network Bandwidth • Sharing natural communication cues 3. Implicit Understanding • Recognizing behaviour and emotion These three trends combined with AR/MR technology can create new types of collaborative experiences
- 10. Content Capture Realism Time Photo Film Live Video Panorama 360 Video 3D Space 1850 1900 1940 1990 2000 2010
- 11. Capturing Reality • Panoramas (1790s) • Immersive paintings • Photography (1820-30s) • Oldest surviving photo (Niépce, 1826) • Stereo imagery (1830s) • Wheatstone (1832) • Brewster (1851) • Movies (1870s) • Muybridge (1878) • Roundhay Garden Scene (1888)
- 12. Sharing Faces to Sharing Spaces • Moving from desktop to handheld cameras
- 13. Capturing 360 images Kodak 360 Fly 360 Gear 360 Theta S Nikon LG 360 Pointgrey Ladybug Panono 360 Bublcam
- 14. 360 Stereo Capture Samsung Project Beyond Jaunt VR camera https://www.youtube.com/watch?v=FnBIvq8jzds
- 15. 3D Space Capture Google Project Tango Occipital Structure Sensor https://www.youtube.com/watch?v=XA7FMoNAK9M
- 16. Content Capture Realism Time Photo Film Live Video Panorama 360 Video 3D Space 1850 1900 1940 1990 2000 2010 2D Static Immersive Live Experience
- 17. Networking Speeds Log (b/s) Time 100 b/s 10 Kb/s 1 Mb/s 1980 1985 1990 1995 2000 2010 100 Mb/s 2005
- 18. Networking Technology • Telephone (1876) • Bell Picture Phone (1955) • B+W live video calls • Mobile Phones (1985) • Desktop Video Conferencing (1990’s) • Netmeeting, Skype https://www.youtube.com/watch?v=OBkcYyenNBM
- 19. Mobile Conferencing (2010-) • Video conferencing • Skype, facebook • Live broadcasting • Meerkat, Periscope • Support for 360 video
- 21. Networking Speeds Log (b/s) Time 100 b/s 10 Kb/s 1 Mb/s 1980 1985 1990 1995 2000 2010 100 Mb/s 2005 Text Audio Natural Video
- 22. Interaction Technology Natural Time Punch Card Keyboard Mouse Speech Gesture Emotion 1950 1960 1980 1990 2000 2010 Thought
- 23. Natural Gesture Input • Freehand input • Depth sensors, rich two handed gestures Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Leichter, D. K. C. R. I., ... & Izadi, S. (2015, April). Accurate, Robust, and Flexible Real-time Hand Tracking. In Proc. CHI (Vol. 8). https://www.youtube.com/watch?v=A-xXrMpOHyc
- 25. Interaction Technology Natural Time Punch Card Keyboard Mouse Speech Gesture Emotion 1950 1960 1980 1990 2000 2010 Thought Implicit Explicit
- 27. Milgram’s Reality-Virtuality Continuum Mixed Reality Reality - Virtuality (RV) Continuum Real Environment Augmented Reality (AR) Augmented Virtuality (AV) Virtual Environment "...anywhere between the extrema of the virtuality continuum." P. Milgram and A. F. Kishino, Taxonomy of Mixed Reality Visual Displays IEICE Transactions on Information and Systems, E77-D(12), pp. 1321-1329, 1994.
- 28. Mixed Reality Conferencing in SciFi
- 29. AR Video Conferencing (2001) • Bringing conferencing into real world • Using AR video textures of remote people • Attaching AR video to real objects Billinghurst, M., & Kato, H. (2002). Collaborative augmented reality. Communications of the ACM, 45(7), 64-70.
- 30. Demo: AR Video Conferencing
- 31. MR Point Cloud Sharing (2016) • Use video-see through HMD • Occulus Rift • Depth camera on HMD • Softkinetic • Create point cloud • Remote hand gestures • User’s space
- 32. Demo: Point Cloud Capture
- 33. Holoportation (2016) • Augmented Reality + 3D capture + high bandwidth • http://research.microsoft.com/en-us/projects/holoportation/
- 35. Social VR • Facebook Social Virtual Reality, AltspaceVR • Bringing Avatars into VR space • Natural social interaction
- 36. Demo: Facebook Social VR (2016) https://www.youtube.com/watch?v=yxHwWHHg4Vs
- 37. AR/VR Collaboration (2017) • Make 3D copy of real space • AR user in real world, VR user in 3D copy of real space • Hololens for AR user, HTC Vive for VR user • Share virtual body cues (head, hands, gaze information) Real World Virtual World
- 38. Virtual Cues • AR/VR displays • Gesture input • Leap Motion • Room scale tracking
- 43. Empathy “Seeing with the Eyes of another, Listening with the Ears of another, and Feeling with the Heart of another..” Alfred Adler
- 44. Empathic Computing Systems that allow us to share what we are seeing, hearing and feeling with others..
- 45. Example: Google Glass • Camera + Processing + Display + Connectivity • Ego-Vision Collaboration (But with FixedView)
- 46. Current Collaboration onWearables • First person remote conferencing/hangouts • Limitations • Single POV, no spatial cues, no annotations, etc
- 47. Social Panoramas (ISMAR 2014) • Capture and share social spaces in real time • Supports independent views into Panorama Reichherzer, C., Nassani, A., & Billinghurst, M. (2014, September). [Poster] Social panoramas using wearable computers. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on (pp. 303-304). IEEE.
- 48. Implementation • Google Glass • Capture live image panorama (compass + camera) • Remote device (tablet) • Immersive viewing, live annotation
- 49. User Interfaces Glass View Tablet View
- 51. JackIn – Live Immersive Video Streaming • Jun Rekimoto – University of Tokyo/Sony CSL
- 52. JackIn Hardware • Wide angle cameras – 360 degree video capture • Live video stitching
- 54. • zxcz
- 55. JospehTame – Tokyo Marathon • Live streaming from Tokyo marathon • http://josephta.me/en/tokyo-marathon/
- 56. Empathy Glasses (CHI 2016) • Combine together eye-tracking, display, face expression • Impicit cues – eye gaze, face expression ++ Pupil Labs Epson BT-200 AffectiveWear Masai, K., Sugimoto, M., Kunze, K., & Billinghurst, M. (2016, May). Empathy Glasses. In Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM.
- 57. AffectiveWear – Emotion Glasses • Photo sensors to recognize expression • User calibration • Machine learning • Recognizing 8 face expressions
- 58. Empathy Glasses in Use • Eye gaze pointer and remote pointing • Face expression display • In future integrated eye-tracking/display
- 60. Empathic VR Environments (2017) • Player and Viewer • Viewer slaved to player • Share emotional signals • Heart rate, GSR • Remote affect measuring
- 61. Sharing VR Experiences • HTC Vive HMD • Empathic glove • Empatica E4
- 62. VR Environments • Butterfly World: calm scene, collect butterflies • Zombie Attack: scary scene, fighting zombies
- 64. AR and VR for Empathic Computing • VR systems are ideal for trying experiences: • Strong story telling medium • Provide total immersion/3D experience • Easy to change virtual body scale and representation • AR systems are idea for live sharing: • Allow overlay on real world view/can share viewpoints • Support remote annotation/communication • Enhance real world task
- 65. CONCLUSIONS
- 66. Conclusions •Trend towards Empathic Computing • Understanding, Experiencing, Sharing •AR/VR Enables Empathic Experiences • Changes perspective • Sharing space/experience • Supports annotation/communication •Many directions for future research