Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Touching More than 3 Dimensions Research Into Novel Interfaces – three dimensions and above

Work on interactive tabletops and surfaces has focused mostly on two-dimensional issues, such as multi-finger gestures and tangible interaction. Interesting as it is, however this picture is missing several dimensions. I will describe work on 2D and 3D semi-immersive environments and present novel on-and-above-the-surface techniques based on bi-manual models to take advantage of the continuous interaction space for creating and editing 3D models in stereoscopic environments. I will also discuss means to allow for more expressive interactions, including novel uses of sound and combining hand and finger tracking in the space above the table with multitouch gestures on its surface continuously. These combinations can provide alternative design environments and allow novel interaction modalities.

  • Be the first to comment

Touching More than 3 Dimensions Research Into Novel Interfaces – three dimensions and above

  1. 1. Touching More than 3 Dimensions Research Into Novel Interfaces – three dimensions and above Joaquim Jorge INESC-ID & IST / U Lisboa Portugal IP / Leiria Nov 2015
  2. 2. Joaquim Jorge Instituto Superior Técnico Universidade de Lisboa Visualization and Multimodal Interfaces @ INESC-ID Lisboa ACM/SIGGRAPH Small Conferences Committee Chair Editor Computers & GraphicsResearch Interests: Calligraphic Interaction, Multimodal Interfaces, Graphical Modeling
  5. 5. Natural Interfaces ?
  6. 6. Dassault Systèmes Vision
  7. 7. NATURAL INTERACTION Ivan Sutherland – 52 years ago
  8. 8. Ivan Sutherland SketchPad Video
  9. 9. Beyond 2D + Space Continuous Above + Hands Bimanual Interactions + Dimensions 3D UIs + Expressiveness Sound, Kicking + Proxemics Eery Space
  10. 10. INTERACT 2011 Interaction Techniques Unifying Touch and Gesture On and Above a Digital Surface
  11. 11. The Continuous Interaction Space Video
  12. 12. Exploring Bimanual Interaction on a TableTop
  13. 13. Pedro Lopes, Daniel Mendes, Bruno Araújo, Joaquim A. Jorge Combining bimanual manipulation and pen-based input for 3D modeling, SBIM 2011
  14. 14. Key Points Table Architecture Combining Pen and Gestures Asymetric Guiard Model
  15. 15. Table Architecture
  16. 16. MOVIE See Movie at:
  17. 17. Evaluation Test Procedure Free drawing
  18. 18. Evaluation More sketching, less camera
  19. 19. Evaluation Camera operations: more fluid (seconds) 44% faster
  20. 20. Evaluation Hand Role Distribution 9% 91% 3% 80% 17%
  21. 21. Evaluation Users prefer touch for all manipulations Imagem e benefits of direct vs indirect
  22. 22. Evaluation Prefer Combined technique
  23. 23. Observations Bimanual workspace and hand role
  24. 24. Observations Natural resting position of NDH
  25. 25. Observations Pen-only often forces re-arrangement
  26. 26. Observations Pen-only causes too much DH motion
  27. 27. Observations Rotate gesture was frail, swapped pen to NDH
  28. 28. Observations Combined gestures were naturally used
  29. 29. Getting it together: Mockup Builder
  30. 30. Setup Multitouch 120Hz Touch 16.25 DPI 3D SVGA 120Hz Active Stereo Kinect (30Hz) Gametrak (120Hz)
  31. 31. Gametrak
  32. 32. User Model Active User Kinect Head Position Kinect Hand Position (Right/Left) Kinect Finger touches Table 4 Fingers tracked in Space (Gametrak) Finger X from Hand Y on or above the surface from user Z
  33. 33. BIMANUAL Asymmetric DH : SketchingNDH : Manipulation Modes  Hands (WIMP)
  35. 35. NDH: Context Menus DH: Selects
  36. 36. ON & ABOVE THE SURFACE Snapping Discrete Constraint Continuous Constraints On the fly Active as long as needed
  37. 37. Preliminary Results expert architect Rhino3D 5’20” 2’45” 5’41” 3’46” xpert User our modeler
  38. 38. Touching The 3D Dimension, Dagstuhl Feb. 2012 Modeling On and Above a Stereoscopic Multitouch Display, (3DCHI): May, 2012. Mockup Builder: Direct 3D Modeling On and Above the Surface in a Continuous Interaction Space (GI'2012), May 28 - May 30, 2012. Special Section C&G (January 2013) ACM Spatial User Interfaces Symposia (SUI)
  39. 39. Adding Sound to Multimodal Interfaces ITS 2011
  40. 40. Take-Home message Touch signature Impact Intensity Interaction space Tangibles
  41. 41. Take-Home message Touch signature Impact Intensity Interaction space Tangibles
  42. 42. Take-Home message Touch signature Impact Intensity Interaction space Tangibles
  43. 43. Take-Home message Touch signature Impact Intensity Interaction space Tangibles
  44. 44. Acoustics of Touch
  45. 45. Acoustics of Touch
  46. 46. Acoustic Sensing Setup
  47. 47. Interaction Opportunities
  48. 48. See Video
  49. 49. Acoustic Sensing Conclusions Affordable Setup
  50. 50. Acoustic Sensing Conclusions Understand impact (amplitude)
  51. 51. Acoustic Sensing Conclusions Recognize contact parts (timbre)
  52. 52. Acoustic Sensing Conclusions Non-tracked Tangibles
  53. 53. Acoustic Sensing Conclusions Multiplicity of Input
  54. 54. Let’s Kick It CHI 2014 Exploring New Interaction Space Large Scale Displays Interacting with the bottom of the screen
  55. 55. Beyond Situated Displays Eery Space
  56. 56. Remote Proxemics: Person to Device Interactions
  57. 57. Social Interactions
  58. 58. Proxemics Distances [Hall, 1966]
  59. 59. Eery Space
  60. 60. Providing Awareness: Floor Circles
  61. 61. Issues Working / Virtual Space Simple Operators / Dialogues Implicit Mode Changes Continuous Transition Asymmetric Bimanual Modes
  62. 62. Take Home Message Extending Touch With Other Modalities Bimanual Interaction Explore Surface Above Table Use Sound in Novel Ways Explore New Modalities Proxemics as tool for implicit interactions
  63. 63. Research Issues
  64. 64. Content Creation in VEs Sketch Interpretation (3D) New Modalities Self Disclosure (Not!) Understandability
  65. 65. Thank You! Joaquim Jorge Bruno Rodrigues De Araújo Maurício de Sousa Ricardo Jorge Jota Costa Pedro Lopes Daniel Mendes