Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

COMP 4026 Lecture4: Processing and Advanced Interface Technology

738 views

Published on

Lecture 4 from the 2016 COMP 4026 course on Advanced Human Computer Interaction taught at the University of South Australia. Taught by Mark Billinghurst, and containing material about Processing and various advanced Human Computer Interfaces.

Published in: Technology
  • Did u try to use external powers for studying? Like ⇒ www.WritePaper.info ⇐ ? They helped me a lot once.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Gov't Seized Cars-95% Off US Largest & Most Trusted Source to Gov't Seized Car Auctions, See Why! ●●● https://w.url.cn/s/Ayy1McS
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • As a student athlete I’m always on the grind at basketball practice and I’ve been really short on time all through high school. I usually order a research paper or English essay here and there. The website is called ⇒ www.HelpWriting.net ⇐ and they really help me out, man. Don’t know where I’d be without it.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • My brother found Custom Writing Service ⇒ www.HelpWriting.net ⇐ and ordered a couple of works. Their customer service is outstanding, never left a query unanswered.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • There are over 16,000 woodworking plans that comes with step-by-step instructions and detailed photos, Click here to take a look ✔✔✔ https://t.cn/A62YeZUX
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

COMP 4026 Lecture4: Processing and Advanced Interface Technology

  1. 1. LECTURE 4: PROCESSING AND ADVANCED INTERFACES COMP 4026 – Advanced HCI Semester 5 - 2016 Mark Billinghurst University of South Australia August 18th 2016
  2. 2. RECAP
  3. 3. App Inventor •  http://appinventor.mit.edu/ •  http://appinventor.org/ •  Visual Programming for Android Apps •  Features •  Access to Android Sensors •  Multimedia output •  Drag and drop web based interface •  Designer view – app layout •  Blocks view – program logic/control
  4. 4. App Inventor DesignerView
  5. 5. App Inventor BlocksView
  6. 6. Interactive Coding - Processing ▪ Programming tool for Artists/Designers ▪ http://processing.org ▪ Easy to code, Free, Open source, Java based ▪ 2D, 3D, audio/video support ▪ Processing For Android ▪ http://wiki.processing.org/w/Android ▪ Strong Android support, builds .apk file
  7. 7. Buchenau, M., & Suri, J. F. (2000, August). Experience prototyping. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (pp. 424-433). ACM.
  8. 8. Experience Prototyping The experience of even simple artifacts does not exist in a vacuum but, rather, in dynamic relationship with other people, places and objects. Additionally, the quality of people’s experience changes over time as it is influenced by variations in these multiple contextual factors.
  9. 9. Role Playing
  10. 10. littleBits - http://littlebits.cc/ •  Plug and play hardware components •  Sensors, input, output •  Rapid design with hardware
  11. 11. Little Bits Demo https://www.youtube.com/watch?v=wDa3dOERxvA
  12. 12. Interaction Design Process Evaluate (Re)Design Identify needs/ establish requirements Build an interactive version Final Product
  13. 13. When to evaluate? • Once the product has been developed •  pros : rapid development, small evaluation cost •  cons : rectifying problems • During design and development •  pros : find and rectify problems early •  cons : higher evaluation cost, longer development design implementationevaluation redesign & reimplementation design implementation
  14. 14. Four evaluation paradigms • ‘quick and dirty’ • usability testing (lab studies) • field studies • predictive evaluation
  15. 15. Characteristics of approaches Usability testing Field studies Predictive Users do task natural not involved Location controlled natural anywhere When prototype early prototype Data quantitative qualitative problems Feed back measures & errors descriptions problems Type applied naturalistic expert
  16. 16. Evaluation approaches and methods Method Usability testing Field studies Predictive Observing x x Asking users x x Asking experts x x Testing x Modeling x
  17. 17. CASE STUDY
  18. 18. Interaction Design Process Evaluate (Re)Design Identify needs/ establish requirements Build an interactive version Final Product
  19. 19. MOBILE AUGMENTED REALITY FOR SPATIAL NAVIGATION Sharon Brosnan 0651869 Bachelor of Science in Digital Media Design
  20. 20. STORYBOARD
  21. 21. INITIAL SKETCHES Pros: •  Good for idea genera,on •  Cheap •  Concepts seem feasible Cons: •  Not great feedback gained •  Photoshop not fast enough for making changes
  22. 22. POST IT PROTOTYPING First Dra. Camera View with 3D Second Dra. Third Dra. •  Selec,on highlighted in blue •  Home bu<on added for easy naviga,on to main menu
  23. 23. POWERPOINT PROTOTYPING Benefits •  Used for User Tes,ng •  Interac,ve •  Func,onali,es work when following the story of Scenario 1 •  Quick •  Easy arrangement of slides User Tes<ng •  Par,cipants found •  15 minute sessions screen captured •  ‘Talk Allowed’ technique used •  Notes taken •  Post-Interview
  24. 24. WIKITUDE •  Popular augmented reality browser for mobile devices •  Mapping •  Point of Interest abilities •  Multiplatform •  Shows the points of interest of Bunratty Folk Park •  Markers can be selected in and an information pop-up appears
  25. 25. VIDEO PROTOTYPE !  Flexible tool for capturing the use of an interface !  Elaborate simula,on of how the naviga,onal aid will work !  Does not need to be realis,c in every detail !  Gives a good idea of how the finished system will work
  26. 26. MORE PROCESSING
  27. 27. Processing - Notes •  Language of Interaction •  Physical Manipulation •  Input using code •  Mouse Manipulation •  Presence, location, image •  Haptic interfaces and multi-touch •  Gesture •  Voice and Speech
  28. 28. Basic Parts of a Sketch /* Notes comment */! //set up global variables! float moveX = 50;! ! //Initialize the Sketch! void setup (){! }! ! //draw every frame! void draw(){! }!
  29. 29. Sample Drawing int m = 0;! float s = 0;! ! void setup(){! size(512,512);! background(255);! }! ! void draw (){! fill(255,0,0);! ellipse(mouseX,mouseY,s,s);! }! ! void mouseMoved(){! s = 40 + 20*sin(++m/10.0f);! }!
  30. 30. Drawing •  draw() gets called as fast as possible, unless a frameRate is specified •  stroke() sets color of drawing outline •  fill() sets inside color of drawing •  mousePressed is true if mouse is down •  mouseX, mouseY - mouse position !void draw() { ! !stroke(255); ! !if(mousePressed) {! ! !line(mouseX, mouseY, pmouseX, pmouseY);! ! !}! !}!
  31. 31. Processing and Drawing •  Basic Shapes rect(x, y, width, height)! ellipse(x, y, width, height)! line(x1, y1, x2, y2), line(x1, y1, x2, y2, z1, z2)! •  Filling shapes - fill( ) fill(int gray), fill(color color), fill(color color, int alpha)! •  Curve •  Draws curved lines •  Vertex •  Creates shapes (beginShape, endShape)
  32. 32. Vertex Demo void setup(){! size(400,400);! }! ! void draw(){! background(255);! fill(0);! beginShape();! vertex(0,0);! vertex(400,400);! vertex(mouseX,mouseY);! endShape();! }!
  33. 33. Curve Demo void setup(){! size(400,400);! }! ! void draw(){! background(255);! fill(0);! ! int xVal = mouseX*3-100;! int yVal = mouseY*3-100;! ! curve(xVal, yVal, 100, 100, 100, 300, xVal, yVal);! curve(xVal, yVal, 100, 300, 300, 300, xVal, yVal);! curve(xVal, yVal, 300, 300, 300, 100, xVal, yVal);! curve(xVal, yVal, 300, 100, 100, 100, xVal, yVal);! ! }!
  34. 34. Class and Objects • see http://processing.org/learning/objects/ • Object •  grouping of multiple related properties and functions • Objects are defined by Object classes • Eg Car object •  Data •  colour, location, speed •  Functions •  drive(), draw()
  35. 35. Classes • four elements: name, data, constructor, and methods. • Name class myName { }! • Data •  collection of class variables • Constructor •  run when object created • Methods •  class functions
  36. 36. Car Class
  37. 37. Class Usage // Step 1. Declare an object.! Car myCar;! ! void setup() { ! // Step 2. Initialize object.! myCar = new Car(); ! } ! ! void draw() { ! background(255); ! // Step 3. Call methods on the object. ! myCar.drive(); ! myCar.display(); ! }!
  38. 38. Constructing Objects •  One Car Car myCar= new Car(); ! •  Two Cars !// Creating two car objects ! !Car myCar1 = new Car(); ! !Car myCar2 = new Car(); ! •  One car with initial values Car myCar = new Car(color(255,0,0),0,100,2); !
  39. 39. Modifying Constructor Car(color tempC, float tempXpos, float tempYpos, float tempXspeed) ! { ! c = tempC; ! xpos = tempXpos; ! ypos = tempYpos; ! xspeed = tempXspeed; ! }!
  40. 40. Mouse Interaction • Mouse position •  mouseX, mouseY variables • Mouse Interaction •  mousePressed() •  mouseReleased() •  mouseDragged() • Add in own code void mouseDragged(){! line(pmouseX, pmouseY, mouseX, mouseY);! }!
  41. 41. Keyboard Interaction • Check keyPressed variable in draw() method !void draw(){! ! !if(keyPressed){! ! ! !print(" you pressed " +key);! ! !}! }! • Use keyPressed() method !void keyPressed(){! ! !print(" you're pressing "+key);! !}!
  42. 42. Importing Libraries •  Can add functionality by Importing Libraries •  java archives - .jar files •  Include import code import processing.opengl.*;! •  Popular Libraries •  Minim - audio library •  OCD - 3D camera views •  Physics - physics engine •  bluetoothDesktop - bluetooth networking
  43. 43. http://toxiclibs.org/
  44. 44. Graphical Controls • Use ControlP5 Library •  http://www.sojamo.de/libraries/controlP5/ • Add graphical controls •  Buttons, sliders, etc •  Support for OSC (Open Sound Controller) • Use ControlP5 class import controlP5.*;! addButton(name, value, x, y, width, height);! • Event Handing
  45. 45. Interface Elements • Interfascia • http://www.superstable.net/interfascia/ • GUI Library for Processing • Buttons • Check boxes • Textfields • Progress bar
  46. 46. Graphical Controls •  Use ControlP5 Library •  http://www.sojamo.de/libraries/controlP5/ •  Add graphical controls •  Buttons, sliders, etc •  Support for OSC (Open Sound Controller) •  Use ControlP5 class import controlP5.*;! addButton(name, value, x, y, width, height);! •  Event Handing
  47. 47. P5 Example Controls
  48. 48. Programming Graphics • Transformations • Creating motion and animation • Bitmaps and pixels • Textures
  49. 49. 3D Graphics • Two options • P3D Library • OpenGL Library • P3D • Simple, integrated directly into processing • Lightweight 3D • OpenGL • Full graphics support • Complex
  50. 50. Shapes •  beginShape(SHAPE); •  DefineVertices •  SHAPES: QUADS, QUAD_STRIP,TRIANGLE_FAN •  endShape(); •  Eg: Quads !beginShape(QUADS);! !fill(0, 1, 1); vertex(-1, 1, 1);! !fill(1, 1, 1); vertex( 1, 1, 1);! !fill(1, 0, 1); vertex( 1, -1, 1);! !fill(0, 0, 1); vertex(-1, -1, 1);! !endShape();!
  51. 51. P3D Scene size(640, 360, P3D); ! background(0);! lights();! ! noStroke();! pushMatrix();! !translate(130, height/2, 0);! !rotateY(1.25);! !rotateX(-0.4);! !box(100);! popMatrix();! ! noFill();! stroke(255);! pushMatrix();! !translate(500, height*0.35, -200);! !sphere(280);! popMatrix();!
  52. 52. Vertices Demo • RGB Cube •  Vetices and vertex fills • VertexDemo •  Different types of quad strips •  User defined drawing function
  53. 53. Transformations •  Rotation ! !rotateX(a), rotateY(a * 2.0), rotateZ(a)! •  Translation ! !translate(X,Y); translate(X,Y,Z);! •  Scale •  Push and Pop functions •  Push - Saving current coordinate system •  Pop – Restores previous coordinate system •  Eg: PushPopCubes
  54. 54. OpenGL • Import opengl library import processing.opengl.*;! • Create drawing context !void draw(){! ! !PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;! ! !GL g = pgl.beginGL();! ! !//use GL methods! ! !pgl.endGL(); //end opengl calls! }!
  55. 55. 3D Model Loading •  Using saito model loader •  Loads obj files •  http://code.google.com/p/saitoobjloader/downloads •  Code fragment import saito.objloader.*;! OBJModel model ;! model = new OBJModel(this, "sa.obj", "absolute", TRIANGLES);!
  56. 56. OtherTools • GLGraphics •  http://glgraphics.sourceforge.net/ • Library that extends the Processing OpenlGL renderer •  opengl textures, •  image post-processing filters, •  3D Models, and shaders in GLSL, •  Cg and CgFX
  57. 57. ADVANCED INTERFACE TECHNOLOGY
  58. 58. Advanced Interface Technology • Wearable Computers • Augmented Reality • Virtual Reality • Invisible Interfaces • Environment Sensing • Physiological Sensing
  59. 59. Class Project 1.  Pick Advanced Technology 2.  Brainstorm use case 3.  Develop conceptual design 4.  Prototype interface/experience design 5.  Conduct user evaluation 6.  Repeat steps 3-5 7.  Write report
  60. 60. WEARABLE COMPUTERS
  61. 61. Major changes in computing
  62. 62. Wearable Computing ▪  Computer on the body that is: ▪  Always on ▪  Always accessible ▪  Always connected ▪  Other attributes ▪  Augmenting user actions ▪  Aware of user and surroundings
  63. 63. Wearable Attributes ▪  fafds
  64. 64. Google Glass
  65. 65. ViewThrough Google Glass
  66. 66. Research Problems • Hardware •  Power, networking, display • User Interaction •  User input, speech, gesture, gaze, etc •  Novel interaction methods • Social Acceptance •  Privacy, social factors • Novel Applications •  Collaboration •  Intelligent assistance
  67. 67. AUGMENTED REALITY
  68. 68. 1977 – StarWars
  69. 69. Augmented Reality Definition • Defining Characteristics [Azuma 97] • Combines Real andVirtual Images • Both can be seen at the same time • Interactive in real-time • The virtual content can be interacted with • Registered in 3D • Virtual objects appear fixed in space Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
  70. 70. 2008 - CNN
  71. 71. •  Put AR pictures here Augmented Reality Examples
  72. 72. Typical Demo Application https://www.youtube.com/watch?v=UOfN1plW_Hw
  73. 73. Research Problems • Low level hardware/software •  Head mounted displays •  Tracking systems • User Interaction •  Gesture based interaction •  Multimodal input (speech, gesture) • Novel Applications •  Face to face collaboration •  Authoring tools
  74. 74. VIRTUAL REALITY
  75. 75. Virtual Reality •  1985…
  76. 76. Virtual Reality • ImmersiveVR •  Head mounted display, gloves •  Separation from the real world
  77. 77. Occulus Rift • $300 USD • 360 degree head tracking • 100 degree field of view
  78. 78. Wearable Virtual Reality •  Samsung Gear VR •  See virtual reality on your phone •  Using phone display, compass
  79. 79. Gear VR Demo https://www.youtube.com/watch?v=CjpGnh2PDoU
  80. 80. AR vsVR
  81. 81. Research Problems • Low level •  Wide area tracking •  Development tools • User Interaction •  Intuitive input (gesture, controllers) •  Avatar control and representation •  Techniques for navigation/manipulation • Novel Applications •  Massive multi-user environments •  Content capture and sharing
  82. 82. INVISIBLE INTERFACES
  83. 83. Early Examples •  Interaction without devices: •  BodySpace [Strachan 2007]: Functions to body position •  Abracadabra [Harrison 2007]: Magnets on finger tips •  GesturePad [Rekimoto 2001]: Capacitive sensing in clothing •  Palm-based Interaction •  Haptic Hand [Kohli 2005]: Using non-dominant hand in VR •  Sixth Sense [Mistry 2009]: Projection on hand •  Brainy Hand [Tamaki 2009]: Head worn projector/camera
  84. 84. Unobtrusive Input Devices ▪  GesturePad ▪  Capacitive multilayered touchpads ▪  Supports interactive clothing
  85. 85. ImaginaryPhone •  Gustafson, S., Holz, C., & Baudisch, P. [2011]
  86. 86. Imaginary Phone Demo https://www.youtube.com/watch?v=xtbRen9RYx4
  87. 87. Transfer Learning
  88. 88. Invisible Interfaces – Gestures in Space •  Gustafson, S., Bierwirth, D., & Baudisch, P. [2010] •  Using a non-dominant hand stabilized interface.
  89. 89. Imaginary Interfaces https://www.youtube.com/watch?v=718RDJeISNA
  90. 90. Project Soli •  Using Radar to support free-hand spatial input
  91. 91. Project Soli https://www.youtube.com/watch?v=0QNiZfSsPc0 https://www.youtube.com/watch?v=jWNebDDmuXc
  92. 92. Research Gaps • Free-hand interfaces using relative input • Combining invisible interface + mobile device • Multimodal interaction •  speech + gesture input • Affordances and discoverability • Interaction frameworks
  93. 93. Research Problems • Hardware •  Power, networking, display • User Interaction •  User input, speech, gesture, gaze, etc •  Novel interaction methods • Social Acceptance •  Privacy, social factors • Novel Applications •  Collaboration •  Intelligent assistance
  94. 94. ENVIRONMENT SENSING
  95. 95. Environmental Sensor • New sensors track and capture real environment •  Navigation, 3D modeling, user tracking • Depth Sensors •  Microsoft Kinect, Intel RealSense • Integrated Devices •  Google Tango
  96. 96. Google Tango • Tablet based system • Android OS • Multiple sensors • RGBD Sensor • IR Structured light • Inertial sensors • High end graphics • Nvidia tegra chip
  97. 97. Google Tango
  98. 98. Research Problems • Content creation •  Creating better 3D models •  Segmenting objects • User Interaction •  Interaction with real world •  Interacting with multiple devices • Novel Applications •  AR notes/real world tagging •  Social networking
  99. 99. PHYSIOLOGICAL SENSING
  100. 100. Physiological Sensors • Sensing user state •  Body worn devices • Multiple possible sensors •  Physical activity •  Eye tracking, gaze •  Heart rate •  GSR •  Breathing •  Etc
  101. 101. Tobii Eye Tracker • Wearable eye tracking system •  Natural data capture •  Scene camera capture •  Recording/streaming eye gaze, 60 Hz sampling
  102. 102. Tobii Demo •  https://www.youtube.com/watch?v=hDG1mRFFusc
  103. 103. Research Problems • User Interaction •  Implicit vs. Explicit interaction •  Measuring cognitive • Social Acceptance •  Privacy, social factors • Novel Applications •  Collaboration •  Intelligent assistance
  104. 104. www.empathiccomputing.org @marknb00 mark.billinghurst@unisa.edu.au

×