Designing for Wearables

DESIGNING FOR WEARABLES
Mark Billinghurst
mark.billinghurst@unisa.edu.au
Oct 17th 2015
AWE Asia 2015

Mark Billinghurst
▪  Professor of HCI, University of
South Australia
▪  PhD Univ. Washington
▪  Research on AR, mobile HCI,
Collaborative Interfaces
▪  More than 300 papers in AR, VR,
interface design, wearable
▪  Sabbatical in Glass team at
Google [x] in 2013

Goals and Schedule
Goals
Learn simple interface design guidelines
Learn useful prototyping tools
Learn where further resources are
Schedule
4:15 Introduction (5 minutes)
4:20 Design Guidelines (15 minutes)
4:35 Prototyping Tools (20 minutes)
4:55 Resources (5 minutes)

1. INTRODUCTION

Room Desk Lap Hand Head

What is a Wearable Computer?
▪  Computer on the body that is:
▪  Always on
▪  Always accessible
▪  Always connected
▪  Other attributes
▪  Augmenting user actions
▪  Aware of user and surroundings
Starner, T. E. (1999). Wearable computing and contextual awareness
(Doctoral dissertation, Massachusetts Institute of Technology).

Evolution of Wearables
•  asdfad

Second Gen. Systems
• Recon (2010 - )
•  Recon Jet, Snow
•  Head worn displays for sports
•  Acquired by Intel (2015)
• Google (2011 - )
•  Google Glass
•  Consumer focus

Smart Glass in 2015

How do you Design for this?

2. DESIGN GUIDELINES

Universal Design Principles
Flexibility
Equitable use
Easy to perceive
Simple and intuitive
Low physical effort
High tolerance for error

Designing for Wearables
•  Wearables are intimate on-body devices, so
interface design for wearables, means:

•  Designing for Attention

•  Designing for Interruption

•  Designing for User Experience

•  Designing for Social Interaction

Attention

Micro-Interactions
Using mobile phone people split their attention
between the display and the real world

Time Looking at Screen
Oulasvirta, A. (2005). The fragmentation of attention in mobile
interaction, and what to do with it. interactions, 12(6), 16-18.

Using Micro Interactions
Quick micro-interactions reduce divided attention
and allow people to spend more time in real world

It's
like
a
rear
view

mirror

Don't
overload
the
user.
S2ck
to
the

absolutely
essen2al,
avoid
long

interac2ons.
Be
explicit.

Make it glanceable
Seek to rigorously reduce information density.
Successful designs afford for recognition, not reading.
Bad Good

Reduce the number of info chunks
You are designing for recognition, not reading.
Reducing the total # of information chunks will greatly
increase the glanceability of your design.
1
2
3
1
2
3
4
5 (6)
Test done by Morten Just using a watch

Design Single Interactions < 4 sec
Eye movements
For 1: 1 230ms
For 2: 1 230ms
For 3: 1 230ms
For 4: 3 690ms
For 5: 2 460ms
~1,840ms
Eye movements
For 1: 1-2 460ms
For 2: 1 230ms
For 3: 1 230ms
~920ms
1
2
3
1
2
3
4
5 (6)
Test done by Morten Just using a watch

Test the glanceability of your design

Design for MicroInteractions
▪  Design interactions less than a few seconds

•  Tiny bursts of interaction

•  One task per interaction

•  One input per interaction

▪  Beneﬁts

•  Use limited input

•  Minimize interruptions

•  Reduce attention fragmentation

Rule of Thumb: Design for
limited attention/
microinteractions - no more
than 4 seconds to complete a
given step in the interaction

Interruption

Designing for Interruptions
•  Assume user is engaged in critical real world task
•  Use context to filter interruptions (is it necessary?)
•  Interrupt in way that consumes least attention
•  Allow user to dismiss interruption with minimal effort
•  Progressively disclose information and increase interaction

Example:Interruptions on Glass
▪  Gradually increase engagement and attention load
▪  Respond to user engagement
Receiving SMS on Glass
“Bing”
Tap
Swipe
Glass
Show Message Start Reply
User
Look
Up
Say
Reply

Rule of Thumb: Design for
interruption and low
cognitive load that can be
increased as needed.

User Experience

ConsiderYour User
• Wearable User
• Probably Mobile
• One/no hand interaction
• Short application use
• Need to be able to multitask
• Use in outdoor or indoor environment
• Want to enhance interaction with real world

How would you take a note?

Example:Glass Pictures
• On Glass there are three ways to take a picture
1/Voice commands – “Ok Glass,Take a Picture”
2/ Touch navigation through menu
3/ Winking with right eye
•  Which you use depends on context
•  Riding a bike outdoors – voice commands
•  During a meeting – winking

Rule of Thumb: Provide
multiple ways of accessing
functionality.

Design For Device
• Simple, relevant information
• Complement existing devices

Glass User Interface
•  dfasdf

Do one thing at a time

Test your design indoors + outdoors

Design for Context

Design for Ecosystem of Wearables
User have multiple devices
- phone, watch
- fitness band, HMD
Each device should be used
when it’s most relevant and
when it’s the easiest interaction
available.

Interface Guidelines
▪  Design for device
•  Use multiple input options
•  Do one thing at a time
•  Consider user context
▪  Design for indoor and outdoor use
▪  Design for device ecosystem

Social Interaction

SocialAcceptance
• People don’t want to look silly
•  Only 12% of 4,600 adults would be willing to wear AR glasses
•  20% of mobile AR browser users experience social issues
• Acceptance more due to Social than Technical issues
•  Needs further study (ethnographic, field tests, longitudinal)

Rule of Thumb: Fashion First

- It DOES NOT MATTER what
the device does unless the user is
willing to put it on the ﬁrst time

Rule of Thumb: Make the
interface not just for the user
but also for the people
around the user, both
physically and socially.

Attention: least visual-manual attention
necessary, < 2 second access time
Interruption: assume busy, ramped attention
Wearable: long term, unobtrusive, fashionable
User Experience: design for device, multiple
ways of information access, all environments
Social: graceful interfaces, design for others
Summary

3. PROTOTYPING

How can we quickly
prototype Wearable
experiences with little
or no coding?

Why Prototype?
▪  Quick visual design
▪  Capture key interactions
▪  Focus on user experience
▪  Communicate design ideas
▪  “Learn by doing/experiencing”

Typical Development Steps
▪  Sketching
▪  Storyboards
▪  UI Mockups
▪  Interaction Flows
▪  Video Prototypes
▪  Interactive Prototypes
▪  Final Native Application
Increased
Fidelity &
Interactivity
Static
Low fidelity
Interactive
High fidelity

Sketched Interfaces
▪  Sketch + Powerpoint/Photoshop/Illustrator

Paper Prototype
•  Use sketched interface in template

Glass UI Templates
▪  Google Glass Photoshop Templates
▪  http://glass-ui.com/
▪  http://dsky9.com/glassfaq/the-google-glass-psd-template/

Smart Watch Templates
•  Eg https://dribbble.com/jaysuthar/buckets/260235-watch

Application Flow

Glassware Flow Designer
•  Quick flow layout tool
•  https://glassware-flow-designer.appspot.com/

Static Limitations
▪  Positives
▪  Good for documenting screens
▪  Can show application flow
▪  Negatives
▪  No interactivity/transitions
▪  Can’t be used for testing
▪  Can’t deploy on wearable
▪  Can be time consuming to create

▪ Series of still photos in a movie format.
▪ Demonstrates the experience of the product
▪ Show transitions between states
▪ Discover where concept needs fleshing out.
▪ Communicate experience and interface
▪ You can use whatever tools, from Flash to iMovie.
Video Sketching

UI Concept Movies

Interactive Wireframing
▪  Developing interactive interfaces/wireframes
▪  Transitions, user feedback, interface design
▪  Web based tools
▪  UXpin - http://www.uxpin.com/
▪  proto.io - http://www.proto.io/
▪  Native tools
▪  Justinmind - http://www.justinmind.com/
▪  Axure - http://www.axure.com/

Proto.io - http://www.proto.io/
▪  Web based mobile prototyping tool
▪  Features
▪  Prototype for multiple devices
▪  Gesture input, touch events, animations
▪  Share with collaborators
▪  Test on device

Proto.io Demo Video

Proto.io Android Wear Demo
•  https://proto.io/showcase/android-wear/

Wireframe Limitations
▪  Can’t deploy on Device
▪  No access to sensor data
▪  Camera, orientation sensor
▪  No multimedia playback
▪  Audio, video
▪  Simple transitions
▪  No conditional logic
▪  No networking

Processing
▪  Programming tool for Artists/Designers
▪  http://processing.org
▪  Easy to code, Free, Open source, Java based
▪  2D, 3D, audio/video support
▪  Sketching with code
▪  Processing For Android
▪  http://wiki.processing.org/w/Android
▪  Strong Android support
▪  Generates Android .apk file

http://processing.org/

Basic Processing Sketch
/* Notes comment */
//set up global variables
float moveX = 50;
//Initialize the Sketch
void setup (){
}
//draw every frame
void draw(){
}

Processing and Wearables
▪  One of the easiest ways to build rich
interactive wearable applications
▪  focus on interactivity, not coding
▪  Collects all sensor input
▪  camera, accelerometer, touch
▪  Can build native Android .apk files
▪  Side load onto Glass

Hello World Image
PImage img; // Create an image variable
void setup() {
size(640, 360);
//load the ok glass home screen image
img = loadImage("okGlass.jpg"); // Load the image
}
void draw() {
// Displays the image at point (0,0)
image(img, 0, 0);
}

Touch Pad Input
▪  Tap recognized as DPAD input
void keyPressed() {
if (key == CODED){
if (keyCode == DPAD) {
// Do something ..
▪  Java code to capture rich motion events
▪  import android.view.MotionEvent;

Motion Event
//Glass Touch Events - reads from touch pad
public boolean dispatchGenericMotionEvent(MotionEvent event) {
float x = event.getX(); // get x/y coords
float y = event.getY();
int action = event.getActionMasked(); // get code for action
switch (action) { // let us know which action code shows up
case MotionEvent.ACTION_DOWN:
touchEvent = "DOWN";
fingerTouch = 1;
break;
case MotionEvent.ACTION_MOVE:
touchEvent = "MOVE";
xpos = myScreenWidth-x*touchPadScaleX;
ypos = y*touchPadScaleY;
break;

Sensors
▪  Ketai Library for Processing
▪  https://code.google.com/p/ketai/
▪  Support all phone sensors
▪  GPS, Compass, Light, Camera, etc
▪  Include Ketai Library
▪  import ketai.sensors.*;
▪  KetaiSensor sensor;

Using Sensors
▪  Setup in Setup( ) function
▪  sensor = new KetaiSensor(this);
▪  sensor.start();
▪ sensor.list();
▪  Event based sensor reading
void onAccelerometerEvent(…)
{
accelerometer.set(x, y, z);
}

Sensor Demo

Physical Input Devices
▪  Can we develop unobtrusive input devices ?
▪  Reduce need for speech, touch pad input
▪  Socially more acceptable
▪  Examples
▪  Ring,
▪  pendant,
▪  bracelet,
▪  gloves, etc

Prototyping Platform
Arduino Kit Bluetooth Shield Google Glass

Example: Glove Input
▪  Buttons on fingertips
▪  Map touches to commands

Example: Ring Input
▪  Touch strip, button, accelerometer
▪  Tap, swipe, flick actions

How it works
Bracelet
Armband
Gloves
1,2,
3,4
Values/
output

Summary
• Prototyping for wearables is similar to mobiles
•  Tools for UI design, storyboarding, wireframing
• Android tools to create interactive prototypes
•  Processing, WearScript, etc
• Arduino can be used for hardware prototypes
• Once prototyped Native Apps can be built
•  Android + SDK for each platform

4. Resources

Designing For Glass Video
•  https://www.youtube.com/watch?v=6ERgbIJ6pCM

Glass UI Design Guidelines
• More guidelines
– https://developers.google.com/glass/design/index

GlassSim – http://glasssim.com/
▪  Simulate the view through Google Glass
▪  Multiple card templates

GlassSim Card Builder
▪  Use HTML for card details
▪  Multiple templates
▪  Change background
▪  Own image
▪  Camera view

GlassSim Samples

Pop - https://popapp.in/
• Take pictures of sketches
• Link pictures together
• Combining sketching and interactivity on mobiles

Using Pop

Viewing Design on Device
• Android Design Preview
•  https://github.com/romannurik/AndroidDesignPreview
• View a portion of your desktop on Android device
•  Select region of screen
•  Mirror it on Android Device
• Use to view mock-ups on target device
•  Eg Powerpoint for wearable mockups

Other Tools
▪  Wireframing
▪  pidoco
▪  FluidUI
▪  Rapid Development
▪  Phone Gap
▪  AppMachine
▪  Interactive
▪  App Inventor
▪  WearScript

WearScript
• WearScript
•  Combines power of Android development on Glass
with the learning curve of a website
•  Use a cloud IDE to write JavaScript that runs instantly
on Glass inside a WebView
•  Get started at http://wearscript.com
• Support for advanced features
•  Augmented Reality
•  Eye tracking
•  Arduino input

How it Works

Books
▪ Programming Google Glass
▪ Eric Redmond
▪ Rapid Android Development:
Build Rich, Sensor-Based
Applications with Processing
▪ Daniel Sauter

• Beginning Google
Glass Development
by Jeff Tang

• Microinteractions:
Designing with Details
• Dan Saffer
• http://microinteractions.com/

CHI Wearables Exhibit
Online at http://wcc.gatech.edu/exhibition

Glass Resources
▪  Main Developer Website
▪  https://developers.google.com/glass/
▪  Glass Apps Developer Site
▪  http://glass-apps.org/glass-developer
▪  Google Design Guidelines Site
▪  https://developers.google.com/glass/design/
index?utm_source=tuicool
▪  Google Glass Emulator
▪  http://glass-apps.org/google-glass-emulator

Recon Jet
•  Developer Website
•  http://www.reconinstruments.com/developers/getting-started/
reconsdk/
•  Application Prototyping
•  http://www.reconinstruments.com/developers/resources/
application-prototyping/

Other Resources
▪  AR for Glass Website
▪  http://www.arforglass.org/
▪  Vandrico Database of wearable devices
▪  http://vandrico.com/database

http://openprocessing.org/

http://toxiclibs.org/

Books
▪  Programming Google Glass
▪  Eric Redmond
▪  Rapid Android
Development: Build Rich,
Sensor-Based Applications
with Processing
▪  Daniel Sauter

www.empathiccomputing.org
@marknb00
mark.billinghurst@unisa.edu.au

Designing for Wearables

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Designing for Wearables

Similar to Designing for Wearables (20)

More from Mark Billinghurst

More from Mark Billinghurst (20)

Recently uploaded

Recently uploaded (20)

Designing for Wearables