Presented at O'Reilly Design Conference on Jan 22, 2016.

1. I HEAR VOICES
karen kaushansky
@kjkausha
EXPLORATIONS OF
MULTI-DEVICE
EXPERIENCES WITH
CONVERSATIONAL
ASSISTANTS

5. JUST LIKE ATMS AND BANK TELLERS, PEOPLE WILL
REALIZE IT’S A MUCH MORE EFFICIENT WAY TO GET THE
INFORMATION YOU WANT AND WE WILL SPEND MUCH OF
OUR TIME TALKING TO MACHINES.
John Markoff
ON SPEECH RECOGNITION IN GENERAL

6. ACCORDING TO APPLE,
SIRI ALREADY HANDLES
MORE THAN A BILLION
SPOKEN REQUESTS PER
WEEK
Source:NY TImes

7. IT’S A FUNNY THING, TRYING TO MAKE SENSE OF A
TECHNOLOGY THAT HAS NO BUILT-IN VISUAL INTERFACE…
THIS GETS AT A DEEPER TRUTH ABOUT CONVERSATIONAL
TECH: YOU ONLY DISCOVER ITS CAPABILITIES IN THE COURSE
OF A PERSONAL RELATIONSHIP WITH IT.
Wired
NO VISUAL INTERFACE

8. PERSONA
▸ “A standardized mental image of a personality or character
that users infer from the application’s voice and language
choices.” It is a vehicle by which companies can brand a
service or project.
▸ Whether implicitly designed or not, users will perceive a
personality. “It is not advisable to leave the perceptions to
chance, especially because branding and image are at stake.”
Source: Voice User Interface Design
by Cohen, Giangola, Balogh

9. WHAT CAN BE ACHIEVED WITH A PERSONA?
Increase in engagement time
Increase in number of interactions
Higher overall rating of the product
Higher likelihood of recommending
device to a friend
Increase in monetization
Leads to a purchase of another device/
brand loyalty
Source: NY Times

10. Google Now and Siri currently
represent two dramatically different
types of Human-Computer
Interaction styles. While Siri
intentionally and successfully mimics
a human, complete with a wry sense
of humor, Google Now opts instead to
function as a pure informational
oracle, devoid of personality or
humanity.

11. OUR ONLINE CONVERSATIONS WILL INCREASINGLY BE
MEDIATED BY CONVERSATION ASSISTANTS WHO WILL
HELP US LAUGH AND BE MORE PRODUCTIVE.
Lili Cheng
THIS IS JUST THE START

12. 1
PERSONA
?
PERSONAS
1
DEVICE
MULTIPLE
DEVICES

13. Source: CNN

15. CORTANA
▸ Cortana is confident, caring, competent, loyal; helpful, but not bossy.
▸ “Not only is she AI, she's self-aware, and that principle of transparency
informs a lot of how we handle error messages, our capabilities, tasks
and chitchat. You'll have more faith and trust in us if we do that for
you.”
▸ “As soon as the team gave Cortana a boost in confidence, people
immediately began responding to her more positively.”
Source: Engadget

16. CORTANA
▸ Cortana places importance on audio
feedback to help complete tasks
▸ Uses audio when processing speech
▸ Speaks like a real person “Sound
good?”
▸ Not as formal as some other
personas, uses contractions, is
chitchatty

18. SIRI
▸ Siri has “Occasionally a light attitude”.
▸ It has been noted “What can Siri do
better? Have an emotional relationship
with a user.”
▸ More formal “Shall I create it?”
▸ Siri uses visual feedback for specific
details unless plugged into a headset

19. GOOGLE NOW
▸ Google Now does a great job
anticipating needs, and gathering
contextual information.
▸ Its persona is a reflection of its
capabilities and strengths: “Google
Now does not attempt personality,
merely efficiency”
▸ All business, no chitchat

20. ALEXA
▸ “ ‘Sorry, I didn’t understand the
question I heard’ is her favorite
response, though honestly she really
doesn’t sound very sorry.”
▸ She is just smart enough to be useful.
And she keeps getting smarter.
▸ “She’s like a genie in a sci-fi-looking
bottle – one not quite at the peak of her
powers, and with a tiny bit of an
attitude.”

23. ERRORS
▸ Cortana: “I’m sorry, the internet
and I aren’t talking right now”
▸ “Siri not available. Connect to the
internet.”
▸ Google: No audio – visual says
“Can’t reach Google at the
moment”

24. As a toy, Hello Barbie needs to be both fun, leading girls through imaginative games,
and funny, telling jokes and being goofy. But Mattel also wanted Barbie to have an
empathetic, affirming sensibility aimed at young girls. - NYTimes

25. IF WE HAD MORE OF A FRAMING
“SHE’S THE PERFECT PERSON TO ASK
ABOUT RESTAURANTS OR SEARCHING
FOR DIRECTIONS” THEN SHE WOULD
WORK 95% OF THE TIME RATHER THAN
FAILING YOU 40% OF THE TIME.
David Rose

26. SHORTCOMINGS
▸ In many cases it is still a layer on top of
the OS; only goes one layer deep

27. Source: Techcrunch

28. A great conversational agent is only fully useful when it’s everywhere, when it can
get to know you in multiple contexts—learning your habits, your likes and dislikes,
your routine and schedule. The way to get there is to have your AI colonize as many
apps and devices as possible.
- Wired

29. THE NEW SIRI IS PAVING THE WAY TO WHAT YOU MIGHT
CALL “AMBIENT COMPUTING” — A FUTURE IN WHICH
ROBOTIC ASSISTANTS ARE ALWAYS ON HAND TO ANSWER
QUESTIONS, TAKE NOTES, TAKE ORDERS OR OTHERWISE
FUNCTION AS AUXILIARY BRAINS TO WHOM YOU MIGHT
OFFLOAD MANY OF YOUR CHORES.
NY Times
IPHONE 6S’S HANDS-FREE SIRI IS AN OMEN OF THE FUTURE

30. Yes?
Yes?
Yes?
Yes?
Yes?
Yes?
Yes?

32. ALL DEVICES ALL THE TIME

35. Which one should listen?

36. The one that can provide the best result based
on context

37. Actually I don’t care who listens…just do the
right thing.

38. So what that means is they should all listen,
and negotiate between them who answers.

39. All pervasive
Hey Watch, do this.
Hey TV, do that.
Hey car, navigate.
Get me X.

40. Source: UX Mag

41. DEVICE SHIFTING
Start on one device and shift based on capabilities
of the device.
Hey Siri, what was the
score of the Habs game?

42. LINKED ACTIONS
Just do what you need it to do no matter how many
“apps” and devices it takes.
Liane Schinasi
Send this to my Mom and
add to favorites.

43. USER DIRECTED
DEVICE SHIFTING
Direct the output when it’s not the obvious choice.
And then ability to surface it
Do you still need
scissors?
Yep
Tweezerman Baby
Nail Scissors
Save this to my watch.

44. TRUE REMOTE CONTROL
Remote as in “on my way home”
Hey Siri, get the next
House of Cards tee’d up
for tonight

45. Amazon is throwing $100M in the form of Alexa Fund to lure developers,
manufacturers, and startups to create voice-driven applications and devices based on
Echo. Companies such as Orange Chef, Scout Alarm, Toymail and, Mojio got the seed
investment from Alexa Fund.
▸ “Your customers can simply speak
to Alexa through the microphone
on your device and Alexa will
respond through your device's
speakers
- Forbes

47. THANK YOU
(WE’RE HIRING ALL TYPES OF DESIGNERS)
@KJKAUSHA
kk@zoox.com