The Haptic Creature Project: Social Human-Robot Interaction through Affective Touch. Presented at the Young Pioneers Workshop at HRI '10: The 5th ACM/IEEE International Conference on Human-Robot Interaction. Osaka, Japan; March 2-5 2010. More info at http://yohanan.org/steve/projects/haptic-creature/.

1. Steve Yohanan
Karon MacLean
SPIN Research Group
Dept of Computer Science
University of British Columbia
Vancouver Canada
The Haptic Creature Project
Social Human-Robot Interaction
through Affective Touch

2. HRI 2010 Young Pioneers Workshop - Steve Yohanan 2
Overview
● Background
● Research Questions
● The Haptic Creature
● User Studies

3. HRI 2010 Young Pioneers Workshop - Steve Yohanan 3
Overview
➢ Background
● Research Questions
● The Haptic Creature
● User Studies
 Research Areas
 SIR & Affect Display
 Affective Touch & HAI

4. HRI 2010 Young Pioneers Workshop - Steve Yohanan 4
Background
Research Areas

5. HRI 2010 Young Pioneers Workshop - Steve Yohanan 5
Background
Socially Interactive Robotics & Affect Display
Facial Expression, Speech Prosody
FACS (Ekman)FACS (Ekman) Kismet (Breazeal)Kismet (Breazeal)

6. HRI 2010 Young Pioneers Workshop - Steve Yohanan 6
Background
Affective Touch & Human-Animal Interaction
Modality Much Less Studied

7. HRI 2010 Young Pioneers Workshop - Steve Yohanan 7
Overview
● Background
➢ Research Questions
● The Haptic Creature
● User Studies
 Expression
 Recognition
 Influence

8. HRI 2010 Young Pioneers Workshop - Steve Yohanan 8
Research Questions
Expression
● In what manner might a robot express its emotional
state through touch to a human?
● In what manner might a human express his/her
emotional state through touch to a robot?

9. HRI 2010 Young Pioneers Workshop - Steve Yohanan 9
Research Questions
Recognition
● Can a robot recognize a human's touch gestures?
– Can these gestures be used to infer the emotional state of a
human?
● Can a human recognize a robot's emotional state
through touch?

10. HRI 2010 Young Pioneers Workshop - Steve Yohanan 10
Research Questions
Influence
● How does the interaction between human and robot
(through affective touch) influence the human?

11. HRI 2010 Young Pioneers Workshop - Steve Yohanan 11
Overview
● Background
● Research Questions
➢ The Haptic Creature
● User Studies
 Design Considerations
 Hardware
 Software
 Emotion Model
 Related Robots

12. HRI 2010 Young Pioneers Workshop - Steve Yohanan 12
The Haptic Creature
Design Considerations
Touch, Organic, Zoomorphism

13. HRI 2010 Young Pioneers Workshop - Steve Yohanan 13
The Haptic Creature
Hardware

14. HRI 2010 Young Pioneers Workshop - Steve Yohanan 14
The Haptic Creature
Hardware

15. HRI 2010 Young Pioneers Workshop - Steve Yohanan 15
The Haptic Creature
Software
Higher-Order Processing on Host Computer

16. HRI 2010 Young Pioneers Workshop - Steve Yohanan 16
The Haptic Creature
Emotion Model
Circumplex Model of Affect (Russell 1980)

17. HRI 2010 Young Pioneers Workshop - Steve Yohanan 17
The Haptic Creature
Related Robots
Zoomorphism, Affect, Touch
Paro (Shibata)Paro (Shibata)
Huggable (MIT)Huggable (MIT)Aibo (Sony)Aibo (Sony)
Probo (VUB)Probo (VUB)

18. HRI 2010 Young Pioneers Workshop - Steve Yohanan 18
Overview
● Background
● Research Questions
● The Haptic Creature
➢ User Studies
 Interaction
Decomposition
 Preliminary Investigation
 Human Affect Display
 Creature Affect Display
 Influence of Affective
Touch

19. HRI 2010 Young Pioneers Workshop - Steve Yohanan 19
User Studies
Interaction Decomposition

20. HRI 2010 Young Pioneers Workshop - Steve Yohanan 20
User Studies
Preliminary Investigation
Affective Touch Proof of Concept

21. HRI 2010 Young Pioneers Workshop - Steve Yohanan 21
User Studies
Preliminary Investigation
The Hapticat
● Wizard of Oz puppet
● Same degrees of freedom
as Haptic Creature
● Hand actuated
● No touch sensing

22. HRI 2010 Young Pioneers Workshop - Steve Yohanan 22
User Studies
Preliminary Investigation
Research Questions Addressed
● Expression
– In what manner might a robot express its emotional state
through touch to a human?
● Recognition
– Can a human recognize a robot's emotional state through
touch?
● Influence
– How does the interaction between human and robot (through
affective touch) influence the human?

23. HRI 2010 Young Pioneers Workshop - Steve Yohanan 23
User Studies
Preliminary Investigation
Results
● Some agreement between our pre-determined
mapping and participants' mapping of gesture to
creature affective response.
● Participants accurately recognized 3 of 5 creature's
affective states: upset, playing dead, asleep.
Confused happy and content.
● Observed shift in participants' affective state as a
result of the interaction.

24. HRI 2010 Young Pioneers Workshop - Steve Yohanan 24
User Studies
Human Affect Display
Creature Recognition of Human Affect

25. HRI 2010 Young Pioneers Workshop - Steve Yohanan 25
User Studies
Human Affect Display
Research Questions Addressed
● Expression
– In what manner might a human express his/her emotional
state through touch to a robot?
● Recognition
– Can a robot recognize a human's touch gestures?
– Can these gestures be used to infer the emotional state of a
human?

26. HRI 2010 Young Pioneers Workshop - Steve Yohanan 26
User Studies
Human Affect Display
Barrett-Russell Emotion Measure (1998)
Activated
aroused
alert
hyperactivated
Unpleasant
miserable
troubled
unhappy
Deactivated
sleepy
still
quiet
Pleasant
happy
pleased
content

27. HRI 2010 Young Pioneers Workshop - Steve Yohanan 27
User Studies
Human Affect Display
1. contact, no movement
2. fingers, idle movement
3. grab
4. hit
5. hold
6. hug
7. kiss
8. lift
9. massage
10. nuzzle
21. slap
22. squeeze
23. stroke
24. swing
25. tap
26. tickle
27. toss
28. tremble
Touch Dictionary
11. pat
12. pick
13. pinch
14. poke
15. press
16. pull
17. push
18. rub
19. scratch
20. shake

28. HRI 2010 Young Pioneers Workshop - Steve Yohanan 28
User Studies
Human Affect Display
Procedure
● Participants will be presented with an emotion word group
alongside the touch dictionary.
● Participants will rank the likelihood that s/he would use each
form of touch to communicate the particular emotion (group).
● Participants will then perform a subset of these gestures
communicating the emotion, so that sensor data can be
recorded.
● Repeat for all emotion word groups.

29. HRI 2010 Young Pioneers Workshop - Steve Yohanan 29
User Studies
Human Affect Display
Affect Communication and Data Capture
● Affect Communication
– Participants will perform each gesture. They will provide
feedback as to which emotion they are conveying.
● Data Capture
– Participants perform each gesture solely for purposes of
recording sensor data. Data will be used for refinement of
the Recognizer. We do not intend to be able to accurately
recognize all gestures, but still want to record them all.

30. HRI 2010 Young Pioneers Workshop - Steve Yohanan 30
User Studies
Creature Affect Display
Human Recognition of Creature Affect

31. HRI 2010 Young Pioneers Workshop - Steve Yohanan 31
User Studies
Creature Affect Display
Research Questions Addressed
● Expression
– In what manner might a robot express its emotional state
through touch to a human?
● Recognition
– Can a human recognize a robot's emotional state through
touch?

32. HRI 2010 Young Pioneers Workshop - Steve Yohanan 32
User Studies
Creature Affect Display
Procedure
● Creature presents 25
different emotions
● Participants rank each
using 12 emotion words
(Barrett-Russell 1998)
● Repeated 3 times

33. HRI 2010 Young Pioneers Workshop - Steve Yohanan 33
User Studies
Influence of Affective Touch
Human Influenced by Interaction

34. HRI 2010 Young Pioneers Workshop - Steve Yohanan 34
User Studies
Influence of Affective Touch
Research Questions Addressed
● Influence
– How does the interaction between human and robot (through
affective touch) influence the human?

35. HRI 2010 Young Pioneers Workshop - Steve Yohanan 35
User Studies
Influence of Affective Touch
Manipulating the Creature's Response
● Observe any changes in emotional state of the human
when the robot presents a response that is:
– mirrored
– sympathetic
– opposed
– random
– inactive

36. HRI 2010 Young Pioneers Workshop - Steve Yohanan 36
Thank You
Questions?