Slides from CHI 2013 presentation of TOCHI paper "Interaction design for and with the lived body: Some implications of merleau-ponty's phenomenology"

1. Dag Svanæs
Interaction Design for and with
the Lived Body –
Some Implications of Merleau-
Ponty’s Phenomenology
Dag Svanæs
NTNU, Trondheim, Norway
ITU, Copenhagen, Denmark
Presentation at CHI 2013, Paris

2. Dag Svanæs
Overview
• Motivations for a theory of the body?
• The body in early HCI theory.
• Merleau-Ponty: The lived body.
• The interactive user experience.
• Reframing interaction as active embodied
perception.
• Examples of active perception.
• Designing with the intelligent body.
• Implications for design.
• Future work.

3. Dag Svanæs
Motivations for a theory of the body
• Current digital products
are designed for active
bodies. They are often
held, worn or carried.
• Their success depends
on an effortless
integration with the living
body of the user.
• The success of a product
like Google Glass is
about more than
ergonomics and usability.

4. Dag Svanæs
The body in early HCI theory
• Card, Moran, and Newell: “The
Psychology of Human-
Computer Interaction”, 1983.
• Cognitive science: The body is
external to cognition. The body
makes possible input and
output, just like the peripherals
of a computer.
• Mind “has” a body.
• Blind spots: The way “body”
shapes “mind” and vise versa.

5. Dag Svanæs
Maurice Merleau-Ponty (1908-61)
• Merleau-Ponty: “The Phenomenology of Perception”,
1945.
• Heidegger refers to the distinction between Körper
and Leib in German:
– Körper is the body as a material object. (3rd person
perspective).
– Leib is the body as that through which I live my life (1st person
perspective).
• Merleau-Ponty: The lived body = Leib. (We are living
bodies).
• Merleau-Ponty starts out with a phenomenological
analysis of perception from a 1st person perspective.

6. Dag Svanæs
Perception is active and directed
Eye Movement Studies (Yarbus, 1967):
– We actively construct our inner
image of the world.
– Perception is always directed
towards something.
– Millisecond level (“pre-cognitive”,
faster than cognition).
Perception!
(passive)!
Action!
(active)!
Early cognitive psychology:
– Perception is passive.
– Information paradigm.
– Action is different from
perception.
– Second and minute level.

7. Dag Svanæs
Task/intention colours perception
• Same image, three different tasks.
• Different viewing patterns.
• Perception is not separate from cognition.
• We live 1/12 second in the past.

8. Dag Svanæs
Skills colour perception
Eye tracking of layperson (left) vs. artist (right) seeing the same work of art.
Training as an artist changes the way people see pictures. Artists “see” all
areas of the picture, while most of us focus on faces and objects in the scene.
Layperson Artist

9. Dag Svanæs
Perception involves the whole body
• We explore objects with
many senses.
• The exploration is an
active process.
• We move, rotate,
touch, smell, taste,
squeeze the object and
change our viewpoint.

10. Dag Svanæs
Instrument
• Merleau-Ponty uses “instrument”
to denote artefacts that enable
the body to both act on the world
with them and to sense the world
through them.
• A blind man's stick (white cane).
• Perception through the cane
requires active perception.
• The cane becomes an extension
of the body (from the 1st person
perspective).

11. Dag Svanæs
Body schema
• The body constantly
maintains knowledge
about its position (the
limbs etc.), its structure,
and its potential for
action.
• This is the body schema.
• The structure includes
the instruments/tools that
have been integrated
into its lived body.

12. Dag Svanæs
The body in everyday coping
• Moving around and
dealing with objects in our
environment does not
require “thinking”.
• The body learns new skills
and easily integrates new
tools/instruments.
• The body is intelligent!
• Challenge: Interaction
design for the intelligent
body.

13. Dag Svanæs
Exploring Interactivity
”13 Rectangles”
Kandinsky
Form + Color Form + Color + Interaction
Abstract interactive
squares
• What stories do people tell?
• What metaphors emerge?
• What dimensions emerge?

14. Dag Svanæs
Experiment ”Square world”!
15 high school students (age 16-17)."
Explore the gadgets"
Think aloud"
Implicit metaphors (Lakoff & Johnson)"
“Understanding Interactivity”, Svanæs, 2000.

15. Dag Svanæs
Interaction patterns – very rapid
Millisecond scale

16. Dag Svanæs
Interaction = Active perception
• When we interact with an
interactive artefact, this is
a kind of active
perception.
• This involves active
perception at two levels.
1. Eyeballs: The visual
image is actively
constructed.
2. Eye-hand-artefact: The
interaction is a sequence
of action-reaction pairs.
Interaction
gestalt

17. Dag Svanæs
Interaction as Perception: Scrolling
• The scrolling wheel allows
for active perception.
• Using the scrolling wheel is
very different from
dragging the scrollbar:
Active perception vs.
Action.
• Active perception is
automated. Little cognitive
workload.

18. Dag Svanæs
Interaction as Perception: Reading
• Turning pages
in a book while
reading is part
of active
embodied
perception.
• eBook
readers: Do
they support
active
perception?

19. Dag Svanæs
Electronic Medical Records
• Turning pages in a
paper-based medical
record is automated.
• It is part of the
embodied active
reading process.
• Gives good eye
contact with patient.
• Both smartphones
and laptops-on-
wheels required
“actions”.
• Gives little eye contact
with patient.
Paper
Laptop on wheels
Smartphone

20. Dag Svanæs
Gaze-controlled scrolling
• Automatic vertical
scrolling of text,
controlled by eye
tracker during reading.
• Feels like reading an
infinite text!
• The scrolling
mechanism becomes
an extension of the
sensory apparatus (an
instrument).
EyeScroll. Kumar & Winograd, 2007.

21. Dag Svanæs
Designing with the body
• Merleau-Ponty: Abstract
vs. concrete movements.
• Concrete movements:
Done as part of an
activity in a context, no
focus on the movement
as such.
• Abstract movements:
Movements done “out of
context”, to try out, to
illustrate, to
communicate.

22. Dag Svanæs
Nintendo Wii in physical
rehabilitation

23. Dag Svanæs
Participatory design workshop
• Lessons learned:
• The body is an important
resource in the design
process.
• Acting out different design
alternatives opens up for
kinaesthetic creativity.
• Designing for the intelligent
body should be done with
the intelligent body, not
through design
representations like
drawings and text.
Physiotherapists inventing a
Nintendo Wii game for their
patients through acting out.

24. Dag Svanæs
Implications for design
• Reframe interaction: For many applications
it makes sense to think of interaction as
active perception.
• Speed: Consider interaction techniques that
allow for very rapid coupling between user
actions and system feedback. Design at the
millisecond level: faster than cognition.
• Mapping: Fluid integration with the intelligent
body requires action-reaction mappings that
are easily “understood” by the body.
• Running prototypes: Technology for the
intelligent body requires high-fidelity
prototypes .
• Use the body in design: Interaction design
for the intelligent body should primarily be
done with the intelligent body – not through
design representations.

25. Dag Svanæs
Future work: Designing for the body
• Research-through-
design project:
– Exploring the design
space of artificial
human tails.
– Applying the design
principles from
Merleau-Ponty.
– What does it take to
make an artificial tail
that is fully
incorporated into the
human body schema?
Shippo by
Neurowear
Monkey with
long tail

26. Dag Svanæs
Future work: Designing for the body
• Research-through-
design project:
– Exploring the design
space of artificial
human tails.
– Applying the design
principles from
Merleau-Ponty.
– What does it take to
make an artificial tail
that is fully
incorporated into the
human body schema?
Shippo by
Neurowear
Monkey with
long tail
Questions?