This is my 3rd year review seminar. I presented an overview of the two studies I have conducted so far.

1. Supporting
Bodily Communication
in Video Consultations of
Physiotherapy
Deepti Aggarwal
Friday, 10 March 17

2. Video Consultations
van Dyk 2014; Yellowlees 2005
Friday, 10 March 17

3. Bodily Communication
Argyle 2013; Heath 1986, 2002
Friday, 10 March 17

4. Bodily Communication in Physiotherapy
Friday, 10 March 17

5. Research Gaps
Friday, 10 March 17

6. Main Research Question
How can interactive technologies
support physiotherapists in
understanding bodily information
during video consultations?
Friday, 10 March 17

7. Research Phases
#1 Observations
#2 Designing & Lab study
#3 Field Deployments
Friday, 10 March 17

8. Research Phases
#1 Observations
#2 Designing & Lab study
#3 Field Deployments
Friday, 10 March 17

9. Research Question
How do physiotherapists
interpret bodily information in
current practices of video
consultations?
Friday, 10 March 17

10. #1 Observations
Face-to-Face (3) Video (7)
2 physiotherapists & 5 patients with chronic pain
Prescribed exercises: tip-toes, squats
Friday, 10 March 17

11. ACM DIS 2016 (full paper)
Friday, 10 March 17

12. Face-to-Face Video
1. Opening
2. History taking
3. Examination
4. Diagnosis
5. Treatment
6. Closing
Byrne & Long 1976
Friday, 10 March 17

13. Face-to-Face Video
Appearance
(Full body)
Posture
Movement
(Walking)
Orientation
Appearance
(Upper Torso)
Posture
Movement
Orientation
1. Opening
Friday, 10 March 17

14. Face-to-Face Video
Facial Expressions
(Tears, red cheeks, tensed
eyes)
Quality of
Movements
(Depth, range, smoothness)
Eye Contacts
(for encouragement,
willingness to engage)
2. History Taking
Eye Contacts
(encouragement,
willingness to engage)
Quality of
Movements
(depth, range, smoothness)
Facial Expressions
(Tears, red cheeks, tensed
eyes)
Friday, 10 March 17

15. Face-to-Face Video
Tactile feedback
(Tightness, inflammation,
temperature)
Response to touch
(Fear, protective spasm,
facial expression)
Touch on
patient’s body
3 & 4. Examination &
Diagnosis
Response to touch
(Fear, protective spasm,
facial expression)
Tactile feedback
(Tightness, inflammation,
temperature)
Touch on own
body
Friday, 10 March 17

16. Face-to-Face Video
Touch on
patient’s body
5. Treatment
Touch on own
body
Body posture
Tone of speech
(pitch)
Efforts
(fatigue, tremor)
Body posture
Tone of speech
(pitch)
Efforts
(fatigue, tremor)
Friday, 10 March 17

17. Face-to-Face Video6. Closing
Body language
Facial expressions
Body language
Facial expressions
Friday, 10 March 17

18. Key Takeaways
Lower limb movements are
difficult to observe
Less specific treatment
Video limits a wide range
of bodily cues
Friday, 10 March 17

19. Research Phases
#1 Observations
#2 Designing & Lab study
#3 Field Deployments
Friday, 10 March 17

20. Key aspects of lower body movements
Weight
Distribution
Range of
Movements
Foot
Orientation
Friday, 10 March 17

21. Research Question
How can we collect and visualize
lower body movements to support
physiotherapists during video
consultations?
Friday, 10 March 17

22. Wii-Fit board
McGough et al. 2012
Friday, 10 March 17

23. Sensoria socks
Sarnow et al. 1994
Friday, 10 March 17

24. SoPhy
SOcks for PHYsiotherapy
Socks with sensors Web interface
Friday, 10 March 17

25. SoPhy socks
IMU
Pressure
Sensors
Bluetooth
Shield
Arduino
Pro Mini
Friday, 10 March 17

26. SoPhy web-interface
Friday, 10 March 17

27. Friday, 10 March 17

28. Research Question
How does SoPhy help
physiotherapists in assessing
lower body movements during
video consultations?
Friday, 10 March 17

29. #2 Lab study
Clinician’s room
10 physiotherapists & 1 actor (patient)
Patient’s room
Friday, 10 March 17

30. Study Design (2x2)
Extreme pain Low pain
Friday, 10 March 17

31. Tasks for
Participants
Assess the patient’s exercises
Organize 4 video sessions
Report confidence in a
questionnaire
Friday, 10 March 17

32. Tasks for Actor
Dorsiflexion &
Plantarflexion
1-leg tip-toes &
2-leg tip-toes
Squats
Friday, 10 March 17

33. Increase in Confidence
Without SoPhy, I may not be able to pick up
things just from video. Like I thought, ‘Ooh!
that foot looks tilted outside’, but then
whether it has any relation with their weight
distribution or not, I can’t tell just from the
video. (P2)
Exercises with SoPhy without SoPhy
Squats
F(1,36) = 10.97, p<.01
M = 5.75,
SD = 1.06
M = 4.17,
SD = 1.13
Friday, 10 March 17

34. Fewer Repetitions of Exercises
Exercises with SoPhy
without
SoPhy
Dorsiflexion
F(1,36) = 6.99, p<.05
M = 8.10,
SD = 3.21
M = 11.25,
SD = 4.54
Plantarflexion
F(1,36) = 6.14, p<.05
M = 7.45,
SD = 3.88
M = 10.60,
SD = 4.24
Squats
F(1,36) = 8.36, p<.01
M = 6.05,
SD = 2.48
M = 8.05,
SD = 3.18
Friday, 10 March 17

35. Fewer Repetitions of Exercises
Over video, I can see the person only from
one direction. [...] The sock system provides
me this detailed information irrespective of
how the person is standing or sitting. I did not
ask the patient to turn backwards or sideways
when I had the sock data - the system was
already doing it for me. (P3)
Friday, 10 March 17

36. Deeper Insights on Weight Distribution
It’s always challenging to understand weight
bearing because the pressure points are not
visible. It’s easy in cases when the person is
putting more weight on one foot than other.
But it is difficult to understand how the
pressure is distributed across the foot, is it on
the heels, or on the balls. (P8)
Friday, 10 March 17

37. Challenges
Interference to visual
observation
Alignment with clinical practice
Friday, 10 March 17

38. ACM CHI 2017 (full paper)
Friday, 10 March 17

39. Limitations
Controlled lab study
Focus on assessment
Friday, 10 March 17

40. Research Phases
#1 Observations
#2 Designing & Lab study
#3 Field Deployments
Friday, 10 March 17

41. Research Question
How does SoPhy support
physiotherapists in assessing
and treating patients, with lower
limb issues, during naturally
occurring video consultations?
Friday, 10 March 17

42. Revised SoPhy socks
Use of bright colored socks to support visual observations
Friday, 10 March 17

43. Revised SoPhy web-interface
To align with clinical practice
Friday, 10 March 17

44. Study Design
2 physiotherapists & 5-7 patients with lower limb issues
Simulated video sessions at Royal Children’s Hospital
Hospital Room #1 Hospital Room #2
Friday, 10 March 17

45. Study Design
Face-to-Face
(60 mins)
Video
Consultation
(45 - 50 mins)
Face-to-Face
follow up
(15 - 10 mins)
First session Second session
Participant Observations & Interviews
Friday, 10 March 17

46. Study Setup
Friday, 10 March 17

47. Session in Action
Friday, 10 March 17

48. Video without SoPhy
(study 1)
1. Opening
2. History taking
3. Examination
4. Diagnosis
5. Treatment
6. Closing
Video with SoPhy
(study 3)
Friday, 10 March 17

49. Study 1
Video with
SoPhy
Appearance
(Upper Torso)
Posture
(Upper Torso)
Movement
(sitting)
Orientation
Appearance
(Upper Torso)
Posture
(upper torso + feet)
Movement
(walking, sitting)
Orientation
1. Opening
Friday, 10 March 17

50. Thesis Contributions
Conceptual understanding of bodily communication
in video consultations of physiotherapy
Designed and studied a technology to support
physiotherapists during video consultations
Friday, 10 March 17

51. Dissertation Status
Commenced study 3
Finished study 1, 2
Thesis Chapters 4 & 6 - first
draft finished
Friday, 10 March 17

52. Completion Plan
Time Tasks
March - April
2017
Continue study 3
Finish Chapters 2, 3 & 5
Revise Chapters 2 to 6
May - June
2017
Attend CHI 2017
Finish Study 3
Finish Chapters 7, 8, 9
July - August
2017
Finish Chapter 1
Revise complete thesis
Thesis Submission
Friday, 10 March 17

53. Thank you
Friday, 10 March 17