ENG 5 Q4 WEEk 1 DAY 1 Restate sentences heard in one’s own words. Use appropr...
Ruzena Bajcsy - Personalized Modeling for HRI
1. Personalized Modeling for
Human-Robot Interaction
Ruzena Bajcsy
Electrical Engineering & Computer Sciences
University of California, Berkeley
October 23, 2015
2. Our Lab’s Motivation
We wish to understand the mechanics of human-robot interactions
and to design algorithms for control-sharing between humans and
autonomous systems.
Why is this problem important?
– A person is a complex kinematic/dynamic system with many
degrees of freedom and parameters which vary from person
to person
– Not all degrees of freedom are used in all activities
4. Human Model Classes: Kinematic
Musculoskeletal
Kinematic/
Dynamic
Kinematic
Agent
Interaction
Goal
Model human motion using a rigid-body
kinematic model
Application
Automated Coaching/
Quantitative Outcome Measures
Contributors: Qifei Wang, Gregorij Kurillo, and Ferda Ofli
5. Human Model Classes: Kinematic/Dynamic
Musculoskeletal
Kinematic/
Dynamic
Kinematic
Agent
Interaction
Goal
Model human motion using a rigid-body
kinematic/dynamic model
Applications
Human-Robot
Collaborative
Manipulation
Human Dynamic
Stability Analysis
Contributors: Aaron Bestick and Victor Shia
6. Application: Collaborative Manipulation
Goal: Enable intelligent
control of robots
providing direct physical
assistance to humans
• Create unified model
of the human-robot
coupled mechanical
system
• Predict intent of
human operator
based on physical cues
7. Application: Collaborative Manipulation
Personalized Human
Mechanical Models
Coupled Human-
Robot Dynamical
Models
Optimal Robot
Control
Human
Constraints
Robot
Constraints
Task
Constraints
Human
Ergonomic
Cost
𝜏∗
= arg min
𝜏∈𝑈
𝑓 𝜃, 𝜏
𝑠. 𝑡. 𝑔 𝜃, 𝜏 ≤ 0,
ℎ 𝜃 = 0
Key Neglected Aspect: Differences in constraints and cost functions
between individual humans (e.g. age, disabilities, natural variation)
Implicit Assumption: Differences between humans not a significant
contributor to task variability
Need personalized models
8. Human Model Classes: Musculoskeletal
Musculoskeletal
Kinematic/
Dynamic
Kinematic
Discrete
States
Goal
Combine a kinematic/dynamic model with a
nonlinear model of muscle characteristics to
predict biomechanical properties throughout
the human’s workspace
Application
Medical Diagnostics
9. Dynamic Human Musculoskeletal Modeling
• Data Acquisition
– MRI Scans
– DICOM Images with
Segmentation
– Motion Capture
– EMG Data
• Modeling
– 2D/3D Visualization
– Interactive Cleaning
– Static, Kinematic, Dynamic
Scenarios
– Physics based Dynamic
Deformation
MRI Data segmentation
10. Ultrasound for Muscle Observation
• External motion capture for
pose of ultrasound
• Muscle and tendon outlines
visible
Muscle
11. Ultrasound for Muscle Observation
• External motion capture for
pose of ultrasound
• Muscle and tendon outlines
visible
Muscle
12. Ultrasound for Muscle Observation
• External motion capture for
pose of ultrasound
• Muscle and tendon outlines
visible
Tendon
13. Ultrasound for Muscle Observation
• External motion capture for
pose of ultrasound
• Muscle and tendon outlines
visible
Tendon