my presentation at IEEE ROBIO Dec. 13, 2018 at Kuala Lumpur, Malaysia

1. Searching Behavior of a Simple Manipulator
only with Sense of Touch Generated by
Probabilistic Flow Control
Ryuichi Ueda
Chiba Institute of Technology

2. decision making under uncertainty
• The real world is essentially uncertain.
• sensor noises
• situations where some sensors are invalid
• ex.: RGB camera in darkness
• Animals and human beings decide their actions in
uncertain situations.
• ex.: a person who goes to his/her bedroom in darkness
Dec. 13, 2018 ROBIO2018 2
problem: how to model such intelligent behavior

3. our past work1: real-time Q-MDP [Ueda 2005]
• method
• preparing a value function (a precise potential function)
beforehand under the assumption of no uncertainty
• calculating the expected improvement of the potential
with particles of MCL (Monte Carlo localization)
• decision making with consideration
of localization uncertainty
• certain -> keeping the position
in the goal
• uncertain -> rushing to the ball
Dec. 13, 2018 ROBIO2018 3
x 2
local minima problem unsuitable for simple navigation

4. our past work2:
probabilistic flow control (PFC) [Ueda 2015]
• giving large weights to particles that have good values
• reduction of local minima
• generating a kind of search behavior
Dec. 13, 2018 ROBIO2018 4
x
value
goal
deadlock
particles
real-time Q-MDP
x
goal
PFC
weighted by
the value

5. search behavior by PFC
• motion that compensates for incomplete self-localization
Dec. 13, 2018 ROBIO2018 5
only one
landmark
goal
The robot
may exist in
this area.
The robot must
go here.
(Is it possible??) robot: dragged by particles
→ search behavior

6. purpose
•to find another search behavior
• with another type robot
• evaluation of various rates of weights
Dec. 13, 2018 ROBIO2018 6
x
goal
low rate
x
goal
high rate

7. searching rod problem
• a simple robot manipulator
and a fixed rod in the environment
• task of the robot: to get the rod in its hand
• restrictions of observation
• When the robot touches the rod,
it feels the rod somewhere in its body.
• When the rod enters in the hand,
the robot notices the task completion.
• The angles of the joints are known.
Dec. 13, 2018 ROBIO2018 7
x
y
fixed rod
(unknown position)

8. applying PFC
• off-line motion planning with known positions of the rod
• solving V(𝜽, 𝒙rod)
• V(𝜽, 𝒙rod): number of steps to the goal
• 𝜽: joint angles (two dimensional, known)
• 𝒙rod: position of the rod (two dimensional, treated as known)
• on-line
• localization of a rod with the touch sense
• decision making
• 𝑎 = argmin
𝑎
𝒙rod
𝑃(𝒙rod)
V(𝜽, 𝒙rod) 𝑚 V 𝜽′, 𝒙rod
Dec. 13, 2018 ROBIO2018 8
probability distribution
where the rod is
action of the robot posterior joint angles by 𝑎rate of weight

9. generated motion
• known rod position
• just the shortest time motion
• PFC (unknown rod position):
• The robot shows behavior like
• searching the rod
• tapping the rod
• changing the folding direction of the arms
Dec. 13, 2018 ROBIO2018 9
optimal control with
the known rod position
(background red color: probability distribution of the rod’s position)

10. • Parameters of the previous works cause local minima.
• Large 𝑚 values prevent the local minima problem.
• Large 𝑚 values delay the task completion.
success rates
effect of the rate of weights 𝑚
Dec. 13, 2018 ROBIO2018 10
Q-MDP
PFC [Ueda 2015]
number of steps
large priority

11. conclusion
• Searching behavior of a manipulator can be
generated with PFC.
• future works
• applying PFC to more practical cases
• making 𝑚 variable
Dec. 13, 2018 ROBIO2018 11