My presentation of the paper "Open-World Mission Specification for Reactive Robotics" in the "Formal Methods I" session of the 2014 International Conference on Robotics and Automation, Hong Kong, China. (The 3 embedded videos won't play on Slideshare.)
Sheet Pile Wall Design and Construction: A Practical Guide for Civil Engineer...
Open-World Mission Specification for Reactive Robots - ICRA 2014
1. Open-World Mission Specification
for Reactive Robots
Spyros Maniatopoulos
Matthew Blair, Cameron Finucane, Hadas Kress-Gazit
Sibley School of Mechanical and Aerospace Engineering
Cornell University
sm2296@cornell.edu
verifiablerobotics.com
5. “Closed” vs. “Open” Worlds
5
“Closed-World Assumption”
Only elements of the world modeled a priori
can be taken into account during execution.
Robotic assembly by
DENSO’s robot arms.
Source: www.robots.com
6. “Closed” vs. “Open” Worlds
6
“Open-World”
New elements and objectives arise during execution.
Photo: Panagiotis Papadakis
Source: spectrum.ieee.org
7. Open-World Challenges
• Not all mission variables known a priori
• Mission objectives change during execution
• React to unexpected events during execution
• Incorporate/learn new functionality on-the-fly
7
8. Related Work
8
S. C. Livingston, R. M. Murray, and J. W. Burdick, “Backtracking temporal logic
synthesis for uncertain environments,” ICRA, 2012.
S. C. Livingston, P. Prabhakar, A. B. Jose, and R. M. Murray, “Patching task-level
robot controllers based on a local μ-calculus formula,” ICRA, 2013.
M. Guo, K. H. Johansson, and D. V. Dimarogonas, “Revising motion planning
under linear temporal logic specifications in partially known workspaces,”
ICRA, 2013.
A. I. Medina Ayala, S. B. Andersson, and C. Belta, “Temporal logic motion
planning in unknown environments,” RSS, 2013.
S. Sarid, B. Xu, and H. Kress-Gazit, “Guaranteeing high-level behaviors while
exploring partially known maps,” RSS, 2012.
10. Logic-based Mission Specification
10
Boolean Operators Temporal Operators
Linear Temporal Logic
Robot starts in r1
If you are sensing alarm then visit r2
Structure
d
English
“Starting in region r1,
repeatedly visit the region r2 if you are not sensing an alarm”
GR(1)
16. Open-World Abstractions – Groups
16
Group Letters is letter_Spyros,
letter_Matt, letter_Cameron
Groups of Propositions
Letters (sensors)
letter_Spyros
letter_Matt
letter_Cameron
Offices (regions)
office_Spyros
office_Matt
office_Cameron
17. Open-World Abstractions – Quantifiers
17
Quantification over Groups of propositions – all
Letters
letter_Spyros
letter_Matt
letter_Cameron
all Letters
18. Open-World Abstractions – Quantifiers
18
Quantification over Groups of propositions – any
Letters
letter_Spyros
letter_Matt
letter_Cameron
any Letters
19. Open-World Abstractions – Quantifiers
19
Quantification over Groups of propositions – each
Offices
office_Spyros
office_Matt
office_Cameron
Go to each Office
• Go to office_Spyros
• Go to office_Matt
• Go to office_Cameron
21. Open-World Abstractions – Correspondence
21
Letters correspond to Offices
If you are sensing any Letter then
go to the corresponding Office
Correspondence between Propositions
27. Resynthesis
27
If you are sensing newLetter then
add to group Letters and resynthesize
Global Resynthesis
S. Sarid, B. Xu, and H. Kress-Gazit, “Guaranteeing high-level
behaviors while exploring partially known maps,” RSS, 2012
32. Summary & Future Work
32
Summary
• Open-world abstractions allow implicit reference to propositions
• New elements are automatically incorporated in the specification
• Changes reflected in robot’s controller via (global) resynthesis
Future Work
• Automatic removal of propositions
• Efficient resynthesis
• Proposition grounding
33. Open-World Mission Specification
for Reactive Robots
Spyros Maniatopoulos
Matthew Blair, Cameron Finucane, Hadas Kress-Gazit
Sibley School of Mechanical and Aerospace Engineering
Cornell University
sm2296@cornell.edu
verifiablerobotics.com
Editor's Notes
Hi all, thank you for sticking around. I’m all that’s left between you and the banquet!
My name is … work done in collaboration with …
Today I will be talking about OWMS
But before I get to that, let me first tell you what our group does.
Our group is interested in high-level mission planning.
Given, … automatically synthesize a correct-by-construction reactive controller.
In order to continuously implement the discrete strategy, we construct a hybrid controller. Then execute in simulation or experimentally
Tell a story: “Last summer, we decided get together as a team and work on a big experimental demo.
“We decided to work on an autonomous robotic courier, a mailbot!” “It consists of an Aldebaran Nao humanoid robot mounted on a Segway-based platform.”
“Its task was to pickup letters, like the one it’s holding, and deliver them to their recipients’ offices.
The mission was specified based on letter recipients and their offices. Those variables were hardcoded a priori in the mission specification.
However, what if a new professor moves in our department building? In the current setting, the user would have to pause execution, manually re-write the mission specification to incorporate this new letter recipient, synthesize a new controller, and then resume execution. That doesn’t sound very autonomous.
Reason? World open wrt letter recipients. What do I mean by “open”? Well, let’s look at what “closed” means first.
Assembly line tasks:
the environment is predictable,
the workspace highly structured,
and the objectives are mostly fixed and repeated over and over
Search & Rescue: workspaces are partially unknown and highly unstructured, and the mission objectives change on-the-fly
Other open-world missions can be found in Autonomous space and planetary exploration, Military scenarios, and Personal robots
In such settings, the closed-world assumption no longer holds.
In general, these are the challenges an open-world presents the robot with …
So, within our high-level mission specification and planning framework, we want to account for these situations in a systematic and autonomous fashion.
This work is but a first step towards tackling these challenges.
Before I get to the approach, I want to give you a brief overview of recent work related to this paper.
Prior work within our group, before I joined. We use a result of that work in our approach. That work involves …
Scott Livingston at Caltech has been working on patching reactive controllers in order to account for local topological changes in the robot’s workspace. The approach is complementary to ours. Specifically, Scott’s work deals with the controller synthesis level, while what I’m about to present deals with the mission specification level.
Also, at KTH in Sweden, they’ve worked on updating a robot’s plan as it’s navigating a partially-known workspace.
Finally, this paper by Calin Belta’s group at BU provides an interesting perspective. There, a robot starts by exploring its unknown workspace without falsifying its mission specification. Once it has enough information, it proceeds to satisfy it.
Set of Atomic Propositions (binary variables)
Explain what each type of proposition is abstracting, in terms of the mailbot scenario.
Use atomic propositions to write LTL formulas.
We are working with the GR(1) fragment of LTL.
“A simple reactive robot task would be …”
In our group’s framework, we also use Structured English, which parses directly to the GR(1) fragment of LTL. Same example “…” Our framework also supports Natural Language, but this is not demonstrated in this presentation.
Overview of our approach.
It consists of 4 main components.
Let’s start with the first one; modeling new elements of the open world.
We define a special class of sensor propositions, detectors propositions.
For example, the mailbot would have 1 such detector, which would trigger if the robot is handed a letter addressed to a previously unknown recipient.
Function D creates a new proposition (at the discrete level) and grounds it to the new element of the world (in the continuous domain).
For example, in the mailbot scenario, a letter proposition would be grounded to a radial barcode.
Quick aside on notation: t denotes the execution time step, whereas k is only incremented when a new element is detected.
When a detector proposition becomes true, a new proposition is generated …
… and added to the set of atomic propositions
Now, we want to augment the specification language s.t. the robot’s mission can be automatically updated when a new proposition is added.
We will augment the specification language with open-world abstractions.
You can think of it as adding elements of first-order logic.
There are three types of open-world abstractions.
The first one is groups of propositions.
All Group is substituted by the logical conjunction of all propositions in the group.
Any Group is substituted by the logical disjunction of all propositions in the group.
The task has to happen for each proposition in the group.
Finally, we define correspondence between propositions.
If the propositions in two groups have a 1-to-1 correspondence, we also say that the groups themselves correspond to each other.
In a mission specification written in Structured English, this is how we would declare and use correspondence.
We first declare correspondence between groups, and then use the corresponding operator to specify a task.
And in general, this is the substitution that takes place.
Using these open-world abstractions, we can now specify missions without explicitly referring to individual propositions.
…
But this still doesn’t solve the problem of dealing with new propositions…
1) … to accomplish that, we’ll need a mechanism that allows us to incorporate new propositions in the mission specification.
This mechanism is the add_to_group operator.
Going back to the mailbot scenario, when a new letter recipient is detected, a new proposition is generated.
This operator adds this new proposition to the appropriate group.
However, this new proposition now has to correspond to something in the other group.
So the add_to_group operator not only adds a new proposition to a group, but it also generates more propositions, if necessary, and adds them to the corresponding groups.
I’m later expecting questions about proposition grounding.
In a mission specification, we would use this operator as such: …
Quick clarification: the detector proposition “newLetter” doesn’t trigger upon seeing an envelope. Rather it abstracts the sensing of a letter addressed to a previously unknown recipient!
So far we’ve seen how the newly detected elements of the world are incorporated in the mission specification.
The final step is to ensure that the changes to the mission are reflected in the robot’s controller.
To accomplish that, we leverage work done previously in our group.
Namely, global resynthesis, which involves saving the current mission state, synthesizing a new robot controller, and resuming execution.
Notice that, in the mission specification, the user has control over when resynthesis will take place, by appropriately specifying that task.
1) Quick summary …
Let’s see how a simplified version of the mailbot mission plays out in simulation.
We use LTLMoP, our group’s Python toolkit for mission specification, controller synthesis, and simulation.
Explain spec …
Explain buttons: red = false, green = true
After resynthesis, there’s a new letter proposition and a new mission objective.
So that was in simulation, and you saw how we queried the user in order to ground the propositions.
Each radial barcode corresponds to a letter recipient and address.
And the addresses are grounded to region propositions
The recipient and address fields on the letter are used instead of a user query.