ROS Ecosystem & Applications [HyphaROS] -- by HaoChih

Website: https://hypharosworkshop.wordpress.com/contact/
FB Page: https://www.facebook.com/HyphaROS/
Email: hypha.ros@gmail.com
ROS Ecosystem & Applications
Lecturer : HaoChih, LIN
林浩鋕

https://hypharosworkshop.wordpress.com/
Copyright © 2017 Hypha ROS Workshop. All rights reserved Lecturer: HaoChih, LIN
● About us
● ROS Introduction
● SLAM Package
● Applications
(ARDrone Indoor monoSLAM & Navigation)
● ROS Industrial
● ROS 2.0
● Q & A

About us
● Introduction
○ Website: https://hypharosworkshop.wordpress.com/
○ Github: https://github.com/Hypha-ROS/hypha-racecar
○ Facebook: https://www.facebook.com/HyphaROS/
○ Youtube: https://www.youtube.com/watch?v=jgS6gpmWPoY
○ Youku: http://i.youku.com/hypha

About us
● Introduction
○ Established date: April-2017
○ Self-organized Workshop: 3
○ Speech & Training: 6
○ Technical supports: > 10 universities/labs & > 5 companies
○ HyphaROS RaceCar ver 1.0 released (sold 30 units)
4

About us
● Introduction
○ Lecturer in “ROS Summer School in China 2017”
○ Presenter in “2017 WRC ROS Forum”
○ Product & Service [ROS]: Workshop, Technical Consultant, AGVs, Home robots, etc..
○ Cooperate with Chips Company (China), will attend CES 2018 (if everything goes right...)

About us
ROS.Taipei 2018 開發者年會
Official Website: https://ros-taipei.wixsite.com/2018
FB Page: https://www.facebook.com/groups/ros.taipei/

ROS Introduction

ROS Introduction
Source: http://www.ros.org/history/

ROS Introduction
● Package
○ include/package_name: C++ include headers
○ msg/: Folder containing Message (msg) types
○ src/: Source files
○ srv/: Folder containing Service (srv) types
○ scripts/: executable scripts
○ CMakeLists.txt: CMake build file (see catkin/CMakeLists.txt)
○ package.xml: Package catkin/package.xml
● Node
● Message [A message is simply a data structure, comprising typed fields (much like C structs).]
● Master [provides name registration and lookup to the rest of the Computation Graph]
● Parameter Server
● Topics
● Service [for request / reply interactions]
● Rosbag
● Actionlib [A common, topic-based interface for preemptible tasks in ROS.]
● Plugins [provides a library for dynamically loading libraries in C++ code.]

ROS Introduction
● [concept] Establishing a topic connection
○ Subscriber starts.
○ Publisher starts.
○ Subscriber registers with the Master. (XMLRPC)
○ Publisher registers with the Master. (XMLRPC)
○ Master informs Subscriber of new Publisher. (XMLRPC)
○ Subscriber contacts Publisher to request a topic
connection and negotiate the transport protocol. (XMLRPC)
○ Publisher sends Subscriber the settings
for the selected transport protocol. (XMLRPC)
○ Subscriber connects to Publisher using
the selected transport protocol. (TCPROS, etc...)
figure source: http://wiki.ros.org/ROS/Technical%20Overview

ROS Introduction
• Standard Linux tools:
Compilers, Debuggers, Loggers, IDEs
• Multiple language support:
C/C++, Python, Java, Lisp, Go, Scratch, etc.
• Standard libraries:
Boost, MySQL, XML (whatever you can imagine)
• Modern GUI tools
• Many tools missing from industrial development packages
Source: ROS-Industrial Introduction slides 2013, Shaun Edwards

ROS Introduction
c
Source: Navigation Illumination Slides, David V. Lu 2014
David V. Lu

ROS Introduction
2014 2016 2017
Source: ROS metrics-report-2017-07

ROS Introduction
Source: ROS metrics-report-2017-07

ROS Introduction
Source: RosCon2017 Opening Remark

ROS Introduction
Source: https://software.intel.com/en-us/articles/build-an-autonomous-mobile-robot-with-the-intel-realsense-camera-ros-and-sawr
IntelROSProject
Realsense
Movidius

ROS Introduction
IntelROSProject
Realsense
Movidius

• Website
1. ROS Install (Kinetic): http://wiki.ros.org/kinetic/Installation/Ubuntu
2. ROS Wiki (Tutorial): http://wiki.ros.org/ROS/Tutorials
3. CLEARPATH Tutorial: http://www.clearpathrobotics.com/assets/guides/ros/
• Youtube Video
1. ROS Tutorial:
https://www.youtube.com/playlist?list=PLSzYQGCXRW1HLWHdJ7ehZPA-nn7R9UKPa
2. Erle Robotics:
https://www.youtube.com/playlist?list=PL39WpgKDjDfVfiNVG47DBi93wsh2XHKVO
3. The Construct: http://www.theconstructsim.com/
ROS Introduction

2D Laser SLAM

● Gmapping
○ needs external odom (in our case: laser odom fused with imu0)
○ Rao-Blackwellized Particle Filter (RBPF)
○ Loop closure
● Hector SLAM
○ Gauss-Newton approach
○ External odom is not necessary
○ No loop closure
○ Requires high quality sensor (e.g. UTM-30LX)
○ Supports EKF fusion (imu)
● MRPT-ICP SLAM
○ ICP algorithm
○ For small to mid-sized maps
○ External odom is not necessary
2D Laser SLAM

2D Laser SLAM
gmapping (laser odom) mrpt icp hector slam

● Google cartographer
○ http://wiki.ros.org/cartographer
● MRPT-ICP SLAM
○ https://github.com/ethz-asl/ethzasl_icp_mapping
2D Laser SLAM

3D SLAM

● 3D Laser
○ google cartographer
○ ETHZ-ASL libpointmatcher
○ ETHZ-ASL laser_slam
● Visual (RGBD)
○ Rtabmap
○ RGBD SLAM (v1, v2)
● Visual Stereo
○ Duo-VIO
● Visual Monocular
○ PTAM
○ LSD_SLAM (semi-dense)
○ SVO (semi-dense)
○ ORB-SLAM (feature based)
● Visual Inertial Odometry/SLAM
○ Loosely coupled: MSF_EKF
○ Tightly coupled: OKVIS, ROVIO, Duo-VIO
○ Framework: ETHZ-ASL maplab
3D SLAM & Odometry
Source: https://github.com/ethz-asl/maplab

● RtabMap
3D SLAM & Odometry
Source: http://wiki.ros.org/rtabmap_ros
Real-Time Appearance-Based Mapping
http://introlab.github.io/rtabmap/

3D SLAM & Odometry
Tutorials
1.RGB-D Hand-Held Mapping
2.This tutorial shows how to use rtabmap_ros out-of-the-box with a Kinect-like sensor in mapping mode or localization mode.
3.Stereo Hand-Held Mapping
4.This tutorial shows how to use rtabmap_ros out-of-the-box with a stereo camera in mapping mode or localization mode.
5.Remote Mapping
6.This tutorial shows how to do mapping on a remote computer.
7.Stereo Outdoor Mapping
8.This tutorial shows how to do stereo mapping with RTAB-Map.
9.Stereo Outdoor Navigation
10.This tutorial shows how to integrate autonomous navigation with RTAB-Map in context of outdoor stereo mapping.
11.Setup RTAB-Map on Your Robot!
12.This tutorial shows multiple RTAB-Map configurations that can be used on your robot.
13.Mapping and Navigation with Turtlebot
14.This tutorial shows how to use RTAB-Map with Turtlebot for mapping and navigation.
15.Tango ROS Streamer
16.Tutorial to get Tango ROS Streamer working with rtabmap_ros
17.Advanced Parameter TuningThis tutorial tells you which parameter to change to improve performances
18.Wifi Signal Strength Mapping (User Data Usage)
19.This tutorial shows how to add user data during mapping that will be saved directly in RTAB-Map's database for convenience.
● RtabMap Real-Time Appearance-Based Mapping

3D SLAM & Odometry

3D SLAM & Odometry
● 3D mapping on Odroid XU4 with Realsense R200
ORB vs GFTT/ORB (BRISK is not good enough)
320 * 240
loop closure (g2o)
Vis + ICP
PCL filter and octomap
The IMU issue (mpu6050 -> gy85)
Supports multi-mapping

3D SLAM & Odometry
● ORBSLAM v1 & v2 http://webdiis.unizar.es/~raulmur/orbslam/

3D SLAM & Odometry
● SVO 1 & 2 http://rpg.ifi.uzh.ch/svo2.html
1. Parrot-SenseFly Albris drone:
https://www.youtube.com/watch?v=mYKrR...
2. Virtual reality:
https://www.youtube.com/watch?v=k0MLs...
3. iPhone app 3D around:
https://3daround.dacuda.com/
4. iPhone app Staramba3D:
https://itunes.apple.com/us/app/stara...
5. Zurich-Eye (now Facebook-Oculus VR):
http://www.zurich-eye.com/

3D SLAM & Odometry
● SVO 1 & 2
http://rpg.ifi.uzh.ch/svo2.htmlhttps://github.com/uzh-rpg/rpg_svo

● ETHZ-ASL libpointmatcher
3D SLAM & Odometry
Source: https://github.com/ethz-asl/libpointmatcher
An "Iterative Closest Point" library for 2-D/3-D mapping in Robotics

● ETHZ-ASL OKVIS & ROVIO
3D SLAM & Odometry
https://github.com/ethz-asl/okvis_ros https://github.com/ethz-asl/rovio

● ETHZ ASL MapLap
3D SLAM & Odometry
Source: https://github.com/ethz-asl/maplab
An Open Framework for Research in Visual-inertial Mapping and Localization

HyphaROS RaceCar

HyphaROS RaceCar: Introduction
● The requirements of the platform
○ ROS fully support
○ Able to implement 2D Navigation stack (laser based)
○ Low cost (ARM SBC, low-cost lidar, mems imu, etc)
○ High speed/performance
○ Robust & Safety
○ Algorithms evaluation/comparison
○ Modularization & Extensibility

● The origin of RACECAR
MIT: https://mit-racecar.github.io/
Youtube: https://www.youtube.com/watch?v=9fzzp6oxid4
1/10-scale Traxxis Rally Car
Nvidia Jetson TX 1
Hokuyo UST-10LX laser range finder
Stereolabs ZED stereo camera
Structure.io depth camera
Sparkfun IMU
Encoder + optical flow
realtime onboard 4 m/s
Total cost: around 3,600 USD

● Similar projects
Penn: http://f1tenth.org/ [without slam, NAV]
UCB: http://www.barc-project.com/projects/ [without laser]
Georgia Tech: https://github.com/AutoRally [for outdoor]
CMU: https://github.com/mrsd16teamd/loco_car [indoor]

MIT RaceCar

Georgia Tech AutoRally

● Our Solution
○ Open-Source, Open-Hardware
○ Low-cost (~ 600 USD), High-speed (~ 3 m/s)
○ Step by step tutorial for beginner (available on github: Hypha-ROS)
Version 1 Version 2

HyphaROS RaceCar: Major Functions
HyphaROS
RaceCar
Localization
Odometry
SLAM
Base
Controller
Navigation
Sensors/
Actuators
Drivers

HyphaROS RaceCar: Hardware Architecture
Odroid XU4
Arduino
Nano
USB
Hub
Gy85
Servo
Motor ESC
Arduino
UNO
RPLidar
(A1M8)
FTDI
USB
USB
I2C
PWM
PWM
Wifi
TX/RX
USB
Ethernet
7.4V BatteyBEC
DC 2 DC
Core
Motor
BEC

HyphaROS RaceCar: Software Architecture
move_base
(path planning)
robot_localization
(EKF)
rf2o
(laser odom)
PySerial
(imu reading)
Filtered
Odometry • Pure Pursuit
• TEB control
• MPC
rosserial
(to arduino)
rplidar_ros
(laser scan)
Ackerman
(rpm odom)
GPS
(optional)

HyphaROS RaceCar: Software Architecture

HyphaROS RaceCar: Github
HyphaROS
VM image
Desktop sharing
(vino + remmina)
Terminal or File
manager (ssh)
All source code can be found
on Hypha-ROS github:
https://github.com/Hypha-
ROS/hypha-racecar
(including udev setting, Arduino
files, necessary documents)

Drone Applications

Drone Applications
● Simulation
○ Gazebo (with PX4)
○ AirSim Simulation
● Well known Projects
○ APM
○ PX4 & Dronecode
○ PaparazziUAV
○ CrazySwarm
○ Erlerobot gym-gazebo
○ DJI SDK
○ ARDrone SDK

Drone Applications
● PX4 X Gazebo https://dev.px4.io/en/simulation/gazebo.html

Drone Applications
● AirSim https://github.com/Microsoft/AirSim

Drone Applications
● CrazySwarm http://crazyswarm.readthedocs.io/en/latest/

Drone Applications
● PaparazziUAV https://blog.paparazziuav.org/

MCU
(Pixhawk, APM, Arduino)SBC (Odroid XU4, Nvidia TX1/2 or NUC)
Drone Applications
● General Architecture (Loosely coupled)
MoveIt
(octomap &
Path)
Trajectory
Generator
(optional)
Serial
e.g. MAVROS
PySerial
VO/SLAM
Sensors
Imu, gps
MSF-
EKF
RGB
camera
High level
Controller
Platform
(HW)
Low level
Controller

ARDrone Indoor Navigation
(LSD-SLAM)

LSD-SLAM
PTAM (IMU)
Least
square
MoveIt
(octomap & RRT*)
Trajectory
Generator
NDI
Controller
In: RGB image
In:
• RGB image
• IMU
• Sonar
Out: scaled pcl Out: way points Out: trajectory
Out: cmd
Could be
replaced by
MSF-EKF

●LSD-SLAM (Large-Scale Direct Monocular SLAM)
As direct method, LSD-SLAM uses all information in the image, including
e.g. edges – while keypoint-based approaches can only use small
patches around corners. This leads to higher accuracy and more
robustness in sparsely textured environments (e.g. indoors), and a much
denser 3D reconstruction.

Monocular SLAM non scaled issue

𝑀 𝑞𝑙 𝑖
𝛿𝑞 − 𝑞𝑡 𝑖
2
Am+1 = Am + 𝑀 𝑇 𝑞𝑙 𝑚+1
𝑀(𝑞𝑙 𝑚+1
)
ym+1 = ym + 𝑀 𝑇 𝑞𝑙 𝑚+1
𝑞𝑡 𝑚+1
[𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑖
𝛿𝑥
𝛼
− 𝑥 𝑡 𝑖
|2
Am+1 = Am + 𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑚+1
𝑇
𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑚+1
ym+1 = ym + 𝐼3×3 𝑅 𝛿𝑞 𝑥𝑙 𝑚+1
𝑇
𝑥 𝑡𝑖
Recursive least-squares
M is quaternion multiplication matrix
R is quaternion rotation matrix
Monocular SLAM non scaled issue

Let 𝑝 𝑛, denote the coefficients of a polynomial P of degree N such that
P t = 𝑃𝑛 𝑡 𝑁
+ 𝑃𝑛−1 𝑡 𝑛−1
. . . +𝑝0 =
𝑛=0
𝑁
𝑃𝑛 𝑡 𝑛
We are interested in optimizing the coefficients of P to minimize cost functions (lateral offset
from a nominal path) of the form
J =
0
𝜏
𝑐0 𝑃 𝑡 2
+ 𝑐1 𝑃′
𝑡 2
+ 𝑐2 𝑃′′
𝑡 2
+ . . . +𝑐 𝑁 𝑃 𝑁
𝑡 2
𝑑𝑡
which can be written in quadratic form as, J = 𝑝 𝑇
𝑄 𝑝
where
Q =
𝑟=0
𝑁
𝑐 𝑟 𝑄 𝑟
Qr
𝑖𝑙
=
2(
𝑚=0
𝑟−1
(𝑖 − 𝑚)(𝑙 − 𝑚))
𝜏 𝑖+𝑙−2𝑟+1
𝑖 + 𝑙 − 2𝑟 + 1
; 𝑖 ≥ 𝑟 ∧ 𝑙 ≥ 𝑟
0; 𝑖 < 𝑟 ∨ 𝑙 < 𝑟
Control, Estimation, and Planning Algorithms for Aggressive Flight using Onboard Sensing, MIT, Adam Parker Bry
Trajectory Generating

Piecewise Polynomial Joint Optimization
We now have a quadratic program (QP) of the form: min
𝑥
𝑥 𝑇
𝑄𝑥
s. t. Ax − b = 0
Since we wish to "correct" for derivative discontinuities in Dubins curves it is necessary
to jointly optimize multiple polynomial segments
Control, Estimation, and Planning Algorithms for Aggressive Flight using Onboard Sensing, MIT, Adam Parker Bry

The higher order time differential constraints for curve-segment of Dubins curves:

The higher order time differential constraints for line-segment of Dubins curves:

𝑥 = 𝑥 𝑤 𝑦 𝑤 zw 𝑢 𝑤 𝑣 𝑤 𝑞1 𝑞2 𝑞3 𝑞4
𝑇
𝑢 = 𝜙 𝜃 𝜓 𝑧 = [𝑢1 𝑢2 𝑢3 𝑢4]
𝑉 = 𝑧 𝑤 = 𝑢4
𝑢1 = tan−1
𝑥
𝑔 − 𝑧
𝑢2 = tan−1
𝑦𝑐𝑜𝑠𝑢1
𝑔 − 𝑧
𝑢3 = 𝜓
Nonlinear Dynamic Inversion (NDI)

ROS Industrial

Source: Source: ROS-Industrial slides 2017, 邱强 (上海交大)
ROS Industrial

Source: Source: ROS-Industrial slides 2017, 邱强 (上海交大)
Inverse
Kinematics
(KDL, IK-FAST)
Collision
Checking
(FCL)
Motion
Planning
(OMPL: RRT)
Forward
Kinematics
(URDF, TF)
ROS Industrial

Source: 04-Mirko_Bordignon_WRC2017_ROS
ROS Industrial

Robot Arms
• Motoman
• Adept
• Universal
• Fanuc
• ABB
• Kuka
• Techman (Taiwan)
• More to come…
Source: ROS-Industrial Introduction slides 2013, Shaun Edwards
ROS Industrial

Industrial_Robot_Driver_Spec http://wiki.ros.org/Industrial/Industrial_Robot_Driver_Spec
ROS Industrial

ROS Industrial
Source: ROS-Industrial Website
ROS-industrial Conference ROS-industrial Workshops

Source: ROS-Industrial Website
ROS Industrial
ROS-industrial Consortium Worldwide Membership, December 2017

rtt_lwr: OROCOS/ROS Components for Light Weight Robots
ROS Industrial
Source: http://rtt-lwr.readthedocs.io/en/latest/index.html

●New Horizon 2020 robotics projects from 2016
https://ec.europa.eu/programmes/horizon2020/en/news/new-horizon-
2020-robotics-projects-2016
The two biggest projects are ROS related !
RoBMoSys and ROSIN (15.5 MEUR in total)
http://rosin-project.eu/
http://robmosys.eu/ € 8,000,000.00
€ 7,504,236.25
• ABB AB Sweden
• Delft University of Technology
• FH Aachen University of Applied Sciences
• Fraunhofer-Institut für Produktionstechnik und Automatisierung IPA
• Fundación Tecnalia Research & Innovations
• IT University of Copenhagen
ROS Industrial

Melonee Wise
http://fetchrobotics.com/fetchers/
ROS Industrial

https://github.com/
ApolloAuto
ROS Industrial

ROS 2.0

Known Issue of ROS 1.0
• Realtime performance
• Node management
• Master (roscore)
• Multi-robots capability (Swarm)
• Small processors
• Communication Priority
• Communication Security
• Unreliable network connectivity
Source: http://blog.ascens-ist.eu/wp-content/uploads/2011/03/13marxbots.jpg
ROS 2.0

The Goals
of ROS 2
Source: https://roscon.ros.org/2014/wp-content/uploads/2014/07/ROSCON-2014-Next-Generation-of-ROS-on-top-of-DDS.pdf
ROS 2.0

Source: https://roscon.ros.org/2014/wp-content/uploads/2014/07/ROSCON-2014-Next-Generation-of-ROS-on-top-of-DDS.pdf
=
less time
spent here
more time to spent here
ROS 2.0

ROS 1 ROS 2
ROS 2.0

User Code
ROS Client Lib
rclpy rclcpp Third Party
ROS Middleware
OpenSplice
RMW Impl.
FastRTPS
RMW Impl.
Third Party
RMW Impl.
PrismTech
OpenSplice
eProsima
FastRTPS
Third Party
DDS
User
Applications
Client
Wrapper
Middleware
DDS
DDS
agnostic
ROS
agnostic
ROS 2.0

Alpha 1
31 Aug 2015
Beta 1
31 Aug
2015
Beta 2
31 Aug
2015
Beta 3
13 Sep
2017
Ver. 1
13 Dec
2017
ROS 2.0
Ardent Apalone

Q & A
Website: https://hypharosworkshop.wordpress.com/
Github: https://github.com/Hypha-ROS/hypha-racecar
FB Page: https://www.facebook.com/HyphaROS/
Youku: http://i.youku.com/hypha
Gmail: hypha.ros@gmail.com
WeChat (ID): HyphaROS

ROS Ecosystem & Applications [HyphaROS] -- by HaoChih

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to ROS Ecosystem & Applications [HyphaROS] -- by HaoChih

Similar to ROS Ecosystem & Applications [HyphaROS] -- by HaoChih (20)

Recently uploaded

Recently uploaded (20)

ROS Ecosystem & Applications [HyphaROS] -- by HaoChih