Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to Henrik Christensen - Vision for Co-robot Applications
Similar to Henrik Christensen - Vision for Co-robot Applications (20)
More from Daniel Huber
More from Daniel Huber (9)
Recently uploaded
Recently uploaded (20)
Henrik Christensen - Vision for Co-robot Applications
- 1. Vision for Co-Robot Applications • Henrik I Christensen KUKA Chair of Robotics Robotics @ Georgia Tech Atlanta, Georgia Henrik.Christensen@gatech.edu
- 2. Co-X Robotics • Co-Worker Next Generation Manufacturing • Co-Inhabitants Assistance to People in Daily Lives • Co-Protectors Support for core industries
- 4. Perception Tasks • Pick-and-place task • Robots moving from controlled settings to unstructured environments • Robust object perception is crucial
- 5. Motivations Depth sensors are everywhere! 40 million Kinects sold Google ProjectTango Apple + PrimeSenseOccipital, Inc
- 6. Motivations 3D object models have been accumulated on the Internet! Known 3D object model was strong assumption
- 7. Google 3D warehouse (about 2.5 million models) Motivations
- 8. 3D Object Models Pose Estimation & Tracking Object ID & Pose ImageCamera The Problem
- 9. Challenges 1. Object with and withoutTextures 2. Background Clutter 3. Object Discontinuities 4. Real-time Constraints
- 10. Challenge 1: Texture • ... •Textured objects •Photometric: color, keypoints, edges or textures from surfaces •Textureless objects •Geometric: point coordinates, surface normals, depth discontinuities Handling both textured and textureless objects Employ both photometric and geometric features
- 11. Challenge 2: Clutter •False measurements •False pose estimates •Stuck in local minima •No table-top assumption Controlled environments Unstructured environments Difficulties = Degree of Clutter Multiple pose hypotheses frameworks: particle filtering for pose tracking and voting process for pose estimation
- 13. Challenge 3: Discontinuities • ... •Ideal vs Reality •Occluded by other objects, human, or robots •Object goes out of the camera’s field of view •Blurred in images •Re-initialization problem BlurOut of FOVOcclusions A re-initialization scheme by combining pose estimation and tracking
- 14. Challenge 4: Real-time •Constrained by timing limitations •Scarcely see real-time state-of-the-art Exploiting the power of parallel computation on GPU
- 15. Approaches • 2DVisual Information (Monocular Camera) – Combining Keypoint and Edge Features – HandlingTextureless Objects • 3DVisual Information (RGB-D Camera) – Voting-based Pose Estimation using Pair Features – Object PoseTracking photometric geometric
- 16. Overview Georgia Institute of Technology Atlanta, GA 30332, USA {cchoi,hic}@cc.gatech.edu h for 3D real-time ctly applicable to res for the initial n initial estimate hese two comple- bust tracking so- ncludes: 1) While e used simplified ned models, our model. To achieve are automatically sually invisible in en they constitute a fully automatic t of the previous ose initialization mes drift because tors the tracking tracking results rate our system’s Image Acquisition Model Rendering Edge Detection Pose Update with IRLS Error Calculation CAD Model Keyframes Keypoint Matching Pose Estimation !"#$%&'()*$+, -+)".+'/),$'0,12.1)34)%.+'/),$'0,12.1)3 ()3)%5+.6'7.2$6. Fig. 1: Overall system flow. We use a monocular camera. The initial pose of the object is estimated by using the SURF keypoint matching in the Global Pose Estimation (GPE). Using the initial pose, the Local Pose Estimation (LPE) consecutively estimates poses of the object utilizing RAPiD style tracking. keyframes and CAD model are employed as models by the GPE and LPE, respectively. The model are generated offline.16
- 17. 17our approach keypoint only edges model rendering
- 18. Particle Filter • Posterior p.d.f. as a set of weighted particles • non-linear, non-Gaussian, multi-modal • widely adopted in robotics, computer vision, etc
- 19. AR Dynamics • Instead of Gaussian random walk models • Linear prediction based on previous states • Propagate particles more effectively we mics ngs, cles ires tro- ure- nted zed II- d in are e of nge AR state dynamics is a good alternative since it is flexible, yet simple to implement. In (1), the term A(X, t) determines the state dynamics. A trivial case, A(X, t) = 0, is a random walk model. [13] modeled this via the first-order AR process on the Aff(2) as: Xt = Xt 1 · exp(At 1 + dWt ⌅ t), (3) At 1 = a log(X 1 t 2Xt 1) (4) where a is the AR process parameter. Since the SE(3) is a compact connected Lie group, the AR process model also holds on the SE(3) group [21]. C. Particle Initialization using keypoint Correspondences Most of the particle filter-based trackers assume that initial states are given. In practice, initial particles are crucial to ensure convergence to a true state. Several trackers [15], [14] search for the true state from scratch, but it is desirable to initialize particle states by using other information. Using
- 20. Re-initialization • Effective number of particle size, objects. In these cases, the tracker is required to re-in the tracking. In general sequential Monte Carlo metho effective particle size Neff has been introduced as a s measure of degeneracy [27]. Since it is hard to evaluate exactly, an alternative estimate [Neff is defined [27]: [Neff = 1 N i=1(˜(i))2 Often it has been used as a measure to execute the resam procedure. But, in our tracker we resample particles frame, and hence we use [Neff as a measure to initialization. When the number of effective particles is a fixed threshold Nthres, the re-initialization proced performed. The overall algorithm is shown in Algorit III. EXPERIMENTAL RESULTS In this section, we validate our proposed particle based tracker via various experiments. First, we co the performance of our approach with the previous 0 200 400 600 800 1000 1200 1400 0 50 100 Frame number N eff 0 200 400 600 800 1000 1200 1400 0 50 100 Frame number Neff Effective number of particle size
- 21. Experiments Single vs. Multiple pose hypotheses with vs. without AR state dynamics Reinitialization exp. 2D Monocular > Combining Keypoint and Edge Features [IJRR’12]
- 23. Robotic Assembly
- 26. 26
- 27. Real Sequence 27Ours PCL tracking Ours PCL tracking
- 28. 3D models on the Web 28
- 29. 29 real-time
- 30. 30
- 32. Credits • Students / Postdocs • Changhyun Choi, now MIT • Heni Ben Amor, now ASU • Samarth Brahmbhatt, GT • Ana Huaman, GT • Sponsors: • Boeing, GM, PSA, • ARL & NSF
- 33. Summary • The next revolution in robotics will be driven by software not the traditional hardware. • Vision for closing loop control • Cloud based services for sharing of knowledge • Systems that collaborate with people