|
|
 |
| 1.10.2021 - 31.12.2025 Förderkennzeichen: 16IS21080 |
The objective of this project is to learn human-like grasping with a
highly sensorized, compliant anthropomorphic hand from cluttered
containers. Grasping policies will be based not only on feedback from
RGB-D sensors, but also on force-torque measurements in the wrist,
force measurements in the fingers, and tactile measurements on the
hand surface. The learned grasping strategies will be interactive, and
thus include non-prehensile manipulation actions, like pushing,
rearranging, and singulating objects, and multi-step plans like
in-hand object reorientation during grasping and regrasping for
caging. In order to achieve data-efficient learning from limited
experience, we will develop a structured approach for a scene parsing,
prediction and tracking that is based on visual and haptic
measurements. Interactive grasping strategies will be learned using
deep reinforcement learning methods in a physics-based simulation and
with the real hand-arm system.
Malte Mosbach and Sven Behnke:
Grasp
Anything: Combining Teacher-Augmented Policy Gradient Learning
with Instance Segmentation to Grasp Arbitrary Objects
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Yokohama, Japan, May 2024.
[Video] [Project
Page]
Arul Selvam Periyasamy and Sven Behnke:
MOTPose: Multi-object 6D Pose Estimation for Dynamic Video
Sequences using Attention-based Temporal Fusion
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Yokohama, Japan, May 2024.
[Video] [Project
Page]
Efficient
Multi-Object Pose Estimation using Multi-Resolution Deformable
Attention and Query Aggregation
In Proceedings of IEEE International Conference on Robotic Computing
(IRC), Laguna Hills, USA, December 2023.
Arul Selvam Periyasamy, Arash Amini, Vladimir Tsaturyan, and Sven
Behnke:
YOLOPose
V2: Understanding and Improving Transformer-based 6D Pose
Estimation
Robotics and Autonomous Systems, 168:104490, Elsevier, 2023.
Angel Villar-Corrales, Ismail Wahdan, and Sven Behnke:
Object-centric Video Prediction via Decoupling of Object Dynamics
and Interactions
In Proceedings of IEEE International Conference on Image Processing
(ICIP), pp. 570-574, Kuala Lumpur, Malaysia, October 2023.
[Video] [Project
Page]
Malte Mosbach and Sven Behnke:
Learning
Generalizable Tool Use with Non-rigid Grasp-pose Registration
In Proceedings of IEEE 19th International Conference on Automation
Science and Engineering (CASE), Auckland, New Zealand, August 2023.
Best Conference Paper Award.
[Project
Page]
Dmytro Pavlichenko and Sven Behnke:
Deep
Reinforcement Learning of Dexterous Pre-grasp Manipulation for
Human-like Functional Categorical Grasping
In Proceedings of IEEE 19th International Conference on Automation
Science and Engineering (CASE), Auckland, New Zealand, August 2023.
[Video]
Arul Selvam Periyasamy and Sven Behnke:
Towards
3D Scene Understanding Using Differentiable Rendering
SN Computer Science 4(3):245, March 2023.
Arul Selvam Periyasamy, Luis Denninger, and Sven Behnke:
Learning
Implicit Probability Distribution Functions for Symmetric
Orientation Estimation from RGB Images Without Pose Labels
In Proceedings of 6th IEEE International Conference on Robotic
Computing (IRC), Naples, Italy, December 2022.
Malte Mosbach and Sven Behnke:
Efficient
Representations
of Object Geometry for Reinforcement Learning of Interactive
Grasping Policies
In Proceedings of 6th IEEE International Conference on Robotic
Computing (IRC), Naples, Italy, December 2022.
[Project
Page]
Malte Mosbach, Kara Moraw, and Sven Behnke:
Accelerating
Interactive Human-like Manipulation Learning with GPU-based
Simulation and High-quality Demonstrations
In Proceedings of IEEE-RAS 21st International Conference on Humanoid
Robots (Humanoids), Ginowan, Okinawa, Japan, November 2022.
[Project
Page]
Angel Villar-Corrales, Ani Karapetyan, Andreas Boltres, and Sven
Behnke:
MSPred:
Video Prediction at Multiple Spatio-Temporal Scales with
Hierarchical Recurrent Networks
In Proceedings of 33rd British Machine Vision Conference (BMVC),
London, UK, November 2022.
[Project Page]
[Supplementary
Material]
Benedikt T. Imbusch, Max Schwarz, and Sven Behnke:
Synthetic-to-Real
Domain Adaptation using Contrastive Unpaired Translation
In Proceedings of 18th IEEE International Conference on Automation
Science and Engineering (CASE), Mexico City, Mexico, August 2022.
[Video]
Arash Amini, Arul Selvam Periyasamy, and Sven Behnke:
YOLOPose: Transformer-based Multi-Object 6D Pose Estimation using
Keypoint Regression
In Proceedings of 17th International Conference on Intelligent
Autonomous Systems (IAS), Zagreb, Croatia, June 2022.
Best Paper Award.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Iterative
3D Deformable Registration from Single-View RGB Images Using
Differentiable Rendering
In Proceedings of 17th International Conference on Computer Vision
Theory and Applications (VISAPP), February 2022.
Finalist for Student Paper Award.
Diego Rodriguez, Florian Huber, and Sven Behnke:
Category-level Part-based 3D Object Non-rigid Registration
In Proceedings of 17th International Conference on Computer Vision
Theory and Applications (VISAPP), February 2022.
Best Poster Award. [Poster]
2018-2021 Amazon supported our research on perception of bin picking scenes with Amazon Research Awards.
Amazon continues to support our research on interpreting bin picking
scenes with a 2019
Amazon
Research Award.
In the new project, we go beyond the passive observation of scenes by
considering scene changes caused by the actions of a
robotic manipulator arm. We will learn predictive models of the
effects of manipulation actions and use these to maintain the
structured scene model while the robot is manipulating.
Supported by an 2018 Amazon Research Award, we continued research in the area of interpreting cluttered bin picking scenes.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
SynPick: A Dataset for Dynamic Bin Picking Scene Understanding
In Proceedings of 17th IEEE International Conference on Automation
Science and Engineering (CASE), Lyon, France, August 2021.
[Projekt page with
data set, code, and videos}
Moritz Zappel, Simon Bultmann, and Sven Behnke:
6D
Object Pose Estimation using Keypoints and Part Affinity Fields
In Proceedings of 24th RoboCup International Symposium, June 2021.
Finalist for Best Paper Award
[Video]
Diego Rodriguez, Florian Huber, and Sven Behnke:
Category-Level
3D
Non-Rigid Registration from Single-View RGB Images
In Proceedings of IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS), Las Vegas, USA, October 2020.
[Video]
Sven Behnke and Max Schwarz:
Generalizing
Scene
Parsing for Cluttered Bin Picking
Presentation at Amazon Research Award Symposium (ARA), Boston, MA,
USA, October 2019.
Max Schwarz and Sven Behnke:
Stillleben:
Realistic
Scene Synthesis for Deep Learning in Robotics
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Paris, France, May 2020.
[Source code] [Presentation video]
Catherine Capellen, Max Schwarz, and Sven Behnke:
ConvPoseCNN:
Dense
Convolutional 6D Object Pose Estimation
In Proceedings of 15th International Conference on Computer Vision
Theory and Applications (VISAPP), pp. 162-172, Valletta, Malta,
February 2020.
Umashankar Deekshith, Nishit Gajjar, Max Schwarz, and Sven Behnke:
Visual
Descriptor
Learning from Monocular Video
In Proceedings of 15th International Conference on Computer Vision
Theory and Applications (VISAPP), pp. 444-451, Valletta, Malta,
February 2020.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Refining
6D Object Pose Predictions using Abstract Render-and-Compare
In Proceedings of IEEE-RAS International Conference on Humanoid Robots
(Humanoids), Toronto, Canada, October 2019.
[Video]
Starting from our work in the Amazon Picking and Robotics Challenges, we investigated the parsing of complex bin picking scenes with support of an Amazon Research Award.
Sven Behnke and Max Schwarz:
Learning
Semantic
Perception for Cluttered Bin Picking
Presentation at Amazon Research Awards Fall Symposium (ARA), Boston,
MA, USA, October 2018.
Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Robust
6D Object Pose Estimation in Cluttered Scenes using Semantic
Segmentation and Pose Regression Networks
In Proceedings of IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS), Madrid, Spain, October 2018.
[Video]
Our team NimbRo Picking came in second at the Amazon Picking Challenge 2016 and the Amazon Robotics Challenge 2017.
Volker Krueger, Francesco Rovida, Bjarne Grossmann, Ronald Petrick,
Matthew Crosby, Arnaud Charzoule, German Martin
Garcia, Sven Behnke, Cesar Toscano, Germano Veiga:
Testing
the Vertical and Cyber-Physical Integration of Cognitive Robots in
Manufacturing
Robotics and Computer-Integrated Manufacturing, vol. 57, pp. 213-229,
Elsevier, June 2019.
Max Schwarz, Christian Lenz, German Martin Garcia, Seongyong Koo,
Arul Selvam Periyasamy, Michael Schreiber, and Sven Behnke:
Fast
Object Learning and Dual-arm Coordination for Cluttered Stowing,
Picking, and Packing
Accepted for IEEE International Conference on Robotics and Automation
(ICRA), Brisbane, Australia, May 2018.
Max Schwarz, Anton Milan, Arul Selvam Periyasamy, and Sven Behnke:
RGB-D
Object Detection and Semantic Segmentation for Autonomous
Manipulation in Clutter
International Journal of Robotics Research (IJRR), Sage Publications,
available online,
June 2017.
Max Schwarz, Anton Milan, Christian Lenz, Aura Munoz, Arul Selvam
Periyasamy, Michael Schreiber, Sebastian Schüller, and Sven Behnke:
NimbRo
Picking: Versatile Part Handling for Warehouse Automation
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Singapore, May 2017.
[Video]
 |
This project is funded by the FP7 ICT GA 610917. |
The goal of STAMINA is to develop a fleet of autonomous and mobile industrial robots for jointly solving three logistic and handling tasks: de-palletizing, bin-picking, and kitting. The main task of our group is the grasping of parts from transport containers. This includes the detection and pose estimation of parts as well as the grasp and arm motion planning for a robot manipulator.
Volker Krueger, Arnaud Charzoule, Bjarne Grossmann, Cesar Toscano,
Francesco Rovida, German Martin Garcia, Germano Veiga, Matthew Crosby,
Ronald Petrick, and Sven Behnke:
Testing the Vertical and
Cyber-Physical Integration of Cognitive Robots in Manufacturing
Accepted for Robotics and Computer-Integrated Manufacturing, Elsevier,
to appear 2019.
Dmytro Pavlichenko, Germán Martín García, Seongyong Koo, and Sven
Behnke:
KittingBot: A Mobile Manipulation Robot for Collaborative Kitting
in Automotive Logistics
Accepted for 15th International Conference on Intelligent Autonomous
Systems (IAS-15), Baden-Baden, Germany, June 2018.
Seongyong Koo, Grzegorz Ficht, Germán Martín García, Dmytro
Pavlichenko, Martin Raak, and Sven Behnke:
Robolink
Feeder: Reconfigurable Bin-Picking and Feeding with a Lightweight
Cable-Driven Manipulator
In Proceedings of 13th IEEE International Conference on Automation
Science and Engineering (CASE), Xi'an, China, August 2017.
[Video]
Dirk Holz and Sven Behnke:
Fast
Edge-Based Detection and Localization of Transport Boxes and
Pallets in RGB-D Images for Mobile Robot Bin Picking
In Proceedings of 47th International Symposium on Robotics (ISR),
Munich, Germany, June 2016.
Dirk Holz, Alexandru-Eugen Ichim, Federico Tombari, Radu B. Rusu, and
Sven Behnke:
Registration with the Point
Cloud Library - A Modular Framework for Aligning in 3-D
IEEE Robotics and Automation Magazine, 22(4):110-124, December 2015.
Dirk Holz, Angeliki Topalidou-Kyniazopoulou, Joerg Stueckler, and
Sven Behnke:
Real-Time
Object Detection, Localization and Verification for Fast Robotic
Depalletizing
In Proceedings of IEEE/RSJ International Conference
on Intelligent Robots and Systems (IROS), Hamburg, Germany, September
2015.
[Video]
Dirk Holz, Angeliki Topalidou-Kyniazopoulou, Francesco Rovida,Mikkel
Rath Pedersen, Volker Krüger, and Sven Behnke:
A
Skill-Based System for Object Perception and Manipulation for
Automating Kitting Tasks
In Proceedings of 20th IEEE International Conference on
Emerging Technologies and Factory Automation (ETFA), Luxemburg,
September 2015.
Dirk Holz and Sven Behnke:
Approximate
Surface Reconstruction and Registration for RGB-D SLAM
In Proceedings of European
Conference on Mobile Robots (ECMR), Lincoln, UK, September
2015.
Dirk Holz, Matthias Nieuwenhuisen, David Droeschel, Jörg Stückler,
Alexander Berner, Jun Li, Reinhard Klein, and Sven Behnke:
Active
Recognition
and Manipulation for Mobile Robot Bin Picking
In: Gearing up and accelerating cross-fertilization between academic
and industrial robotics research in Europe - Technology transfer
experiments from the ECHORD project, vo. 94 of Springer Tracts in
Advanced Robotics (STAR), pp. 133-153, 2014.
Alexander Berner, Jun Li, Dirk Holz, Jörg Stückler, Sven Behnke, and
Reinhard Klein:
Combining
Contour
and Shape Primitives for Object Detection and Pose Estimation of
Prefabricated Parts
In Proceedings of IEEE International Conference on Image Processing
(ICIP), Melbourne, Australia, September 2013.
Matthias Nieuwenhuisen, David Droeschel, Dirk Holz, Joerg Stückler,
Alexander Berner, Jun Li, Reinhard Klein, and Sven Behnke:
Mobile Bin Picking with an
Anthropomorphic Service Robot
In Proceedings of IEEE International Conference on Robotics and
Automation (ICRA), Karlsruhe, May 2013.
Matthias Nieuwenhuisen, Jörg Stückler, Alexander
Berner, Reinhard Klein, and Sven Behnke:
Shape-Primitive
Based
Object Recognition and Grasping
In Proceedings of 7th German Conference on Robotics (ROBOTIK),
Munich, May 2012.
Dirk Holz, Matthias Nieuwenhuisen, David Droeschel, Jörg Stückler,
Sven Behnke, Alexander Berner, and Reinhard Klein:
Active
Recognition
and Manipulation of Objects Based on Shape Primitives
Poster at 5th International Conference on Cognitive Systems (CogSys)
Vienna, February 2012.
University of Bonn, Institute for Computer Science, Computer Science VI - Intelligent Systems and Robotics Imprint Data Privacy Statement ; Robotics
![[Poster]](../papers/VISAPP_2022_Rodriguez_Poster.svg){kind=link}
