Our proposal for the Amazon Research Award project "Learning Scene Parsing for Cluttered Bin Picking" was funded.
Our team NimbRo Picking won the 2nd price in the Pick task and came in 2nd in the final Stow and Pick task.
Invited presentation at the ICRA 2018 Workshop on Advances in Robotic Warehouse Automation, Brisbane, Australia:
"Team NimbRo Picking at ARC 2017: Fast Learning Semantic Perception and Coordinating Two Arms"
Max Schwarz, Christian Lenz, German Martin Garcia, Seongyong Koo, Arul Selvam Periyasamy, Michael Schreiber, and Sven Behnke:
Fast Object Learning and Dual-arm Coordination for Cluttered Stowing, Picking, and Packing
has been accepted for the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, to appear May 2018.
In the Finals, both stow and pick needed to be performed by the robotic system. Our robot stored 14 of the 16 items provided in a tote into our storage system, where additional 16 items were placed manually before. Afterwards, it picked eight of the possible nine requested items and placed them in the target card boxes. This resulted in 235 points, second only to ACRV from Australia, who reached 272 points. The missing ninth item – a mesh cup made from thin wires – was grasped by one of the manipulators, but then put back into the storage system due to a failed weight check.
Our team had a great run in the Picking task evaluation. All requested items were picked and placed successfully in the three target card boxes. The score reached was 245, second only to team Nanyang (Singapore, score 257).
After recovering from a serious system breakdown, which occurred due to an electrical short cut, our system performed the Stow task evaluation. The task was to stow items provided chaotically in a tote into the storage system. Due to an issue in the weight file, the measured weights of the picked objects were inconsistent with the expected weights and the manipulators kept dropping the picked items back into the tote, resulting in a score of only 20 points.
In the official practice run, our team reached the top score (150), followed by team IFL PiRo (KIT, Germany, 140), and team Nanyang (Nanyang Technological University, Singapore, 120). Teams could chose the practice task and the scores do not count towards the competition results. The score is promising, but there is still room for improvement.
Our system performed some practice runs. Many items were sucessfully picked and placed, but there is still much room for improvement.
Our team arrived at the RoboCup venue in Nagoya, Japan, and started to assemble and set-up the developed robotic system. Our system consist of two Universal Robots UR5 manipulators which are equipped with a flexible suction finger and a second finger for pinch grasps, RGB-D gantry sensors consisting of a Nikon DSLR camera and a Photoneo® PhoXi® 3D Scanner XL, a GPU server with four NVidia Titan GPUs, and scales below totes and boxes.
This year, as final the Stow and Pick has been added, where the robotic system must pick the items that it stored before. Teams can design their own storage system, subject to space restrictions. The density of items in the tote and the storage systems has been increased. Maybe the most challenging aspect is that not all items are known in advance, but half of the items are introduced to the system only few minutes before the competition run.
Our Amazon Picking Challenge 2016 software was released.
The paper Max Schwarz, Anton Milan, Arul Selvam Periyasamy, and Sven Behnke:
RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Clutter
has been accepted for International Journal of Robotics Research (IJRR), Sage Publications, to apprear 2017.
The abstract Max Schwarz and Sven Behnke:
Data-efficient Deep Learning for RGB-D Object Perception in Cluttered Bin Picking
has been accepted for the Warehouse Picking Automation Workshop (WPAW), IEEE International Conference on Robotics and Automation (ICRA), Singapore, May 2017.
The paper by Max Schwarz, Anton Milan, Christian Lenz, Aura
Periyasamy, Michael Schreiber, Sebastian Schüller, and Sven Behnke: NimbRo Picking:
Versatile Part Handling for Warehouse Automation
accepted for the IEEE International Conference on Robotics and
(ICRA), Singapore, May 2017.
Finalist for Best Automation Paper Award.
We presented our approach to the APC tasks in the CASE Workshop on Automation
for Warehouse Logistics, Fort Worth, Texas, USA.
In the Picking task, our robot picked 10 of the 12 items from the shelf and placed them in the tote. It received 97 points, coming in third in the overall ranking, next to TU Delft and the Japanese Team PFN.
Our team came in second in the Stowing task of the Amazon Picking Challenge, next only to the team of TU Delft. All but one object has been stowed.
We used our practice run for the Stowing task. Our system was working quite well. NimbRo recived 123 points, followed by Duke (15 points) and MIT (10 points).
Our team made good progress in setting up the robot for the
challenge. Here is a first video of picking from the tote.
Our team NimbRo Picking arrived at Leipziger Messe and is setting up the robot for the Amazon Picking Challenge.
Our team NimbRo Picking is qualified for the Amazon Picking Challenge 2016, which will take place June 28th - July 3rd, 2016 at Leipziger Messe, colocated with RoboCup 2016. The challenge tasks consist of picking a large variety of objects from a shelf and placing them in a tote, and of picking unordered objects from a tote and stowing them in the shelf.