Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching

This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses a category-agnostic affordance prediction algorithm to select and execute among four different grasping primitive behaviors. It then recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional training data. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT-Princeton Team system that took 1st place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.eduComment: Project webpage: http://arc.cs.princeton.edu Summary video: https://youtu.be/6fG7zwGfIk

SegICP: Integrated Deep Semantic Segmentation and Pose Estimation

Recent robotic manipulation competitions have highlighted that sophisticated robots still struggle to achieve fast and reliable perception of task-relevant objects in complex, realistic scenarios. To improve these systems' perceptive speed and robustness, we present SegICP, a novel integrated solution to object recognition and pose estimation. SegICP couples convolutional neural networks and multi-hypothesis point cloud registration to achieve both robust pixel-wise semantic segmentation as well as accurate and real-time 6-DOF pose estimation for relevant objects. Our architecture achieves 1cm position error and <5^\circ$ angle error in real time without an initial seed. We evaluate and benchmark SegICP against an annotated dataset generated by motion capture.Comment: IROS camera-read

Collision-free inverse kinematics of the redundant seven-link manipulator used in a cucumber picking robot

The paper presents results of research on an inverse kinematics algorithm that has been used in a functional model of a cucumber-harvesting robot consisting of a redundant P6R manipulator. Within a first generic approach, the inverse kinematics problem was reformulated as a non-linear programming problem and solved with a Genetic Algorithm (GA). Although solutions were easily obtained, the considerable calculation time needed to solve the problem prevented on-line implementation. To circumvent this problem, a second, less generic, approach was developed which consisted of a mixed numerical-analytic solution of the inverse kinematics problem exploiting the particular structure of the P6R manipulator. Using the latter approach, calculation time was considerably reduced. During the early stages of the cucumber-harvesting project, this inverse kinematics algorithm was used off-line to evaluate the ability of the robot to harvest cucumbers using 3D-information obtained from a cucumber crop in a real greenhouse. Thereafter, the algorithm was employed successfully in a functional model of the cucumber harvester to determine if cucumbers were hanging within the reachable workspace of the robot and to determine a collision-free harvest posture to be used for motion control of the manipulator during harvesting. The inverse kinematics algorithm is presented and demonstrated with some illustrative examples of cucumber harvesting, both off-line during the design phase as well as on-line during a field test

Construction safety and digital design: a review

As digital technologies become widely used in designing buildings and infrastructure, questions arise about their impacts on construction safety. This review explores relationships between construction safety and digital design practices with the aim of fostering and directing further research. It surveys state-of-the-art research on databases, virtual reality, geographic information systems, 4D CAD, building information modeling and sensing technologies, finding various digital tools for addressing safety issues in the construction phase, but few tools to support design for construction safety. It also considers a literature on safety critical, digital and design practices that raises a general concern about ‘mindlessness’ in the use of technologies, and has implications for the emerging research agenda around construction safety and digital design. Bringing these strands of literature together suggests new kinds of interventions, such as the development of tools and processes for using digital models to promote mindfulness through multi-party collaboration on safet