Opening Cabinets and Drawers in the Real World using a Commodity Mobile Manipulator

Pulling open cabinets and drawers presents many difficult technical challenges in perception (inferring articulation parameters for objects from onboard sensors), planning (producing motion plans that conform to tight task constraints), and control (making and maintaining contact while applying forces on the environment). In this work, we build an end-to-end system that enables a commodity mobile manipulator (Stretch RE2) to pull open cabinets and drawers in diverse previously unseen real world environments. We conduct 4 days of real world testing of this system spanning 31 different objects from across 13 different real world environments. Our system achieves a success rate of 61% on opening novel cabinets and drawers in unseen environments zero-shot. An analysis of the failure modes suggests that errors in perception are the most significant challenge for our system. We will open source code and models for others to replicate and build upon our system.Comment: Project webpage: https://arjung128.github.io/opening-cabinets-and-drawer

Robust and adaptive door operation with a mobile robot

The version of record is available online at: http://dx.doi.org/10.1007/s11370-021-00366-7The ability to deal with articulated objects is very important for robots assisting humans. In this work, a framework to robustly and adaptively operate common doors, using an autonomous mobile manipulator, is proposed. To push forward the state of the art in robustness and speed performance, we devise a novel algorithm that fuses a convolutional neural network with efficient point cloud processing. This advancement enables real-time grasping pose estimation for multiple handles from RGB-D images, providing a speed up improvement for assistive human-centered applications. In addition, we propose a versatile Bayesian framework that endows the robot with the ability to infer the door kinematic model from observations of its motion and learn from previous experiences or human demonstrations. Combining these algorithms with a Task Space Region motion planner, we achieve an efficient door operation regardless of the kinematic model. We validate our framework with real-world experiments using the Toyota human support robot.Peer ReviewedPostprint (author's final draft

Robot learning of everyday object manipulations via human demonstration

We deal with the problem of teaching a robot to manipulate everyday objects through human demonstration. We first design a task descriptor which encapsulates important elements of a task. The design originates from observations that manipulations involved in many everyday object tasks can be considered as a series of sequential rotations and translations, which we call manipulation primitives. We then propose a method that enables a robot to decompose a demonstrated task into sequential manipulation primitives and construct a task descriptor. We also show how to transfer a task descriptor learned from one object to similar objects. In the end, we argue that this framework is highly generic. Particularly, it can be used to construct a robot task database that serves as a manipulation knowledge base for a robot to succeed in manipulating everyday objects

Motion Primitives and Planning for Robots with Closed Chain Systems and Changing Topologies

When operating in human environments, a robot should use predictable motions that allow humans to trust and anticipate its behavior. Heuristic search-based planning offers predictable motions and guarantees on completeness and sub-optimality of solutions. While search-based planning on motion primitive-based (lattice-based) graphs has been used extensively in navigation, application to high-dimensional state-spaces has, until recently, been thought impractical. This dissertation presents methods we have developed for applying these graphs to mobile manipulation, specifically for systems which contain closed chains. The formation of closed chains in tasks that involve contacts with the environment may reduce the number of available degrees-of-freedom but adds complexity in terms of constraints in the high-dimensional state-space. We exploit the dimensionality reduction inherent in closed kinematic chains to get efficient search-based planning. Our planner handles changing topologies (switching between open and closed-chains) in a single plan, including what transitions to include and when to include them. Thus, we can leverage existing results for search-based planning for open chains, combining open and closed chain manipulation planning into one framework. Proofs regarding the framework are introduced for the application to graph-search and its theoretical guarantees of optimality. The dimensionality-reduction is done in a manner that enables finding optimal solutions to low-dimensional problems which map to correspondingly optimal full-dimensional solutions. We apply this framework to planning for opening and navigating through non-spring and spring-loaded doors using a Willow Garage PR2. The framework motivates our approaches to the Atlas humanoid robot from Boston Dynamics for both stationary manipulation and quasi-static walking, as a closed chain is formed when both feet are on the ground