Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planning

Robotic manipulation tasks, such as object rearrangement, play a crucial role in enabling robots to interact with complex and arbitrary environments. Existing work focuses primarily on single-level rearrangement planning and, even if multiple levels exist, dependency relations among substructures are geometrically simpler, like tower stacking. We propose Structural Concept Learning (SCL), a deep learning approach that leverages graph attention networks to perform multi-level object rearrangement planning for scenes with structural dependency hierarchies. It is trained on a self-generated simulation data set with intuitive structures, works for unseen scenes with an arbitrary number of objects and higher complexity of structures, infers independent substructures to allow for task parallelization over multiple manipulators, and generalizes to the real world. We compare our method with a range of classical and model-based baselines to show that our method leverages its scene understanding to achieve better performance, flexibility, and efficiency. The dataset, supplementary details, videos, and code implementation are available at: https://manavkulshrestha.github.io/sclComment: Accepted to Conference on Robot Learning (CoRL) 202

Placement generation and hybrid planning for robotic rearrangement on cluttered surfaces

Rearranging multiple moving objects across surfaces, e.g. from a table to kitchen shelves as it arises in the context of service robotics, is a challenging problem. The rearrangement problem consists of two subproblems: placement generation and rearrangement planning. Firstly, the collision-free goal poses of the objects to be moved need to be determined subject to the arbitrary geometries of the objects and the state of the surface that already includes movable objects (clutter) and immovable obstacles on it. Secondly, after the goal poses of all objects have been determined, a plan of physical actions must be computed to achieve these goal poses. Computation of such a rearrangement plan is difficult in that it necessitates not only high-level task planning, but also low-level feasibility checks to be integrated with this task plan to ensure that each step of the plan is collision-free. In this thesis, we propose a general solution to the rearrangement of multiple arbitrarily-shaped objects on a cluttered flat surface with multiple movable objects and obstacles. In particular, we introduce a novel method to solve the object placement problem, utilizing nested local searches guided by intelligent heuristics to efficiently perform multi-objective optimizations. The solutions computed by our method satisfy the collision-freeness constraint, and involves minimal movements of the clutter. Based on such a solution, we introduce a hybrid method to generate an optimal feasible rearrangement plan, by integrating ASP-based high-level task planning with low-level feasibility checks. Our hybrid planner is capable of solving challenging non-monotone rearrangement planning instances that cannot be solved by the existing geometric rearrangement approaches. The proposed algorithms have been systematically evaluated in terms of computational efficiency, solution quality, success rate, and scalability. Furthermore, several challenging benchmark instances have been introduced that demonstrate the capabilities of these methods. The real-life applicability of the proposed approaches have also been verified through physical implementation using a Baxter robot

Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planning

International audienceWe address the problem of visually guided rearrangement planning with many movable objects, i.e., finding a sequence of actions to move a set of objects from an initial arrangement to a desired one, while relying on visual inputs coming from an RGB camera. To do so, we introduce a complete pipeline relying on two key contributions. First, we introduce an efficient and scalable rearrangement planning method, based on a Monte-Carlo Tree Search exploration strategy. We demonstrate that because of its good trade-off between exploration and exploitation our method (i) scales well with the number of objects while (ii) finding solutions which require a smaller number of moves compared to the other state-of-the-art approaches. Note that on the contrary to many approaches, we do not require any buffer space to be available. Second, to precisely localize movable objects in the scene, we develop an integrated approach for robust multi-object workspace state estimation from a single uncalibrated RGB camera using a deep neural network trained only with synthetic data. We validate our multi-object visually guided manipulation pipeline with several experiments on a real UR-5 robotic arm by solving various rearrangement planning instances, requiring only 60 ms to compute the plan to rearrange 25 objects. In addition, we show that our system is insensitive to camera movements and can successfully recover from external perturbations. Supplementary video, source code and pre-trained models are available at https://ylabbe.github.io/rearrangement-planning/