Integrating Visual Foundation Models for Enhanced Robot Manipulation and Motion Planning: A Layered Approach

This paper presents a novel layered framework that integrates visual foundation models to improve robot manipulation tasks and motion planning. The framework consists of five layers: Perception, Cognition, Planning, Execution, and Learning. Using visual foundation models, we enhance the robot's perception of its environment, enabling more efficient task understanding and accurate motion planning. This approach allows for real-time adjustments and continual learning, leading to significant improvements in task execution. Experimental results demonstrate the effectiveness of the proposed framework in various robot manipulation tasks and motion planning scenarios, highlighting its potential for practical deployment in dynamic environments.Comment: 3 pages, 2 figures, IEEE Worksho

ACS-PRM: Adaptive Cross Sampling Based Probabilistic Roadmap for Multi-robot Motion Planning

International audienceIn this paper we present a novel approach for multi-robot motion planning by using a probabilistic roadmap (PRM) based on adaptive cross sampling (ACS). The proposed approach, we call ACS-PRM, consists of three steps, which are C-space sampling, roadmap building and motion planning. Firstly, an adequate number of points should be generated in C-space on an occupancy grid map by using an adaptive cross sampling method. Secondly, a roadmap should be built while the potential targets and the milestones are extracted by second learning the result of sampling. Finally, the motion of robots should be planned by querying the constructed roadmap. In contrast to previous approaches, our ACS-PRM approach is designed to plan separate kinematic paths for multiple robots to minimize the problem of congestion and collision in an effective way so as to improve the planning efficiency. Our approach has been implemented and evaluated in simulation. The experimental results demonstrate the total planning time can be significantly reduced by our ACS-PRM approach compared with previous approaches