Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

Combining task and motion planning for mobile manipulators

Aplicat embargament des de la data de defensa fins el dia 31/12/2019Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit d’Enginyeria IndustrialThis thesis addresses the combination of task and motion planning which deals with different types of robotic manipulation problems. Manipulation problems are referred to as mobile manipulation, collaborative multiple mobile robots tasks, and even higher dimensional tasks (like bi-manual robots or mobile manipulators). Task and motion planning problems needs to obtain a geometrically feasible manipulation plan through symbolic and geometric search space. The combination of task and motion planning levels has emerged as a challenging issue as the failure leads robots to dead-end tasks due to geometric constraints. In addition, task planning is combined with physics-based motion planning and information to cope with manipulation tasks in which interactions between robots and objects are required, or also a low-cost feasible plan in terms of power is looked for. Moreover, combining task and motion planning frameworks is enriched by introducing manipulation knowledge. It facilitates the planning process and aids to provide the way of executing symbolic actions. Combining task and motion planning can be considered under uncertain information and with human-interaction. Uncertainty can be viewed in the initial state of the robot world or the result of symbolic actions. To deal with such issues, contingent-based task and motion planning is proposed using a perception system and human knowledge. Also, robots can ask human for those tasks which are difficult or infeasible for the purpose of collaboration. An implementation framework to combine different types of task and motion planning is presented. All the required modules and tools are also illustrated. As some task planning algorithms are implemented in Prolog or C++ languages and our geometric reasoner is developed in C++, the flow of information between different languages is explained.Aquesta tesis es centra en les eines de planificació combinada a nivell de tasca i a nivell de moviments per abordar diferents problemes de manipulació robòtica. Els problemes considerats són de navegació de robots mòbil enmig de obstacles no fixes, tasques de manipulació cooperativa entre varis robots mòbils, i tasques de manipulació de dimensió més elevada com les portades a terme amb robots bi-braç o manipuladors mòbils. La planificació combinada de tasques i de moviments ha de cercar un pla de manipulació que sigui geomètricament realitzable, a través de d'un espai de cerca simbòlic i geomètric. La combinació dels nivells de planificació de tasca i de moviments ha sorgit com un repte ja que les fallades degudes a les restriccions geomètriques poden portar a tasques sense solució. Addicionalment, la planificació a nivell de tasca es combina amb informació de la física de l'entorn i amb mètodes de planificació basats en la física, per abordar tasques de manipulació en les que la interacció entre el robot i els objectes és necessària, o també si es busca un pla realitzable i amb un baix cost en termes de potència. A més, el marc proposat per al combinació de la planificació a nivell de tasca i a nivell de moviments es millora mitjançant l'ús de coneixement, que facilita el procés de planificació i ajuda a trobar la forma d'executar accions simbòliques. La combinació de nivells de planificació també es pot considerar en casos d'informació incompleta i en la interacció humà-robot. La incertesa es considera en l'estat inicial i en el resultat de les accions simbòliques. Per abordar aquest problema, es proposa la planificació basada en contingències usant un sistema de percepció i el coneixement de l'operari humà. Igualment, els robots poden demanar col·laboració a l'operari humà per a que realitzi aquelles accions que són difícils o no realitzables pel robot. Es presenta també un marc d'implementació per a la combinació de nivells de planificació usant diferents mètodes, incloent tots els mòduls i eines necessàries. Com que alguns algorismes estan implementats en Prolog i d'altres en C++, i el mòdul de raonament geomètric proposat està desenvolupat en C++, es detalla el flux d'informació entre diferents llenguatges.Award-winningPostprint (published version

Path Planning and Evolutionary Optimization of Wheeled Robots

Probabilistic roadmap methods (PRM) have been a well-known solution for solving motion planning problems where we have a fixed set of start and goal configurations in a workspace. We define a configuration space with static obstacles. We implement PRM to find a feasible path between start and goal for car-like robots. We further extend the concept of path planning by incorporating evolutionary optimization algorithms to tune the PRM parameters. The theory is demonstrated with simulations and experiments. Our results show that there is a significant improvement in the performance metrics of PRM after optimizing the PRM parameters using biogeography-based optimization, which is an evolutionary optimization algorithm. The performance metrics (namely path length, number of hops, number of loops and fail-rate) show 34.91 , 23.18 , 52.21 and 21.21 improvement after using optimized PRM parameters. We also experimentally demonstrate the application of path planning using PRM to mobile car-like robot

Path Planning and Evolutionary Optimization of Wheeled Robots

Probabilistic roadmap methods (PRM) have been a well-known solution for solving motion planning problems where we have a fixed set of start and goal configurations in a workspace. We define a configuration space with static obstacles. We implement PRM to find a feasible path between start and goal for car-like robots. We further extend the concept of path planning by incorporating evolutionary optimization algorithms to tune the PRM parameters. The theory is demonstrated with simulations and experiments. Our results show that there is a significant improvement in the performance metrics of PRM after optimizing the PRM parameters using biogeography-based optimization, which is an evolutionary optimization algorithm. The performance metrics (namely path length, number of hops, number of loops and fail-rate) show 34.91 , 23.18 , 52.21 and 21.21 improvement after using optimized PRM parameters. We also experimentally demonstrate the application of path planning using PRM to mobile car-like robot

Analysis and Development of Computational Intelligence based Navigational Controllers for Multiple Mobile Robots

Navigational path planning problems of the mobile robots have received considerable attention over the past few decades. The navigation problem of mobile robots are consisting of following three aspects i.e. locomotion, path planning and map building. Based on these three aspects path planning algorithm for a mobile robot is formulated, which is capable of finding an optimal collision free path from the start point to the target point in a given environment. The main objective of the dissertation is to investigate the advanced methodologies for both single and multiple mobile robots navigation in highly cluttered environments using computational intelligence approach. Firstly, three different standalone computational intelligence approaches based on the Adaptive Neuro-Fuzzy Inference System (ANFIS), Cuckoo Search (CS) algorithm and Invasive Weed Optimization (IWO) are presented to address the problem of path planning in unknown environments. Next two different hybrid approaches are developed using CS-ANFIS and IWO-ANFIS to solve the mobile robot navigation problems. The performance of each intelligent navigational controller is demonstrated through simulation results using MATLAB. Experimental results are conducted in the laboratory, using real mobile robots to validate the versatility and effectiveness of the proposed navigation techniques. Comparison studies show, that there are good agreement between them. During the analysis of results, it is noticed that CS-ANFIS and IWO-ANFIS hybrid navigational controllers perform better compared to other discussed navigational controllers. The results obtained from the proposed navigation techniques are validated by comparison with the results from other intelligent techniques such as Fuzzy logic, Neural Network, Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO) and other hybrid algorithms. By investigating the results, finally it is concluded that the proposed navigational methodologies are efficient and robust in the sense, that they can be effectively implemented to solve the path optimization problems of mobile robot in any complex environment

Collaborative Control for a Robotic Wheelchair: Evaluation of Performance, Attention, and Workload

Powered wheelchair users often struggle to drive safely and effectively and in more critical cases can only get around when accompanied by an assistant. To address these issues, we propose a collaborative control mechanism that assists the user as and when they require help. The system uses a multiple–hypotheses method to predict the driver’s intentions and if necessary, adjusts the control signals to achieve the desired goal safely. The main emphasis of this paper is on a comprehensive evaluation, where we not only look at the system performance, but, perhaps more importantly, we characterise the user performance, in an experiment that combines eye–tracking with a secondary task. Without assistance, participants experienced multiple collisions whilst driving around the predefined route. Conversely, when they were assisted by the collaborative controller, not only did they drive more safely, but they were able to pay less attention to their driving, resulting in a reduced cognitive workload. We discuss the importance of these results and their implications for other applications of shared control, such as brain–machine interfaces, where it could be used to compensate for both the low frequency and the low resolution of the user input

A Survey on Passing-through Control of Multi-Robot Systems in Cluttered Environments

This survey presents a comprehensive review of various methods and algorithms related to passing-through control of multi-robot systems in cluttered environments. Numerous studies have investigated this area, and we identify several avenues for enhancing existing methods. This survey describes some models of robots and commonly considered control objectives, followed by an in-depth analysis of four types of algorithms that can be employed for passing-through control: leader-follower formation control, multi-robot trajectory planning, control-based methods, and virtual tube planning and control. Furthermore, we conduct a comparative analysis of these techniques and provide some subjective and general evaluations.Comment: 18 pages, 19 figure

Sampling-Based Motion Planning: A Comparative Review

Sampling-based motion planning is one of the fundamental paradigms to generate robot motions, and a cornerstone of robotics research. This comparative review provides an up-to-date guideline and reference manual for the use of sampling-based motion planning algorithms. This includes a history of motion planning, an overview about the most successful planners, and a discussion on their properties. It is also shown how planners can handle special cases and how extensions of motion planning can be accommodated. To put sampling-based motion planning into a larger context, a discussion of alternative motion generation frameworks is presented which highlights their respective differences to sampling-based motion planning. Finally, a set of sampling-based motion planners are compared on 24 challenging planning problems. This evaluation gives insights into which planners perform well in which situations and where future research would be required. This comparative review thereby provides not only a useful reference manual for researchers in the field, but also a guideline for practitioners to make informed algorithmic decisions.Comment: 25 pages, 7 figures, Accepted for Volume 7 (2024) of the Annual Review of Control, Robotics, and Autonomous System