Learning Team-Based Navigation: A Review of Deep Reinforcement Learning Techniques for Multi-Agent Pathfinding

Multi-agent pathfinding (MAPF) is a critical field in many large-scale robotic applications, often being the fundamental step in multi-agent systems. The increasing complexity of MAPF in complex and crowded environments, however, critically diminishes the effectiveness of existing solutions. In contrast to other studies that have either presented a general overview of the recent advancements in MAPF or extensively reviewed Deep Reinforcement Learning (DRL) within multi-agent system settings independently, our work presented in this review paper focuses on highlighting the integration of DRL-based approaches in MAPF. Moreover, we aim to bridge the current gap in evaluating MAPF solutions by addressing the lack of unified evaluation metrics and providing comprehensive clarification on these metrics. Finally, our paper discusses the potential of model-based DRL as a promising future direction and provides its required foundational understanding to address current challenges in MAPF. Our objective is to assist readers in gaining insight into the current research direction, providing unified metrics for comparing different MAPF algorithms and expanding their knowledge of model-based DRL to address the existing challenges in MAPF.Comment: 36 pages, 10 figures, published in Artif Intell Rev 57, 41 (2024

Self-adaptive decision-making mechanisms to balance the execution of multiple tasks for a multi-robots team

This work addresses the coordination problem of multiple robots with the goal of finding specific hazardous targets in an unknown area and dealing with them cooperatively. The desired behaviour for the robotic system entails multiple requirements, which may also be conflicting. The paper presents the problem as a constrained bi-objective optimization problem in which mobile robots must perform two specific tasks of exploration and at same time cooperation and coordination for disarming the hazardous targets. These objectives are opposed goals, in which one may be favored, but only at the expense of the other. Therefore, a good trade-off must be found. For this purpose, a nature-inspired approach and an analytical mathematical model to solve this problem considering a single equivalent weighted objective function are presented. The results of proposed coordination model, simulated in a two dimensional terrain, are showed in order to assess the behaviour of the proposed solution to tackle this problem. We have analyzed the performance of the approach and the influence of the weights of the objective function under different conditions: static and dynamic. In this latter situation, the robots may fail under the stringent limited budget of energy or for hazardous events. The paper concludes with a critical discussion of the experimental results

Mobil robotların yol planması için metasezgisel hibrit algoritmalar geliştirilmesi ve uygulanması

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Anahtar kelimeler: Meta-sezgisel algoritmalar, Mobil Robotlar, Yol planlama, Matlab, ROS (Robot Operating System) Günümüzde teknolojik gelişmelere paralel olarak, robot teknolojisinin tüm sektörlerde yaygın olarak kullanıldığı görülmektedir. Endüstriyel ve teknik uygulamalarda mobil robotlar güvenilirlik, erişilebilirlik ve maliyet gibi sahip olduğu üstün özelliklerle ön plana çıkmaktadır. Mobil robotlar hareket halinde olduklarından diğer robotlara göre enerji tüketimi daha fazla önem kazanmaktadır. Dolayısıyla mobil robotlar için hedefe ulaşabilecek en kısa yolun belirlenmesi önemli bir çalışma alanı olmuştur. Bu kapsamda, mobil robotlarda yol planlama algoritmalarının geliştirilmesi üzerine çalışmalara odaklanılmış olup konu hala güncelliğini korumaktadır. Bu tez çalışmasının amacı, meta-sezgisel algoritma tabanlı mobil robot yol planlaması için özgün bir hibrit algoritma geliştirmektir. İlk olarak meta-sezgisel tabanlı yol planlama algoritmaları incelenerek daha etkili ve özgün bir hibrit algoritmanın geliştirilmesi için zemin hazırlandı. Kullanılan algoritmaların amacı meta-sezgisel algoritmaların verimliliğini ve yol planlamada yolun minimize edilerek belirlenen zaman, maliyet gibi en önemli performans kriterlerini geliştirmektir. Bu çalışmada öncelikle tek bir tepe kamerası kullanılarak görüntü düzleminden alınan görüntü işlenerek mobil robotların ve engellerin koordinatları alınmaktadır. Daha sonra üzerinde engeller bulunan statik bir ortamda tekli ve çoklu mobil robotların yol planlaması için özgün hibrit bir algoritma geliştirildi ve uygulandı. PSO(Parçacık Sürü Optimizasyonu), FA (Ateş Böceği Algoritması), CS (Guguk Kuşu Algoritması) gibi meta-sezgisel algoritmalardan her biri ayrı ayrı kullanılarak elde edilen sonuçlar benzer çalışmalarla karşılaştırıldı. Daha sonra aynı ortam şartlarında robotlar için yol planlamalar yapmak üzere çalışmaya özgü CS-PSO(Guguk Kuşu AlgoritmasıParçacık Sürü Optimizasyonu), CS-FA(Guguk Kuşu Algoritması-Ateş Böceği Algoritması), CS-PSO-FA (Guguk Kuşu Algoritması-Parçacık Sürü OptimizasyonuAteş Böceği Algoritması) gibi hibrit algoritmalar geliştirildi. Kameradan alınan görüntüyü işleyerek harita oluşturma, algoritmalar ile yol bulma, robot ile haberleşme ve robotun algoritmalar ile belirleyeceği yol bilgisine göre ilerleyebilmesi gibi tüm faaliyetleri kolaylıkla gerçekleştirebilmek amacıyla MATLAB-GUI (Graphical User Interface) tabanlı bir ara yüz tasarlanmıştır. Ayrıca geliştirilen algoritmaları doğrulamak üzere fiziksel ortamda çeşitli uygulamalar gerçekleştirilmekte ve elde edilen sonuçların karşılaştırılması yapılmaktadır. Son olarak geliştirilen CS-PSO-FA hibrit algoritmasıyla elde edilen yolun diğer algoritmalara göre daha kısa olduğu ve böylece daha yüksek bir performansa sahip olduğu kanıtlanmıştır. DEVELOPMENT AND APPLICATION OF META-HEURISTIC HYBRID ALGORITHMS FOR PATH PLANNING OF MOBIL ROBOTSKeywords: Meta-heuristic algorithm, Mobil Robots, Path Planning, Matlab, ROS (Robot Operating System) Today, parallel to technological developments, robot technology seems to be widely used in all sectors. In industrial and technical applications, mobile robots come into prominence with superior features such as reliability, accessibility and cost. Energy consumption is more important for mobile robots than other robots because mobile robots are in motion. Therefore, determining the shortest path to reach the target for mobile robots has become an important field of study. In this context, in mobile robots the development of road planning algorithms has been focused and the issue keep up-to-date.The purpose of this thesis is to develop an authentic hybrid algorithm for meta-heuristic algorithm based mobile robot path planning. Firstly, metaheuristic-based path planning algorithms were examined and the basis for developing a more efficient and authentic hybrid algorithm was prepared. The purpose of the algorithms used is to improve the efficiency of meta-heuristic algorithms and the most important performance criteria such as time, cost, etc., determined by minimizing the path in the path planning. In this study, coordinates of mobile robots and obstacles are taken by processing the image taken from the image plane using a single top camera. In a static environment with obstacles on it, an authentic hybrid algorithm for path planning of single and multiple mobile robots has been developed and implemented. The results were obtained using every meta-heuristic algorithm such as PSO (Particle Swarm Optimization), FA (Firefly Algorithm) and CS (Cuckoo Search Algorithm) separately and these results were compared with similar studies. Then, hybrid algorithms specific to study such as CS-PSO (Cuckoo Search Algorithm - Particle Swarm Optimization), CS-FA (Cuckoo Search Algorithm - Firefly Algorithm) and CS-PSO-FA (Cuckoo Search Algorithm - Particle Swarm Optimization - Firefly Algorithm) were developed to make path planning for robots in the same environment conditions. A MATLAB-GUI (Graphical User Interface) based interface was designed in order to easily perform all activities such as generating maps by processing the images taken from the camera, finding paths with algorithms, communicating with robots, and navigating according to path information that the robot determines with algorithms. In addition, various applications are performed in the physical environment to verify the developed algorithms and the obtained results are compared. In conclusion, it is proved that the path obtained with the developed CS-PSO-FA hybrid algorithm is shorter than the other algorithms and thus has a higher performance

Particle Swarm Optimization

Particle swarm optimization (PSO) is a population based stochastic optimization technique influenced by the social behavior of bird flocking or fish schooling.PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms (GA). The system is initialized with a population of random solutions and searches for optima by updating generations. However, unlike GA, PSO has no evolution operators such as crossover and mutation. In PSO, the potential solutions, called particles, fly through the problem space by following the current optimum particles. This book represents the contributions of the top researchers in this field and will serve as a valuable tool for professionals in this interdisciplinary field

Application of machine learning and artificial intelligence techniques to improve autonomy in maritime surveillance radar systems

Executive Summary. Current maritime radar surveillance missions are typically carried out using an airborne platform with one or more operators on board. The workload of human operators is a bottleneck in surveillance performance as they can only perform on a single platform discontinuously. Additionally, with progress being made towards the use of remotely operated UAVs for radar surveillance missions, an increase in radar operational autonomy is required to maximise the UAV’s surveillance potential. Consequently, the focus of this research is to improve the autonomy of current maritime radar surveillance missions. By reducing the workload of the current radar operator, then surveillance missions can be performed for longer. This research breaks the autonomy of the radar surveillance mission into two aspects: the platform operator autonomy and the radar operator autonomy. However, the implemented autonomous methods must “complement rather than compete with one another". In order to implement algorithms for the platform operator autonomy and radar operator autonomy, a maritime radar surveillance simulation and user interface is required. Consequently, this work outlines a real-time maritime surveillance radar simulation and graphical user interface which can be used to carry out missions in the same manner as an operator would with a real system. For the platform operator autonomy aspect, there is a trade-off between maximising information obtained from the surveillance search area and minimising fuel consumption. The research presented here provides an approach for the optimisation of a UAV’s trajectory for maritime radar wide area persistent surveillance to simultaneously minimise fuel consumption, maximise mean probability of detection, and minimise mean revisit time. Quintic polynomials are used to generate UAV trajectories due to their ability to provide complete and complex solutions while requiring few inputs. A wide area search radar model is used within this article in conjunction with a discretised grid in order to determine the search area’s mean probability of detection and mean revisit time. The trajectory generation method is then used in conjunction with a multi-objective particle swarm optimisation algorithm to obtain a global optimum in terms of path, airspeed (and thus time), and altitude. The performance of the approach is then tested over two common maritime surveillance scenarios and compared to an industry recommended baseline. In terms of the radar operator autonomy, imitation learning, as opposed to other forms of machine learning, are advantageous as they act in the same manner as the operator, thus reducing the deviation from the current operational standard and allowing for easier system qualification and human operator interaction. The developed radar simulation and interface is used to obtain operator decision data from a human operator. The operator data is then used with two imitation learning methods, namely Bayesian networks and inverse reinforcement learning, with the methods used in place of the operator with their performance compared and their suitability discussed

Shortest Route at Dynamic Location with Node Combination-Dijkstra Algorithm

Abstract— Online transportation has become a basic requirement of the general public in support of all activities to go to work, school or vacation to the sights. Public transportation services compete to provide the best service so that consumers feel comfortable using the services offered, so that all activities are noticed, one of them is the search for the shortest route in picking the buyer or delivering to the destination. Node Combination method can minimize memory usage and this methode is more optimal when compared to A* and Ant Colony in the shortest route search like Dijkstra algorithm, but can’t store the history node that has been passed. Therefore, using node combination algorithm is very good in searching the shortest distance is not the shortest route. This paper is structured to modify the node combination algorithm to solve the problem of finding the shortest route at the dynamic location obtained from the transport fleet by displaying the nodes that have the shortest distance and will be implemented in the geographic information system in the form of map to facilitate the use of the system. Keywords— Shortest Path, Algorithm Dijkstra, Node Combination, Dynamic Location (key words

Data bases and data base systems related to NASA's Aerospace Program: A bibliography with indexes

This bibliography lists 641 reports, articles, and other documents introduced into the NASA scientific and technical information system during the period January 1, 1981 through June 30, 1982. The directory was compiled to assist in the location of numerical and factual data bases and data base handling and management systems

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field