Dynamic pathfinding for a swarm intelligence based UAV control model using particle swarm optimisation

In recent years unmanned aerial vehicles (UAVs) have become smaller, cheaper, and more efficient, enabling the use of multiple autonomous drones where previously a single, human-operated drone would have been used. This likely includes crisis response and search and rescue missions. These systems will need a method of navigating unknown and dynamic environments. Typically, this would require an incremental heuristic search algorithm, however, these algorithms become increasingly computationally and memory intensive as the environment size increases. This paper used two different Swarm Intelligence (SI) algorithms: Particle Swarm Optimisation and Reynolds flocking to propose an overall system for controlling and navigating groups of autonomous drones through unknown and dynamic environments. This paper proposes Particle Swarm Optimisation Pathfinding (PSOP): a dynamic, cooperative algorithm; and, Drone Flock Control (DFC): a modular model for controlling systems of agents, in 3D environments, such that collisions are minimised. Using the Unity game engine, a realtime application, simulation environment, and data collection apparatus were developed and the performances of DFC-controlled drones—navigating with either the PSOP algorithm or a D* Lite implementation—were compared. The simulations do not consider UAV dynamics. The drones were tasked with navigating to a given target position in environments of varying size and quantitative data on pathfinding performance, computational and memory performance, and usability were collected. Using this data, the advantages of PSO-based pathfinding were demonstrated. PSOP was shown to be more memory efficient, more successful in the creation of high quality, accurate paths, more usable and as computationally efficient as a typical incremental heuristic search algorithm when used as part of a SI-based drone control model. This study demonstrated the capabilities of SI approaches as a means of controlling multi-agent UAV systems in a simple simulation environment. Future research may look to apply the DFC model, with the PSOP algorithm, to more advanced simulations which considered environment factors like atmospheric pressure and turbulence, or to real-world UAVs in a controlled environment

Optimizing UAV Navigation: A Particle Swarm Optimization Approach for Path Planning in 3D Environments

This study explores the application of Particle Swarm Optimization (PSO) in Unmanned Aerial Vehicle (UAV) path planning within a simulated three-dimensional environment. UAVs, increasingly prevalent across various sectors, demand efficient navigation solutions that account for dynamic and unpredictable elements. Traditional pathfinding algorithms often fall short in complex scenarios, hence the shift towards PSO, a bio-inspired algorithm recognized for its adaptability and robustness. We developed a Python-based framework to simulate the UAV path planning scenario. The PSO algorithm was tasked to navigate a UAV from a starting point to a predetermined destination while avoiding spherical obstacles. The environment was set within a 3D grid with a series of waypoints, marking the UAV's trajectory, generated by the PSO to ensure obstacle avoidance and path optimization. The PSO parameters were meticulously tuned to balance the exploration and exploitation of the search space, with an emphasis on computational efficiency. A cost function penalizing proximity to obstacles guided the PSO in real-time decision-making, resulting in a collision-free and optimized path. The UAV's trajectory was visualized in both 2D and 3D perspectives, with the analysis focusing on the path's smoothness, length, and adherence to spatial constraints. The results affirm the PSO's effectiveness in UAV path planning, successfully avoiding obstacles and minimizing path length. The findings highlight PSO's potential for practical UAV applications, emphasizing the importance of parameter optimization. This research contributes to the advancement of autonomous UAV navigation, indicating PSO as a viable solution for real-world path planning challenges

Inheritance-Based Diversity Measures for Explicit Convergence Control in Evolutionary Algorithms

Diversity is an important factor in evolutionary algorithms to prevent premature convergence towards a single local optimum. In order to maintain diversity throughout the process of evolution, various means exist in literature. We analyze approaches to diversity that (a) have an explicit and quantifiable influence on fitness at the individual level and (b) require no (or very little) additional domain knowledge such as domain-specific distance functions. We also introduce the concept of genealogical diversity in a broader study. We show that employing these approaches can help evolutionary algorithms for global optimization in many cases.Comment: GECCO '18: Genetic and Evolutionary Computation Conference, 2018, Kyoto, Japa

The theoretical and methodological toolbox of urban economics: from and towards where?

After more than five decades since the recognition of the importance of a branch in Economics called Regional and Urban Economics, there is for sure sufficient scientific material for an ex-post evaluation of what has been achieved so far, where is the scientific frontier in this field, and what are the main open conceptual questions. The present contribution aims at providing such a "picture", by underlining, in a critical way, the results achieved and the challenges that still remain to be faced. It is not at all a first attempt in this direction: especially in the last decade, some doubts on the scientific achievements in the fields of Regional and Urban Economics, and more widely of Regional Science, were stressed, especially in the American academic world. These sciences were interpreted as going through a deep "scientific crisis", interpreted as a sort of downward slope in their "life cycle". Some scientific in-depth analyses have already been provided by regional scientists, rightly claiming that still much can be said and be produced in terms of both theoretical and empirical (modelling) elements. Our paper will start from those considerations, and provide a step further in the interpretation of the problems encountered by Regional and Urban Economics in the actual scientific world, and will strongly suggest that there are important signs of a reaffirmation of the discipline, given the recently strong renewed interest around the conept of territory, developed: - within other branches of the Economic theory. A clear example of this is the emergence of the "New Economic Geography" theory, widely anchored to some well known regional economic concepts. The same can be said for the recent concept of knowledge spillover of the Industrial Economists, in which the innovative diffusion process is largely dependent on geographical distance among innovative actors; - with traditionally related scientific fields, such as Urban Planning and Geography, for the development of a unified framework of analysis for territorial economic processes; - with sociologists, on the concept of "social capital", related to the interaction between social and economic elements for the explanation of the processes of knowledge creation at the local level. In all these cases, there is still place for a more in-depth cooperation among scientists, with the aim to achieve a more in-depth knowledge of theoretical concepts; moreover, in all these cases, there is still place for regional economists to strengthen their role within the international interdisciplinary arena. The paper will highlight these cases, and strengthen the reasons for this statement.

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

Toward human-like pathfinding with hierarchical approaches and the GPS of the brain theory

Pathfinding for autonomous agents and robots has been traditionally driven by finding optimal paths. Where typically optimality means finding the shortest path between two points in a given environment. However, optimality may not always be strictly necessary. For example, in the case of video games, often computing the paths for non-player characters (NPC) must be done under strict time constraints to guarantee real time simulation. In those cases, performance is more important than finding the shortest path, specially because often a sub-optimal path can be just as convincing from the point of view of the player. When simulating virtual humanoids, pathfinding has also been used with the same goal: finding the shortest path. However, humans very rarely follow precise shortest paths, and thus there are other aspects of human decision making and path planning strategies that should be incorporated in current simulation models. In this thesis we first focus on improving performance optimallity to handle as many virtual agents as possible, and then introduce neuroscience research to propose pathfinding algorithms that attempt to mimic humans in a more realistic manner.In the case of simulating NPCs for video games, one of the main challenges is to compute paths as efficiently as possible for groups of agents. As both the size of the environments and the number of autonomous agents increase, it becomes harder to obtain results in real time under the constraints of memory and computing resources. For this purpose we explored hierarchical approaches for two reasons: (1) they have shown important performance improvements for regular grids and other abstract problems, and (2) humans tend to plan trajectories also following an topbottom abstraction, focusing first on high level location and then refining the path as they move between those high level locations. Therefore, we believe that hierarchical approaches combine the best of our two goals: improving performance for multi-agent pathfinding and achieving more human-like pathfinding. Hierarchical approaches, such as HNA* (Hierarchical A* for Navigation Meshes) can compute paths more efficiently, although only for certain configurations of the hierarchy. For other configurations, the method suffers from a bottleneck in the step that connects the Start and Goal positions with the hierarchy. This bottleneck can drop performance drastically.In this thesis we present different approaches to solve the HNA* bottleneck and thus obtain a performance boost for all hierarchical configurations. The first method relies on further memory storage, and the second one uses parallelism on the GPU. Our comparative evaluation shows that both approaches offer speed-ups as high as 9x faster than A*, and show no limitations based on hierarchical configuration. Then we further exploit the potential of CUDA parallelism, to extend our implementation to HNA* for multi-agent path finding. Our method can now compute paths for over 500K agents simultaneously in real-time, with speed-ups above 15x faster than a parallel multi-agent implementation using A*. We then focus on studying neurosience research to learn about the way that humans build mental maps, in order to propose novel algorithms that take those finding into account when simulating virtual humans. We propose a novel algorithm for path finding that is inspired by neuroscience research on how the brain learns and builds cognitive maps. Our method represents the space as a hexagonal grid, based on the GPS of the brain theory, and fires memory cells as counters. Our path finder then combines a method for exploring unknown environments while building such a cognitive map, with an A* search using a modified heuristic that takes into account the GPS of the brain cognitive map.El problema de Pathfinding para agentes autónomos o robots, ha consistido tradicionalmente en encontrar un camino óptimo, donde por óptimo se entiende el camino de distancia mínima entre dos posiciones de un entorno. Sin embargo, en ocasiones puede que no sea estrictamente necesario encontrar una solución óptima. Para ofrecer la simulación de multitudes de agentes autónomos moviéndose en tiempo real, es necesario calcular sus trayectorias bajo condiciones estrictas de tiempo de computación, pero no es fundamental que las soluciones sean las de distancia mínima ya que, con frecuencia, el observador no apreciará la diferencia entre un camino óptimo y un sub-óptimo. Por tanto, suele ser suficiente con que la solución encontrada sea visualmente creíble para el observado. Cuando se simulan humanoides virtuales en aplicaciones de realidad virtual que requieren avatares que simulen el comportamiento de los humanos, se tiende a emplear los mismos algoritmos que en video juegos, con el objetivo de encontrar caminos de distancia mínima. Pero si realmente queremos que los avatares imiten el comportamiento humano, tenemos que tener en cuenta que, en el mundo real, los humanos rara vez seguimos precisamente el camino más corto, y por tanto se deben considerar otros aspectos que influyen en la toma de decisiones de los humanos y la selección de rutas en el mundo real. En esta tesis nos centraremos primero en mejorar el rendimiento de la búsqueda de caminos para poder simular grandes números de humanoides virtuales autónomos, y a continuación introduciremos conceptos de investigación con mamíferos en neurociencia, para proponer soluciones al problema de pathfinding que intenten imitar con mayor realismo, el modo en el que los humanos navegan el entorno que les rodea. A medida que aumentan tanto el tamaño de los entornos virtuales como el número de agentes autónomos, resulta más difícil obtener soluciones en tiempo real, debido a las limitaciones de memoria y recursos informáticos. Para resolver este problema, en esta tesis exploramos enfoques jerárquicos porque consideramos que combinan dos objetivos fundamentales: mejorar el rendimiento en la búsqueda de caminos para multitudes de agentes y lograr una búsqueda de caminos similar a la de los humanos. El primer método presentado en esta tesis se basa en mejorar el rendimiento del algoritmo HNA* (Hierarchical A* for Navigation Meshes) incrementando almacenamiento de datos en memoria, y el segundo utiliza el paralelismo para mejorar drásticamente el rendimiento. La evaluación cuantitativa realizada en esta tesis, muestra que ambos enfoques ofrecen aceleraciones que pueden llegar a ser hasta 9 veces más rápidas que el algoritmo A* y no presentan limitaciones debidas a la configuración jerárquica. A continuación, aprovechamos aún más el potencial del paralelismo ofrecido por CUDA para extender nuestra implementación de HNA* a sistemas multi-agentes. Nuestro método permite calcular caminos simultáneamente y en tiempo real para más de 500.000 agentes, con una aceleración superior a 15 veces la obtenida por una implementación paralela del algoritmo A*. Por último, en esta tesis nos hemos centrado en estudiar los últimos avances realizados en el ámbito de la neurociencia, para comprender la manera en la que los humanos construyen mapas mentales y poder así proponer nuevos algoritmos que imiten de forma más real el modo en el que navegamos los humanos. Nuestro método representa el espacio como una red hexagonal, basada en la distribución de ¿place cells¿ existente en el cerebro, e imita las activaciones neuronales como contadores en dichas celdas. Nuestro buscador de rutas combina un método para explorar entornos desconocidos mientras construye un mapa cognitivo hexagonal, utilizando una búsqueda A* con una nueva heurística adaptada al mapa cognitivo del cerebro y sus contadores

Artificial and Computational Intelligence in Games (Dagstuhl Seminar 12191)

This report documents the program and the outcomes of Dagstuhl Seminar 12191 "Artificial and Computational Intelligence in Games". The aim for the seminar was to bring together creative experts in an intensive meeting with the common goals of gaining a deeper understanding of various aspects of artificial and computational intelligence in games, to help identify the main challenges in game AI research and the most promising venues to deal with them. This was accomplished mainly by means of workgroups on 14 different topics (ranging from search, learning, and modeling to architectures, narratives, and evaluation), and plenary discussions on the results of the workgroups. This report presents the conclusions that each of the workgroups reached. We also added short descriptions of the few talks that were unrelated to any of the workgroups