Evolutionary Robotics

info:eu-repo/semantics/publishedVersio

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Dynamic path planning of multiple mobile robots

Ph.DDOCTOR OF PHILOSOPH

ES-ENAS: Blackbox Optimization over Hybrid Spaces via Combinatorial and Continuous Evolution

We consider the problem of efficient blackbox optimization over a large hybrid search space, consisting of a mixture of a high dimensional continuous space and a complex combinatorial space. Such examples arise commonly in evolutionary computation, but also more recently, neuroevolution and architecture search for Reinforcement Learning (RL) policies. Unfortunately however, previous mutation-based approaches suffer in high dimensional continuous spaces both theoretically and practically. We thus instead propose ES-ENAS, a simple joint optimization procedure by combining Evolutionary Strategies (ES) and combinatorial optimization techniques in a highly scalable and intuitive way, inspired by the one-shot or supernet paradigm introduced in Efficient Neural Architecture Search (ENAS). Through this relatively simple marriage between two different lines of research, we are able to gain the best of both worlds, and empirically demonstrate our approach by optimizing BBOB functions over hybrid spaces as well as combinatorial neural network architectures via edge pruning and quantization on popular RL benchmarks. Due to the modularity of the algorithm, we also are able incorporate a wide variety of popular techniques ranging from use of different continuous and combinatorial optimizers, as well as constrained optimization.Comment: 22 pages. See https://github.com/google-research/google-research/tree/master/es_enas for associated cod

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

An Approach Based on Particle Swarm Optimization for Inspection of Spacecraft Hulls by a Swarm of Miniaturized Robots

The remoteness and hazards that are inherent to the operating environments of space infrastructures promote their need for automated robotic inspection. In particular, micrometeoroid and orbital debris impact and structural fatigue are common sources of damage to spacecraft hulls. Vibration sensing has been used to detect structural damage in spacecraft hulls as well as in structural health monitoring practices in industry by deploying static sensors. In this paper, we propose using a swarm of miniaturized vibration-sensing mobile robots realizing a network of mobile sensors. We present a distributed inspection algorithm based on the bio-inspired particle swarm optimization and evolutionary algorithm niching techniques to deliver the task of enumeration and localization of an a priori unknown number of vibration sources on a simplified 2.5D spacecraft surface. Our algorithm is deployed on a swarm of simulated cm-scale wheeled robots. These are guided in their inspection task by sensing vibrations arising from failure points on the surface which are detected by on-board accelerometers. We study three performance metrics: (1) proximity of the localized sources to the ground truth locations, (2) time to localize each source, and (3) time to finish the inspection task given a 75% inspection coverage threshold. We find that our swarm is able to successfully localize the present so

Evolutionary Reinforcement Learning: A Survey

Reinforcement learning (RL) is a machine learning approach that trains agents to maximize cumulative rewards through interactions with environments. The integration of RL with deep learning has recently resulted in impressive achievements in a wide range of challenging tasks, including board games, arcade games, and robot control. Despite these successes, there remain several crucial challenges, including brittle convergence properties caused by sensitive hyperparameters, difficulties in temporal credit assignment with long time horizons and sparse rewards, a lack of diverse exploration, especially in continuous search space scenarios, difficulties in credit assignment in multi-agent reinforcement learning, and conflicting objectives for rewards. Evolutionary computation (EC), which maintains a population of learning agents, has demonstrated promising performance in addressing these limitations. This article presents a comprehensive survey of state-of-the-art methods for integrating EC into RL, referred to as evolutionary reinforcement learning (EvoRL). We categorize EvoRL methods according to key research fields in RL, including hyperparameter optimization, policy search, exploration, reward shaping, meta-RL, and multi-objective RL. We then discuss future research directions in terms of efficient methods, benchmarks, and scalable platforms. This survey serves as a resource for researchers and practitioners interested in the field of EvoRL, highlighting the important challenges and opportunities for future research. With the help of this survey, researchers and practitioners can develop more efficient methods and tailored benchmarks for EvoRL, further advancing this promising cross-disciplinary research field

A generalized laser simulator algorithm for optimal path planning in constraints environment

Path planning plays a vital role in autonomous mobile robot navigation, and it has thus become one of the most studied areas in robotics. Path planning refers to a robot's search for a collision-free and optimal path from a start point to a predefined goal position in a given environment. This research focuses on developing a novel path planning algorithm, called Generalized Laser Simulator (GLS), to solve the path planning problem of mobile robots in a constrained environment. This approach allows finding the path for a mobile robot while avoiding obstacles, searching for a goal, considering some constraints and finding an optimal path during the robot movement in both known and unknown environments. The feasible path is determined between the start and goal positions by generating a wave of points in all directions towards the goal point with adhering to constraints. A simulation study employing the proposed approach is applied to the grid map settings to determine a collision-free path from the start to goal positions. First, the grid mapping of the robot's workspace environment is constructed, and then the borders of the workspace environment are detected based on the new proposed function. This function guides the robot to move toward the desired goal. Two concepts have been implemented to find the best candidate point to move next: minimum distance to goal and maximum index distance to the boundary, integrated by negative probability to sort out the most preferred point for the robot trajectory determination. In order to construct an optimal collision-free path, an optimization step was included to find out the minimum distance within the candidate points that have been determined by GLS while adhering to particular constraint's rules and avoiding obstacles. The proposed algorithm will switch its working pattern based on the goal minimum and boundary maximum index distances. For static obstacle avoidance, the boundaries of the obstacle(s) are considered borders of the environment. However, the algorithm detects obstacles as a new border in dynamic obstacles once it occurs in front of the GLS waves. The proposed method has been tested in several test environments with different degrees of complexity. Twenty different arbitrary environments are categorized into four: Simple, complex, narrow, and maze, with five test environments in each. The results demonstrated that the proposed method could generate an optimal collision-free path. Moreover, the proposed algorithm result are compared to some common algorithms such as the A* algorithm, Probabilistic Road Map, RRT, Bi-directional RRT, and Laser Simulator algorithm to demonstrate its effectiveness. The suggested algorithm outperforms the competition in terms of improving path cost, smoothness, and search time. A statistical test was used to demonstrate the efficiency of the proposed algorithm over the compared methods. The GLS is 7.8 and 5.5 times faster than A* and LS, respectively, generating a path 1.2 and 1.5 times shorter than A* and LS. The mean value of the path cost achieved by the proposed approach is 4% and 15% lower than PRM and RRT, respectively. The mean path cost generated by the LS algorithm, on the other hand, is 14% higher than that generated by the PRM. Finally, to verify the performance of the developed method for generating a collision-free path, experimental studies were carried out using an existing WMR platform in labs and roads. The experimental work investigates complete autonomous WMR path planning in the lab and road environments using live video streaming. The local maps were built using data from live video streaming s by real-time image processing to detect the segments of the lab and road environments. The image processing includes several operations to apply GLS on the prepared local map. The proposed algorithm generates the path within the prepared local map to find the path between start-to-goal positions to avoid obstacles and adhere to constraints. The experimental test shows that the proposed method can generate the shortest path and best smooth trajectory from start to goal points in comparison with the laser simulator