An empirical evaluation of evolutionary controller design methods for collective gathering task

This research aims to evaluate the performance of evolutionary controller design methods for developing a collective behaviour for a team of robots. The methods tested in this research are NEAT which is capable of finding minimal solution quickly, and SANE which maintains high genetic diversity through neuron level evolution. The task chosen for these methods was a collective gathering task which required a team of robots to cooperate in finding and retrieving item of interest. Our results showed that NEAT consistently produced better controllers compared to SANE

Reinforcement Learning for UAV Attitude Control

Autopilot systems are typically composed of an "inner loop" providing stability and control, while an "outer loop" is responsible for mission-level objectives, e.g. way-point navigation. Autopilot systems for UAVs are predominately implemented using Proportional, Integral Derivative (PID) control systems, which have demonstrated exceptional performance in stable environments. However more sophisticated control is required to operate in unpredictable, and harsh environments. Intelligent flight control systems is an active area of research addressing limitations of PID control most recently through the use of reinforcement learning (RL) which has had success in other applications such as robotics. However previous work has focused primarily on using RL at the mission-level controller. In this work, we investigate the performance and accuracy of the inner control loop providing attitude control when using intelligent flight control systems trained with the state-of-the-art RL algorithms, Deep Deterministic Gradient Policy (DDGP), Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO). To investigate these unknowns we first developed an open-source high-fidelity simulation environment to train a flight controller attitude control of a quadrotor through RL. We then use our environment to compare their performance to that of a PID controller to identify if using RL is appropriate in high-precision, time-critical flight control.Comment: 13 pages, 9 figure

Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Decision tree learning for intelligent mobile robot navigation

The replication of human intelligence, learning and reasoning by means of computer algorithms is termed Artificial Intelligence (Al) and the interaction of such algorithms with the physical world can be achieved using robotics. The work described in this thesis investigates the applications of concept learning (an approach which takes its inspiration from biological motivations and from survival instincts in particular) to robot control and path planning. The methodology of concept learning has been applied using learning decision trees (DTs) which induce domain knowledge from a finite set of training vectors which in turn describe systematically a physical entity and are used to train a robot to learn new concepts and to adapt its behaviour. To achieve behaviour learning, this work introduces the novel approach of hierarchical learning and knowledge decomposition to the frame of the reactive robot architecture. Following the analogy with survival instincts, the robot is first taught how to survive in very simple and homogeneous environments, namely a world without any disturbances or any kind of "hostility". Once this simple behaviour, named a primitive, has been established, the robot is trained to adapt new knowledge to cope with increasingly complex environments by adding further worlds to its existing knowledge. The repertoire of the robot behaviours in the form of symbolic knowledge is retained in a hierarchy of clustered decision trees (DTs) accommodating a number of primitives. To classify robot perceptions, control rules are synthesised using symbolic knowledge derived from searching the hierarchy of DTs. A second novel concept is introduced, namely that of multi-dimensional fuzzy associative memories (MDFAMs). These are clustered fuzzy decision trees (FDTs) which are trained locally and accommodate specific perceptual knowledge. Fuzzy logic is incorporated to deal with inherent noise in sensory data and to merge conflicting behaviours of the DTs. In this thesis, the feasibility of the developed techniques is illustrated in the robot applications, their benefits and drawbacks are discussed

Navigation and coordination of autonomous mobile robots with limited resources

The use of autonomous robots in complex exploration tasks is rapidly increasing. Indeed, robots can provide speed and cost effectiveness in many tasks, as well as allow operation in environments that are hostile to humans. In this dissertation we: 1) provide two adaptive navigation algorithms; 2) develop a coordination mechanism; 3) develop a dynamic partnership formation mechanism; and 4) demonstrate the use of algorithms in a hardware implementation. The two adaptive navigation algorithms are neuro-evolution and policy gradient, where the results show that effective, adaptive navigation techniques can be developed for mobile robots in an exploration domain when the robots have limited capabilities. In addition, we show that policy gradient approaches thrive on short-term objective values, whereas neuro-evolutionary approaches provide more robust results with a time-extended objective value. Finally, we show that summing short-term values to generate a time-extended value does not capture the complexities of some real world exploration tasks. Coordinating multi-robot systems to maximize global information collection in these exploration domains presents additional challenges. In particular, in many multi-robot domains where communication is expensive, the coordination must be achieved in a passive manner. This is done in this dissertation via objective design on a hierarchical control scheme where both a navigation algorithm and coordination algorithm are operating simultaneously. We then extend results on such multi-robot coordination algorithms to domains where the robots cannot achieve the required tasks without forming teams. We investigate team formation where: i) robots must perform a task together; ii) there is an optimal number of robots; and iii) individuals vary, forming heterogeneous teams. The results show that using neuro-evolutionary robot teams with objective functions that are aligned with the global objective and locally computable significantly improve over robots using the global objective directly, particularly in dynamic environments. Finally, we develop a path to implementation of all of the coordination research done to date into robot hardware. The design represents a stable, robust robotic platform on which navigation and coordination algorithms can be run in the fashion they were developed and intricacies of real-world operation can be analyzed. Functional experiments show that the platform operates as expected and performs similarly to algorithm work done in simulation

Multi-Robot Systems: Challenges, Trends and Applications

This book is a printed edition of the Special Issue entitled “Multi-Robot Systems: Challenges, Trends, and Applications” that was published in Applied Sciences. This Special Issue collected seventeen high-quality papers that discuss the main challenges of multi-robot systems, present the trends to address these issues, and report various relevant applications. Some of the topics addressed by these papers are robot swarms, mission planning, robot teaming, machine learning, immersive technologies, search and rescue, and social robotics