12 research outputs found

    Advances in Reinforcement Learning

    Get PDF
    Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

    Bio-Inspired Robotics

    Get PDF
    Modern robotic technologies have enabled robots to operate in a variety of unstructured and dynamically-changing environments, in addition to traditional structured environments. Robots have, thus, become an important element in our everyday lives. One key approach to develop such intelligent and autonomous robots is to draw inspiration from biological systems. Biological structure, mechanisms, and underlying principles have the potential to provide new ideas to support the improvement of conventional robotic designs and control. Such biological principles usually originate from animal or even plant models, for robots, which can sense, think, walk, swim, crawl, jump or even fly. Thus, it is believed that these bio-inspired methods are becoming increasingly important in the face of complex applications. Bio-inspired robotics is leading to the study of innovative structures and computing with sensory–motor coordination and learning to achieve intelligence, flexibility, stability, and adaptation for emergent robotic applications, such as manipulation, learning, and control. This Special Issue invites original papers of innovative ideas and concepts, new discoveries and improvements, and novel applications and business models relevant to the selected topics of ``Bio-Inspired Robotics''. Bio-Inspired Robotics is a broad topic and an ongoing expanding field. This Special Issue collates 30 papers that address some of the important challenges and opportunities in this broad and expanding field

    Implementation of the New Control Methods in Simplification of a Multidimensional Control and Optimization of a Control System Parameters.

    Get PDF
    The main purpose of this text is to present application of the Largest Lyapunov Exponent (LLE) as a criterion for optimization of the new type of simple controller parameters. Investigated controller is the part of numerically simulated control system. The calculation of LLE was done with a new method [2]. Introduction contains reference to previous publications on inverted pendulum control and Lyapunov stability. Application of the new simple formula for LLE estimation in control systems is discussed. In the next part simulated dynamical system is described and new type of simple controller allowing to control multidimensional system is introduced. In the last part results of the simulation are shown along with conclusions to whole dynamics analysis. Comparison of the proposed regulator with the linearquadratic regulator (LQR) was verified and its better effectiveness with respect to LQR was proved

    Reinforcement learning in continuous state- and action-space

    Get PDF
    Reinforcement learning in the continuous state-space poses the problem of the inability to store the values of all state-action pairs in a lookup table, due to both storage limitations and the inability to visit all states sufficiently often to learn the correct values. This can be overcome with the use of function approximation techniques with generalisation capability, such as artificial neural networks, to store the value function. When this is applied we can select the optimal action by comparing the values of each possible action; however, when the action-space is continuous this is not possible. In this thesis we investigate methods to select the optimal action when artificial neural networks are used to approximate the value function, through the application of numerical optimization techniques. Although it has been stated in the literature that gradient-ascent methods can be applied to the action selection [47], it is also stated that solving this problem would be infeasible, and therefore, is claimed that it is necessary to utilise a second artificial neural network to approximate the policy function [21, 55]. The major contributions of this thesis include the investigation of the applicability of action selection by numerical optimization methods, including gradient-ascent along with other derivative-based and derivative-free numerical optimization methods,and the proposal of two novel algorithms which are based on the application of two alternative action selection methods: NM-SARSA [40] and NelderMead-SARSA. We empirically compare the proposed methods to state-of-the-art methods from the literature on three continuous state- and action-space control benchmark problems from the literature: minimum-time full swing-up of the Acrobot; Cart-Pole balancing problem; and a double pole variant. We also present novel results from the application of the existing direct policy search method genetic programming to the Acrobot benchmark problem [12, 14]

    Energy-Economical Heuristically Based Control of Compass Gait Walking on Stochastically Varying Terrain

    Get PDF
    Investigation uses simulation to explore the inherent tradeoffs ofcontrolling high-speed and highly robust walking robots while minimizing energy consumption. Using a novel controller which optimizes robustness, energy economy, and speed of a simulated robot on rough terrain, the user can adjust their priorities between these three outcome measures and systematically generate a performance curveassessing the tradeoffs associated with these metrics

    Energy-Economical Heuristically Based Control of Compass Gait Walking on Stochastically Varying Terrain

    Get PDF
    Investigation uses simulation to explore the inherent tradeoffs ofcontrolling high-speed and highly robust walking robots while minimizing energy consumption. Using a novel controller which optimizes robustness, energy economy, and speed of a simulated robot on rough terrain, the user can adjust their priorities between these three outcome measures and systematically generate a performance curveassessing the tradeoffs associated with these metrics

    Enhanced Bees Algorithm with fuzzy logic and Kalman filtering

    Get PDF
    The Bees Algorithm is a new population-based optimisation procedure which employs a combination of global exploratory and local exploitatory search. This thesis introduces an enhanced version of the Bees Algorithm which implements a fuzzy logic system for greedy selection of local search sites. The proposed fuzzy greedy selection system reduces the number of parameters needed to run the Bees Algorithm. The proposed algorithm has been applied to a number of benchmark function optimisation problems to demonstrate its robustness and self-organising ability. The Bees Algorithm in both its basic and enhanced forms has been used to optimise the parameters of a fuzzy logic controller. The purpose of the controller is to stabilise and balance an under-actuated two-link acrobatic robot (ACROBOT) in the upright position. Kalman filtering, as a fast convergence gradient-based optimisation method, is introduced as an alternative to random neighbourhood search to guide worker bees speedily towards the optima of local search sites. The proposed method has been used to tune membership functions for a fuzzy logic system. Finally, the fuzzy greedy selection system is enhanced by using multiple independent criteria to select local search sites. The enhanced fuzzy selection system has again been used with Kalman filtering to speed up the Bees Algorithm. The resulting algorithm has been applied to train a Radial Basis Function (RBF) neural network for wood defect identification. The results obtained show that the changes made to the Bees Algorithm in this research have significantly improved its performance. This is because these enhancements maintain the robust global search attribute of the Bees Algorithm and improve its local search procedure.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

    A small spiking neural network with LQR control applied to the acrobot

    Get PDF
    This paper presents the results of a computer simulation which, combined a small network of spiking neurons with linear quadratic regulator (LQR) control to solve the acrobot swing-up and balance task. To our knowledge, this task has not been previously solved with spiking neural networks. Input to the network was drawn from the state of the acrobot, and output was torque, either directly applied to the actuated joint, or via the switching of an LQR controller designed for balance. The neural network’s weights were tuned using a (μ + λ)-evolution strategy without recombination, and neurons’ parameters, were chosen to roughly approximate biological neurons

    Enhanced Bees Algorithm with fuzzy logic and Kalman filtering

    Get PDF
    The Bees Algorithm is a new population-based optimisation procedure which employs a combination of global exploratory and local exploitatory search. This thesis introduces an enhanced version of the Bees Algorithm which implements a fuzzy logic system for greedy selection of local search sites. The proposed fuzzy greedy selection system reduces the number of parameters needed to run the Bees Algorithm. The proposed algorithm has been applied to a number of benchmark function optimisation problems to demonstrate its robustness and self-organising ability. The Bees Algorithm in both its basic and enhanced forms has been used to optimise the parameters of a fuzzy logic controller. The purpose of the controller is to stabilise and balance an under-actuated two-link acrobatic robot (ACROBOT) in the upright position. Kalman filtering, as a fast convergence gradient-based optimisation method, is introduced as an alternative to random neighbourhood search to guide worker bees speedily towards the optima of local search sites. The proposed method has been used to tune membership functions for a fuzzy logic system. Finally, the fuzzy greedy selection system is enhanced by using multiple independent criteria to select local search sites. The enhanced fuzzy selection system has again been used with Kalman filtering to speed up the Bees Algorithm. The resulting algorithm has been applied to train a Radial Basis Function (RBF) neural network for wood defect identification. The results obtained show that the changes made to the Bees Algorithm in this research have significantly improved its performance. This is because these enhancements maintain the robust global search attribute of the Bees Algorithm and improve its local search procedure
    corecore