Computation Approaches for Continuous Reinforcement Learning Problems

Optimisation theory is at the heart of any control process, where we seek to control the behaviour of a system through a set of actions. Linear control problems have been extensively studied, and optimal control laws have been identified. But the world around us is highly non-linear and unpredictable. For these dynamic systems, which don’t possess the nice mathematical properties of the linear counterpart, the classic control theory breaks and other methods have to be employed. But nature thrives by optimising non-linear and over-complicated systems. Evolutionary Computing (EC) methods exploit nature’s way by imitating the evolution process and avoid to solve the control problem analytically. Reinforcement Learning (RL) from the other side regards the optimal control problem as a sequential one. In every discrete time step an action is applied. The transition of the system to a new state is accompanied by a sole numerical value, the “reward” that designate the quality of the control action. Even though the amount of feedback information is limited into a sole real number, the introduction of the Temporal Difference method made possible to have accurate predictions of the value-functions. This paved the way to optimise complex structures, like the Neural Networks, which are used to approximate the value functions. In this thesis we investigate the solution of continuous Reinforcement Learning control problems by EC methodologies. The accumulated reward of such problems throughout an episode suffices as information to formulate the required measure, fitness, in order to optimise a population of candidate solutions. Especially, we explore the limits of applicability of a specific branch of EC, that of Genetic Programming (GP). The evolving population in the GP case is comprised from individuals, which are immediately translated to mathematical functions, which can serve as a control law. The major contribution of this thesis is the proposed unification of these disparate Artificial Intelligence paradigms. The provided information from the systems are exploited by a step by step basis from the RL part of the proposed scheme and by an episodic basis from GP. This makes possible to augment the function set of the GP scheme with adaptable Neural Networks. In the quest to achieve stable behaviour of the RL part of the system a modification of the Actor-Critic algorithm has been implemented. Finally we successfully apply the GP method in multi-action control problems extending the spectrum of the problems that this method has been proved to solve. Also we investigated the capability of GP in relation to problems from the food industry. These type of problems exhibit also non-linearity and there is no definite model describing its behaviour

Definition study for photovoltaic residential prototype system

A site evaluation was performed to assess the relative merits of different regions of the country in terms of the suitability for experimental photovoltaic powered residences. Eight sites were selected based on evaluation criteria which included population, photovoltaic systems performance and the cost of electrical energy. A parametric sensitivity analysis was performed for four selected site locations. Analytical models were developed for four different power system implementation approaches. Using the model which represents a direct (or float) charge system implementation the performance sensitivity to the following parameter variations is reported: (1) solar roof slope angle; (2) ratio of the number of series cells in the solar array to the number of series cells in the lead-acid battery; and (3) battery size. For a Cleveland site location, a system with no on site energy storage and with a maximum power tracking inverter which feeds back excess power to the utility was shown to have 19 percent greater net system output than the second place system. The experiment test plan is described. The load control and data acquisition system and the data display panel for the residence are discussed

Intelligent Systems

This book is dedicated to intelligent systems of broad-spectrum application, such as personal and social biosafety or use of intelligent sensory micro-nanosystems such as "e-nose", "e-tongue" and "e-eye". In addition to that, effective acquiring information, knowledge management and improved knowledge transfer in any media, as well as modeling its information content using meta-and hyper heuristics and semantic reasoning all benefit from the systems covered in this book. Intelligent systems can also be applied in education and generating the intelligent distributed eLearning architecture, as well as in a large number of technical fields, such as industrial design, manufacturing and utilization, e.g., in precision agriculture, cartography, electric power distribution systems, intelligent building management systems, drilling operations etc. Furthermore, decision making using fuzzy logic models, computational recognition of comprehension uncertainty and the joint synthesis of goals and means of intelligent behavior biosystems, as well as diagnostic and human support in the healthcare environment have also been made easier

Using an Inductive Learning Algorithm to Improve Antibody Generation in a Single Packet Computer Defense Immune System

Coherent optical sources in the mid-infrared region (mid-IR) are important fundamental tools for infrared countermeasures and battlefield remote sensing. Nonlinear optical effects can be applied to convert existing near-IR laser sources to radiate in the mid-IR. This research focused on achieving such a conversion with a quasi-phase matched optical parametric oscillators using orientation-patterned gallium arsenide (OPGaAs), a material that can be quasi-phased matched by periodically reversing the crystal structure during the epitaxial growth process. Although non-linear optical conversion was not ultimately achieved during this research, many valuable lessons were learned from working with this material. This thesis reviews the theory of nonlinear optics and explores the importance of accurate refractive index measurements to proper structure design. The details of four nonlinear optical experiments are presented recommendations are offered for the design of future OPGaAs crystals. Recommendations are also made for improved experimental techniques

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion