In situ Distributed Genetic Programming: An Online Learning Framework for Resource Constrained Networked Devices

This research presents In situ Distributed Genetic Programming (IDGP) as a framework for distributively evolving logic while attempting to maintain acceptable average performance on highly resource-constrained embedded networked devices. The framework is motivated by the proliferation of devices employing microcontrollers with communications capability and the absence of online learning approaches that can evolve programs for them. Swarm robotics, Internet of Things (IoT) devices including smart phones, and arguably the most constrained of the embedded systems, Wireless Sensor Networks (WSN) motes, all possess the capabilities necessary for the distributed evolution of logic - specifically the abilities of sensing, computing, actuation and communications. Genetic programming (GP) is a mechanism that can evolve logic for these devices using their “native” logic representation (i.e. programs) and so technically GP could evolve any behaviour that can be coded on the device. IDGP is designed, implemented, demonstrated and analysed as a framework for evolving logic via genetic programming on highly resource-constrained networked devices in real-world environments while achieving acceptable average performance. Designed with highly resource-constrained devices in mind, IDGP provides a guide for those wishing to implement genetic programming on such systems. Furthermore, an implementation on mote class devices is demonstrated to evolve logic for a time-varying sense-compute-act problem and another problem requiring the evolution of primitive communications. Distributed evolution of logic is also achieved by employing the Island Model architecture, and a comparison of individual and distributed evolution (with the same and slightly different goals) presented. This demonstrates the advantage of leveraging the fact that such devices often reside within networks of devices experiencing similar conditions. Since GP is a population-based metaheuristic which relies on the diversity of the population to achieve learning, many, if not most, programs within the population exhibit poor performance. As such, the average observed performance (pool fitness) of the population using the standard GP learning mechanism is unlikely to be acceptable for online learning scenarios. This is suspected to be the reason why no previous attempts have been made to deploy standard GP as an online learning approach. Nonetheless, the benefits of GP for evolving logic on such devices are compelling and motivated the design of a novel satisficing heuristic called Fitness Importance (FI). FI is population-based heuristic used to bias the evaluation of candidate solutions such that an “acceptable” average fitness (AAF) is achieved while also achieving ongoing, though diminished, learning capacity. This trade off motivated further investigation into whether dynamically adjusting the average performance in response to AAF would be superior to a constant, balanced, performing-learning approach. Dynamic and constant strategies were compared on a simple problem where the AAF target was changed during evolution, revealing that dynamically tracking the AAF target can yield a higher success rate in meeting the AAF. The combination of IDGP and FI offers a novel approach for achieving online learning with GP on highly resource-constrained embedded systems. Furthermore, it simultaneously considers the acceptable average performance of the system which may change during the operational lifetime. This approach could be applied to swarm and cooperative robot systems, WSN motes or IoT devices allowing them to cooperatively learn and adapt their logic locally to meet dynamic performance requirements

Autonomy in the real real-world: A behaviour based view of autonomous systems control in an industrial product inspection system

The thesis presented in this dissertation appears in two sequential parts that arose from an exploration of the use of Behaviour Based Artificial Intelligence (BBAI) techniques in a domain outside that of robotics, where BBAI is most frequently used. The work details a real-world physical implementation of the control and interactions of an industrial product inspection system from a BBAI perspective. It concentrates particularly on the control of a number of active laser scanning sensor systems (each a subsystem of a larger main inspection system), using a subsumption architecture. This industrial implementation is in itself a new direction for BBAI control and an important aspect of this thesis. However, the work has also led on to the development of a number of key ideas which contribute to the field of BBAI in general. The second part of the thesis concerns the nature of physical and temporal constraints on a distributed control system and the desirability of utilising mechanisms to provide continuous, low-level learning and adaptation of domain knowledge on a sub-behavioural basis. Techniques used include artificial neural networks and hill-climbing state-space search algorithms. Discussion is supported with examples from experiments with the laser scanning inspection system. Encouraging results suggest that concerted design effort at this low level of activity will benefit the whole system in terms of behavioural robustness and reliability. Relevant aspects of the design process that should be of value in similar real-world projects are identified and emphasised. These issues are particularly important in providing a firm foundation for artificial intelligence based control systems

Computer Aided Verification

This open access two-volume set LNCS 10980 and 10981 constitutes the refereed proceedings of the 30th International Conference on Computer Aided Verification, CAV 2018, held in Oxford, UK, in July 2018. The 52 full and 13 tool papers presented together with 3 invited papers and 2 tutorials were carefully reviewed and selected from 215 submissions. The papers cover a wide range of topics and techniques, from algorithmic and logical foundations of verification to practical applications in distributed, networked, cyber-physical, and autonomous systems. They are organized in topical sections on model checking, program analysis using polyhedra, synthesis, learning, runtime verification, hybrid and timed systems, tools, probabilistic systems, static analysis, theory and security, SAT, SMT and decisions procedures, concurrency, and CPS, hardware, industrial applications

HIERARCHICAL HYBRID-MODEL BASED DESIGN, VERIFICATION, SIMULATION, AND SYNTHESIS OF MISSION CONTROL FOR AUTONOMOUS UNDERWATER VEHICLES

The objective of modeling, verification, and synthesis of hierarchical hybrid mission control for underwater vehicle is to (i) propose a hierarchical architecture for mission control for an autonomous system, (ii) develop extended hybrid state machine models for the mission control, (iii) use these models to verify for logical correctness, (iv) check the feasibility of a simulation software to model the mission executed by an autonomous underwater vehicle (AUV) (v) perform synthesis of high-level mission coordinators for coordinating lower-level mission controllers in accordance with the given mission, and (vi) suggest further design changes for improvement. The dissertation describes a hierarchical architecture in which mission level controllers based on hybrid systems theory have been, and are being developed using a hybrid systems design tool that allows graphical design, iterative redesign, and code generation for rapid deployment onto the target platform. The goal is to support current and future autonomous underwater vehicle (AUV) programs to meet evolving requirements and capabilities. While the tool facilitates rapid redesign and deployment, it is crucial to include safety and performance verification into each step of the (re)design process. To this end, the modeling of the hierarchical hybrid mission controller is formalized to facilitate the use of available tools and newly developed methods for formal verification of safety and performance specifications. A hierarchical hybrid architecture for mission control of autonomous systems with application to AUVs is proposed and a theoretical framework for the models that make up the architecture is outlined. An underwater vehicle like any other autonomous system is a hybrid system, as the dynamics of the vehicle as well as its vehicle level control is continuous whereas the mission level control is discrete, making the overall system a hybrid system i.e., one possessing both continuous and discrete states. The hybrid state machine models of the mission controller modules is derived from their implementation done using TEJA, a software for representing hybrid systems with support for auto code generation. The verification of their logical correctness properties has been done using UPPAAL, a software tool for verification of timed automata a special kind of hybrid system. A Teja to Uppaal converter, called dem2xml, has been created at Applied Reserarch Lab that converts a hybrid (timed) autonomous system description in Teja to an Uppaal system description. Verification work involved developing abstract models for the lower level vehicle controllers with which the mission controller modules interact and follow a hierarchical approach: Assuming the correctness of level-zero or vehicle controllers, we establish the correctness of level-one mission controller modules, and then the correctness of level-two modules, etc. The goal of verification is to show that any valid meaning for a mission formalized in our research verifies the safe and correct execution of actions. Simulation of the sequence of actions executed for each of the operations give a better view of the combined working of the mission coordinators and the low level controllers. So we next looked into the feasibility of simulating the operations executed during a mission. A Perl program has been developed to convert the UPPAAL files in .xml format to OpenGL graphic files. The graphic files simulate the steps involved in the execution of a sequence of operations executed by an AUV. The highest level coordinators send mission orders to be executed by the lower level controllers. So a more generalized design of the highest level controllers would help to incorporate the execution of a variety of missions for a vast field of applications. Initially, we consider manually synthesized mission coordinator modules. Later we design automated synthesis of coordinators. This method synthesizes mission coordinators which coordinate the lower level controllers for the execution of the missions ordered and can be used for any autonomous system

Computer Aided Verification

This open access two-volume set LNCS 10980 and 10981 constitutes the refereed proceedings of the 30th International Conference on Computer Aided Verification, CAV 2018, held in Oxford, UK, in July 2018. The 52 full and 13 tool papers presented together with 3 invited papers and 2 tutorials were carefully reviewed and selected from 215 submissions. The papers cover a wide range of topics and techniques, from algorithmic and logical foundations of verification to practical applications in distributed, networked, cyber-physical, and autonomous systems. They are organized in topical sections on model checking, program analysis using polyhedra, synthesis, learning, runtime verification, hybrid and timed systems, tools, probabilistic systems, static analysis, theory and security, SAT, SMT and decisions procedures, concurrency, and CPS, hardware, industrial applications

The 1995 Goddard Conference on Space Applications of Artificial Intelligence and Emerging Information Technologies

This publication comprises the papers presented at the 1995 Goddard Conference on Space Applications of Artificial Intelligence and Emerging Information Technologies held at the NASA/Goddard Space Flight Center, Greenbelt, Maryland, on May 9-11, 1995. The purpose of this annual conference is to provide a forum in which current research and development directed at space applications of artificial intelligence can be presented and discussed

Craniofacial Growth Series Volume 56

https://deepblue.lib.umich.edu/bitstream/2027.42/153991/1/56th volume CF growth series FINAL 02262020.pdfDescription of 56th volume CF growth series FINAL 02262020.pdf : Proceedings of the 46th Annual Moyers Symposium and 44th Moyers Presymposiu

The 1989 Goddard Conference on Space Applications of Artificial Intelligence

The following topics are addressed: mission operations support; planning and scheduling; fault isolation/diagnosis; image processing and machine vision; data management; and modeling and simulation

Drama, a connectionist model for robot learning: experiments on grounding communication through imitation in autonomous robots

The present dissertation addresses problems related to robot learning from demonstra¬ tion. It presents the building of a connectionist architecture, which provides the robot with the necessary cognitive and behavioural mechanisms for learning a synthetic lan¬ guage taught by an external teacher agent. This thesis considers three main issues: 1) learning of spatio-temporal invariance in a dynamic noisy environment, 2) symbol grounding of a robot's actions and perceptions, 3) development of a common symbolic representation of the world by heterogeneous agents.We build our approach on the assumption that grounding of symbolic communication creates constraints not only on the cognitive capabilities of the agent but also and especially on its behavioural capacities. Behavioural skills, such as imitation, which allow the agent to co-ordinate its actionn to that of the teacher agent, are required aside to general cognitive abilities of associativity, in order to constrain the agent's attention to making relevant perceptions, onto which it grounds the teacher agent's symbolic expression. In addition, the agent should be provided with the cognitive capacity for extracting spatial and temporal invariance in the continuous flow of its perceptions. Based on this requirement, we develop a connectionist architecture for learning time series. The model is a Dynamical Recurrent Associative Memory Architecture, called DRAMA. It is a fully connected recurrent neural network using Hebbian update rules. Learning is dynamic and unsupervised. The performance of the architecture is analysed theoretically, through numerical simulations and through physical and simulated robotic experiments. Training of the network is computationally fast and inexpensive, which allows its implementation for real time computation and on-line learning in a inexpensive hardware system. Robotic experiments are carried out with different learning tasks involving recognition of spatial and temporal invariance, namely landmark recognition and prediction of perception-action sequence in maze travelling.The architecture is applied to experiments on robot learning by imitation. A learner robot is taught by a teacher agent, a human instructor and another robot, a vocabulary to describe its perceptions and actions. The experiments are based on an imitative strategy, whereby the learner robot reproduces the teacher's actions. While imitating the teacher's movements, the learner robot makes similar proprio and exteroceptions to those of the teacher. The learner robot grounds the teacher's words onto the set of common perceptions they share. We carry out experiments in simulated and physical environments, using different robotic set-ups, increasing gradually the complexity of the task. In a first set of experiments, we study transmission of a vocabulary to designate actions and perception of a robot. Further, we carry out simulation studies, in which we investigate transmission and use of the vocabulary among a group of robotic agents. In a third set of experiments, we investigate learning sequences of the robot's perceptions, while wandering in a physically constrained environment. Finally, we present the implementation of DRAMA in Robota, a doll-like robot, which can imitate the arms and head movements of a human instructor. Through this imitative game, Robota is taught to perform and label dance patterns. Further, Robota is taught a basic language, including a lexicon and syntactical rules for the combination of words of the lexicon, to describe its actions and perception of touch onto its body

Reinforcement Learning in Autonomous Robots: An Empirical Investigation of the Role of Emotions

Institute of Perception, Action and BehaviourThis thesis presents a study of the provision of emotions for artificial agents with the ultimate aim of enhancing their autonomy, i.e. making them more exible, robust and self-sufficient. In recent years, the importance of emotions and their assistance to cognition has been increasingly acknowledged. Emotions are no longer considered undesirable or simply useless. Their role in various aspects of human and animal cog- nition like perception, attention, memory, decision-making and social interaction has been recognised as essential. The importance of emotions is much more evident in social interaction and therefore much of the emotions research done in artificial systems focuses on the expression and recognition of emotions. However, recent neurophysiological research suggests that emotions also play a crucial part in cognition itself. This thesis investigates ways in which artificial emotions can improve autonomous behaviour in the domain of a simple, but complete, solitary learning agent. For this purpose, a non-symbolic emotion model was designed and implemented. It takes the form of a recurrent artificial neural network where emotions influence the perception of the state of the world, on which they ultimately depend. This is done through a hormone system that acts as a persistence mechanism. This model is somewhat more sophisticated than those usually found in equivalent non-symbolic systems, yet the emotions themselves were restricted to a few simplified emotions that do not try to mimic the complexity of the human counterparts, but are afforded by the agent's interaction with the environment. Several hypotheses were investigated of how the emotion model above could be integrated in a reinforcement learning framework which, by itself, provides the base for the adaptiveness necessary for autonomy. Experiments were carried out in a realistic robot simulator that compared the performance of emotional with non-emotional agents in a survival task that consists of maintaining adequate energy levels in an environment with obstacles and energy sources. One of the most common roles attributed to emotions is as source of reinforcement and was therefore examined first. In experiments with a controller that selects between primitive actions, the reinforcement provided by emotions was found inappropriate because of the time scale discrepancies introduced by the emotion model. The reinforcement provided by emotions proved to be much more successful when used by a controller that selects between behaviours rather than actions, achieving equivalent performance to that of a standard reinforcement function. One of the crucial issues for efficient and productive learning, highlighted by the latter experiments, is to determine exactly when the controller should re-evaluate its decision concerning which behaviour to activate. The emotions proved to be particularly helpful in this role, enabling better performance with substantially less computational effort than the best suited interruption mechanism using regular time intervals. The modulation of learning parameters such as learning rate and the exploration vs. exploitation ratio was also explored. Experiments suggested that emotions might also be useful for this purpose. This research led to the conclusion that artificial emotions are a useful construct to have in the domain of behaviour-based autonomous agents, because they provide a unifying way to tackle different issues of control, analogous to natural systems' emotions