On the Combination of Game-Theoretic Learning and Multi Model Adaptive Filters

This paper casts coordination of a team of robots within the framework of game theoretic learning algorithms. In particular a novel variant of fictitious play is proposed, by considering multi-model adaptive filters as a method to estimate other players’ strategies. The proposed algorithm can be used as a coordination mechanism between players when they should take decisions under uncertainty. Each player chooses an action after taking into account the actions of the other players and also the uncertainty. Uncertainty can occur either in terms of noisy observations or various types of other players. In addition, in contrast to other game-theoretic and heuristic algorithms for distributed optimisation, it is not necessary to find the optimal parameters a priori. Various parameter values can be used initially as inputs to different models. Therefore, the resulting decisions will be aggregate results of all the parameter values. Simulations are used to test the performance of the proposed methodology against other game-theoretic learning algorithms.</p

To Adapt or Not to Adapt: A Quantification Technique for Measuring an Expected Degree of Self-Adaptation

Self-adaptation and self-organization (SASO) have been introduced to the management of technical systems as an attempt to improve robustness and administrability. In particular, both mechanisms adapt the system’s structure and behavior in response to dynamics of the environment and internal or external disturbances. By now, adaptivity has been considered to be fully desirable. This position paper argues that too much adaptation conflicts with goals such as stability and user acceptance. Consequently, a kind of situation-dependent degree of adaptation is desired, which defines the amount and severity of tolerated adaptations in certain situations. As a first step into this direction, this position paper presents a quantification approach for measuring the current adaptation behavior based on generative, probabilistic models. The behavior of this method is analyzed in terms of three application scenarios: urban traffic control, the swidden farming model, and data communication protocols. Furthermore, we define a research roadmap in terms of six challenges for an overall measurement framework for SASO systems

Improving just-in-time delivery performance of IoT-enabled flexible manufacturing systems with AGV based material transportation

Autonomous guided vehicles (AGVs) are driverless material handling systems used for transportation of pallets and line side supply of materials to provide flexibility and agility in shop-floor logistics. Scheduling of shop-floor logistics in such systems is a challenging task due to their complex nature associated with the multiple part types and alternate material transfer routings. This paper presents a decision support system capable of supporting shop-floor decision-making activities during the event of manufacturing disruptions by automatically adjusting both AGV and machine schedules in Flexible Manufacturing Systems (FMSs). The proposed system uses discrete event simulation (DES) models enhanced by the Internet-of-Things (IoT) enabled digital integration and employs a nonlinear mixed integer programming Genetic Algorithm (GA) to find near-optimal production schedules prioritising the just-in-time (JIT) material delivery performance and energy efficiency of the material transportation. The performance of the proposed system is tested on the Integrated Manufacturing and Logistics (IML) demonstrator at WMG, University of Warwick. The results showed that the developed system can find the near-optimal solutions for production schedules subjected to production anomalies in a negligible time, thereby supporting shop-floor decision-making activities effectively and rapidly

Improving just-in-time delivery performance of IoT-enabled flexible manufacturing systems with AGV based material transportation

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. Autonomous guided vehicles (AGVs) are driverless material handling systems used for transportation of pallets and line side supply of materials to provide flexibility and agility in shop-floor logistics. Scheduling of shop-floor logistics in such systems is a challenging task due to their complex nature associated with the multiple part types and alternate material transfer routings. This paper presents a decision support system capable of supporting shop-floor decision-making activities during the event of manufacturing disruptions by automatically adjusting both AGV and machine schedules in Flexible Manufacturing Systems (FMSs). The proposed system uses discrete event simulation (DES) models enhanced by the Internet-of-Things (IoT) enabled digital integration and employs a nonlinear mixed integer programming Genetic Algorithm (GA) to find near-optimal production schedules prioritising the just-in-time (JIT) material delivery performance and energy efficiency of the material transportation. The performance of the proposed system is tested on the Integrated Manufacturing and Logistics (IML) demonstrator at WMG, University of Warwick. The results showed that the developed system can find the near-optimal solutions for production schedules subjected to production anomalies in a negligible time, thereby supporting shop-floor decision-making activities effectively and rapidly

Reinforcement Learning for Energy-Storage Systems in Grid-Connected Microgrids: An Investigation of Online vs. Offline Implementation

Grid-connected microgrids consisting of renewable energy sources, battery storage, and load require an appropriate energy management system that controls the battery operation. Traditionally, the operation of the battery is optimised using 24 h of forecasted data of load demand and renewable energy sources (RES) generation using offline optimisation techniques, where the battery actions (charge/discharge/idle) are determined before the start of the day. Reinforcement Learning (RL) has recently been suggested as an alternative to these traditional techniques due to its ability to learn optimal policy online using real data. Two approaches of RL have been suggested in the literature viz. offline and online. In offline RL, the agent learns the optimum policy using predicted generation and load data. Once convergence is achieved, battery commands are dispatched in real time. This method is similar to traditional methods because it relies on forecasted data. In online RL, on the other hand, the agent learns the optimum policy by interacting with the system in real time using real data. This paper investigates the effectiveness of both the approaches. White Gaussian noise with different standard deviations was added to real data to create synthetic predicted data to validate the method. In the first approach, the predicted data were used by an offline RL algorithm. In the second approach, the online RL algorithm interacted with real streaming data in real time, and the agent was trained using real data. When the energy costs of the two approaches were compared, it was found that the online RL provides better results than the offline approach if the difference between real and predicted data is greater than 1.6%

Reinforcement Learning for Energy Storage Systems in Grid-Connected Microgrids: An Investigation of Online versus Offline Implementation

This is the final version. Available on open access from MDPI via the DOI in this recordGrid-connected microgrids consisting of renewable energy sources, battery storage, and load, require an appropriate energy management system that controls the battery operation. Traditionally, the operation of the battery is optimised using 24-hours of forecasted data of load demand and renewable energy sources (RES) generation using offline optimisation techniques, where the battery actions (charge/discharge/idle) are determined before the start of the day. Reinforcement Learning (RL) has recently been suggested as an alternative to these traditional techniques due to its ability to learn optimal policy online using real data. Two approaches of RL have been suggested in the literature viz. offline and online. In offline RL the agent learns the optimum policy using predicted generation and load data. Once convergence is achieved, battery commands are dispatched in real-time. This method is similar to traditional methods because it relies on forecasted data. In online RL, on the other hand, the agent learns the optimum policy by interacting with the system in real time using real data. This paper investigates the effectiveness of both the approaches. White Gaussian noise with different standard deviations was added to real data to create synthetic predicted data to validate the method. In the first approach, the predicted data was then used by an offline RL algorithm. In the second approach, the online RL algorithm interacted with real streaming data in real time and the agent was trained using real data. When energy costs of the two approaches were compared, it was found that the online RL provides better results than the offline approach if the difference between real and predicted data is greater than 1.6%.Engineering and Physical Sciences Research Council (EPSRC

Agents and Robots for Reliable Engineered Autonomy

This book contains the contributions of the Special Issue entitled "Agents and Robots for Reliable Engineered Autonomy". The Special Issue was based on the successful first edition of the "Workshop on Agents and Robots for reliable Engineered Autonomy" (AREA 2020), co-located with the 24th European Conference on Artificial Intelligence (ECAI 2020). The aim was to bring together researchers from autonomous agents, as well as software engineering and robotics communities, as combining knowledge from these three research areas may lead to innovative approaches that solve complex problems related to the verification and validation of autonomous robotic systems

Digital Transformation in Healthcare

This book presents a collection of papers revealing the impact of advanced computation and instrumentation on healthcare. It highlights the increasing global trend driving innovation for a new era of multifunctional technologies for personalized digital healthcare. Moreover, it highlights that contemporary research on healthcare is performed on a multidisciplinary basis comprising computational engineering, biomedicine, biomedical engineering, electronic engineering, and automation engineering, among other areas

Safe Reinforcement Learning Using Formally Verified Abstract Policies

Reinforcement learning (RL) is an artificial intelligence technique for finding optimal solutions for sequential decision-making problems modelled as Markov decision processes (MDPs). Objectives are represented as numerical rewards in the model where positive values represent achievements and negative values represent failures. An autonomous agent explores the model to locate rewards with the goal to learn behaviour which will cumulate the largest reward possible. Despite RL successes in applications ranging from robotics and planning systems to sensing, it has so far had little appeal in mission- and safety-critical systems where unpredictable agent actions could lead to mission failure, risks to humans, itself or other systems, or violations of legal requirements. This is due to the difficulty of encoding non-trivial requirements of agent behaviour through rewards alone. This thesis introduces assured reinforcement learning (ARL), a safe RL approach that restricts agent actions, during and after learning. This restriction is based on formally verified policies synthesised for a high-level, abstract MDP that models the safety-relevant aspects of the RL problem. The resulting actions form overall solutions whose properties satisfy strict safety and optimality requirements. Next, ARL with knowledge revision is introduced, allowing ARL to still be used if the initial knowledge for generating action constraints proves to be incorrect. Additionally, two case studies are introduced to test the efficacy of ARL: the first is an adaptation of the benchmark flag collection navigation task and the second is an assisted-living planning system. Finally, an architecture for runtime ARL is proposed to allow ARL to be utilised in real-time systems. ARL is empirically evaluated and is shown to successfully satisfy strict safety and optimality requirements and, furthermore, with knowledge revision and action reuse, it can be successfully applied in environments where initial information may prove incomplete or incorrect