Autonomic management of multiple non-functional concerns in behavioural skeletons

We introduce and address the problem of concurrent autonomic management of different non-functional concerns in parallel applications build as a hierarchical composition of behavioural skeletons. We first define the problems arising when multiple concerns are dealt with by independent managers, then we propose a methodology supporting coordinated management, and finally we discuss how autonomic management of multiple concerns may be implemented in a typical use case. The paper concludes with an outline of the challenges involved in realizing the proposed methodology on distributed target architectures such as clusters and grids. Being based on the behavioural skeleton concept proposed in the CoreGRID GCM, it is anticipated that the methodology will be readily integrated into the current reference implementation of GCM based on Java ProActive and running on top of major grid middleware systems.Comment: 20 pages + cover pag

Integrated Dynamic Facade Control with an Agent-based Architecture for Commercial Buildings

Dynamic façades have significant technical potential to minimize heating, cooling, and lighting energy use and peak electric demand in the perimeter zone of commercial buildings, but the performance of these systems is reliant on being able to balance complex trade-offs between solar control, daylight admission, comfort, and view over the life of the installation. As the context for controllable energy-efficiency technologies grows more complex with the increased use of intermittent renewable energy resources on the grid, it has become increasingly important to look ahead towards more advanced approaches to integrated systems control in order to achieve optimum life-cycle performance at a lower cost. This study examines the feasibility of a model predictive control system for low-cost autonomous dynamic façades. A system architecture designed around lightweight, simple agents is proposed. The architecture accommodates whole building and grid level demands through its modular, hierarchical approach. Automatically-generated models for computing window heat gains, daylight illuminance, and discomfort glare are described. The open source Modelica and JModelica software tools were used to determine the optimum state of control given inputs of window heat gains and lighting loads for a 24-hour optimization horizon. Penalty functions for glare and view/ daylight quality were implemented as constraints. The control system was tested on a low-power controller (1.4 GHz single core with 2 GB of RAM) to evaluate feasibility. The target platform is a low-cost ($35/unit) embedded controller with 1.2 GHz dual-core cpu and 1 GB of RAM. Configuration and commissioning of the curtainwall unit was designed to be largely plug and play with minimal inputs required by the manufacturer through a web-based user interface. An example application was used to demonstrate optimal control of a three-zone electrochromic window for a south-facing zone. The overall approach was deemed to be promising. Further engineering is required to enable scalable, turnkey solutions

Closing the loop between neural network simulators and the OpenAI Gym

Since the enormous breakthroughs in machine learning over the last decade, functional neural network models are of growing interest for many researchers in the field of computational neuroscience. One major branch of research is concerned with biologically plausible implementations of reinforcement learning, with a variety of different models developed over the recent years. However, most studies in this area are conducted with custom simulation scripts and manually implemented tasks. This makes it hard for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. This toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments of varying complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym

Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page