A Multi-Agent Reinforcement Learning Approach to Price and Comfort Optimization in HVAC-Systems

This paper addresses the challenge of minimizing training time for the control of Heating, Ventilation, and Air-conditioning (HVAC) systems with online Reinforcement Learning (RL). This is done by developing a novel approach to Multi-Agent Reinforcement Learning (MARL) to HVAC systems. In this paper, the environment formed by the HVAC system is formulated as a Markov Game (MG) in a general sum setting. The MARL algorithm is designed in a decentralized structure, where only relevant states are shared between agents, and actions are shared in a sequence, which are sensible from a system’s point of view. The simulation environment is a domestic house located in Denmark and designed to resemble an average house. The heat source in the house is an air-to-water heat pump, and the HVAC system is an Underfloor Heating system (UFH). The house is subjected to weather changes from a data set collected in Copenhagen in 2006, spanning the entire year except for June, July, and August, where heat is not required. It is shown that: (1) When comparing Single Agent Reinforcement Learning (SARL) and MARL, training time can be reduced by 70% for a four temperature-zone UFH system, (2) the agent can learn and generalize over seasons, (3) the cost of heating can be reduced by 19% or the equivalent to 750 kWh of electric energy per year for an average Danish domestic house compared to a traditional control method, and (4) oscillations in the room temperature can be reduced by 40% when comparing the RL control methods with a traditional control method

Vision-Based Corrosion Identification Using Data-Driven Semantic Segmentation Techniques

Corrosion is a natural process that degrades metal-made materials. Its detection is of primordial importance for quality control and for ensuring longevity of metal-made objectsin various contexts, in particular in industrial environments. Different techniques for corrosion identification including ultrasonic testing, radio-graphic testing, and magnetic flux leakage have been proposed in the past. However, these require the use of costlyand heavy equipment onsite for successful data acquisition. An under-explored alternative is to deploy conventional lightweight and inexpensive camera systems and computer vision based methods to tackle the former problem. In this work we present a detailed benchmark of four state-of-the-art supervised semantic segmentation techniques, for vision-based pixel-level corrosion identification. We focus our study on four, recently proposed deep learning architectures which have surpassed human-level accuracy on various visual tasks. The results demonstrate that the former approaches may be used for the problem of segmenting highly irregular patterns in industrial settings, such as corrosion, with high accuracy rates