Enhancing the Performance of Multi-Agent Reinforcement Learning for Controlling HVAC Systems

Systems for heating, ventilation and air-conditioning (HVAC) of buildings are traditionally controlled by a rule-based approach. In order to reduce the energy consumption and the environmental impact of HVAC systems more advanced control methods such as reinforcement learning are promising. Reinforcement learning (RL) strategies offer a good alternative, as user feedback can be integrated more easily and presence can also be incorporated. Moreover, multi-agent RL approaches scale well and can be generalized. In this paper, we propose a multi-agent RL framework based on existing work that learns reducing on one hand energy consumption by optimizing HVAC control and on the other hand user feedback by occupants about uncomfortable room temperatures. Second, we show how to reduce training time required for proper RL-agent-training by using parameter sharing between the multiple agents and apply different pretraining techniques. Results show that our framework is capable of reducing the energy by around 6% when controlling a complete building or 8% for a single room zone. The occupants complaints are acceptable or even better compared to a rule-based baseline. Additionally, our performance analysis show that the training time can be drastically reduced by using parameter sharing

Enhancing Grid Operation with Electric Vehicle Integration in Automatic Generation Control

Wind energy has been recognized as a clean energy source with significant potential for reducing carbon emissions. However, its inherent variability poses substantial challenges for power system operators due to its unpredictable nature. As a result, there is an increased dependence on conventional generation sources to uphold the power system balance, resulting in elevated operational costs and an upsurge in carbon emissions. Hence, an urgent need exists for alternative solutions that can reduce the burden on traditional generating units and optimize the utilization of reserves from non-fossil fuel technologies. Meanwhile, vehicle-to-grid (V2G) technology integration has emerged as a remedial approach to rectify power capacity shortages during grid operations, enhancing stability and reliability. This research focuses on harnessing electric vehicle (EV) storage capacity to compensate for power deficiencies caused by forecasting errors in large-scale wind energy-based power systems. A real-time dynamic power dispatch strategy is developed for the automatic generation control (AGC) system to integrate EVs and utilize their reserves optimally to reduce reliance on conventional power plants and increase system security. The results obtained from this study emphasize the significant prospects associated with the fusion of EVs and traditional power plants, offering a highly effective solution for mitigating real-time power imbalances in large-scale wind energy-based power systems

Smart Microgrids: Optimizing Local Resources toward Increased Efficiency and a More Sustainable Growth

Smart microgrids are a possibility to reduce complexity by performing local optimization of power production, consumption and storage. We do not envision smart microgrids to be island solutions but rather to be integrated into a larger network of microgrids that form the future energy grid. Operating and controlling a smart microgrid involves optimization for using locally generated energy and to provide feedback to the user when and how to use devices. This chapter shows how these issues can be addressed starting with measuring and modeling energy consumption patterns by collecting an energy consumption dataset at device level. The open dataset allows to extract typical usage patterns and subsequently to model test scenarios for energy management algorithms. Section 3 discusses means for analyzing measured data and for providing detailed feedback about energy consumption to increase customers’ energy awareness. Section 4 shows how renewable energy sources can be integrated in a smart microgrid and how energy production can be accurately predicted. Section 5 introduces a self-organizing local energy system that autonomously coordinates production and consumption via an agent-based energy auction system. The final section discusses how the proposed methods contribute to sustainable growth and gives an outlook to future research

Indirect monitoring of energy efficiency in a simulated chemical process

Abstract. Energy efficiency is an important part of chemical process sustainability. Wasted energy contributes significantly to process costs and overall emissions. Therefore, contributions to improving energy efficiency in chemical processes are of value. The main objective of this thesis is the exploration of indirect energy efficiency monitoring methods and their compilation into a generalized framework. As part of the proposed framework, data-based modelling methods were explored and used to identify a model for estimating energy efficiency in a simulated process. The proposed framework can act as a potential tool in different practical applications with energy efficiency improvements as an objective. As a simulated test process for this thesis, the Tennessee Eastman process was utilized. This process is widely used in research, especially regarding fault diagnosis and control design. The process includes slow dynamics and nonlinearity, providing interesting challenges for research. Even though the process has been studied extensively, the energy efficiency aspect of the process has not been taken into account in research. The results of the thesis show that data-based models provide sufficient accuracy for real-time estimation of energy efficiency for the Tennessee Eastman process. Parts of the proposed framework were tested with the explored methods, but some areas were beyond the scope of this thesis. As such, further research, for example prediction of the energy efficiency horizon, fault diagnosis and advanced process control, could be beneficial.Energiatehokkuuden epäsuora monitorointi simuloidussa kemiallisessa prosessissa. Tiivistelmä. Energiatehokkuus on tärkeä osa kemiallisen teollisuuden kestävyyttä. Energian käytön tehottomuus näkyy merkittävästi kasvavina prosessikustannuksina ja kokonaispäästöinä. Toimet energiatehokkuuden nostamiseksi ovat siksi merkityksellisiä. Diplomityön päätavoitteena on erilaisten epäsuorien energiatehokkuuden seurantamenetelmien tutkiminen ja niiden kokoaminen yleistettävään menetelmäkehykseen. Datapohjaisia mallinnusmenetelmiä tutkitaan osana esitettyä kehystä, ja niitä hyödynnetään energiatehokkuutta arvioivan mallin muodostuksessa. Esitetty menetelmäkehys voi toimia mahdollisena työkaluna erilaisissa käyttökohteissa, joissa energiatehokkuuden parantaminen on päämääränä. Tutkittavana kohteena diplomityössä käytettiin simuloitua Tennessee Eastman prosessimallia. Vaikka prosessia on tutkittu laajasti, energiatehokkuuden tarkempi tarkastelu on jäänyt vajaaksi. Simuloitua prosessidataa hyödynnettiin tässä työssä prosessin energiatehokkuuden mallipohjaisen arvion muodostuksessa. Työssä analysoitiin myös mallinnuksen luotettavuuteen vaikuttavia tekijöitä, kuten opetusdatan rajallisuutta ja siitä seuraavaa mallin ekstrapolointia. Diplomityön tulokset osoittavat, että Tennessee Eastman prosessin energiatehokkuuden reaaliaikainen arviointi datapohjaisilla menetelmillä onnistuu riittävällä tarkkuudella. Esitetyn menetelmäkehyksen osia testattiin tutkituilla menetelmillä, mutta jotkin alueet jäivät työn ulkopuolelle. Tulevaisuuden mahdollisiin tutkimusalueisiin kuuluukin energiatehokkuuden ennustaminen, vikadiagnostiikka ja niitä yhdistävä kehittynyt prosessisäätö