545 research outputs found
Learning Agent for a Heat-Pump Thermostat With a Set-Back Strategy Using Model-Free Reinforcement Learning
The conventional control paradigm for a heat pump with a less efficient
auxiliary heating element is to keep its temperature set point constant during
the day. This constant temperature set point ensures that the heat pump
operates in its more efficient heat-pump mode and minimizes the risk of
activating the less efficient auxiliary heating element. As an alternative to a
constant set-point strategy, this paper proposes a learning agent for a
thermostat with a set-back strategy. This set-back strategy relaxes the
set-point temperature during convenient moments, e.g. when the occupants are
not at home. Finding an optimal set-back strategy requires solving a sequential
decision-making process under uncertainty, which presents two challenges. A
first challenge is that for most residential buildings a description of the
thermal characteristics of the building is unavailable and challenging to
obtain. A second challenge is that the relevant information on the state, i.e.
the building envelope, cannot be measured by the learning agent. In order to
overcome these two challenges, our paper proposes an auto-encoder coupled with
a batch reinforcement learning technique. The proposed approach is validated
for two building types with different thermal characteristics for heating in
the winter and cooling in the summer. The simulation results indicate that the
proposed learning agent can reduce the energy consumption by 4-9% during 100
winter days and by 9-11% during 80 summer days compared to the conventional
constant set-point strategyComment: Submitted to Energies - MDPI.co
Designing a generalised reward for Building Energy Management Reinforcement Learning agents
The reduction of the carbon footprint of buildings is a challenging task, partly due to the conflicting goals of maximising occupant comfort and minimising energy consumption. An intelligent management of Heating, Ventilation and Air Conditioning (HVAC) systems is creating a promising research line in which the creation of suitable algorithms could reduce energy consumption maintaining occupants' comfort. In this regard, Reinforcement Learning (RL) approaches are giving a good balance between data requirements and intelligent operations to control building systems. However, there is a gap concerning how to create a generalised reward signal that can train RL agents without delimiting the problem to a specific or controlled scenario. To tackle it, an analysis and discussion is presented about the necessary requirements for the creation of generalist rewards, with the objective of laying the foundations that allow the creation of generalist intelligent agents for building energy management.The work described in this paper was partially supported by the Basque Government under ELKARTEK project (LANTEGI4.0 KK-2020/00072)
Data-driven control of micro-climate in buildings: an event-triggered reinforcement learning approach
Smart buildings have great potential for shaping an energy-efficient,
sustainable, and more economic future for our planet as buildings account for
approximately 40% of the global energy consumption. Future of the smart
buildings lies in using sensory data for adaptive decision making and control
that is currently gloomed by the key challenge of learning a good control
policy in a short period of time in an online and continuing fashion. To tackle
this challenge, an event-triggered -- as opposed to classic time-triggered --
paradigm, is proposed in which learning and control decisions are made when
events occur and enough information is collected. Events are characterized by
certain design conditions and they occur when the conditions are met, for
instance, when a certain state threshold is reached. By systematically
adjusting the time of learning and control decisions, the proposed framework
can potentially reduce the variance in learning, and consequently, improve the
control process. We formulate the micro-climate control problem based on
semi-Markov decision processes that allow for variable-time state transitions
and decision making. Using extended policy gradient theorems and temporal
difference methods in a reinforcement learning set-up, we propose two learning
algorithms for event-triggered control of micro-climate in buildings. We show
the efficacy of our proposed approach via designing a smart learning thermostat
that simultaneously optimizes energy consumption and occupants' comfort in a
test building
Improvement of Refrigeration Efficiency by Combining Reinforcement Learning with a Coarse Model
It is paramount to improve operational conversion efficiency in air-conditioning refrigeration. It is noticed that control efficiency for model-based methods highly relies on the accuracy of the mechanism model, and data-driven methods would face challenges using the limited collected data to identify the information beyond. In this study, a hybrid novel approach is presented, which is to integrate a data-driven method with a coarse model. Specifically, reinforcement learning is used to exploit/explore the conversion efficiency of the refrigeration, and a coarse model is utilized to evaluate the reward, by which the requirement of the model accuracy is reduced and the model information is better used. The proposed approach is implemented based on a hierarchical control strategy which is divided into a process level and a loop level. The simulation of a test bed shows the proposed approach can achieve better conversion efficiency of refrigeration than the conventional methods
Scaling energy management in buildings with artificial intelligence
L'abstract è presente nell'allegato / the abstract is in the attachmen
Online implementation of a soft actor-critic agent to enhance indoor temperature control and energy efficiency in buildings
Recently, a growing interest has been observed in HVAC control systems based on Artificial Intelligence, to improve comfort conditions while avoiding unnecessary energy consumption. In this work, a model-free algorithm belonging to the Deep Reinforcement Learning (DRL) class, Soft Actor-Critic, was implemented to control the supply water temperature to radiant terminal units of a heating system serving an office building. The controller was trained online, and a preliminary sensitivity analysis on hyperparameters was performed to assess their influence on the agent performance. The DRL agent with the best performance was compared to a rule-based controller assumed as a baseline during a three-month heating season. The DRL controller outperformed the baseline after two weeks of deployment, with an overall performance improvement related to control of indoor temperature conditions. Moreover, the adaptability of the DRL agent was tested for various control scenarios, simulating changes of external weather conditions, indoor temperature setpoint, building envelope features and occupancy patterns. The agent dynamically deployed, despite a slight increase in energy consumption, led to an improvement of indoor temperature control, reducing the cumulative sum of temperature violations on average for all scenarios by 75% and 48% compared to the baseline and statically deployed agent respectively
- …