231 research outputs found
A systematic literature review on the use of artificial intelligence in energy self-management in smart buildings
Buildings are one of the main consumers of energy in cities, which is why a lot of research has been generated around this problem. Especially, the buildings energy management systems must improve in the next years. Artificial intelligence techniques are playing and will play a fundamental role in these improvements. This work presents a systematic review of the literature on researches that have been done in recent years to improve energy management systems for smart building using artificial intelligence techniques. An originality of the work is that they are grouped according to the concept of "Autonomous Cycles of Data Analysis Tasks", which defines that an autonomous management system requires specialized tasks, such as monitoring, analysis, and decision-making tasks for reaching objectives in the environment, like improve the energy efficiency. This organization of the work allows us to establish not only the positioning of the researches, but also, the visualization of the current challenges and opportunities in each domain. We have identified that many types of researches are in the domain of decision-making (a large majority on optimization and control tasks), and defined potential projects related to the development of autonomous cycles of data analysis tasks, feature engineering, or multi-agent systems, among others.European Commissio
Optimal Economic Gas Turbine Dispatch with Deep Reinforcement Learning
Dispatching strategies for gas turbines (GTs) are changing in modern
electricity grids. A growing incorporation of intermittent renewable energy
requires GTs to operate more but shorter cycles and more frequently on partial
loads. Deep reinforcement learning (DRL) has recently emerged as a tool that
can cope with this development and dispatch GTs economically. The key
advantages of DRL are a model-free optimization and the ability to handle
uncertainties, such as those introduced by varying loads or renewable energy
production. In this study, three popular DRL algorithms are implemented for an
economic GT dispatch problem on a case study in Alberta, Canada. We highlight
the benefits of DRL by incorporating an existing thermodynamic software
provided by Siemens Energy into the environment model and by simulating
uncertainty via varying electricity prices, loads, and ambient conditions.
Among the tested algorithms and baseline methods, Deep Q-Networks (DQN)
obtained the highest rewards while Proximal Policy Optimization (PPO) was the
most sample efficient. We further propose and implement a method to assign GT
operation and maintenance cost dynamically based on operating hours and cycles.
Compared to existing methods, our approach better approximates the true cost of
modern GT dispatch and hence leads to more realistic policies.Comment: This work has been accepted to IFAC for publication under a Creative
Commons Licence CC-BY-NC-N
Data-Driven Transferred Energy Management Strategy for Hybrid Electric Vehicles via Deep Reinforcement Learning
Real-time applications of energy management strategies (EMSs) in hybrid
electric vehicles (HEVs) are the harshest requirements for researchers and
engineers. Inspired by the excellent problem-solving capabilities of deep
reinforcement learning (DRL), this paper proposes a real-time EMS via
incorporating the DRL method and transfer learning (TL). The related EMSs are
derived from and evaluated on the real-world collected driving cycle dataset
from Transportation Secure Data Center (TSDC). The concrete DRL algorithm is
proximal policy optimization (PPO) belonging to the policy gradient (PG)
techniques. For specification, many source driving cycles are utilized for
training the parameters of deep network based on PPO. The learned parameters
are transformed into the target driving cycles under the TL framework. The EMSs
related to the target driving cycles are estimated and compared in different
training conditions. Simulation results indicate that the presented transfer
DRL-based EMS could effectively reduce time consumption and guarantee control
performance.Comment: 25 pages, 12 figure
Eco-driving for Electric Connected Vehicles at Signalized Intersections: A Parameterized Reinforcement Learning approach
This paper proposes an eco-driving framework for electric connected vehicles
(CVs) based on reinforcement learning (RL) to improve vehicle energy efficiency
at signalized intersections. The vehicle agent is specified by integrating the
model-based car-following policy, lane-changing policy, and the RL policy, to
ensure safe operation of a CV. Subsequently, a Markov Decision Process (MDP) is
formulated, which enables the vehicle to perform longitudinal control and
lateral decisions, jointly optimizing the car-following and lane-changing
behaviors of the CVs in the vicinity of intersections. Then, the hybrid action
space is parameterized as a hierarchical structure and thereby trains the
agents with two-dimensional motion patterns in a dynamic traffic environment.
Finally, our proposed methods are evaluated in SUMO software from both a
single-vehicle-based perspective and a flow-based perspective. The results show
that our strategy can significantly reduce energy consumption by learning
proper action schemes without any interruption of other human-driven vehicles
(HDVs)
An Overview of Carbon Footprint Mitigation Strategies. Machine Learning for Societal Improvement, Modernization, and Progress
Among the most pressing issues in the world today is the impact of globalization and energy consumption on the environment. Despite the growing regulatory framework to prevent ecological degradation, sustainability continues to be a problem. Machine learning can help with the transition toward a net-zero carbon society. Substantial work has been done in this direction. Changing electrical systems, transportation, buildings, industry, and land use are all necessary to reduce greenhouse gas emissions. Considering the carbon footprint aspect of sustainability, this chapter provides a detailed overview of how machine learning can be applied to forge a path to ecological sustainability in each of these areas. The chapter highlights how various machine learning algorithms are used to increase the use of renewable energy, efficient transportation, and waste management systems to reduce the carbon footprint. The authors summarize the findings from the current research literature and conclude by providing a few future directions
Software Architecture for Autonomous and Coordinated Navigation of UAV Swarms in Forest and Urban Firefighting
Advances in the field of unmanned aerial vehicles (UAVs) have led to an exponential increase in their market, thanks to the development of innovative technological solutions aimed at a wide range of applications and services, such as emergencies and those related to fires. In addition, the expansion of this market has been accompanied by the birth and growth of the so-called UAV swarms. Currently, the expansion of these systems is due to their properties in terms of robustness, versatility, and efficiency. Along with these properties there is an aspect, which is still a field of study, such as autonomous and cooperative navigation of these swarms. In this paper we present an architecture that includes a set of complementary methods that allow the establishment of different control layers to enable the autonomous and cooperative navigation of a swarm of UAVs. Among the different layers, there are a global trajectory planner based on sampling, algorithms for obstacle detection and avoidance, and methods for autonomous decision making based on deep reinforcement learning. The paper shows satisfactory results for a line-of-sight based algorithm for global path planner trajectory smoothing in 2D and 3D. In addition, a novel method for autonomous navigation of UAVs based on deep reinforcement learning is shown, which has been tested in 2 different simulation environments with promising results about the use of these techniques to achieve autonomous navigation of UAVs.This work was supported by the Comunidad de Madrid Government through the Industrial Doctorates Grants (GRANT IND2017/TIC-7834)
- …