Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

A Distributed Demand-Side Management Framework for the Smart Grid

This paper proposes a fully distributed Demand-Side Management system for Smart Grid infrastructures, especially tailored to reduce the peak demand of residential users. In particular, we use a dynamic pricing strategy, where energy tariffs are function of the overall power demand of customers. We consider two practical cases: (1) a fully distributed approach, where each appliance decides autonomously its own scheduling, and (2) a hybrid approach, where each user must schedule all his appliances. We analyze numerically these two approaches, showing that they are characterized practically by the same performance level in all the considered grid scenarios. We model the proposed system using a non-cooperative game theoretical approach, and demonstrate that our game is a generalized ordinal potential one under general conditions. Furthermore, we propose a simple yet effective best response strategy that is proved to converge in a few steps to a pure Nash Equilibrium, thus demonstrating the robustness of the power scheduling plan obtained without any central coordination of the operator or the customers. Numerical results, obtained using real load profiles and appliance models, show that the system-wide peak absorption achieved in a completely distributed fashion can be reduced up to 55%, thus decreasing the capital expenditure (CAPEX) necessary to meet the growing energy demand

Decentralized Greedy-Based Algorithm for Smart Energy Management in Plug-in Electric Vehicle Energy Distribution Systems

Variations in electricity tariffs arising due to stochastic demand loads on the power grids have stimulated research in finding optimal charging/discharging scheduling solutions for electric vehicles (EVs). Most of the current EV scheduling solutions are either centralized, which suffer from low reliability and high complexity, while existing decentralized solutions do not facilitate the efficient scheduling of on-move EVs in large-scale networks considering a smart energy distribution system. Motivated by smart cities applications, we consider in this paper the optimal scheduling of EVs in a geographically large-scale smart energy distribution system where EVs have the flexibility of charging/discharging at spatially-deployed smart charging stations (CSs) operated by individual aggregators. In such a scenario, we define the social welfare maximization problem as the total profit of both supply and demand sides in the form of a mixed integer non-linear programming (MINLP) model. Due to the intractability, we then propose an online decentralized algorithm with low complexity which utilizes effective heuristics to forward each EV to the most profitable CS in a smart manner. Results of simulations on the IEEE 37 bus distribution network verify that the proposed algorithm improves the social welfare by about 30% on average with respect to an alternative scheduling strategy under the equal participation of EVs in charging and discharging operations. Considering the best-case performance where only EV profit maximization is concerned, our solution also achieves upto 20% improvement in flatting the final electricity load. Furthermore, the results reveal the existence of an optimal number of CSs and an optimal vehicle-to-grid penetration threshold for which the overall profit can be maximized. Our findings serve as guidelines for V2G system designers in smart city scenarios to plan a cost-effective strategy for large-scale EVs distributed energy management

Optimised Residential Loads Scheduling Based on Dynamic Pricing of Electricity : A Simulation Study

This paper presents a simulation study which addresses Demand Side Management (DSM) via scheduling and optimization of a set of residential smart appliances under day-ahead variable pricing with the aim of minimizing the customer’s energy bill. The appliances’ operation and the overall model are subject to the manufacturer and user specific constraints formulated as a constrained linear programming problem. The overall model is simulated using MATLAB and SIMULINK / SimPowerSystems basic blocks. The results comparing Real Time Pricing (RTP) and the Fixed Time Tariff (FTT) demonstrate that optimal scheduling of the residential smart appliances can potentially result in energy cost savings. The extension of the model to incorporate renewable energy resources and storage system is also discussedNon peer reviewedFinal Accepted Versio

A Review on Energy Consumption Optimization Techniques in IoT Based Smart Building Environments

In recent years, due to the unnecessary wastage of electrical energy in residential buildings, the requirement of energy optimization and user comfort has gained vital importance. In the literature, various techniques have been proposed addressing the energy optimization problem. The goal of each technique was to maintain a balance between user comfort and energy requirements such that the user can achieve the desired comfort level with the minimum amount of energy consumption. Researchers have addressed the issue with the help of different optimization algorithms and variations in the parameters to reduce energy consumption. To the best of our knowledge, this problem is not solved yet due to its challenging nature. The gap in the literature is due to the advancements in the technology and drawbacks of the optimization algorithms and the introduction of different new optimization algorithms. Further, many newly proposed optimization algorithms which have produced better accuracy on the benchmark instances but have not been applied yet for the optimization of energy consumption in smart homes. In this paper, we have carried out a detailed literature review of the techniques used for the optimization of energy consumption and scheduling in smart homes. The detailed discussion has been carried out on different factors contributing towards thermal comfort, visual comfort, and air quality comfort. We have also reviewed the fog and edge computing techniques used in smart homes

Optimized Household Demand Management with Local Solar PV Generation

Demand Side Management (DSM) strategies are of-ten associated with the objectives of smoothing the load curve and reducing peak load. Although the future of demand side manage-ment is technically dependent on remote and automatic control of residential loads, the end-users play a significant role by shifting the use of appliances to the off-peak hours when they are exposed to Day-ahead market price. This paper proposes an optimum so-lution to the problem of scheduling of household demand side management in the presence of PV generation under a set of tech-nical constraints such as dynamic electricity pricing and voltage deviation. The proposed solution is implemented based on the Clonal Selection Algorithm (CSA). This solution is evaluated through a set of scenarios and simulation results show that the proposed approach results in the reduction of electricity bills and the import of energy from the grid