Representing Long-term Impact of Residential Building Energy Management using Stochastic Dynamic Programming

Scheduling a residential building short-term to optimize the electricity bill can be difficult with the inclusion of capacity-based grid tariffs. Scheduling the building based on a proposed measured-peak (MP) grid tariff, which is a cost based on the highest peak power over a period, requires the user to consider the impact the current decision-making has in the future. Therefore, the authors propose a mathematical model using stochastic dynamic programming (SDP) that tries to represent the long-term impact of current decision-making. The SDP algorithm calculates non-linear expected future cost curves (EFCC) for the building based on the peak power backwards for each day over a month. The uncertainty in load demand and weather are considered using a discrete Markov chain setup. The model is applied to a case study for a Norwegian building with smart control of flexible loads, and compared against methods where the MP grid tariff is not accurately represented, and where the user has perfect information of the whole month. The results showed that the SDP algorithm performs 0.3 % better than a scenario with no accurate way of presenting future impacts, and performs 3.6 % worse compared to a scenario where the user had perfect information.Comment: Presented at the 16th International Conference on Probabilistic Methods Applied to Power Systems 2020 (PMAPS 2020) August 19th. Awarded the Roy Billinton Student Paper Gold Awar

Policy-based power consumption management in smart energy community using single agent and multi agent Q learning algorithms

Power consumption in residential sector has increased due to growing population, economic growth, invention of many electrical appliances and therefore is becoming a growing concern in the power industry. Managing power consumption in residential sector without sacrificing user comfort has become one of the main research areas recently. The complexity of the power system keeps growing due to the penetration of alternative sources of electric energy such as solar plant, Hydro, Biomass, Geothermal and wind farm to meet the growing demand for electricity. To overcome the challenges due to complexity, the power grid needs to be intelligent in all aspects. As the grid gets smarter and smarter, considerable efforts are being undertaken to make the houses and businesses smarter in consuming the electrical energy to minimize and level the electricity demand which is also known as Demand Side Management (DSM). It also necessitates that the conventional way of modelling, control and energy management in all sectors needs to be enhanced or replaced by intelligent information processing techniques. In our research work, it has been done in several stages. (Purpose of Study and Results) We proposed a policy-based framework which allows intelligent and flexible energy management of home appliances in a smart home which is complex and dynamic in ways that saves energy automatically. We considered the challenges in formalizing the behaviour of the appliances using their states and managing the energy consumption using policies. Policies are rules which are created and edited by a house agent to deal with situations or power problems that are likely to occur. Each time the power problem arises the house agent will refer to policy and one or a set of rules will be executed to overcome that situation. Our policy-based smart home can manage energy efficiently and can significantly participate in reducing peak energy demand (thereby may reduce carbon emission). Our proposed policy-based framework achieves peak shaving so that power consumption adapts to available power, while ensuring the comfort level of the inhabitants and taking device characteristics in to account. Our simulation results on MATLAB indicate that the proposed Policy driven homes can effectively contribute to Demand side power management by decreasing the peak hour usage of the appliances and can efficiently manage energy in a smart home in a user-friendly way. We propounded and developed peak demand management algorithms for a Smart Energy Community using different types of coordination mechanisms for coordination of multiple house agents working in the same environment. These algorithms use centralized model, decentralized model, hybrid model and Pareto resource allocation model for resource allocation. We modelled user comfort for the appliance based on user preference, the power reduction capability and the important activities that run around the house associated with that appliance. Moreover, we compared these algorithms with respect to their peak reduction capability, overall comfort of the community, simplicity of the algorithm and community involvement and finally able to find the best performing algorithm among them. Our simulation results show that the proposed coordination algorithms can effectively reduce peak demand while maintaining user comfort. With the help of our proposed algorithms, the demand for electricity of a smart community can be managed intelligently and sustainably. This work is not only aiming for peak reduction management it aims for achieving it while keeping the comfort level of the inhabitants is minimum. It can learn user’s behaviour and establish the set of optimal rules dynamically. If the available power to a house is kept at a certain level the house agent will learn to use this notional power to operate all the appliances according to the requirements and comfort level of the household. This way the consumers are forced to use the power below the set level which can result in the over-all power consumption be maintained at a certain rate or level which means sustainability is possible or depletion of natural resources for electricity can be reduced. Temporal interactions of Energy Demand by local users and renewable energy sources can also be done more efficiently by having a set of new policy rules to switch between the utility and the renewable source of energy but it is beyond the scope of this thesis. We applied Q learning techniques to a home energy management agent where the agent learns to find the optimal sequence of turning off appliances so that the appliances with higher priority will not be switched off during peak demand period or power consumption management. The policy-based home energy management determines the optimal policy at every instant dynamically by learning through the interaction with the environment using one of the reinforcement learning approaches called Q-learning. The Q-learning home power consumption problem formulation consisting of state space, actions and reward function is presented. The implications of these simulation results are that the proposed Q- learning based power consumption management is very effective and enables the users to have minimum discomfort during participation in peak demand management or at the time when power consumption management is essential when the available power is rationale. This work is extended to a group of 10 houses and three multi agent Q- learning algorithms are proposed and developed for improving the individual and community comfort while at the same time keeping the power consumption below the available power level or electricity price below the set price. The proposed algorithms are weighted strategy sharing algorithm, concurrent Q learning algorithm and cooperative distributive learning algorithm. These proposed algorithms are coded and tested for managing power consumption of a group of 10 houses and the performance of all three algorithms with respect to power management and community comfort is studied and compared. Actual power consumption of a community and modified power consumption curves using Weighted Strategy Sharing algorithm, Concurrent learning and Distributive Q Learning and user comfort results are presented, and the results are analysed in this thesis

Demand response model development for smart households using time of use tariffs and optimal control - the Isle of Wight energy autonomous community case study

Residential variable energy price schemes can be made more effective with the use of a demand response (DR) strategy along with smart appliances. Using DR, the electricity bill of participating customers/households can be minimised, while pursuing other aims such as demand-shifting and maximising consumption of locally generated renewable-electricity. In this article, a two-stage optimization method is used to implement a price-based implicit DR scheme. The model considers a range of novel smart devices/technologies/schemes, connected to smart-meters and a local DR-Controller. A case study with various decarbonisation scenarios is used to analyse the effects of deploying the proposed DR-scheme in households located in the west area of the Isle of Wight (Southern United Kingdom). There are approximately 15,000 households, of which 3000 are not connected to the gas-network. Using a distribution network model along with a load flow software-tool, the secondary voltages and apparent-power through transformers at the relevant substations are computed. The results show that in summer, participating households could export up to 6.4 MW of power, which is 10% of installed large-scale photovoltaics (PV) capacity on the island. Average carbon dioxide equivalent (CO2e) reductions of 7.1 ktons/annum and a reduction in combined energy/transport fuel-bills of 60%/annum could be achieved by participating households

Electrical Demand Modeling of a Household at an Appliance/Occupant Level

Ever increasing demand for electricity in the residential sector has made providers implement demand management strategies to curtail residential electricity consumption during peak demand events. With time, customer satisfaction of such programs dropped, resulting in high dropout rates making utilities rethink about the success of such strategies. The future success of how well these strategies could be implemented is highly dependent on how well the providers understand the effect the strategies have on the customers. This in turn, requires them to study occupant and appliance behavior of each household to have a much better understanding of the problem and this information is not readily available. In this research, a discrete event simulation model is proposed (a Non-Homogenous Markov process), that will help simulate appliance and occupant level demand patterns for residential customers in order to allow for the study of consumer-friendly demand response strategies. To develop a valid model, both occupant behavior patterns and appliance level patterns have been combined to form a data-driven simulation process. The model accounts for various factors like climate, day of week, time of day, number of occupants and nature of the appliances used. The model is validated against real hourly smart meter data and the American Time Use Survey using Simio, a simulation package. Validation was conducted in two stages namely a statistical validation stage, where levels of confidence were calculated for the proposed model during different times of day and a more industry-friendly non-parametric stage, where modern clustering techniques were used to gauge how well the proposed model helps simulate the real-world data

Data-driven load profiles and the dynamics of residential electricity consumption

The dynamics of power consumption constitutes an essential building block for planning and operating sustainable energy systems. Whereas variations in the dynamics of renewable energy generation are reasonably well studied, a deeper understanding of the variations in consumption dynamics is still missing. Here, we analyse highly resolved residential electricity consumption data of Austrian, German and UK households and propose a generally applicable data-driven load model. Specifically, we disentangle the average demand profiles from the demand fluctuations based purely on time series data. We introduce a stochastic model to quantitatively capture the highly intermittent demand fluctuations. Thereby, we offer a better understanding of demand dynamics, in particular its fluctuations, and provide general tools for disentangling mean demand and fluctuations for any given system, going beyond the standard load profile (SLP). Our insights on the demand dynamics may support planning and operating future-compliant (micro) grids in maintaining supply-demand balance

Simulation-based optimisation of the timing of loan recovery across different portfolios

A novel procedure is presented for the objective comparison and evaluation of a bank's decision rules in optimising the timing of loan recovery. This procedure is based on finding a delinquency threshold at which the financial loss of a loan portfolio (or segment therein) is minimised. Our procedure is an expert system that incorporates the time value of money, costs, and the fundamental trade-off between accumulating arrears versus forsaking future interest revenue. Moreover, the procedure can be used with different delinquency measures (other than payments in arrears), thereby allowing an indirect comparison of these measures. We demonstrate the system across a range of credit risk scenarios and portfolio compositions. The computational results show that threshold optima can exist across all reasonable values of both the payment probability (default risk) and the loss rate (loan collateral). In addition, the procedure reacts positively to portfolios afflicted by either systematic defaults (such as during an economic downturn) or episodic delinquency (i.e., cycles of curing and re-defaulting). In optimising a portfolio's recovery decision, our procedure can better inform the quantitative aspects of a bank's collection policy than relying on arbitrary discretion alone.Comment: Accepted by the journal "Expert Systems with Applications". 25 pages (including appendix), 9 figures. arXiv admin note: text overlap with older arXiv:1907.1261

State-of-the-art on research and applications of machine learning in the building life cycle

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science