Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

Modelling and analysis of demand response implementation in the residential sector

Demand Response (DR) eliminates the need for expensive capital expenditure on the electricity distribution, transmission and the generation systems by encouraging consumers to alter their power usage through electricity pricing or incentive programs. However, modelling of DR programs for residential consumers is complicated due to the uncertain consumption behavious of consumers and the complexity of schedulling a large number of household appliances. This thesis has investigated the design and the implementation challenges of the two most commonly used DR components in the residential sector, i.e., time of use (TOU) and direct load control (DLC) programs for improving their effectiveness and implementation with innovative strategies to facilitate their acceptance by both consumers and utilities. In price-based DR programs, the TOU pricing scheme is one of the most attractive and simplest approaches for reducing peak electricity demand in the residential sector. This scheme has been adopted in many developed countries because it requires less communication infrastructure for its implementation. However, the implementation of TOU pricing in low and lower-middle income economies is less appealing, mainly due to a large number of low-income consumers, as traditional TOU pricing schemes may increase the cost of electricity for low income residential consumers and adversely affect their comfort levels. The research in this thesis proposes an alternative TOU pricing strategy for the residential sector in developing countries in order to manage peak demand problems while ensuring a low impact on consumers’ monthly energy bills and comfort levels. In this study, Bangladesh is used as an example of a lower-to-middle income developing country. The DLC program is becoming an increasingly attractive solution for utilities in developed countries due to advances in the construction of communication infrastructures as part of the smart grid concept deployment. One of the main challenges of the DLC program implementation is ensuring optimal control over a large number of different household appliances for managing both short and long intervals of voltage variation problems in distribution networks at both medium voltage (MV) and low voltage (LV) networks, while simultaneously enabling consumers to maintain their comfort levels. Another important challenge for DLC implementation is achieving a fair distribution of incentives among a large number of participating consumers. This thesis addresses these challenges by proposing a multi-layer load control algorithm which groups the household appliances based on the intervals of the voltage problems and coordinates with the reactive power from distributed generators (DGs) for the effective voltage management in MV networks. The proposed load controller takes into consideration the consumption preference of individual appliance, ensuring that the consumer’s comfort level is satisfied as well as fairly incentivising consumers based on their contributions in network voltage and power loss improvement. Another significant challenge with the existing DLC strategy as it applies to managing voltage in LV networks is that it does not take into account the network’s unbalance constraints in the load control algorithm. In LV distribution networks, voltage unbalance is prevalent and is one of the main power quality problems of concern. Unequal DR activation among the phases may cause excessive voltage unbalance in the network. In this thesis, a new load control algorithm is developed with the coordination of secondary on-load tap changer (OLTC) transformer for effective management of both voltage magnitude and unbalance in the LV networks. The proposed load control algorithm minimises the disturbance to consumers’ comfort levels by prioritising their consumption preferences. It motivates consumers to participate in DR program by providing flexibility to bid their participation prices dynamically in each DR event. The proposed DR programs are applicable for both developed and developing countries based on their available communication infrastructure for DR implementation. The main benefits of the proposed DR programs can be shared between consumers and their utilities. Consumers have flexibility in being able to prioritise their comfort levels and bid for their participation prices or receive fair incentives, while utilities effectively manage their network peak demand and power quality problems with minimum compensation costs. As a whole, consumers get the opportunity to minimise their electricity bills while utilities are able to defer or avoid the high cost of their investment in network reinforcements

Managing Flexible Loads in Residential Areas

Load flexibility in households is a promising option for efficient and reliable operation of future power systems. Due to the distributed nature of residential demand, coordination mechanisms have to cope with a large number of flexible units. This thesis provides a model for demand response analysis and proposes different mechanisms for coordinating flexible loads. In particular, the potential to match intermittent output of renewable generators with electricity demand is investigated

Scenarios for the development of smart grids in the UK: synthesis report

‘Smart grid’ is a catch-all term for the smart options that could transform the ways society produces, delivers and consumes energy, and potentially the way we conceive of these services. Delivering energy more intelligently will be fundamental to decarbonising the UK electricity system at least possible cost, while maintaining security and reliability of supply. Smarter energy delivery is expected to allow the integration of more low carbon technologies and to be much more cost effective than traditional methods, as well as contributing to economic growth by opening up new business and innovation opportunities. Innovating new options for energy system management could lead to cost savings of up to £10bn, even if low carbon technologies do not emerge. This saving will be much higher if UK renewable energy targets are achieved. Building on extensive expert feedback and input, this report describes four smart grid scenarios which consider how the UK’s electricity system might develop to 2050. The scenarios outline how political decisions, as well as those made in regulation, finance, technology, consumer and social behaviour, market design or response, might affect the decisions of other actors and limit or allow the availability of future options. The project aims to explore the degree of uncertainty around the current direction of the electricity system and the complex interactions of a whole host of factors that may lead to any one of a wide range of outcomes. Our addition to this discussion will help decision makers to understand the implications of possible actions and better plan for the future, whilst recognising that it may take any one of a number of forms

Creative User-Centered Visualization Design for Energy Analysts and Modelers

We enhance a user-centered design process with techniques that deliberately promote creativity to identify opportunities for the visualization of data generated by a major energy supplier. Visualization prototypes developed in this way prove effective in a situation whereby data sets are largely unknown and requirements open – enabling successful exploration of possibilities for visualization in Smart Home data analysis. The process gives rise to novel designs and design metaphors including data sculpting. It suggests: that the deliberate use of creativity techniques with data stakeholders is likely to contribute to successful, novel and effective solutions; that being explicit about creativity may contribute to designers developing creative solutions; that using creativity techniques early in the design process may result in a creative approach persisting throughout the process. The work constitutes the first systematic visualization design for a data rich source that will be increasingly important to energy suppliers and consumers as Smart Meter technology is widely deployed. It is novel in explicitly employing creativity techniques at the requirements stage of visualization design and development, paving the way for further use and study of creativity methods in visualization design

Changing demand: flexibility of energy practices in households with children (final report)

This report summarises the findings and recommendations from the ‘Changing Demand: Flexibility of energy practices in households with children’ research project funded by the Consumer Advocacy Panel. The project aimed to understand how households with children may be affected by electricity market reforms and demand management initiatives, such as cost-reflective pricing. The study involved 44 in-depth interviews, home tours and observations and a national survey with over 500 Australian households with children. Overview In households with children, many of the practices which use energy are coordinated and concentrated in the late afternoon and early evening on weekdays. Parents’ reliance on routine to manage the demands of family life limits the flexibility of energy use. With limited ability to shift practices to other times of the day, and priorities such as ‘doing what’s best for children’ and ‘using time efficiently’ taking precedence, households with children risk financial disadvantage under pricing strategies such as Time-of-Use (TOU) pricing. Financial insecurity is widespread in, but not limited to, low-income and sole parent households. Health concerns, thermally inefficient housing and appliances, housing tenure, safety and noise concerns, and widespread tariff confusion also restrict the capacity of households with children to manage energy use and costs. Many parents had little time, interest or trust to investigate tariff choice and available energy information. As such, increasing choice and complexity in electricity market offerings does not meet the needs of these households and TOU pricing is unlikely to achieve its aims with this household group. Family routines were more amenable to disruption on an occasional basis for non-financial reasons. For example, 85 per cent of survey respondents said they would reduce electricity use for a ‘peak alert’ in hot weather.  Acting for the ‘common good’ appealed to most parents, for example to prevent an electricity outage and/or be part of a community effort. Household activities considered inflexible for a hypothetical TOU tariff, such as home cooling, television and computer activities and cooking, were considered.  Recommendations from this study include reassessing the energy policy focus on price signals, tariff choice and information to address issues of household demand in Australia. Several alternatives are proposed such as peak alerts, and affordable access to public cooling during hot peak days.&nbsp

Statistical Predictions of Electric Load Profiles in the UK Domestic Buildings.

yesThis paper presents a method of generating realistic electricity load profile data for the UK domestic buildings. The domestic space heating and domestic hot water have been excluded in this study. The information and results of previous investigations and works that is available in public reports and statistics have been used as input data when modeling of domestic energy consumption. A questionnaire survey was conducted to find out what occupants do in different times of the day in order to get probabilistic estimates of usage of electrical household. The daily energy demand load profile of each appliance can be predicted using this method. A measured data set is also applied for comparison, and verification. Our analysis shows that the generated load profiles have a good agreement with real data. The daily load profile from individual dwelling to community can be predicted using this method

Identifying residential consumption patterns using data-mining techniques: A large-scale study of smart meter data in Chengdu, China

The fine-grained electricity consumption data created by advanced metering technologies offers an opportunity to understand residential demand from new angles. Although there exists a large body of research on demand response in short- and long-term forecasting, a comprehensive analysis to identify household consumption behaviour in different scenarios has not been conducted. The study’s novelty lies in its use of unsupervised machine learning tools to explore residential customers’ demand patterns and response without the assistance of traditional survey tools. We investigate behavioural response in three different contexts: 1) seasonal (using weekly consumption profiles); 2) holidays/festivals; and 3) extreme weather situations. The analysis is based on the smart metering data of 2,000 households in Chengdu, China over three years from 2014 to 2016. Workday/weekend profiles indicate that there are two distinct groups of households that appear to be white-collar or relatively affluent families. Demand patterns at the major festivals in China, especially the Spring Festival, reveal various types of lifestyle and households. In terms of extreme weather response, the most striking finding was that in summer, at night-time, over 72% of households doubled (or more) their electricity usage, while consumption changes in winter do not seem to be significant. Our research offers more detailed insight into Chinese residential consumption and provides a practical framework to understand households’ behaviour patterns in different settings