Demand Response Strategy Based on Reinforcement Learning and Fuzzy Reasoning for Home Energy Management

As energy demand continues to increase, demand response (DR) programs in the electricity distribution grid are gaining momentum and their adoption is set to grow gradually over the years ahead. Demand response schemes seek to incentivise consumers to use green energy and reduce their electricity usage during peak periods which helps support grid balancing of supply-demand and generate revenue by selling surplus of energy back to the grid. This paper proposes an effective energy management system for residential demand response using Reinforcement Learning (RL) and Fuzzy Reasoning (FR). RL is considered as a model-free control strategy which learns from the interaction with its environment by performing actions and evaluating the results. The proposed algorithm considers human preference by directly integrating user feedback into its control logic using fuzzy reasoning as reward functions. Q-learning, a RL strategy based on a reward mechanism, is used to make optimal decisions to schedule the operation of smart home appliances by shifting controllable appliances from peak periods, when electricity prices are high, to off-peak hours, when electricity prices are lower without affecting the customer’s preferences. The proposed approach works with a single agent to control 14 household appliances and uses a reduced number of state-action pairs and fuzzy logic for rewards functions to evaluate an action taken for a certain state. The simulation results show that the proposed appliances scheduling approach can smooth the power consumption profile and minimise the electricity cost while considering user’s preferences, user’s feedbacks on each action taken and his/her preference settings. A user-interface is developed in MATLAB/Simulink for the Home Energy Management System (HEMS) to demonstrate the proposed DR scheme. The simulation tool includes features such as smart appliances, electricity pricing signals, smart meters, solar photovoltaic generation, battery energy storage, electric vehicle and grid supply.Peer reviewe

Integration via Meaning: Using the Semantic Web to deliver Web Services

Presented at the CRIS2002 Conference in Kassel.-- 9 pages.-- Contains: Conference paper (PDF) + PPT presentation.The major developments of the World Wide Web (WWW) in the last two years have been Web Services and the Semantic Web. The former allows the construction of distributed systems across the WWW by providing a lightweight middleware architecture. The latter provides an infrastructure for accessing resources on the WWW via their relationships with respect to conceptual descriptions. In this paper, I shall review the progress undertaken in each of these two areas. Further, I shall argue that in order for the aims of both the Semantic Web and the Web Services activities to be successful, then the Web Service architecture needs to be augmented by concepts and tools of the Semantic Web. This infrastructure will allow resource discovery, brokering and access to be enabled in a standardised, integrated and interoperable manner. Finally, I survey the CLRC Information Technology R&D programme to show how it is contributing to the development of this future infrastructure

Mechanism design for spatio-temporal request satisfaction in mobile networks

Mobile agents participating in geo-presence-capable crowdsourcing applications should be presumed rational, competitive, and willing to deviate from their routes if given the right incentive. In this paper, we design a mechanism that takes into consideration this rationality for request satisfaction in such applications. We propose the Geo-temporal Request Satisfaction (GRS) problem to be that of finding the optimal assignment of requests with specific spatio-temporal characteristics to competitive mobile agents subject to spatio-temporal constraints. The objective of the GRS problem is to maximize the total profit of the system subject to our rationality assumptions. We define the problem formally, prove that it is NP-Complete, and present a practical solution mechanism, which we prove to be convergent, and which we evaluate experimentally.National Science Foundation (1012798, 0952145, 0820138, 0720604, 0735974

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion