Estimating and Incentivizing Imperfect-Knowledge Agents with Hidden Rewards

In practice, incentive providers (i.e., principals) often cannot observe the reward realizations of incentivized agents, which is in contrast to many principal-agent models that have been previously studied. This information asymmetry challenges the principal to consistently estimate the agent's unknown rewards by solely watching the agent's decisions, which becomes even more challenging when the agent has to learn its own rewards. This complex setting is observed in various real-life scenarios ranging from renewable energy storage contracts to personalized healthcare incentives. Hence, it offers not only interesting theoretical questions but also wide practical relevance. This paper explores a repeated adverse selection game between a self-interested learning agent and a learning principal. The agent tackles a multi-armed bandit (MAB) problem to maximize their expected reward plus incentive. On top of the agent's learning, the principal trains a parallel algorithm and faces a trade-off between consistently estimating the agent's unknown rewards and maximizing their own utility by offering adaptive incentives to lead the agent. For a non-parametric model, we introduce an estimator whose only input is the history of principal's incentives and agent's choices. We unite this estimator with a proposed data-driven incentive policy within a MAB framework. Without restricting the type of the agent's algorithm, we prove finite-sample consistency of the estimator and a rigorous regret bound for the principal by considering the sequential externality imposed by the agent. Lastly, our theoretical results are reinforced by simulations justifying applicability of our framework to green energy aggregator contracts.Comment: 72 pages, 6 figures. arXiv admin note: text overlap with arXiv:2304.0740

Artificial Intelligence and Machine Learning Approaches to Energy Demand-Side Response: A Systematic Review

Recent years have seen an increasing interest in Demand Response (DR) as a means to provide flexibility, and hence improve the reliability of energy systems in a cost-effective way. Yet, the high complexity of the tasks associated with DR, combined with their use of large-scale data and the frequent need for near real-time de-cisions, means that Artificial Intelligence (AI) and Machine Learning (ML) — a branch of AI — have recently emerged as key technologies for enabling demand-side response. AI methods can be used to tackle various challenges, ranging from selecting the optimal set of consumers to respond, learning their attributes and pref-erences, dynamic pricing, scheduling and control of devices, learning how to incentivise participants in the DR schemes and how to reward them in a fair and economically efficient way. This work provides an overview of AI methods utilised for DR applications, based on a systematic review of over 160 papers, 40 companies and commercial initiatives, and 21 large-scale projects. The papers are classified with regards to both the AI/ML algorithm(s) used and the application area in energy DR. Next, commercial initiatives are presented (including both start-ups and established companies) and large-scale innovation projects, where AI methods have been used for energy DR. The paper concludes with a discussion of advantages and potential limitations of reviewed AI techniques for different DR tasks, and outlines directions for future research in this fast-growing area

Artificial intelligence for decision making in energy demand-side response

This thesis examines the role and application of data-driven Artificial Intelligence (AI) approaches for the energy demand-side response (DR). It follows the point of view of a service provider company/aggregator looking to support its decision-making and operation. Overall, the study identifies data-driven AI methods as an essential tool and a key enabler for DR. The thesis is organised into two parts. It first provides an overview of AI methods utilised for DR applications based on a systematic review of over 160 papers, 40 commercial initiatives, and 21 large-scale projects. The reviewed work is categorised based on the type of AI algorithm(s) employed and the DR application area of the AI methods. The end of the first part of the thesis discusses the advantages and potential limitations of the reviewed AI techniques for different DR tasks and how they compare to traditional approaches. The second part of the thesis centres around designing machine learning algorithms for DR. The undertaken empirical work highlights the importance of data quality for providing fair, robust, and safe AI systems in DR — a high-stakes domain. It furthers the state of the art by providing a structured approach for data preparation and data augmentation in DR to minimise propagating effects in the modelling process. The empirical findings on residential response behaviour show better response behaviour in households with internet access, air-conditioning systems, power-intensive appliances, and lower gas usage. However, some insights raise questions about whether the reported levels of consumers’ engagement in DR schemes translate to actual curtailment behaviour and the individual rationale of customer response to DR signals. The presented approach also proposes a reinforcement learning framework for the decision problem of an aggregator selecting a set of consumers for DR events. This approach can support an aggregator in leveraging small-scale flexibility resources by providing an automated end-to-end framework to select the set of consumers for demand curtailment during Demand-Side Response (DR) signals in a dynamic environment while considering a long-term view of their selection process

Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence

Artificial intelligence (AI) has witnessed a substantial breakthrough in a variety of Internet of Things (IoT) applications and services, spanning from recommendation systems and speech processing applications to robotics control and military surveillance. This is driven by the easier access to sensory data and the enormous scale of pervasive/ubiquitous devices that generate zettabytes of real-time data streams. Designing accurate models using such data streams, to revolutionize the decision-taking process, inaugurates pervasive computing as a worthy paradigm for a better quality-of-life (e.g., smart homes and self-driving cars.). The confluence of pervasive computing and artificial intelligence, namely Pervasive AI, expanded the role of ubiquitous IoT systems from mainly data collection to executing distributed computations with a promising alternative to centralized learning, presenting various challenges, including privacy and latency requirements. In this context, an intelligent resource scheduling should be envisaged among IoT devices (e.g., smartphones, smart vehicles) and infrastructure (e.g., edge nodes and base stations) to avoid communication and computation overheads and ensure maximum performance. In this paper, we conduct a comprehensive survey of the recent techniques and strategies developed to overcome these resource challenges in pervasive AI systems. Specifically, we first present an overview of the pervasive computing, its architecture, and its intersection with artificial intelligence. We then review the background, applications and performance metrics of AI, particularly Deep Learning (DL) and reinforcement learning, running in a ubiquitous system. Next, we provide a deep literature review of communication-efficient techniques, from both algorithmic and system perspectives, of distributed training and inference across the combination of IoT devices, edge devices and cloud servers. Finally, we discuss our future vision and research challenges

Market-based coordination for domestic demand response in low-carbon electricity grids

Efforts towards a low carbon economy are challenging the electricity industry. On the supply-side, centralised carbon-intensive power plants are set to gradually decrease their contribution to the generation mix, whilst distributed renewable generation is to successively increase its share. On the demand-side, electricity use is expected to increase in the future due to the electrification of heating and transport. Moreover, the demand-side is to become more active allowing end-users to invest in generation and storage technologies, such as solar photovoltaics (PV) and home batteries. As a result, some network reinforcements might be needed and instrumentation at the users’ end is to be required, such as controllers and home energy management systems (HEMS). The electricity grid must balance supply and demand at all times in order to maintain technical constraints of frequency, voltage, and current; and this will become more challenging as a result of this transition. Failure to meet these constraints compromises the service and could damage the power grid assets and end-users’ appliances. Balancing generation, although responsive, is carbon-intensive and associated with inefficient asset utilisation, as these generators are mostly used during peak hours and sit idle the rest of the time. Furthermore, energy storage is a potential solution to assist the balancing problem in the presence of non-dispatchable low-carbon generators; however, it is substantially expensive to store energy in large amounts. Therefore, demand response (DR) has been envisioned as a complementary solution to increase the system’s resilience to weather-dependent, stochastic, and intermittent generation along with variable and temperature-correlated electric load. In the domestic setting, operational flexibility of some appliances, such as heaters and electric cars, can be coordinated amongst several households so as to help balance supply and demand, and reduce the need of balancing generators. Against this background, the electricity supply system requires new organisational paradigms that integrate DR effectively. Although some dynamic pricing schemes have been proposed to guide DR, such as time of use (ToU) and real-time pricing (RTP), it is still unclear how to control oscillatory massive responses (e.g., large fleet of electric cars simultaneously responding to a favourable price). Hence, this thesis proposes an alternative approach in which households proactively submit DR offers that express their preferences to their respective retailer in exchange for a discount. This research develops a computational model of domestic electricity use, and simulates appliances with operational flexibility in order to evaluate the effects and benefits of DR for both retailers and households. It provides a representation for this flexibility so that it can be integrated into specific DR offers. Retailers and households are modelled as computational agents. Furthermore, two market-based mechanisms are proposed to determine the allocation of DR offers. More specifically, a one-sided Vickrey-Clarke-Groves (VCG)-based mechanism and penalty schemes were designed for electricity retailers to coordinate their customers’ DR efforts so as to ameliorate the imbalance of their trading schedules. Similarly, a two-sided McAfee-based mechanism was designed to integrate DR offers into a multi-retailer setting in order to reduce zonal imbalances. A suitable method was developed to construct DR block offers that could be traded amongst retailers. Both mechanisms are dominant-strategy incentive-compatible and trade off a small amount of economic efficiency in order to maintain individual rationality, truthful reporting, weak budget balance and tractable computation. Moreover, privacy preserving is achieved by including computational agents from the independent system operator (ISO) as intermediaries between each retailer and its domestic customers, and amongst retailers. The theoretical properties of these mechanisms were proved using worst-case analysis, and their economic effects were evaluated in simulations based on data from a survey of UK household electricity use. In addition, forecasting methods were assessed on the end-users’ side in order to make better DR offers and avoid penalties. The results show that, under reasonable assumptions, the proposed coordination mechanisms achieve significant savings for both end-users and retailers, as they reduce the required amount of expensive balancing generation

A Multiarmed Bandit Incentive Mechanism for Crowdsourcing Demand Response in Smart Grids

Demand response is a critical part of renewable integration and energy cost reduction goals across the world. Motivated by the need to reduce costs arising from electricity shortage and renewable energy fluctuations, we propose a novel multiarmed bandit mechanism for demand response (MAB-MDR) which makes monetary offers to strategic consumers who have unknown response characteristics, to incetivize reduction in demand. Our work is inspired by a novel connection we make to crowdsourcing mechanisms. The proposed mechanism incorporates realistic features of the demand response problem including time varying and quadratic cost function. The mechanism marries auctions, that allow users to report their preferences, with online algorithms, that allow distribution companies to learn user-specific parameters. We show that MAB-MDR is dominant strategy incentive compatible, individually rational, and achieves sublinear regret. Such mechanisms can be effectively deployed in smart grids using new information and control architecture innovations and lead to welcome savings in energy costs