Sub-agent elements for control methods in multi-agent energy management system

Increased penetration of generation and decentralised control are considered to be feasible and effective solution for reducing cost and emissions and hence efficiency associated with power generation and distribution. Distributed generation in combination with the multi-agent technology are perfect candidates for this solution. Pro-active and autonomous nature of multi-agent systems can provide an effective platform for decentralised control whilst improving reliability and flexibility of the grid

Collected notes from the Benchmarks and Metrics Workshop

In recent years there has been a proliferation of proposals in the artificial intelligence (AI) literature for integrated agent architectures. Each architecture offers an approach to the general problem of constructing an integrated agent. Unfortunately, the ways in which one architecture might be considered better than another are not always clear. There has been a growing realization that many of the positive and negative aspects of an architecture become apparent only when experimental evaluation is performed and that to progress as a discipline, we must develop rigorous experimental methods. In addition to the intrinsic intellectual interest of experimentation, rigorous performance evaluation of systems is also a crucial practical concern to our research sponsors. DARPA, NASA, and AFOSR (among others) are actively searching for better ways of experimentally evaluating alternative approaches to building intelligent agents. One tool for experimental evaluation involves testing systems on benchmark tasks in order to assess their relative performance. As part of a joint DARPA and NASA funded project, NASA-Ames and Teleos Research are carrying out a research effort to establish a set of benchmark tasks and evaluation metrics by which the performance of agent architectures may be determined. As part of this project, we held a workshop on Benchmarks and Metrics at the NASA Ames Research Center on June 25, 1990. The objective of the workshop was to foster early discussion on this important topic. We did not achieve a consensus, nor did we expect to. Collected here is some of the information that was exchanged at the workshop. Given here is an outline of the workshop, a list of the participants, notes taken on the white-board during open discussions, position papers/notes from some participants, and copies of slides used in the presentations

Multiagent systems: games and learning from structures

Multiple agents have become increasingly utilized in various fields for both physical robots and software agents, such as search and rescue robots, automated driving, auctions and electronic commerce agents, and so on. In multiagent domains, agents interact and coadapt with other agents. Each agent's choice of policy depends on the others' joint policy to achieve the best available performance. During this process, the environment evolves and is no longer stationary, where each agent adapts to proceed towards its target. Each micro-level step in time may present a different learning problem which needs to be addressed. However, in this non-stationary environment, a holistic phenomenon forms along with the rational strategies of all players; we define this phenomenon as structural properties. In our research, we present the importance of analyzing the structural properties, and how to extract the structural properties in multiagent environments. According to the agents' objectives, a multiagent environment can be classified as self-interested, cooperative, or competitive. We examine the structure from these three general multiagent environments: self-interested random graphical game playing, distributed cooperative team playing, and competitive group survival. In each scenario, we analyze the structure in each environmental setting, and demonstrate the structure learned as a comprehensive representation: structure of players' action influence, structure of constraints in teamwork communication, and structure of inter-connections among strategies. This structure represents macro-level knowledge arising in a multiagent system, and provides critical, holistic information for each problem domain. Last, we present some open issues and point toward future research

BQL-DRS: A Novel Balanced Q-Learning Based Demand Response System for IoT based Smart Grids

The modernization of electricity networks and the integration of renewable energy resources in Internet of Things (IoT) based smart grids have led to increased variability in market prices, necessitating effective demand response (DR) strategies. To address this challenge, this paper proposes a novel Balanced Q-Learning based Demand Response System (BQL-DRS) that combines both optimistic and pessimistic targets in the Q-learning algorithm to achieve a balanced decision-making process in IoT based smart grids. It optimizes DR actions by efficiently managing consumer demand in real-time, considering IoT data from grid conditions, energy prices, and consumer preferences. The significance of the BQL-DRS lies in its ability to handle dynamic and uncertain IoT based grid environments, enabling it to make informed and cautious decisions while pursuing energy efficiency and cost-effectiveness. By effectively addressing both pessimistic and optimistic scenarios, the BQL-DRS ensures grid stability, load balancing, and substantial cost savings compared to representative models

Reviewing agent-based modelling of socio-ecosystems: a methodology for the analysis of climate change adaptation and sustainability

The integrated - environmental, economic and social - analysis of climate change calls for a paradigm shift as it is fundamentally a problem of complex, bottom-up and multi-agent human behaviour. There is a growing awareness that global environmental change dynamics and the related socio-economic implications involve a degree of complexity that requires an innovative modelling of combined social and ecological systems. Climate change policy can no longer be addressed separately from a broader context of adaptation and sustainability strategies. A vast body of literature on agent-based modelling (ABM) shows its potential to couple social and environmental models, to incorporate the influence of micro-level decision making in the system dynamics and to study the emergence of collective responses to policies. However, there are few publications which concretely apply this methodology to the study of climate change related issues. The analysis of the state of the art reported in this paper supports the idea that today ABM is an appropriate methodology for the bottom-up exploration of climate policies, especially because it can take into account adaptive behaviour and heterogeneity of the system's components.Review, Agent-Based Modelling, Socio-Ecosystems, Climate Change, Adaptation, Complexity.

Reinforcement learning method for plug-in electric vehicle bidding

This study proposes a novel multi-agent method for electric vehicle (EV) owners who will take part in the electricity market. Each EV is considered as an agent, and all the EVs have vehicle-to-grid capability. These agents aim to minimise the charging cost and to increase the privacy of EV owners due to omitting the aggregator role in the system. Each agent has two independent decision cores for buying and selling energy. These cores are developed based on a reinforcement learning (RL) algorithm, i.e. Q-learning algorithm, due to its high efficiency and appropriate performance in multi-agent methods. Based on the proposed method, agents can buy and sell energy with the cost minimisation goal, while they should always have enough energy for the trip, considering the uncertain behaviours of EV owners. Numeric simulations on an illustrative example with one agent and a testing system with 500 agents demonstrate the effectiveness of the proposed method

Scheduling Algorithms: Challenges Towards Smart Manufacturing

Collecting, processing, analyzing, and driving knowledge from large-scale real-time data is now realized with the emergence of Artificial Intelligence (AI) and Deep Learning (DL). The breakthrough of Industry 4.0 lays a foundation for intelligent manufacturing. However, implementation challenges of scheduling algorithms in the context of smart manufacturing are not yet comprehensively studied. The purpose of this study is to show the scheduling No.s that need to be considered in the smart manufacturing paradigm. To attain this objective, the literature review is conducted in five stages using publish or perish tools from different sources such as Scopus, Pubmed, Crossref, and Google Scholar. As a result, the first contribution of this study is a critical analysis of existing production scheduling algorithms\u27 characteristics and limitations from the viewpoint of smart manufacturing. The other contribution is to suggest the best strategies for selecting scheduling algorithms in a real-world scenario

Optimal Management of community Demand Response

More than one-third of the electricity produced globally is consumed by the residential sectors [1], with nearly 17% of CO2 emissions, are coming from residential buildings according to reports from 2018 [2] [3]. In order to cope with increase in electricity demand and consumption, while considering the environmental impacts, electricity providers are seeking to implement solutions to help them balance the supply with the electricity demand while mitigating emissions. Thus, increasing the number of conventional generation units and using unreliable renewable source of energy is not a viable investment. That’s why, in recent years research attention has shifted to demand side solutions [4]. This research investigates the optimal management for an urban residential community, that can help in reducing energy consumption and peak and CO2 emissions. This will help to put an agreement with the grid operator for an agreed load shape, for efficient demand response (DR) program implementation. This work uses a framework known as CityLearn [2]. It is based on a Machine Learning branch known as Reinforcement Learning (RL), and it is used to test a variety of intelligent agents for optimizing building load consumption and load shape. The RL agent is used for controlling hot water and chilled water storages, as well as the battery system. When compared to the regular building usage, the results demonstrate that utilizing an RL agent for storage system control can be helpful, as the electricity consumption is greatly reduced when it’s compared to the normal building consumption

Integrating Multi-agent System and Microsimulation for Dynamic Modeling of Urban Freight Transport

This work aims to apply an integration between a multi-agent system and microsimulation to take advantage of the large amount of data generated in urban freight transport to improve the overall performance of the urban supply chain without forgetting the principles of autonomy that govern each of its actors, responding to the different dynamic scenarios that may arise in the operational context. The integration framework produces a satisfactory communication process in those modeling methods measured by two indexes: throughput and latency. The results of this integration show a robust response to dynamic scenarios and allow reacting to the different quantity of changes without dismissing the search for optimum solutions