2,566 research outputs found
Agent-Based Computational Economics
Agent-based computational economics (ACE) is the computational study of economies modeled as evolving systems of autonomous interacting agents. Starting from initial conditions, specified by the modeler, the computational economy evolves over time as its constituent agents repeatedly interact with each other and learn from these interactions. ACE is therefore a bottom-up culture-dish approach to the study of economic systems. This study discusses the key characteristics and goals of the ACE methodology. Eight currently active research areas are highlighted for concrete illustration. Potential advantages and disadvantages of the ACE methodology are considered, along with open questions and possible directions for future research.Agent-based computational economics; Autonomous agents; Interaction networks; Learning; Evolution; Mechanism design; Computational economics; Object-oriented programming.
On Partially Controlled Multi-Agent Systems
Motivated by the control theoretic distinction between controllable and
uncontrollable events, we distinguish between two types of agents within a
multi-agent system: controllable agents, which are directly controlled by the
system's designer, and uncontrollable agents, which are not under the
designer's direct control. We refer to such systems as partially controlled
multi-agent systems, and we investigate how one might influence the behavior of
the uncontrolled agents through appropriate design of the controlled agents. In
particular, we wish to understand which problems are naturally described in
these terms, what methods can be applied to influence the uncontrollable
agents, the effectiveness of such methods, and whether similar methods work
across different domains. Using a game-theoretic framework, this paper studies
the design of partially controlled multi-agent systems in two contexts: in one
context, the uncontrollable agents are expected utility maximizers, while in
the other they are reinforcement learners. We suggest different techniques for
controlling agents' behavior in each domain, assess their success, and examine
their relationship.Comment: See http://www.jair.org/ for any accompanying file
Recommended from our members
Towards Informed Exploration for Deep Reinforcement Learning
In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact
Society-in-the-Loop: Programming the Algorithmic Social Contract
Recent rapid advances in Artificial Intelligence (AI) and Machine Learning
have raised many questions about the regulatory and governance mechanisms for
autonomous machines. Many commentators, scholars, and policy-makers now call
for ensuring that algorithms governing our lives are transparent, fair, and
accountable. Here, I propose a conceptual framework for the regulation of AI
and algorithmic systems. I argue that we need tools to program, debug and
maintain an algorithmic social contract, a pact between various human
stakeholders, mediated by machines. To achieve this, we can adapt the concept
of human-in-the-loop (HITL) from the fields of modeling and simulation, and
interactive machine learning. In particular, I propose an agenda I call
society-in-the-loop (SITL), which combines the HITL control paradigm with
mechanisms for negotiating the values of various stakeholders affected by AI
systems, and monitoring compliance with the agreement. In short, `SITL = HITL +
Social Contract.'Comment: (in press), Ethics of Information Technology, 201
Learning-based social coordination to improve safety and robustness of cooperative autonomous vehicles in mixed traffic
It is expected that autonomous vehicles(AVs) and heterogeneous human-driven
vehicles(HVs) will coexist on the same road. The safety and reliability of AVs
will depend on their social awareness and their ability to engage in complex
social interactions in a socially accepted manner. However, AVs are still
inefficient in terms of cooperating with HVs and struggle to understand and
adapt to human behavior, which is particularly challenging in mixed autonomy.
In a road shared by AVs and HVs, the social preferences or individual traits of
HVs are unknown to the AVs and different from AVs, which are expected to follow
a policy, HVs are particularly difficult to forecast since they do not
necessarily follow a stationary policy. To address these challenges, we frame
the mixed-autonomy problem as a multi-agent reinforcement learning (MARL)
problem and propose an approach that allows AVs to learn the decision-making of
HVs implicitly from experience, account for all vehicles' interests, and safely
adapt to other traffic situations. In contrast with existing works, we quantify
AVs' social preferences and propose a distributed reward structure that
introduces altruism into their decision-making process, allowing the altruistic
AVs to learn to establish coalitions and influence the behavior of HVs.Comment: arXiv admin note: substantial text overlap with arXiv:2202.0088
Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis
Each year, expert-level performance is attained in increasingly-complex
multiagent domains, notable examples including Go, Poker, and StarCraft II.
This rapid progression is accompanied by a commensurate need to better
understand how such agents attain this performance, to enable their safe
deployment, identify limitations, and reveal potential means of improving them.
In this paper we take a step back from performance-focused multiagent learning,
and instead turn our attention towards agent behavior analysis. We introduce a
model-agnostic method for discovery of behavior clusters in multiagent domains,
using variational inference to learn a hierarchy of behaviors at the joint and
local agent levels. Our framework makes no assumption about agents' underlying
learning algorithms, does not require access to their latent states or
policies, and is trained using only offline observational data. We illustrate
the effectiveness of our method for enabling the coupled understanding of
behaviors at the joint and local agent level, detection of behavior
changepoints throughout training, discovery of core behavioral concepts,
demonstrate the approach's scalability to a high-dimensional multiagent MuJoCo
control domain, and also illustrate that the approach can disentangle
previously-trained policies in OpenAI's hide-and-seek domain
Efficient XAI Techniques: A Taxonomic Survey
Recently, there has been a growing demand for the deployment of Explainable
Artificial Intelligence (XAI) algorithms in real-world applications. However,
traditional XAI methods typically suffer from a high computational complexity
problem, which discourages the deployment of real-time systems to meet the
time-demanding requirements of real-world scenarios. Although many approaches
have been proposed to improve the efficiency of XAI methods, a comprehensive
understanding of the achievements and challenges is still needed. To this end,
in this paper we provide a review of efficient XAI. Specifically, we categorize
existing techniques of XAI acceleration into efficient non-amortized and
efficient amortized methods. The efficient non-amortized methods focus on
data-centric or model-centric acceleration upon each individual instance. In
contrast, amortized methods focus on learning a unified distribution of model
explanations, following the predictive, generative, or reinforcement
frameworks, to rapidly derive multiple model explanations. We also analyze the
limitations of an efficient XAI pipeline from the perspectives of the training
phase, the deployment phase, and the use scenarios. Finally, we summarize the
challenges of deploying XAI acceleration methods to real-world scenarios,
overcoming the trade-off between faithfulness and efficiency, and the selection
of different acceleration methods.Comment: 15 pages, 3 figure
- …