257 research outputs found
K-SHAP: Policy Clustering Algorithm for Anonymous State-Action Pairs
Learning agent behaviors from observational data has shown to improve our
understanding of their decision-making processes, advancing our ability to
explain their interactions with the environment and other agents. While
multiple learning techniques have been proposed in the literature, there is one
particular setting that has not been explored yet: multi agent systems where
agent identities remain anonymous. For instance, in financial markets labeled
data that identifies market participant strategies is typically proprietary,
and only the anonymous state-action pairs that result from the interaction of
multiple market participants are publicly available. As a result, sequences of
agent actions are not observable, restricting the applicability of existing
work. In this paper, we propose a Policy Clustering algorithm, called K-SHAP,
that learns to group anonymous state-action pairs according to the agent
policies. We frame the problem as an Imitation Learning (IL) task, and we learn
a world-policy able to mimic all the agent behaviors upon different
environmental states. We leverage the world-policy to explain each anonymous
observation through an additive feature attribution method called SHAP (SHapley
Additive exPlanations). Finally, by clustering the explanations we show that we
are able to identify different agent policies and group observations
accordingly. We evaluate our approach on simulated synthetic market data and a
real-world financial dataset. We show that our proposal significantly and
consistently outperforms the existing methods, identifying different agent
strategies.Comment: ICML 202
Equitable Marketplace Mechanism Design
We consider a trading marketplace that is populated by traders with diverse
trading strategies and objectives. The marketplace allows the suppliers to list
their goods and facilitates matching between buyers and sellers. In return,
such a marketplace typically charges fees for facilitating trade. The goal of
this work is to design a dynamic fee schedule for the marketplace that is
equitable and profitable to all traders while being profitable to the
marketplace at the same time (from charging fees). Since the traders adapt
their strategies to the fee schedule, we present a reinforcement learning
framework for simultaneously learning a marketplace fee schedule and trading
strategies that adapt to this fee schedule using a weighted optimization
objective of profits and equitability. We illustrate the use of the proposed
approach in detail on a simulated stock exchange with different types of
investors, specifically market makers and consumer investors. As we vary the
equitability weights across different investor classes, we see that the learnt
exchange fee schedule starts favoring the class of investors with the highest
weight. We further discuss the observed insights from the simulated stock
exchange in light of the general framework of equitable marketplace mechanism
design
ABIDES-Economist: Agent-Based Simulation of Economic Systems with Learning Agents
We introduce a multi-agent simulator for economic systems comprised of
heterogeneous Households, heterogeneous Firms, Central Bank and Government
agents, that could be subjected to exogenous, stochastic shocks. The
interaction between agents defines the production and consumption of goods in
the economy alongside the flow of money. Each agent can be designed to act
according to fixed, rule-based strategies or learn their strategies using
interactions with others in the simulator. We ground our simulator by choosing
agent heterogeneity parameters based on economic literature, while designing
their action spaces in accordance with real data in the United States. Our
simulator facilitates the use of reinforcement learning strategies for the
agents via an OpenAI Gym style environment definition for the economic system.
We demonstrate the utility of our simulator by simulating and analyzing two
hypothetical (yet interesting) economic scenarios. The first scenario
investigates the impact of heterogeneous household skills on their learned
preferences to work at different firms. The second scenario examines the impact
of a positive production shock to one of two firms on its pricing strategy in
comparison to the second firm. We aspire that our platform sets a stage for
subsequent research at the intersection of artificial intelligence and
economics
ATMS: Algorithmic Trading-Guided Market Simulation
The effective construction of an Algorithmic Trading (AT) strategy often
relies on market simulators, which remains challenging due to existing methods'
inability to adapt to the sequential and dynamic nature of trading activities.
This work fills this gap by proposing a metric to quantify market discrepancy.
This metric measures the difference between a causal effect from underlying
market unique characteristics and it is evaluated through the interaction
between the AT agent and the market. Most importantly, we introduce Algorithmic
Trading-guided Market Simulation (ATMS) by optimizing our proposed metric.
Inspired by SeqGAN, ATMS formulates the simulator as a stochastic policy in
reinforcement learning (RL) to account for the sequential nature of trading.
Moreover, ATMS utilizes the policy gradient update to bypass differentiating
the proposed metric, which involves non-differentiable operations such as order
deletion from the market. Through extensive experiments on semi-real market
data, we demonstrate the effectiveness of our metric and show that ATMS
generates market data with improved similarity to reality compared to the
state-of-the-art conditional Wasserstein Generative Adversarial Network (cWGAN)
approach. Furthermore, ATMS produces market data with more balanced BUY and
SELL volumes, mitigating the bias of the cWGAN baseline approach, where a
simple strategy can exploit the BUY/SELL imbalance for profit
Transparency as Delayed Observability in Multi-Agent Systems
Is transparency always beneficial in complex systems such as traffic networks
and stock markets? How is transparency defined in multi-agent systems, and what
is its optimal degree at which social welfare is highest? We take an
agent-based view to define transparency (or its lacking) as delay in agent
observability of environment states, and utilize simulations to analyze the
impact of delay on social welfare. To model the adaptation of agent strategies
with varying delays, we model agents as learners maximizing the same objectives
under different delays in a simulated environment. Focusing on two agent types
- constrained and unconstrained, we use multi-agent reinforcement learning to
evaluate the impact of delay on agent outcomes and social welfare. Empirical
demonstration of our framework in simulated financial markets shows opposing
trends in outcomes of the constrained and unconstrained agents with delay, with
an optimal partial transparency regime at which social welfare is maximal
- …