17,876 research outputs found
Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning
A fundamental question in any peer-to-peer ridesharing system is how to, both
effectively and efficiently, dispatch user's ride requests to the right driver
in real time. Traditional rule-based solutions usually work on a simplified
problem setting, which requires a sophisticated hand-crafted weight design for
either centralized authority control or decentralized multi-agent scheduling
systems. Although recent approaches have used reinforcement learning to provide
centralized combinatorial optimization algorithms with informative weight
values, their single-agent setting can hardly model the complex interactions
between drivers and orders. In this paper, we address the order dispatching
problem using multi-agent reinforcement learning (MARL), which follows the
distributed nature of the peer-to-peer ridesharing problem and possesses the
ability to capture the stochastic demand-supply dynamics in large-scale
ridesharing scenarios. Being more reliable than centralized approaches, our
proposed MARL solutions could also support fully distributed execution through
recent advances in the Internet of Vehicles (IoV) and the Vehicle-to-Network
(V2N). Furthermore, we adopt the mean field approximation to simplify the local
interactions by taking an average action among neighborhoods. The mean field
approximation is capable of globally capturing dynamic demand-supply variations
by propagating many local interactions between agents and the environment. Our
extensive experiments have shown the significant improvements of MARL order
dispatching algorithms over several strong baselines on the gross merchandise
volume (GMV), and order response rate measures. Besides, the simulated
experiments with real data have also justified that our solution can alleviate
the supply-demand gap during the rush hours, thus possessing the capability of
reducing traffic congestion.Comment: 11 pages, 9 figure
Learning Scheduling Algorithms for Data Processing Clusters
Efficiently scheduling data processing jobs on distributed compute clusters
requires complex algorithms. Current systems, however, use simple generalized
heuristics and ignore workload characteristics, since developing and tuning a
scheduling policy for each workload is infeasible. In this paper, we show that
modern machine learning techniques can generate highly-efficient policies
automatically. Decima uses reinforcement learning (RL) and neural networks to
learn workload-specific scheduling algorithms without any human instruction
beyond a high-level objective such as minimizing average job completion time.
Off-the-shelf RL techniques, however, cannot handle the complexity and scale of
the scheduling problem. To build Decima, we had to develop new representations
for jobs' dependency graphs, design scalable RL models, and invent RL training
methods for dealing with continuous stochastic job arrivals. Our prototype
integration with Spark on a 25-node cluster shows that Decima improves the
average job completion time over hand-tuned scheduling heuristics by at least
21%, achieving up to 2x improvement during periods of high cluster load
Q-Strategy: A Bidding Strategy for Market-Based Allocation of Grid Services
The application of autonomous agents by the provisioning and usage of computational services is an attractive research field. Various methods and technologies in the area of artificial intelligence, statistics and economics are playing together to achieve i) autonomic service provisioning and usage of Grid services, to invent ii) competitive bidding strategies for widely used market mechanisms and to iii) incentivize consumers and providers to use such market-based systems.
The contributions of the paper are threefold. First, we present a bidding agent framework for implementing artificial bidding agents, supporting consumers and providers in technical and economic preference elicitation as well as automated bid generation by the requesting and provisioning of Grid services. Secondly, we introduce a novel consumer-side bidding strategy, which enables a goal-oriented and strategic behavior by the generation and submission of consumer service requests and selection of provider offers. Thirdly, we evaluate and compare the Q-strategy, implemented within the presented framework, against the Truth-Telling bidding strategy in three mechanisms – a centralized CDA, a decentralized on-line machine scheduling and a FIFO-scheduling mechanisms
- …