617 research outputs found
CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms
How to optimally dispatch orders to vehicles and how to tradeoff between
immediate and future returns are fundamental questions for a typical
ride-hailing platform. We model ride-hailing as a large-scale parallel ranking
problem and study the joint decision-making task of order dispatching and fleet
management in online ride-hailing platforms. This task brings unique challenges
in the following four aspects. First, to facilitate a huge number of vehicles
to act and learn efficiently and robustly, we treat each region cell as an
agent and build a multi-agent reinforcement learning framework. Second, to
coordinate the agents from different regions to achieve long-term benefits, we
leverage the geographical hierarchy of the region grids to perform hierarchical
reinforcement learning. Third, to deal with the heterogeneous and variant
action space for joint order dispatching and fleet management, we design the
action as the ranking weight vector to rank and select the specific order or
the fleet management destination in a unified formulation. Fourth, to achieve
the multi-scale ride-hailing platform, we conduct the decision-making process
in a hierarchical way where a multi-head attention mechanism is utilized to
incorporate the impacts of neighbor agents and capture the key agent in each
scale. The whole novel framework is named as CoRide. Extensive experiments
based on multiple cities real-world data as well as analytic synthetic data
demonstrate that CoRide provides superior performance in terms of platform
revenue and user experience in the task of city-wide hybrid order dispatching
and fleet management over strong baselines.Comment: CIKM 201
Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach
The dynamic vehicle dispatching problem corresponds to deciding which
vehicles to assign to requests that arise stochastically over time and space.
It emerges in diverse areas, such as in the assignment of trucks to loads to be
transported; in emergency systems; and in ride-hailing services. In this paper,
we model the problem as a semi-Markov decision process, which allows us to
treat time as continuous. In this setting, decision epochs coincide with
discrete events whose time intervals are random. We argue that an event-based
approach substantially reduces the combinatorial complexity of the decision
space and overcomes other limitations of discrete-time models often proposed in
the literature. In order to test our approach, we develop a new discrete-event
simulator and use double deep q-learning to train our decision agents.
Numerical experiments are carried out in realistic scenarios using data from
New York City. We compare the policies obtained through our approach with
heuristic policies often used in practice. Results show that our policies
exhibit better average waiting times, cancellation rates and total service
times, with reduction in average waiting times of up to 50% relative to the
other tested heuristic policies.Comment: 42 pages, 22 figure
How machine learning informs ride-hailing services: A survey
In recent years, online ride-hailing services have emerged as an important component of urban transportation system, which not only provide significant ease for residents’ travel activities, but also shape new travel behavior and diversify urban mobility patterns. This study provides a thorough review of machine-learning-based methodologies for on-demand ride-hailing services. The importance of on-demand ride-hailing services in the spatio-temporal dynamics of urban traffic is first highlighted, with machine-learning-based macro-level ride-hailing research demonstrating its value in guiding the design, planning, operation, and control of urban intelligent transportation systems. Then, the research on travel behavior from the perspective of individual mobility patterns, including carpooling behavior and modal choice behavior, is summarized. In addition, existing studies on order matching and vehicle dispatching strategies, which are among the most important components of on-line ride-hailing systems, are collected and summarized. Finally, some of the critical challenges and opportunities in ride-hailing services are discussed
CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms
How to optimally dispatch orders to vehicles and how to trade
off between immediate and future returns are fundamental questions for a typical ride-hailing platform. We model ride-hailing as a
large-scale parallel ranking problem and study the joint decisionmaking task of order dispatching and fleet management in online
ride-hailing platforms. This task brings unique challenges in the
following four aspects. First, to facilitate a huge number of vehicles
to act and learn efficiently and robustly, we treat each region cell
as an agent and build a multi-agent reinforcement learning framework. Second, to coordinate the agents from different regions to
achieve long-term benefits, we leverage the geographical hierarchy
of the region grids to perform hierarchical reinforcement learning.
Third, to deal with the heterogeneous and variant action space
for joint order dispatching and fleet management, we design the
action as the ranking weight vector to rank and select the specific
order or the fleet management destination in a unified formulation.
Fourth, to achieve the multi-scale ride-hailing platform, we conduct
the decision-making process in a hierarchical way where a multihead attention mechanism is utilized to incorporate the impacts
of neighbor agents and capture the key agent in each scale. The
whole novel framework is named as CoRide. Extensive experiments
based on multiple cities real-world data as well as analytic synthetic
data demonstrate that CoRide provides superior performance in
terms of platform revenue and user experience in the task of citywide hybrid order dispatching and fleet management over strong
baselines
Data-driven Methodologies and Applications in Urban Mobility
The world is urbanizing at an unprecedented rate where urbanization goes from 39% in 1980 to 58% in 2019 (World Bank, 2019). This poses more and more transportation demand and pressure on the already at or over-capacity old transport infrastructure, especially in urban areas. Along the same timeline, more data generated as a byproduct of daily activity are being collected via the advancement of the internet of things, and computers are getting more and more powerful. These are shown by the statistics such as 90% of the world’s data is generated within the last two years and IBM’s computer is now processing at the speed of 120,000 GPS points per second. Thus, this dissertation discusses the challenges and opportunities arising from the growing demand for urban mobility, particularly in cities with outdated infrastructure, and how to capitalize on the unprecedented growth in data in solving these problems by ways of data-driven transportation-specific methodologies. The dissertation identifies three primary challenges and/or opportunities, which are (1) optimally locating dynamic wireless charging to promote the adoption of electric vehicles, (2) predicting dynamic traffic state using an enormously large dataset of taxi trips, and (3) improving the ride-hailing system with carpooling, smart dispatching, and preemptive repositioning. The dissertation presents potential solutions/methodologies that have become available only recently thanks to the extraordinary growth of data and computers with explosive power, and these methodologies are (1) bi-level optimization planning frameworks for locating dynamic wireless charging facilities, (2) Traffic Graph Convolutional Network for dynamic urban traffic state estimation, and (3) Graph Matching and Reinforcement Learning for the operation and management of mixed autonomous electric taxi fleets. These methodologies are then carefully calibrated, methodically scrutinized under various performance metrics and procedures, and validated with previous research and ground truth data, which is gathered directly from the real world. In order to bridge the gap between scientific discoveries and practical applications, the three methodologies are applied to the case study of (1) Montgomery County, MD, (2) the City of New York, and (3) the City of Chicago and from which, real-world implementation are suggested. This dissertation’s contribution via the provided methodologies, along with the continual increase in data, have the potential to significantly benefit urban mobility and work toward a sustainable transportation system
Dispatching AGVs with Battery Constraints using Deep Reinforcement Learning
This paper considers the problem of real-time dispatching of a fleet of automated guided vehicles (AGVs) with battery constraints. AGVs must be immediately assigned to transport requests, which arrive randomly. In addition, the AGVs must be repositioned and recharged, awaiting future transport requests. Each transport request has a soft time window with late delivery incurring a tardiness cost. This research aims to minimize the total costs, consisting of tardiness costs of transport requests and travel costs of AGVs. We extend the existing literature by making a distinction between parking and charging nodes, where AGVs wait idle for incoming transporting requests and satisfy their charging needs, respectively. Also, we formulate this online decision-making problem as a Markov decision process and propose a solution approach based on deep reinforcement learning. To assess the quality of the proposed approach, we compare it with the optimal solution of a mixed-integer linear programming model that assumes full knowledge of transport requests in hindsight and hence serves as a lower-bound on the costs. We also compare our solution with a heuristic policy used in practice. We assess the performance of the proposed solutions in an industry case study using real-world data
A Systematic Literature Review on Machine Learning in Shared Mobility
Shared mobility has emerged as a sustainable alternative to both private transportation and traditional public transport, promising to reduce the number of private vehicles on roads while offering users greater flexibility. Today, urban areas are home to a myriad of innovative services, including car-sharing, ride-sharing, and micromobility solutions like moped-sharing, bike-sharing, and e-scooter-sharing. Given the intense competition and the inherent operational complexities of shared mobility systems, providers are increasingly seeking specialized decision-support methodologies to boost operational efficiency. While recent research indicates that advanced machine learning methods can tackle the intricate challenges in shared mobility management decisions, a thorough evaluation of existing research is essential to fully grasp its potential and pinpoint areas needing further exploration. This paper presents a systematic literature review that specifically targets the application of Machine Learning for decision-making in Shared Mobility Systems. Our review underscores that Machine Learning offers methodological solutions to specific management challenges crucial for the effective operation of Shared Mobility Systems. We delve into the methods and datasets employed, spotlight research trends, and pinpoint research gaps. Our findings culminate in a comprehensive framework of Machine Learning techniques designed to bolster managerial decision-making in addressing challenges specific to Shared Mobility across various levels
A multi-functional simulation platform for on-demand ride service operations
On-demand ride services or ride-sourcing services have been experiencing fast
development in the past decade. Various mathematical models and optimization
algorithms have been developed to help ride-sourcing platforms design
operational strategies with higher efficiency. However, due to cost and
reliability issues (implementing an immature algorithm for real operations may
result in system turbulence), it is commonly infeasible to validate these
models and train/test these optimization algorithms within real-world ride
sourcing platforms. Acting as a useful test bed, a simulation platform for
ride-sourcing systems will be very important to conduct algorithm
training/testing or model validation through trails and errors. While previous
studies have established a variety of simulators for their own tasks, it lacks
a fair and public platform for comparing the models or algorithms proposed by
different researchers. In addition, the existing simulators still face many
challenges, ranging from their closeness to real environments of ride-sourcing
systems, to the completeness of different tasks they can implement. To address
the challenges, we propose a novel multi-functional and open-sourced simulation
platform for ride-sourcing systems, which can simulate the behaviors and
movements of various agents on a real transportation network. It provides a few
accessible portals for users to train and test various optimization algorithms,
especially reinforcement learning algorithms, for a variety of tasks, including
on-demand matching, idle vehicle repositioning, and dynamic pricing. In
addition, it can be used to test how well the theoretical models approximate
the simulated outcomes. Evaluated on real-world data based experiments, the
simulator is demonstrated to be an efficient and effective test bed for various
tasks related to on-demand ride service operations
Towards More Efficient Shared Autonomous Mobility: A Learning-Based Fleet Repositioning Approach
Shared-use autonomous mobility services (SAMS) present new opportunities for
improving accessible and demand-responsive mobility. A fundamental challenge
that SAMS face is appropriate positioning of idle fleet vehicles to meet future
demand - a problem that strongly impacts service quality and efficiency. This
paper formulates SAMS fleet repositioning as a Markov Decision Process and
presents a reinforcement learning-based repositioning (RLR) approach called
integrated system-agent repositioning (ISR). The ISR learns a scalable fleet
repositioning strategy in an integrated manner: learning to respond to evolving
demand patterns without explicit demand forecasting and to cooperate with
optimization-based passenger-to-vehicle assignment. Numerical experiments are
conducted using New York City taxi data and an agent-based simulation tool. The
ISR is compared to an alternative RLR approach named externally guided
repositioning (EGR) and a benchmark joint optimization (JO) for
passenger-to-vehicle assignment and repositioning. The results demonstrate the
RLR approaches' substantial reductions in passenger wait times, over 50%,
relative to the JO approach. The ISR's ability to bypass demand forecasting is
also demonstrated as it maintains comparable performance to EGR in terms of
average metrics. The results also demonstrate the model's transferability to
evolving conditions, including unseen demand patterns, extended operational
periods, and changes in the assignment strategy
- …