Vehicle Dispatching and Routing of On-Demand Intercity Ride-Pooling Services: A Multi-Agent Hierarchical Reinforcement Learning Approach

The integrated development of city clusters has given rise to an increasing demand for intercity travel. Intercity ride-pooling service exhibits considerable potential in upgrading traditional intercity bus services by implementing demand-responsive enhancements. Nevertheless, its online operations suffer the inherent complexities due to the coupling of vehicle resource allocation among cities and pooled-ride vehicle routing. To tackle these challenges, this study proposes a two-level framework designed to facilitate online fleet management. Specifically, a novel multi-agent feudal reinforcement learning model is proposed at the upper level of the framework to cooperatively assign idle vehicles to different intercity lines, while the lower level updates the routes of vehicles using an adaptive large neighborhood search heuristic. Numerical studies based on the realistic dataset of Xiamen and its surrounding cities in China show that the proposed framework effectively mitigates the supply and demand imbalances, and achieves significant improvement in both the average daily system profit and order fulfillment ratio

A multi-functional simulation platform for on-demand ride service operations

On-demand ride services or ride-sourcing services have been experiencing fast development in the past decade. Various mathematical models and optimization algorithms have been developed to help ride-sourcing platforms design operational strategies with higher efficiency. However, due to cost and reliability issues (implementing an immature algorithm for real operations may result in system turbulence), it is commonly infeasible to validate these models and train/test these optimization algorithms within real-world ride sourcing platforms. Acting as a useful test bed, a simulation platform for ride-sourcing systems will be very important to conduct algorithm training/testing or model validation through trails and errors. While previous studies have established a variety of simulators for their own tasks, it lacks a fair and public platform for comparing the models or algorithms proposed by different researchers. In addition, the existing simulators still face many challenges, ranging from their closeness to real environments of ride-sourcing systems, to the completeness of different tasks they can implement. To address the challenges, we propose a novel multi-functional and open-sourced simulation platform for ride-sourcing systems, which can simulate the behaviors and movements of various agents on a real transportation network. It provides a few accessible portals for users to train and test various optimization algorithms, especially reinforcement learning algorithms, for a variety of tasks, including on-demand matching, idle vehicle repositioning, and dynamic pricing. In addition, it can be used to test how well the theoretical models approximate the simulated outcomes. Evaluated on real-world data based experiments, the simulator is demonstrated to be an efficient and effective test bed for various tasks related to on-demand ride service operations

How machine learning informs ride-hailing services: A survey

In recent years, online ride-hailing services have emerged as an important component of urban transportation system, which not only provide significant ease for residents’ travel activities, but also shape new travel behavior and diversify urban mobility patterns. This study provides a thorough review of machine-learning-based methodologies for on-demand ride-hailing services. The importance of on-demand ride-hailing services in the spatio-temporal dynamics of urban traffic is first highlighted, with machine-learning-based macro-level ride-hailing research demonstrating its value in guiding the design, planning, operation, and control of urban intelligent transportation systems. Then, the research on travel behavior from the perspective of individual mobility patterns, including carpooling behavior and modal choice behavior, is summarized. In addition, existing studies on order matching and vehicle dispatching strategies, which are among the most important components of on-line ride-hailing systems, are collected and summarized. Finally, some of the critical challenges and opportunities in ride-hailing services are discussed

Spatio-temporal Incentives Optimization for Ride-hailing Services with Offline Deep Reinforcement Learning

A fundamental question in any peer-to-peer ride-sharing system is how to, both effectively and efficiently, meet the request of passengers to balance the supply and demand in real time. On the passenger side, traditional approaches focus on pricing strategies by increasing the probability of users' call to adjust the distribution of demand. However, previous methods do not take into account the impact of changes in strategy on future supply and demand changes, which means drivers are repositioned to different destinations due to passengers' calls, which will affect the driver's income for a period of time in the future. Motivated by this observation, we make an attempt to optimize the distribution of demand to handle this problem by learning the long-term spatio-temporal values as a guideline for pricing strategy. In this study, we propose an offline deep reinforcement learning based method focusing on the demand side to improve the utilization of transportation resources and customer satisfaction. We adopt a spatio-temporal learning method to learn the value of different time and location, then incentivize the ride requests of passengers to adjust the distribution of demand to balance the supply and demand in the system. In particular, we model the problem as a Markov Decision Process (MDP)

Optimizing Information Values in Smart Mobility

Smart mobility, enabled by advanced sensing, communication, vehicle, and emerging mobility technologies, has transformed transportation systems. Real-time information shared by public and private entities plays a pivotal role in smart mobility, which facilitates informed decision-making, including effective mode choice, dynamic vehicle control, optimized travel routing, and strategic vehicle relocation. While more information is believed to benefit individual decision makers, it is crucial to acknowledge that the effects of information on transportation network performance are contingent; more information may not always benefit the safety and mobility of the whole system. The goal of this dissertation is to investigate the effects of information shared by public and private transportation entities on system-level performance. The challenges are primarily due to the lack of a unified modeling framework to endogenously reflect the decentralized multi-agent interaction involved in the interconnected transportation networks and the resulting computational complexities arising from non-convexity and high dimensionality. To address these challenges, this dissertation proposes novel modeling frameworks and computational solutions for three cutting-edge smart mobility applications. First, to examine the impact of en-route information on a transportation network, we propose a novel two-stage stochastic traffic equilibrium model to characterize the equilibrium traffic patterns considering adaptive routing behavior when locational en-route traffic information is provided through infrastructure-to-vehicles (I2V) communications. This model is formulated as a convex stochastic optimization problem so that efficient stochastic programming algorithms can be directly leveraged to achieve scalability. Second, to achieve optimal control over real-time variable speed limits information sharing and evaluate its impact on the network, we propose a twin-delayed deep deterministic policy gradient model, which converges more reliably than state-of-the-art deep reinforcement learning models. We investigate the transferability of the control algorithm and conduct comparative analyses of different traffic control strategies and spatial distributions of variable speed limit control (VSLC) deployment. Third, to assess the impacts of information provided by private ride-sourcing companies on transportation network congestion, we propose a Stackelberg framework for spatial pricing of ride-sourcing services considering traffic congestion and convex reformulation strategies under mild conditions. We perform numerical experiments on transportation networks of varying scales and with diverse transportation network company (TNC) objectives, aiming to derive policy insights regarding the implications of spatial pricing information on transportation systems

Balancing the Tradeoff between Profit and Fairness in Rideshare Platforms During High-Demand Hours

Rideshare platforms, when assigning requests to drivers, tend to maximize profit for the system and/or minimize waiting time for riders. Such platforms can exacerbate biases that drivers may have over certain types of requests. We consider the case of peak hours when the demand for rides is more than the supply of drivers. Drivers are well aware of their advantage during the peak hours and can choose to be selective about which rides to accept. Moreover, if in such a scenario, the assignment of requests to drivers (by the platform) is made only to maximize profit and/or minimize wait time for riders, requests of a certain type (e.g. from a non-popular pickup location, or to a non-popular drop-off location) might never be assigned to a driver. Such a system can be highly unfair to riders. However, increasing fairness might come at a cost of the overall profit made by the rideshare platform. To balance these conflicting goals, we present a flexible, non-adaptive algorithm, \lpalg, that allows the platform designer to control the profit and fairness of the system via parameters $\alpha$ and $\beta$ respectively. We model the matching problem as an online bipartite matching where the set of drivers is offline and requests arrive online. Upon the arrival of a request, we use \lpalg to assign it to a driver (the driver might then choose to accept or reject it) or reject the request. We formalize the measures of profit and fairness in our setting and show that by using \lpalg, the competitive ratios for profit and fairness measures would be no worse than $\alpha/e$ and $\beta/e$ respectively. Extensive experimental results on both real-world and synthetic datasets confirm the validity of our theoretical lower bounds. Additionally, they show that \lpalg under some choice of $(\alpha, \beta)$ can beat two natural heuristics, Greedy and Uniform, on \emph{both} fairness and profit

e-Uber\textit{e-Uber}: A Crowdsourcing Platform for Electric Vehicle-based Ride- and Energy-sharing

The sharing-economy-based business model has recently seen success in the transportation and accommodation sectors with companies like Uber and Airbnb. There is growing interest in applying this model to energy systems, with modalities like peer-to-peer (P2P) Energy Trading, Electric Vehicles (EV)-based Vehicle-to-Grid (V2G), Vehicle-to-Home (V2H), Vehicle-to-Vehicle (V2V), and Battery Swapping Technology (BST). In this work, we exploit the increasing diffusion of EVs to realize a crowdsourcing platform called e-Uber that jointly enables ride-sharing and energy-sharing through V2G and BST. e-Uber exploits spatial crowdsourcing, reinforcement learning, and reverse auction theory. Specifically, the platform uses reinforcement learning to understand the drivers' preferences towards different ride-sharing and energy-sharing tasks. Based on these preferences, a personalized list is recommended to each driver through CMAB-based Algorithm for task Recommendation System (CARS). Drivers bid on their preferred tasks in their list in a reverse auction fashion. Then e-Uber solves the task assignment optimization problem that minimizes cost and guarantees V2G energy requirement. We prove that this problem is NP-hard and introduce a bipartite matching-inspired heuristic, Bipartite Matching-based Winner selection (BMW), that has polynomial time complexity. Results from experiments using real data from NYC taxi trips and energy consumption show that e-Uber performs close to the optimum and finds better solutions compared to a state-of-the-art approachComment: Preprint, under revie