Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning

A fundamental question in any peer-to-peer ridesharing system is how to, both effectively and efficiently, dispatch user's ride requests to the right driver in real time. Traditional rule-based solutions usually work on a simplified problem setting, which requires a sophisticated hand-crafted weight design for either centralized authority control or decentralized multi-agent scheduling systems. Although recent approaches have used reinforcement learning to provide centralized combinatorial optimization algorithms with informative weight values, their single-agent setting can hardly model the complex interactions between drivers and orders. In this paper, we address the order dispatching problem using multi-agent reinforcement learning (MARL), which follows the distributed nature of the peer-to-peer ridesharing problem and possesses the ability to capture the stochastic demand-supply dynamics in large-scale ridesharing scenarios. Being more reliable than centralized approaches, our proposed MARL solutions could also support fully distributed execution through recent advances in the Internet of Vehicles (IoV) and the Vehicle-to-Network (V2N). Furthermore, we adopt the mean field approximation to simplify the local interactions by taking an average action among neighborhoods. The mean field approximation is capable of globally capturing dynamic demand-supply variations by propagating many local interactions between agents and the environment. Our extensive experiments have shown the significant improvements of MARL order dispatching algorithms over several strong baselines on the gross merchandise volume (GMV), and order response rate measures. Besides, the simulated experiments with real data have also justified that our solution can alleviate the supply-demand gap during the rush hours, thus possessing the capability of reducing traffic congestion.Comment: 11 pages, 9 figure

Stochastic equilibrium models for generation capacity expansion

Capacity expansion models in the power sector were among the first applications of operations research to the industry. The models lost some of their appeal at the inception of restructuring even though they still offer a lot of possibilities and are in many respect irreplaceable provided they are adapted to the new environment. We introduce stochastic equilibrium versions of these models that we believe provide a relevant context for looking at the current very risky market where the power industry invests and operates. We then take up different questions raised by the new environment. Some are due to developments of the industry like demand side management: an optimization framework has difficulties accommodating them but the more general equilibrium paradigm offers additional possibilities. We then look at the insertion of risk related investment practices that developed with the new environment and may not be easy to accommodate in an optimization context. Specifically we consider the use of plant specific discount rates that we derive by including stochastic discount rates in the equilibrium model. Linear discount factors only price systematic risk. We therefore complete the discussion by inserting different risk functions (for different agents) in order to account for additional unpriced idiosyncratic risk in investments. These different models can be cast in a single mathematical representation but they do not have the same mathematical properties. We illustrate the impact of these phenomena on a small but realistic example.capacity adequacy, risk functions, stochastic equilibrium models, stochastic discount factors

Preventing Strategic Manipulation in Iterative Auctions: Proxy Agents and Price-Adjustment

Iterative auctions have many computational advantages over sealed-bid auctions, but can present new possibilities for strategic manipulation. We propose a two-stage technique to make iterative auctions that compute optimal allocations with myopic best-response bidding strategies more robust to manipulation. First, introduce proxy bidding agents to constrain bidding strategies to (possibly untruthful) myopic bestresponse. Second, after the auction terminates adjust the prices towards those given in the Vickrey auction, a sealedbid auction in which truth-revelation is optimal. We present an application of this methodology to iBundle, an iterative combinatorial auction which gives optimal allocations for myopic best-response agents.Engineering and Applied Science

Real-time Container Transport Planning with Decision Trees based on Offline Obtained Optimal Solutions

Hinterland networks for container transportation require planning methods in order to increase efficiency and reliability of the inland road, rail and waterway connections. In this paper we aim to derive real-time decision rules for suitable allocations of containers to inland services by analysing the solution structure of a centralised optimisation method used offline on historic data. The decision tree can be used in a decision support system (DSS) for instantaneously allocating incoming orders to suitable services, without the need for continuous planning updates. Such a DSS is beneficial, as it is easy to implement in the current practice of container transportation. Earlier proposed centralised methods can find the optimal solution for the intermodal inland transportation problem in retrospect, but are not suitable when information becomes gradually available. The main contributions are threefold: firstly, a structured method for creating decision trees from optimal solutions is proposed. Secondly, an innovative method is used for obtaining multiple equivalent optimal solutions to prevent overfitting of the decision tree. And finally, a structured analysis of three error types is presented for assessing the quality of an obtained tree. A case study illustrates the method’s purpose by comparing the quality of the resulting plan with alternative methods

Algorithms for bundling and pricing trucking services: Deterministic and stochastic approaches

Bundling and pricing trucking services is an important strategic decision for carriers. This is helpful when they consider the incorporation of new businesses to their networks, look for economic and optimal operations, and develop revenue management strategies. Reverse combinatorial auctions for trucking services are real-world examples that illustrate the necessity of such strategies. In these auctions, a shipper asks carriers for quotes to serve combinations of lanes and the carriers have to bundle demand and price it properly. This dissertation explores several dimensions of the problem employing state-of-the-art analytical tools. These dimensions include: Truckload (TL) and less-than-truckload (LTL) operations, behavioral attributes driving the selection of trucking services, and consideration of deterministic and stochastic demand. Analytical tools include: advanced econometrics, network modeling, statistical network analysis, combinatorial optimization, and stochastic optimization. The dissertation is organized as follows. Chapter 1 introduces the problem and related concepts. Chapter 2 studies the attributes driving the selection of trucking services and proposes an econometric model to quantify the shipper willingness to pay using data from a discrete choice experiment. Chapter 3 proposes an algorithm for demand clustering in freight logistics networks using historical data from TL carriers. Chapter 4 develops an algorithmic approach for pricing and demand segmentation of bundles in TL combinatorial auctions. Chapter 5 expands the latter framework to consider stochastic demand. Chapter 6 uses an analytical approach to demonstrate the benefits of in-vehicle consolidation for LTL carriers. Finally, Chapter 7 proposes an algorithm for pricing and demand segmentation of bundles in LTL combinatorial auctions that accounts for stochastic demand. This research provides meaningful negotiation guidance for shippers and carriers, which is supported by quantitative methods. Likewise, numerical experiments demonstrate the benefits and efficiencies of the proposed algorithms, which are transportation modeling contributions

Operational research and simulation methods for autonomous ride-sourcing

Ride-sourcing platforms provide on-demand shared transport services by solving decision problems related to ride-matching and pricing. The anticipated commercialisation of autonomous vehicles could transform these platforms to fleet operators and broaden their decision-making by introducing problems such as fleet sizing and empty vehicle redistribution. These problems have been frequently represented in research using aggregated mathematical programs, and alternative practises such as agent-based models. In this context, this study is set at the intersection between operational research and simulation methods to solve the multitude of autonomous ride-sourcing problems. The study begins by providing a framework for building bespoke agent-based models for ride-sourcing fleets, derived from the principles of agent-based modelling theory, which is used to tackle the non-linear problem of minimum fleet size. The minimum fleet size problem is tackled by investigating the relationship of system parameters based on queuing theory principles and by deriving and validating a novel model for pickup wait times. Simulating the fleet function in different urban areas shows that ride-sourcing fleets operate queues with zero assignment times above the critical fleet size. The results also highlight that pickup wait times have a pivotal role in estimating the minimum fleet size in ride-sourcing operations, with agent-based modelling being a more reliable estimation method. The focus is then shifted to empty vehicle redistribution, where the omission of market structure and underlying customer acumen, compromises the effectiveness of existing models. As a solution, the vehicle redistribution problem is formulated as a non-linear convex minimum cost flow problem that accounts for the relationship of supply and demand of rides by assuming a customer discrete choice model and a market structure. An edge splitting algorithm is then introduced to solve a transformed convex minimum cost flow problem for vehicle redistribution. Results of simulated tests show that the redistribution algorithm can significantly decrease wait times and increase profits with a moderate increase in vehicle mileage. The study is concluded by considering the operational time-horizon decision problems of ride-matching and pricing at periods of peak travel demand. Combinatorial double auctions have been identified as a suitable alternative to surge pricing in research, as they maximise social welfare by relying on stated customer and driver valuations. However, a shortcoming of current models is the exclusion of trip detour effects in pricing estimates. The study formulates a shared-ride assignment and pricing algorithm using combinatorial double auctions to resolve the above problem. The model is reduced to the maximum weighted independent set problem, which is APX-hard. Therefore, a fast local search heuristic is proposed, producing solutions within 10\% of the exact approach for practical implementations.Open Acces