A dynamic approach to rebalancing bike-sharing systems

Bike-sharing services are flourishing in Smart Cities worldwide. They provide a low-cost and environment-friendly transportation alternative and help reduce traffic congestion. However, these new services are still under development, and several challenges need to be solved. A major problem is the management of rebalancing trucks in order to ensure that bikes and stalls in the docking stations are always available when needed, despite the fluctuations in the service demand. In this work, we propose a dynamic rebalancing strategy that exploits historical data to predict the network conditions and promptly act in case of necessity. We use Birth-Death Processes to model the stations' occupancy and decide when to redistribute bikes, and graph theory to select the rebalancing path and the stations involved. We validate the proposed framework on the data provided by New York City's bike-sharing system. The numerical simulations show that a dynamic strategy able to adapt to the fluctuating nature of the network outperforms rebalancing schemes based on a static schedule

The multiple vehicle balancing problem

This paper deals with the multiple vehicle balancing problem (MVBP). Given a fleet of vehicles of limited capacity, a set of vertices with initial and target inventory levels and a distribution network, the MVBP requires to design a set of routes along with pickup and delivery operations such that inventory is redistributed among the vertices without exceeding capacities, and routing costs are minimized. The MVBP is NP\u2010hard, generalizing several problems in transportation, and arising in bike\u2010sharing systems. Using theoretical properties of the problem, we propose an integer linear programming formulation and introduce strengthening valid inequalities. Lower bounds are computed by column generation embedding an ad\u2010hoc pricing algorithm, while upper bounds are obtained by a memetic algorithm that separate routing from pickup and delivery operations. We combine these bounding routines in both exact and matheuristic algorithms, obtaining proven optimal solutions for MVBP instances with up to 25 stations

실시간 동적 계획법 및 강화학습 기반의 공공자전거 시스템의 동적 재배치 전략

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 건설환경공학부, 2020. 8. 고승영.The public bicycle sharing system is one of the modes of transportation that can help to relieve several urban problems, such as traffic congestion and air pollution. Because users can pick up and return bicycles anytime and anywhere a station is located, pickup or return failure can occur due to the spatiotemporal imbalances in demand. To prevent system failures, the operator should establish an appropriate repositioning strategy. As the operator makes a decision based on the predicted demand information, the accuracy of forecasting demand is an essential factor. Due to the stochastic nature of demand, however, the occurrence of prediction errors is inevitable. This study develops a stochastic dynamic model that minimizes unmet demand for rebalancing public bicycle sharing systems, taking into account the stochastic demand and the dynamic characteristics of the system. Since the repositioning mechanism corresponds to the sequential decision-making problem, this study applies the Markov decision process to the problem. To solve the Markov decision process, a dynamic programming method, which decomposes complex problems into simple subproblems to derive an exact solution. However, as a set of states and actions of the Markov decision process become more extensive, the computational complexity increases and it is intractable to derive solutions. An approximate dynamic programming method is introduced to derive an approximate solution. Further, a reinforcement learning model is applied to obtain a feasible solution in a large-scale public bicycle network. It is assumed that the predicted demand is derived from the random forest, which is a kind of machine learning technique, and that the observed demand occurred along the Poisson distribution whose mean is the predicted demand to simulate the uncertainty of the future demand. Total unmet demand is used as a key performance indicator in this study. In this study, a repositioning strategy that quickly responds to the prediction error, which means the difference between the observed demand and the predicted demand, is developed and the effectiveness is assessed. Strategies developed in previous studies or applied in the field are also modeled and compared with the results to verify the effectiveness of the strategy. Besides, the effects of various safety buffers and safety stock are examined and appropriate strategies are suggested for each situation. As a result of the analysis, the repositioning effect by the developed strategy was improved compared to the benchmark strategies. In particular, the effect of a strategy focusing on stations with high prediction errors is similar to the effect of a strategy considering all stations, but the computation time can be further reduced. Through this study, the utilization and reliability of the public bicycle system can be improved through the efficient operation without expanding the infrastructure.공공자전거 시스템은 교통혼잡과 대기오염 등 여러 도시문제를 완화할 수 있는 교통수단이다. 대여소가 위치한 곳이면 언제 어디서든 이용자가 자전거를 이용할 수 있는 시스템의 특성상 수요의 시공간적 불균형으로 인해 대여 실패 또는 반납 실패가 발생한다. 시스템 실패를 예방하기 위해 운영자는 적절한 재배치 전략을 수립해야 한다. 운영자는 예측 수요 정보를 전제로 의사결정을 하므로 수요예측의 정확성이 중요한 요소이나, 수요의 불확실성으로 인해 예측 오차의 발생이 불가피하다. 본 연구의 목적은 공공자전거 수요의 불확실성과 시스템의 동적 특성을 고려하여 불만족 수요를 최소화하는 재배치 모형을 개발하는 것이다. 공공자전거 재배치 메커니즘은 순차적 의사결정 문제에 해당하므로, 본 연구에서는 순차적 의사결정 문제를 모형화할 수 있는 마르코프 결정 과정을 적용한다. 마르코프 결정 과정을 풀기 위해 복잡한 문제를 간단한 부문제로 분해하여 정확해를 도출하는 동적 계획법을 이용한다. 하지만 마르코프 결정 과정의 상태 집합과 결정 집합의 크기가 커지면 계산 복잡도가 증가하므로, 동적 계획법을 이용한 정확해를 도출할 수 없다. 이를 해결하기 위해 근사적 동적 계획법을 도입하여 근사해를 도출하며, 대규모 공공자전거 네트워크에서 가능해를 얻기 위해 강화학습 모형을 적용한다. 장래 공공자전거 이용수요의 불확실성을 모사하기 위해, 기계학습 기법의 일종인 random forest로 예측 수요를 도출하고, 예측 수요를 평균으로 하는 포아송 분포를 따라 수요를 확률적으로 발생시켰다. 본 연구에서는 관측 수요와 예측 수요 간의 차이인 예측오차에 빠르게 대응하는 재배치 전략을 개발하고 효과를 평가한다. 개발된 전략의 우수성을 검증하기 위해, 기존 연구의 재배치 전략 및 현실에서 적용되는 전략을 모형화하고 결과를 비교한다. 또한, 재고량의 안전 구간 및 안전재고량에 관한 민감도 분석을 수행하여 함의점을 제시한다. 개발된 전략의 효과를 분석한 결과, 기존 연구의 전략 및 현실에서 적용되는 전략보다 개선된 성능을 보이며, 특히 예측오차가 큰 대여소를 탐색하는 전략이 전체 대여소를 탐색하는 전략과 재배치 효과가 유사하면서도 계산시간을 절감할 수 있는 것으로 나타났다. 공공자전거 인프라를 확대하지 않고도 운영의 효율화를 통해 공공자전거 시스템의 이용률 및 신뢰성을 제고할 수 있고, 공공자전거 재배치에 관한 정책적 함의점을 제시한다는 점에서 본 연구의 의의가 있다.Chapter 1. Introduction １ 1.1 Research Background and Purposes １ 1.2 Research Scope and Procedure ７ Chapter 2. Literature Review １０ 2.1 Vehicle Routing Problems １０ 2.2 Bicycle Repositioning Problem １２ 2.3 Markov Decision Processes ２３ 2.4 Implications and Contributions ２６ Chapter 3. Model Formulation ２８ 3.1 Problem Definition ２８ 3.2 Markov Decision Processes ３４ 3.3 Demand Forecasting ４０ 3.4 Key Performance Indicator (KPI) ４５ Chapter 4. Solution Algorithms ４７ 4.1 Exact Solution Algorithm ４７ 4.2 Approximate Dynamic Programming ５０ 4.3 Reinforcement Learning Method ５２ Chapter 5. Numerical Example ５５ 5.1 Data Overview ５５ 5.2 Experimental Design ６１ 5.3 Algorithm Performance ６６ 5.4 Sensitivity Analysis ７４ 5.5 Large-scale Cases ７６ Chapter 6. Conclusions ８２ 6.1 Conclusions ８２ 6.2 Future Research ８３ References ８６ 초 록 ９２Docto

An exact algorithm for the static rebalancing problem arising in bicycle sharing systems

Bicycle sharing systems can significantly reduce traffic, pollution, and the need for parking spaces in city centers. One of the keys to success for a bicycle sharing system is the efficiency of rebalancing operations, where the number of bicycles in each station has to be restored to its target value by a truck through pickup and delivery operations. The Static Bicycle Rebalancing Problem aims to determine a minimum cost sequence of stations to be visited by a single vehicle as well as the amount of bicycles to be collected or delivered at each station. Multiple visits to a station are allowed, as well as using stations as temporary storage. This paper presents an exact algorithm for the problem and results of computational tests on benchmark instances from the literature. The computational experiments show that instances with up to 60 stations can be solved to optimality within 2 hours of computing time

A heuristic algorithm for a single vehicle static bike sharing rebalancing problem

The static bike rebalancing problem (SBRP) concerns the task of repositioning bikes among stations in self-service bike-sharing systems. This problem can be seen as a variant of the one-commodity pickup and delivery vehicle routing problem, where multiple visits are allowed to be performed at each station, i.e., the demand of a station is allowed to be split. Moreover, a vehicle may temporarily drop its load at a station, leaving it in excess or, alternatively, collect more bikes from a station (even all of them), thus leaving it in default. Both cases require further visits in order to meet the actual demands of such station. This paper deals with a particular case of the SBRP, in which only a single vehicle is available and the objective is to find a least-cost route that meets the demand of all stations and does not violate the minimum (zero) and maximum (vehicle capacity) load limits along the tour. Therefore, the number of bikes to be collected or delivered at each station must be appropriately determined in order to respect such constraints. We propose an iterated local search (ILS) based heuristic to solve the problem. The ILS algorithm was tested on 980 benchmark instances from the literature and the results obtained are competitive when compared to other existing methods. Moreover, our heuristic was capable of finding most of the known optimal solutions and also of improving the results on a number of open instances

A Study of the Static Bicycle Reposition Problem with a Single Vehicle

The Bicycle Sharing System (BSS), a public service system operated by the government or a private company, provides the convenient use of a bicycle as a temporary method of transportation. More specifically, this system allows people to rent a bike from one location, use it for a short time period and then return it to either to the same or a different location for an inexpensive fee. With the development of IT technology in the 1990s, it became possible to balance the bicycle inventory among the various destinations. In fact, a critical aspect to maintaining a satisfactory BSS is effectively rebalancing bicycle inventory across the various stations. In this research, we focus on the static bicycle repositioning problem with a single vehicle which is abstracted from the operation issue in the bicycle sharing system. The mathematical model for the static bicycle reposition problem had been created and several variations had been analyzed. This research starts to solve the problem from a very restrictive and constrained model and relaxes the constraints step by step to approach the real world case scenario. Several realistic assumptions have been considered in our research, such as a limited working time horizon, multiple visit limitation for the same station, multiple trips used for the vehicle, etc. In this research, we use the variable neighborhood search heuristic algorithm as the basic structure to find the solution for the static bicycle reposition problem. The numeric results indicate that our algorithms can provide good quality result within short solving time. By solving such a problem well, in comparison to benchmark algorithms, this research provides a starting place for dynamic bicycle repositioning and multiple vehicle repositioning

Rebalancing shared mobility systems by user incentive scheme via reinforcement learning

Shared mobility systems regularly suffer from an imbalance of vehicle supply within the system, leading to users being unable to receive service. If such imbalance problems are not mitigated some users will not be serviced. There is an increasing interest in the use of reinforcement learning (RL) techniques for improving the resource supply balance and service level of systems. The goal of these techniques is to produce an effective user incentivization policy scheme to encourage users of a shared mobility system to slightly alter their travel behavior in exchange for a small monetary incentive. These slight changes in user behavior are intended to over time increase the service level of the shared mobility system and improve user experience. In this thesis, two important questions are explored: (1) What state-action representation should be used to produce an effective user incentive scheme for a shared mobility system? (2) How effective are reinforcement learning-based solutions on the rebalancing problem under varying levels of resource supply, user demand, and budget? Our extensive empirical results based on data-driven simulation show that: 1. A state space with predicted user behavior coupled with a simple action mechanism produces an effective incentive scheme under varying environment scenarios. 2. The reinforcement learning-based incentive mechanisms perform at varying degrees of effectiveness under different environmental scenarios in terms of service level

A multiple type bike repositioning problem

This paper investigates a new static bicycle repositioning problem in which multiple types of bikes are considered. Some types of bikes that are in short supply at a station can be substituted by other types, whereas some types of bikes can occupy the spaces of other types in the vehicle during repositioning. These activities provide two new strategies, substitution and occupancy, which are examined in this paper. The problem is formulated as a mixed-integer linear programming problem to minimize the total cost, which consists of the route travel cost, penalties due to unmet demand, and penalties associated with the substitution and occupancy strategies. A combined hybrid genetic algorithm is proposed to solve this problem. This solution algorithm consists of (i) a modified version of a hybrid genetic search with adaptive diversity control to determine routing decisions and (ii) a proposed greedy heuristic to determine the loading and unloading instructions at each visited station and the substitution and occupancy strategies. The results show that the proposed method can provide high-quality solutions with short computing times. Using small examples, this paper also reveals problem properties and repositioning strategies in bike sharing systems with multiple types of bikes.published_or_final_versio

Solving a static repositioning problem in bike-sharing systems using iterated tabu search

In this paper, we study the static bike repositioning problem where the problem consists of selecting a subset of stations to visit, sequencing them, and determining the pick-up/drop-off quantities (associated with each of the visited stations) under the various operational constraints. The objective is to minimize the total penalties incurred at all the stations. We present an iterated tabu search heuristic to solve the described problem. Experimental results show that this simple heuristic can generate high quality solutions using small computing times.postprin