Reinforcement Learning for Scalable Train Timetable Rescheduling with Graph Representation

Train timetable rescheduling (TTR) aims to promptly restore the original operation of trains after unexpected disturbances or disruptions. Currently, this work is still done manually by train dispatchers, which is challenging to maintain performance under various problem instances. To mitigate this issue, this study proposes a reinforcement learning-based approach to TTR, which makes the following contributions compared to existing work. First, we design a simple directed graph to represent the TTR problem, enabling the automatic extraction of informative states through graph neural networks. Second, we reformulate the construction process of TTR's solution, not only decoupling the decision model from the problem size but also ensuring the generated scheme's feasibility. Third, we design a learning curriculum for our model to handle the scenarios with different levels of delay. Finally, a simple local search method is proposed to assist the learned decision model, which can significantly improve solution quality with little additional computation cost, further enhancing the practical value of our method. Extensive experimental results demonstrate the effectiveness of our method. The learned decision model can achieve better performance for various problems with varying degrees of train delay and different scales when compared to handcrafted rules and state-of-the-art solvers

An Integrated Framework Integrating Monte Carlo Tree Search and Supervised Learning for Train Timetabling Problem

The single-track railway train timetabling problem (TTP) is an important and complex problem. This article proposes an integrated Monte Carlo Tree Search (MCTS) computing framework that combines heuristic methods, unsupervised learning methods, and supervised learning methods for solving TTP in discrete action spaces. This article first describes the mathematical model and simulation system dynamics of TTP, analyzes the characteristics of the solution from the perspective of MCTS, and proposes some heuristic methods to improve MCTS. This article considers these methods as planners in the proposed framework. Secondly, this article utilizes deep convolutional neural networks to approximate the value of nodes and further applies them to the MCTS search process, referred to as learners. The experiment shows that the proposed heuristic MCTS method is beneficial for solving TTP; The algorithm framework that integrates planners and learners can improve the data efficiency of solving TTP; The proposed method provides a new paradigm for solving TTP

Solving the single-track train scheduling problem via Deep Reinforcement Learning

Every day, railways experience small inconveniences, both on the network and the fleet side, affecting the stability of rail traffic. When a disruption occurs, delays propagate through the network, resulting in demand mismatching and, in the long run, demand loss. When a critical situation arises, human dispatchers distributed over the line have the duty to do their best to minimize the impact of the disruptions. Unfortunately, human operators have a limited depth of perception of how what happens in distant areas of the network may affect their control zone. In recent years, decision science has focused on developing methods to solve the problem automatically, to improve the capabilities of human operators. In this paper, machine learning-based methods are investigated when dealing with the train dispatching problem. In particular, two different Deep Q-Learning methods are proposed. Numerical results show the superiority of these techniques respect to the classical linear Q-Learning based on matrices.Comment: 12 pages, 4 figures (2 b&w

Hybrid railway vehicle trajectory optimisation using a non‐convex function and evolutionary hybrid forecast algorithm

AbstractThis paper introduces a novel optimisation algorithm for hybrid railway vehicles, combining a non‐linear programming solver with the highly efficient “Mayfly Algorithm” to address a non‐convex optimisation problem. The primary objective is to generate efficient trajectories that enable effective power distribution, optimal energy consumption, and economical use of multiple onboard power sources. By reducing unnecessary load stress on power sources during peak time, the algorithm contributes to lower maintenance costs, reduced downtime, and extended operational life of these sources. The algorithm's design considers various operational parameters, such as power demand, regenerative braking, velocity and additional power requirements, enabling it to optimise the energy consumption profile throughout the journey. Its adaptability to the unique characteristics of hybrid railway vehicles allows for efficient energy management by leveraging its hybrid powertrain capabilities.</jats:p

Interactive reinforcement learning innovation to reduce carbon emissions in railway infrastructure maintenance

Carbon emission is one of the primary contributors to global warming. The global community is paying great attention to this negative impact. The goal of this study is to reduce the negative impact of railway maintenance by applying reinforcement learning (RL) by optimizing maintenance activities. Railway maintenance is a complex process that may not be efficient in terms of environmental aspect. This study is the world's first to use the potential of RL to reduce carbon emission from railway maintenance. The data used to create the RL model are gathered from the field data between 2016–019. The study section is 30 km long. Proximal Policy Optimization (PPO) is applied in the study to develop the RL model. The results demonstrate that using RL reduces carbon emission from railway maintenance by 48%, which generates a considerable amount of carbon emission reduction and reduces railway defects by 68%, which also improves maintenance efficiency significantly