Optimizing transport logistics under uncertainty with simheuristics: concepts, review and trends

Background: Uncertainty conditions have been increasingly considered in optimization problems arising in real-life transportation and logistics activities. Generally, the analysis of complex systems in these non-deterministic environments is approached with simulation techniques. However, simulation is not an optimization tool. Hence, it must be combined with optimization methods when our goal is to: (i) minimize operating costs while guaranteeing a given quality of service; or (ii) maximize system performance using limited resources. When solving NP-hard optimization problems, the use of metaheuristics allows us to deal with large-scale instances in reasonable computation times. By adding a simulation layer to the metaheuristics, the methodology becomes a simheuristic, which allows the optimization element to solve scenarios under uncertainty. Methods: This paper reviews the indexed documents in Elsevier Scopus database of both initial as well as recent applications of simheuristics in the logistics and transportation field. The paper also discusses open research lines in this knowledge area. Results: The simheuristics approaches to solving NP-hard and large-scale combinatorial optimization problems under uncertainty scenarios are discussed, as they frequently appear in real-life applications in logistics and transportation activities. Conclusions: The way in which the different simheuristic components interact puts a special emphasis in the different stages that can contribute to make the approach more efficient from a computational perspective. There are several lines of research that are still open in the field of simheuristics.Peer ReviewedPostprint (published version

Problèmes de tournées en viabilité hivernale utilisant la prévision des volumes d’épandage

RÉSUMÉ : Cette thèse combine deux domaines de recherche différents appliqués au déneigement : la recherche opérationnelle et la science des données. La science des données a été utilisée pour développer un modèle de prédiction de quantité de sel et d’abrasif avec une méthodologie d’apprentissage machine; par la suite, ce modèle est pris en compte pour la confection des tournées de véhicules. La confection des tournées a été élaborée en utilisant des outils de la recherche opérationnelle, qui servent à optimiser les tournées en considérant plusieurs contraintes et en intégrant les données réelles. La thèse est le fruit d’une collaboration avec deux villes québécoises, Granby et Saint-Jean-sur- Richelieu. Elle traite une application réelle en viabilité hivernale, qui est l’opération d’épandage. Cette opération est une activité nécessaire, dont le but est d’assurer une meilleure circulation routière. Cependant, cela se réalise avec un coût économique et environnemental important. Par conséquent, la réduction de ce coût devient une grande préoccupation. Cette thèse contribue significativement aux opérations d’épandage : premièrement, nous prédisons la quantité nécessaire de sel et d’abrasif à épandre afin d’éviter le surépandage; deuxièmement, nous optimisons les tournées des opérations d’épandage en considérant la variation de la quantité. La première contribution de cette thèse consiste en un modèle de prédiction des quantités de sel et d’abrasif pour chaque segment de rue et pour chaque heure, en utilisant des algorithmes d’apprentissage machine. L’importance de cette contribution réside d’une part dans l’intégration des données géomatiques avec les données météo-routières, et d’autre part dans l’extraction des variables importantes (feature engineering) pour le modèle de prédiction. Plusieurs algorithmes d’apprentissage machine ont été évalués : (les forêts aléatoires, les arbres extrêmement aléatoires, les réseaux de neurones artificiels, Adaboost, Gradient Boosting Machine et XGBoost). Le modèle élaboré par XGBoost a réalisé une meilleure performance. Le modèle de prédiction permet non seulement de prédire les quantités de sel et d’abrasif nécessaires à épandre mais aussi, d’identifier les variables les plus importantes pour la prédiction. Cette information représente un outil de décision intéressant pour les gestionnaires. L’identification des variables importantes pourrait améliorer les opérations de déneigement. D’après les résultats trouvés, le facteur humain (conducteur) influence significativement la quantité d’épandage; donc, le contrôle de ce facteur peut améliorer considérablement ces opérations. La deuxième contribution introduit un nouveau problème dans la littérature : le problème de tournées de véhicules générales avec capacité dont la quantité de sel et d’abrasif dépend du temps. Le problème est basé sur l’hypothèse que le modèle de prédiction est capable de fournir la quantité d’épandage pour chaque segment et pour chaque heure avec une bonne précision. Le fait d’avoir cette information pour chaque heure et pour chaque segment de rue, introduit la notion du temps dépendant. Le nouveau problème est modélisé à l’aide d’une formulation mathématique sur le graphe original, ce qui présente un défi de modélisation. En effet, il est difficile d’associer des temps de début et de fin uniques à un arc ou à une arête. Une métaheuristique basée sur la stratégie de destruction et construction a été développée pour résoudre les grandes instances. La métaheuristique est inspirée de SISRs (Slack Induction by String Removals). Elle considère la demande dépendante du temps et la présence des arêtes par la méthode d’évaluation basée sur la programmation dynamique. De nouvelles instances ont été créées à partir des instances des problèmes de tournées de véhicules générales avec contrainte de capacité avec demande fixe. Elles ont été générées à partir de différents types de fonction dont la demande dépend du temps. La troisième contribution propose une nouvelle approche, dans le but de présenter le niveau de priorité des rues (la hiérarchie de service) sous forme d’une fonction linéaire dépendante du temps. Le problème présenté dans cette contribution concerne des tournées de véhicules générales hiérarchiques avec contrainte de capacité sous l’incertitude de la demande. Lorsque les données collectées ne permettent pas de développer un bon modèle de prédiction, la notion de demande dépendante du temps n’est plus valide. L’approche robuste a démontré une grande réussite pour traiter et résoudre les problèmes avec incertitude. Une métaheuristique robuste a été proposée pour résoudre les deux cas réels de Granby et de Saint-Jean-sur-Richelieu. La métaheuristique a été validée par un modèle mathématique sur les petites instances générées à partir des cas réels. La simulation de Monte Carlo a été utilisée pour évaluer les différentes solutions proposées. En outre, elle permet d’offrir aux gestionnaires un outil de décision pour comparer les différentes solutions robustes, et aussi pour comprendre le compromis entre le niveau de robustesse souhaité et d’autres mesures de performances (coût, risque, niveau de service).----------ABSTRACT : This thesis combines two different fields applied to winter road maintenance : operational research and data science. Data science was used to develop a prediction model for the quantity of salt and abrasive with a machine learning methodology, later this model is considered for building vehicles routing. This route planning was developed using operational research which seeks to optimize routes by looking at several constraints and by integrating real data. The thesis which is the fruit of a collaboration with two Canadian cities Granby and Saint-Jean-sur-Richelieu, deals with a real application in winter road maintenance which is the spreading operation. The spreading operation presents an activity necessary for winter road maintenance, in order to ensure better road traffic. However, this road safety comes with a significant economic and environmental cost, which creates a great concern in order to reduce the economic and environmental impact. This thesis contributes significantly in the spreading operations : firstly, predicting the necessary quantity of salt and abrasive to be spread in order to avoid over-spreading, secondly optimizing the spreading operations routes considering quantity variations. The first contribution of this thesis is to develop a prediction model for the quantities of salt and abrasive using machine learning algorithms, for each street segment and for each hour. The importance of this contribution lies in the integration of geomatic data with weather-road data, and also the feature engineering. Several machine learning algorithms were evaluated (Random Forest, Extremely Random Trees, Artificial Neural Networks, Adaboost, Gradient Boosting Machine and XGBoost); ultimately XGBoost performed better. The prediction model not only predicts the amounts of salt and abrasive needed to spread, but also identifies the most important variables in the model. This information presents an interesting decision-making tool for managers. The identification of important variables could improve snow removal operations. According to the results, the human factor (driver) significantly influences the amount of spreading, so controlling this factor can significantly improve the spreading operations.The second contribution introduces a new problem in the literature : the mixed capacitated general routing problem with time-dependent demand; the problem is based on the assumption that the prediction model is able to provide the amount of spreading for each segment and for each hour with good accuracy. Having this information for each hour and for each street segment introduces the concept of time dependency. The new problem was modeled using a mathematical formulation on the original graph, which presents a modeling challenge since it is difficult to associate a unique starting and ending time to an arc or edge. A meta-heuristic based on the destruction and construction strategy has been developed to solve large-scale instances. The meta-heuristic is inspired by SISRs considers time-dependent demand and the presence of edges by an evaluation method based on dynamic programming. New instances were created from the instances of the mixed capacitated general routing problem with fixed demand; the new instances were generated from different types of function where the demand varies with time. The third contribution proposes a new approach to present the service hierarchy or the priority level of streets, as a time-dependent linear function. The problem addressed in this contribution concerns the hierarchical mixed capacitated general routing problems under demand uncertainty. When the collected data does not allow the development of a good prediction model, the concept of time-dependent demand is no longer valid. The robust approach has demonstrated great success in resolving and dealing with problems with uncertainty. A robust meta-heuristic was proposed to solve the two real cases Granby and Saint-Jean-sur-Richelieu, the meta-heuristic was validated by a mathematical model on small instances generated from the real cases. The Monte Carlo simulation was used, on the one hand, to evaluate the different solutions proposed, and, on the other hand, to offer managers a decision tool to compare the different robust solutions and also to understand the trade-off between the desired level of robustness, and other performance measures (cost, risk, level of service)

Coalitional Bargaining via Reinforcement Learning: An Application to Collaborative Vehicle Routing

Collaborative Vehicle Routing is where delivery companies cooperate by sharing their delivery information and performing delivery requests on behalf of each other. This achieves economies of scale and thus reduces cost, greenhouse gas emissions, and road congestion. But which company should partner with whom, and how much should each company be compensated? Traditional game theoretic solution concepts, such as the Shapley value or nucleolus, are difficult to calculate for the real-world problem of Collaborative Vehicle Routing due to the characteristic function scaling exponentially with the number of agents. This would require solving the Vehicle Routing Problem (an NP-Hard problem) an exponential number of times. We therefore propose to model this problem as a coalitional bargaining game where - crucially - agents are not given access to the characteristic function. Instead, we implicitly reason about the characteristic function, and thus eliminate the need to evaluate the VRP an exponential number of times - we only need to evaluate it once. Our contribution is that our decentralised approach is both scalable and considers the self-interested nature of companies. The agents learn using a modified Independent Proximal Policy Optimisation. Our RL agents outperform a strong heuristic bot. The agents correctly identify the optimal coalitions 79% of the time with an average optimality gap of 4.2% and reduction in run-time of 62%.Comment: Accepted to NeurIPS 2021 Workshop on Cooperative A

DeepACO: Neural-enhanced Ant Systems for Combinatorial Optimization

Ant Colony Optimization (ACO) is a meta-heuristic algorithm that has been successfully applied to various Combinatorial Optimization Problems (COPs). Traditionally, customizing ACO for a specific problem requires the expert design of knowledge-driven heuristics. In this paper, we propose DeepACO, a generic framework that leverages deep reinforcement learning to automate heuristic designs. DeepACO serves to strengthen the heuristic measures of existing ACO algorithms and dispense with laborious manual design in future ACO applications. As a neural-enhanced meta-heuristic, DeepACO consistently outperforms its ACO counterparts on eight COPs using a single neural model and a single set of hyperparameters. As a Neural Combinatorial Optimization method, DeepACO performs better than or on par with problem-specific methods on canonical routing problems. Our code is publicly available at https://github.com/henry-yeh/DeepACO.Comment: Accepted at NeurIPS 202

Reinforcement Learning for Solving Stochastic Vehicle Routing Problem with Time Windows

This paper introduces a reinforcement learning approach to optimize the Stochastic Vehicle Routing Problem with Time Windows (SVRP), focusing on reducing travel costs in goods delivery. We develop a novel SVRP formulation that accounts for uncertain travel costs and demands, alongside specific customer time windows. An attention-based neural network trained through reinforcement learning is employed to minimize routing costs. Our approach addresses a gap in SVRP research, which traditionally relies on heuristic methods, by leveraging machine learning. The model outperforms the Ant-Colony Optimization algorithm, achieving a 1.73% reduction in travel costs. It uniquely integrates external information, demonstrating robustness in diverse environments, making it a valuable benchmark for future SVRP studies and industry application

Internet of Vehicles and Real-Time Optimization Algorithms: Concepts for Vehicle Networking in Smart Cities

Achieving sustainable freight transport and citizens’ mobility operations in modern cities are becoming critical issues for many governments. By analyzing big data streams generated through IoT devices, city planners now have the possibility to optimize traffic and mobility patterns. IoT combined with innovative transport concepts as well as emerging mobility modes (e.g., ridesharing and carsharing) constitute a new paradigm in sustainable and optimized traffic operations in smart cities. Still, these are highly dynamic scenarios, which are also subject to a high uncertainty degree. Hence, factors such as real-time optimization and re-optimization of routes, stochastic travel times, and evolving customers’ requirements and traffic status also have to be considered. This paper discusses the main challenges associated with Internet of Vehicles (IoV) and vehicle networking scenarios, identifies the underlying optimization problems that need to be solved in real time, and proposes an approach to combine the use of IoV with parallelization approaches. To this aim, agile optimization and distributed machine learning are envisaged as the best candidate algorithms to develop efficient transport and mobility systems

On the inventory routing problem with stationary stochastic demand rate

One of the most significant paradigm shifts of present business management is that individual businesses no longer participate as solely independent entities, but rather as supply chains (Lambert and Cooper, 2000). Therefore, the management of multiple relationships across the supply chain such as flow of materials, information, and finances is being referred to as supply chain management (SCM). SCM involves coordinating and integrating these multiple relationships within and among companies, so that it can improve the global performance of the supply chain. In this dissertation, we discuss the issue of integrating the two processes in the supply chain related, respectively, to inventory management and routing policies. The challenging problem of coordinating the inventory management and transportation planning decisions in the same time, is known as the inventory routing problem (IRP). The IRP is one of the challenging optimization problems in logis-tics and supply chain management. It aims at optimally integrating inventory control and vehicle routing operations in a supply network. In general, IRP arises as an underlying optimization problem in situations involving simultaneous optimization of inventory and distribution decisions. Its main goal is to determine an optimal distribution policy, consisting of a set of vehicle routes, delivery quantities and delivery times that minimizes the total inventory holding and transportation costs. This is a typical logistical optimization problem that arises in supply chains implementing a vendor managed inventory (VMI) policy. VMI is an agreement between a supplier and his regular retailers according to which retailers agree to the alternative that the supplier decides the timing and size of the deliveries. This agreement grants the supplier the full authority to manage inventories at his retailers'. This allows the supplier to act proactively and take responsibility for the inventory management of his regular retailers, instead of reacting to the orders placed by these retailers. In practice, implementing policies such as VMI has proven to considerably improve the overall performance of the supply network, see for example Lee and Seungjin (2008), Andersson et al. (2010) and Coelho et al. (2014). This dissertation focuses mainly on the single-warehouse, multiple-retailer (SWMR) system, in which a supplier serves a set of retailers from a single warehouse. In the first situation, we assume that all retailers face a deterministic, constant demand rate and in the second condition, we assume that all retailers consume the product at a stochastic stationary rate. The primary objective is to decide when and how many units to be delivered from the supplier to the warehouse and from the warehouse to retailers so as to minimize total transportation and inventory holding costs over the finite horizon without any shortages