8 research outputs found
Deep reinforcement learning approach to solve dynamic vehicle routing problem with stochastic customers
National Research Foundation (NRF) Singapore under Corp Lab @ University scheme; Fujitsu Lt
SOLO: Search Online, Learn Offline for Combinatorial Optimization Problems
We study combinatorial problems with real world applications such as machine
scheduling, routing, and assignment. We propose a method that combines
Reinforcement Learning (RL) and planning. This method can equally be applied to
both the offline, as well as online, variants of the combinatorial problem, in
which the problem components (e.g., jobs in scheduling problems) are not known
in advance, but rather arrive during the decision-making process. Our solution
is quite generic, scalable, and leverages distributional knowledge of the
problem parameters. We frame the solution process as an MDP, and take a Deep
Q-Learning approach wherein states are represented as graphs, thereby allowing
our trained policies to deal with arbitrary changes in a principled manner.
Though learned policies work well in expectation, small deviations can have
substantial negative effects in combinatorial settings. We mitigate these
drawbacks by employing our graph-convolutional policies as non-optimal
heuristics in a compatible search algorithm, Monte Carlo Tree Search, to
significantly improve overall performance. We demonstrate our method on two
problems: Machine Scheduling and Capacitated Vehicle Routing. We show that our
method outperforms custom-tailored mathematical solvers, state of the art
learning-based algorithms, and common heuristics, both in computation time and
performance
Digital twin applications in urban logistics:an overview
Urban traffic attributed to commercial and industrial transportation is observed to largely affect living standards in cities due to external factors like pollution and congestion. To counter this, smart cities deploy technologies such as digital twins (DT)s to achieve sustainability. Research suggests that DTs can be beneficial in optimizing the physical systems they are linked with. The concept has been extensively studied in many technology-driven industries like manufacturing. However, little work has been done with regards to their application in urban logistics. In this paper, we seek to provide a framework by which DTs could be easily adapted to urban logistics applications. To do this, we survey previous research on DT applications in urban logistics as we found that a holistic overview is lacking. Using this knowledge in combination with the identification of key factors in urban logistics, we produce a conceptual model for the general design of an urban logistics DT through a knowledge graph. We provide an illustration on how the conceptual model can be used in solving a relevant problem and showcase the integration of relevant DDO methods. We finish off with a discussion on research opportunities and challenges based on previous research and our practical experience
Digital twin applications in urban logistics:an overview
Urban traffic attributed to commercial and industrial transportation is observed to largely affect living standards in cities due to external factors like pollution and congestion. To counter this, smart cities deploy technologies such as digital twins (DT)s to achieve sustainability. Research suggests that DTs can be beneficial in optimizing the physical systems they are linked with. The concept has been extensively studied in many technology-driven industries like manufacturing. However, little work has been done with regards to their application in urban logistics. In this paper, we seek to provide a framework by which DTs could be easily adapted to urban logistics applications. To do this, we survey previous research on DT applications in urban logistics as we found that a holistic overview is lacking. Using this knowledge in combination with the identification of key factors in urban logistics, we produce a conceptual model for the general design of an urban logistics DT through a knowledge graph. We provide an illustration on how the conceptual model can be used in solving a relevant problem and showcase the integration of relevant DDO methods. We finish off with a discussion on research opportunities and challenges based on previous research and our practical experience
Dynamic service of geographically dispersed time-sensitive demands
This paper presents a new framework that models the novel dynamic vehicle dispatch problem with holding costs (DVDPHC), which focuses on serving stochastic demands at geographically dispersed locations in a timely manner. This framework is applicable, among others, to the post-disaster ambulance bus routing problem, where an ambulance bus must pick up (urgent) patients at geographically dispersed locations and bring them to a centrally-located hospital as quickly as possible. Solving the DVDPHC requires a dynamic decision-making rule at each decision moment for which demands to serve at the current location, and where to direct the vehicle next. We propose a heuristic based on approximate dynamic programming combined with a neural network (ADP-NN) for effectively solving the DVDPHC. Numerical experiments demonstrate that our proposed method is fast, scalable and robust. Furthermore, it keeps up with computationally heavy direct lookahead (DLA) benchmarks on 120 large representative instances, achieving on average 12.77% total cost improvement. Numerical analysis also reveals that our proposed method exhibits complex self-learned flexible behavior, such as waiting near locations in anticipation of new demand