Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach

The dynamic vehicle dispatching problem corresponds to deciding which vehicles to assign to requests that arise stochastically over time and space. It emerges in diverse areas, such as in the assignment of trucks to loads to be transported; in emergency systems; and in ride-hailing services. In this paper, we model the problem as a semi-Markov decision process, which allows us to treat time as continuous. In this setting, decision epochs coincide with discrete events whose time intervals are random. We argue that an event-based approach substantially reduces the combinatorial complexity of the decision space and overcomes other limitations of discrete-time models often proposed in the literature. In order to test our approach, we develop a new discrete-event simulator and use double deep q-learning to train our decision agents. Numerical experiments are carried out in realistic scenarios using data from New York City. We compare the policies obtained through our approach with heuristic policies often used in practice. Results show that our policies exhibit better average waiting times, cancellation rates and total service times, with reduction in average waiting times of up to 50% relative to the other tested heuristic policies.Comment: 42 pages, 22 figure

Hybrid multiobjective optimization model for manufacturing cell formation

O objetivo deste trabalho é propor um procedimento híbrido para a solução do problema de formação de células de manufatura com réplicas de máquinas. Constrói-se um modelo matemático de otimização multiobjetivo cujos valores das funções-objetivo são obtidos por meio da execução de um modelo de simulação de eventos discretos, o qual representa um sistema de manufatura celular. Em seguida, geram-se soluções eficientes segundo o conceito de otimalidade de Pareto através de um processo de busca por valores ótimos executado por um algoritmo genético. Três funções-objetivo conflitantes são consideradas: inventário em processo, movimentação intercelular e investimento total em máquinas. Um algoritmo de análise de agrupamento é utilizado para a redução do conjunto final de soluções. A eficácia do procedimento é avaliada mediante a aplicação a dois casos da literatura. Os resultados obtidos são analisados e comentados. Conclui-se, por fim, que o procedimento é capaz de gerar um conjunto de configurações sub-ótimas equivalentes para as células de manufatura, representando aproximadamente os trade-offs entre as três funções-objetivo.The purpose of this work is to propose a hybrid procedure for solving the manufacturing cell formation problem. A multiobjective optimization model is built whose objective function values are realized by running a discrete-event simulation model, which represents a cellular manufacturing system. Thereafter, efficient solutions are generated following the Pareto optimality concept through a search for optimum values carried out by a genetic algorithm. Three conflicting objective functions are considered, namely, work-in-process, intercell moves and total machine investment. A clustering algorithm is applied to the final solution set so as to reduce it. The procedure efficacy is evaluated via its application to two cases from the literature. The obtained results are analyzed and commented. Finally, it is concluded that the procedure is capable of generating a set of equivalent sub-optimal manufacturing cell configurations, representing approximately the trade-offs betvveen the objective functions adopted

Hybrid multiobjective optimization model for manufacturing cell formation

O objetivo deste trabalho é propor um procedimento híbrido para a solução do problema de formação de células de manufatura com réplicas de máquinas. Constrói-se um modelo matemático de otimização multiobjetivo cujos valores das funções-objetivo são obtidos por meio da execução de um modelo de simulação de eventos discretos, o qual representa um sistema de manufatura celular. Em seguida, geram-se soluções eficientes segundo o conceito de otimalidade de Pareto através de um processo de busca por valores ótimos executado por um algoritmo genético. Três funções-objetivo conflitantes são consideradas: inventário em processo, movimentação intercelular e investimento total em máquinas. Um algoritmo de análise de agrupamento é utilizado para a redução do conjunto final de soluções. A eficácia do procedimento é avaliada mediante a aplicação a dois casos da literatura. Os resultados obtidos são analisados e comentados. Conclui-se, por fim, que o procedimento é capaz de gerar um conjunto de configurações sub-ótimas equivalentes para as células de manufatura, representando aproximadamente os trade-offs entre as três funções-objetivo.The purpose of this work is to propose a hybrid procedure for solving the manufacturing cell formation problem. A multiobjective optimization model is built whose objective function values are realized by running a discrete-event simulation model, which represents a cellular manufacturing system. Thereafter, efficient solutions are generated following the Pareto optimality concept through a search for optimum values carried out by a genetic algorithm. Three conflicting objective functions are considered, namely, work-in-process, intercell moves and total machine investment. A clustering algorithm is applied to the final solution set so as to reduce it. The procedure efficacy is evaluated via its application to two cases from the literature. The obtained results are analyzed and commented. Finally, it is concluded that the procedure is capable of generating a set of equivalent sub-optimal manufacturing cell configurations, representing approximately the trade-offs betvveen the objective functions adopted

A simulation-based evolutionary multiobjective approach to manufacturing cell formation

The purpose of this paper is to propose a multiobjective optimization approach for solving the manufacturing cell formation problem, explicitly considering the performance of this said manufacturing system. Cells are formed so as to simultaneously minimize three conflicting objectives, namely, the level of the work-in-process, the intercell moves and the total machinery investment. A genetic algorithm performs a search in the design space, in order to approximate to the Pareto optimal set. The values of the objectives for each candidate solution in a population are assigned by running a discrete-event simulation, in which the model is automatically generated according to the number of machines and their distribution among cells implied by a particular solution. The potential of this approach is evaluated via its application to an illustrative example, and a case from the relevant literature. The obtained results are analyzed and reviewed. Therefore, it is concluded that this approach is capable of generating a set of alternative manufacturing cell configurations considering the optimization of multiple performance measures, greatly improving the decision making process involved in planning and designing cellular systems. (C) 2010 Elsevier Ltd. All rights reserved.CAPE

Statistical models for the estimation of the origin-destination matrix from traffic counts

In transportation planning, one of the first steps is to estimate the travel demand. The final product of the estimation process is an origin-destination (OD) matrix, whose entries correspond to the number of trips between pairs of origin-destination zones in a study region. In this paper, we review the main statistical models proposed in the literature for the estimation of the OD matrix based on traffic counts. Unlike reconstruction models, statistical models do not aim at estimating the exact OD matrix corresponding to observed traffic volumes, but they rather aim at estimating the parameters of a statistical model of the population of OD matrices. Initially we define the estimation problem, emphasizing its underspecified nature, which has lead to the development of several models based on different approaches. We describe static models whose parameters are estimated by means of maximum likelihood, the method of moments, and Bayesian inference. We also describe  some recent dynamic models. Following that, we discuss research questions related to the underspecification problem, model assumptions and the estimation of the route choice matrix, and indicate promising research directions

Bayesian Inference on Dynamic Linear Models of Day-to-Day Origin-Destination Flows in Transportation Networks

Estimation of origin⁻destination (OD) demand plays a key role in successful transportation studies. In this paper, we consider the estimation of time-varying day-to-day OD flows given data on traffic volumes in a transportation network for a sequence of days. We propose a dynamic linear model (DLM) in order to represent the stochastic evolution of OD flows over time. DLMs are Bayesian state-space models which can capture non-stationarity. We take into account the hierarchical relationships between the distribution of OD flows among routes and the assignment of traffic volumes on links. Route choice probabilities are obtained through a utility model based on past route costs. We propose a Markov chain Monte Carlo algorithm, which integrates Gibbs sampling and a forward filtering backward sampling technique, in order to approximate the joint posterior distribution of mean OD flows and parameters of the route choice model. Our approach can be applied to congested networks and in the case when data are available on only a subset of links. We illustrate the application of our approach through simulated experiments on a test network from the literature

Discussão conceitual sobre métodos de reconstrução de matrizes origem-destino estáticas em redes de transportes

Matrizes origem-destino (OD) quantificam a demanda por transporte em uma região geográfica e constituem peças-chave em estudos de planejamento, projeto e gerenciamento de sistemas de transportes urbanos e regionais. Tradicionalmente, matrizes OD são obtidas por meio da realização de levantamentos amostrais diretos, tais como: pesquisas domiciliares, contagem de placas de veículos e entrevistas em terminais de transporte público. Uma alternativa de menor custo consiste em sintetizar uma matriz OD por meio de métodos matemáticos com o uso de dados provenientes de contagens de tráfego feitas na rede de transportes em estudo, os quais podem ser classificados em métodos de reconstrução e de estimação. Este artigo apresenta uma discussão conceitual com foco nas premissas e limitações dos principais métodos de reconstrução baseados em maximização de entropia, mínimos quadrados generalizados e inferência bayesiana. Descreve-se detalhadamente a fundamentação matemática dos métodos e são feitas recomendações para o avanço e aplicação efetiva dos mesmos

A Genetic Algorithm Integrated with Monte Carlo Simulation for the Field Layout Design Problem

International audienceOil and gas production is moving deeper and further offshore as energy companies seek new sources, making the field layout design problem even more important. Although many optimization models are presented in the revised literature, they do not properly consider the uncertainties in well deliverability. This paper aims at presenting a Monte Carlo simulation integrated with a genetic algorithm that addresses this stochastic nature of the problem. Based on the results obtained, we conclude that the probabilistic approach brings new important perspectives to the field development engineering