Application of support vector machine in a traffic lights control

This article presents the process of adapting support vector machine model’s parameters used for studying the effect of traffic light cycle length parameter’s value on traffic quality. The survey is carried out using data collected during running simulations in author’s traffic simulator. The article shows results of searching for optimum traffic light cycle length parameter’s value

Traffic congestion prevention system

Transport is one of the key elements in the development of any country; it can be a powerful catalyst for economic growth. However, the infrastructure does not give enough to the huge number of vehicles which produces several problems, particularly in terms of road safety, and loss of time and pollution. One of the most significant problems is congestion, this is a major handicap for the road transport system. An alternative would be to use new technologies in the field of communication to send traffic information such as treacherous road conditions and accident sites by communicating, for a more efficient use of existing infrastructure.  In this paper, we present a CPS system, which can help drivers in order to have a better trip. For this raison we find the optimal way to reduce travel time and fuel consumption. This system based on our recent work [1]. It´s new approach aims to avoid congestion and queues, hat assure more efficient and optimal use of the existing road infrastructure. For that we concentrate by analyzing the useful and reliable traffic information collected in real time. The system is simulated in several conditions, Experimental result show that our approach is very effective. In the future work, we try to improve our system by adding more complexity in our system

Road Artery Traffic Light Optimization with Use of the Reinforcement Learning

The basic principle of optimal traffic control is the appropriate real-time response to dynamic traffic flow changes. Signal plan efficiency depends on a large number of input parameters. An actuated signal system can adjust very well to traffic conditions, but cannot fully adjust to stochastic traffic volume oscillation. Due to the complexity of the problem analytical methods are not applicable for use in real time, therefore the purpose of this paper is to introduce heuristic method suitable for traffic light optimization in real time. With the evolution of artificial intelligence new possibilities for solving complex problems have been introduced. The goal of this paper is to demonstrate that the use of the Q learning algorithm for traffic lights optimization is suitable. The Q learning algorithm was verified on a road artery with three intersections. For estimation of the effectiveness and efficiency of the proposed algorithm comparison with an actuated signal plan was carried out. The results (average delay per vehicle and the number of vehicles that left road network) show that Q learning algorithm outperforms the actuated signal controllers. The proposed algorithm converges to the minimal delay per vehicle regardless of the stochastic nature of traffic. In this research the impact of the model parameters (learning rate, exploration rate, influence of communication between agents and reward type) on algorithm effectiveness were analysed as well.</p

Railway traffic scheduling with use of reinforcement learning

The reliability of railway traffic is commonly evaluated with train punctuality, where the\ud deviations of actual train arrivals/departures and train arrivals/departures published in the\ud timetable are compared. Minor train delays can be mitigated or even eliminated with running\ud time supplements, while major delays can lead to so-called secondary delays of other trains\ud on the network. Railway lines with high capacity utilization are more likely subject to delays,\ud since a greater number of trains means a larger number of potential conflicts and more\ud interactions between trains. Consequently, the secondary delays are harder to limit. Railway\ud manager and carrier personnel are responsible for safe, undisturbed and punctual railway\ud traffic. But unforeseen events can lead to delays, which calls for train rescheduling, where\ud new train arrivals and departures are calculated. Train rescheduling is a complex\ud optimization problem, currently solved based on dispatcher’s expert knowledge. With the\ud increasing number of trains the complexity of the problem grows, the need for a decision\ud support system increases. Train rescheduling is considered an NP-complete problem, where\ud conventional mathematical and computer optimization methods fail to find the optimal\ud solution, but artificial intelligence approaches have some measure of success. In this\ud dissertation an algorithm for train rescheduling based on reinforcement learning, more\ud precisely Q-learning, was developed. The Q-learning agent learns from rewards and\ud punishments received from the environment, and looks for the optimal train dispatching\ud strategy depending on the objective function

ONLINE and REAL-TIME TRANSPORTATION SYSTEMS MANAGEMENT and OPERATIONS DECISION SUPPORT WITH INTEGRATED TRAVEL BEHAVIOR and DYNAMIC NETWORK MODELS

The acceleration of urbanization is witnessed all around the world. Both population and vehicle ownership are rapidly growing, and the induced traffic congestion becomes an increasingly pervasive problem in people’s daily life. In 2014, transportation congestion caused $160 billion economic loss in 498 U.S. urban areas, which is 5.5 more than that in 1982. Without effective reactions, this number is expect to grow to$192 billion in 2020. In order to mitigate traffic congestion, many transportation demand management (TDM) strategies (e.g. bus rapid lanes, and flextime policy), and active traffic management (ATM) strategies (e.g. real-time user guidance, and adaptive traffic signal control) have been proposed and implemented. Although TDM and ATM have proved their values in theoretical researches or field implementations, it is still hard for transportation engineers to select the optimal strategy when faced with complex traffic conditions. In the science of transportation engineering, mathematical models are usually expected to help estimate traffic conditions under different scenarios. There have been a number of models that help transportation engineers make decisions. However, many of them are developed for offline use and are not suitable for real-time applications due to computational time issues. With the development of computational technologies and traffic monitoring systems, online transportation network modeling is getting closer and closer to reality. The objective of this dissertation is to develop a large-scale mesoscopic transportation model which is integrated with an agent-based travel behavior model. The ultimate goal is to achieve online (real-time) simulation to estimate and predict the traffic performance of the entire Washington D.C. area. The simulation system is expected to support real-time transportation system managements and operations. One of the most challenging issue for this dissertation is the calibration of online simulation models. Model parameters need to be estimated based on real-time traffic data to reflect the reality. Literature review of previous relevant studies indicates a trade-off between computational speed and calibration accuracy. In order to apply the model onto a real-time horizon, experts usually ignore the inherent mechanism of traffic modeling but rely on fast converging technologies to approximate the model parameters. Differently from previous online transportation simulation approaches, the method proposed in this dissertation focuses more on the mechanism of transportation modeling. With the fundamental understanding of the modeling mechanism, one can quickly determine the gradient of model parameters such that the gap between real-time traffic measures and simulation results is minimized. This research is one of the earliest attempts to introduce both agent-based modeling and gradient-based calibration approach to model real-time large-scale networks. The contribution includes: 1) integrate an agent-based travel behavior model into dynamic transportation network models to enhance the behavior realism; 2) propose a fast online calibration procedure that quickly adjusts model parameters based on real-time traffic data. A number of real-world case studies are illustrated to demonstrate the value of this model for both long-term and real-time applications

Méthodes d'apprentissage de la coordination multiagent : application au transport intelligent

Les problèmes de prise de décisions séquentielles multiagents sont difficiles à résoudre surtout lorsque les agents n'observent pas parfaitement l'état de Y environnement. Les approches existantes pour résoudre ces problèmes utilisent souvent des approximations de la fonction de valeur ou se basent sur la structure pour simplifier la résolution. Dans cette thèse, nous proposons d'approximer un problème de décisions séquentielles multiagent à observation limitée, modélisé par un processus décisionnel markovien décentralisé (DEC-MDP) en utilisant deux hypothèses sur la structure du problème. La première hypothèse porte sur la structure de comportement optimal et suppose qu'il est possible d'approximer la politique optimale d'un agent en connaissant seulement les actions optimales au niveau d'un petit nombre de situations auxquelles l'agent peut faire face dans son environnement. La seconde hypothèse porte, quant à elle, sur la structure organisationnelle des agents et suppose que plus les agents sont éloignés les uns des autres, moins ils ont besoin de se coordonner. Ces deux hypothèses nous amènent à proposer deux approches d'approximation. La première approche, nommée Supervised Policy Reinforcement Learning, combine l'apprentissage par renforcement et l'apprentissage supervisé pour généraliser la politique optimale d'un agent. La second approche se base, quant à elle, sur la structure organisationnelle des agents pour apprendre une politique multiagent dans des problèmes où l'observation est limitée. Pour cela, nous présentons un modèle, le D O F - D E C - M DP (Distance-Observable Factored Decentralized Markov Décision Process) qui définit une distance d'observation pour les agents. A partir de ce modèle, nous proposons des bornes sur le gain de récompense que permet l'augmentation de la distance d'observation. Les résultats empiriques obtenus sur des problèmes classiques d'apprentissage par renforcement monoagents et multiagents montrent que nos approches d'approximation sont capables d'apprendre des politiques proches de l'optimale. Enfin, nous avons testé nos approches sur un problème de coordination de véhicules en proposant une méthode de synchronisation d'agents via la communication dans un cadre à observation limitée

Urban Traffic Control Based on Learning Agents

Abstract — The optimization of traffic light control systems is at the heart of work in traffic management. Many of the solutions considered to design efficient traffic signal patterns rely on controllers that use pre-timed stages. Such systems are unable to identify dynamic changes in the local traffic flow and thus cannot adapt to new traffic conditions. An alternative, novel approach proposed by computer scientists in order to design adaptive traffic light controllers relies on the use of intelligents agents. The idea is to let autonomous entities, named agents, learn an optimal behavior by interacting directly in the system. By using machine learning algorithms based on the attribution of rewards according to the results of the actions selected by the agents, we can obtain a control policy that tries to optimize the urban traffic flow. In this paper, we will explain how we designed an intelligent agent that learns a traffic light control policy. We will also compare this policy with results from an optimal pre-timed controller. I