Comparison of Game Theoretical Strategy and Reinforcement Learning in Traffic Light Control

Many traffic models and control methods have already been utilized in the public transportation system due to the increasing traffic congestion. Thus, an intelligent traffic model is formalized and presented to control multiple traffic light simultaneously and efficiently according to the distribution of vehicles from each incoming link (i.e. sections) in this paper. Compared with constant strategy, two methods are proposed for traffic light control, i.e., game theoretical strategy and reinforcement learning methods. Game theoretical strategy is generated in a game theoretical framework where incoming links are regarded as players and the combination of the status of traffic lights can be regarded as decisions made by these players. The cost function is evaluated and the strategy is produced with Nash equilibrium for passing maximum vehicles in an intersection. The other one is Single-Agent Reinforcement Learning (SARL), specifically with the Q-learning algorithm in this case, which is usually used in such a dynamic environment to control traffic flow so the traffic problem could be improved. Generally, the intersection is regarded as the centralized agent and controlling signal status is considered as the actions of the agent. The performance of these two methods is compared after simulated and implemented in a junction

Bio-inspired Computing and Smart Mobility

Por último, se aborda la predicción de plazas libres de aparcamiento utilizando técnicas de aprendizaje automático, tales como series temporales, agrupamiento, etc., incluyendo un prototipo de aplicación web. La tercera parte de esta tesis doctoral se enfoca en el diseño y evaluación de un nuevo algoritmo inspirado en la epigénesis, el Algoritmo Epigenético. Luego de la descripción del modelo en el que se basa y de sus partes, se utiliza este nuevo algoritmo para la resolución del problema de la mochila multidimensional y se comparan sus resultados con los de otros algoritmos del estado de arte. Por último se emplea también el Algoritmo Epigenético para la optimización de la arquitectura Yellow Swarm, un problema de movilidad inteligente resuelto por un nuevo algoritmo bioinspirado. A lo largo de esta tesis doctoral se han descrito los problemas de movilidad inteligente y propuesto nuevas herramientas para su optimización. A partir de los experimentos realizados se concluye que estas herramientas, basadas en algoritmos bioinspirados, son eficientes para abordar estos problemas, obteniendo resultados competitivos comparados con los del estado del arte, los cuales han sido validados estadísticamente. Esto representa un aporte científico pero también una serie de mejoras para la sociedad toda, tanto en su salud como en el aprovechamiento de su tiempo libre. Fecha de lectura de Tesis: 01 octubre 2018.Esta tesis doctoral propone soluciones a problemas de movilidad inteligente, concretamente la reducción de los tiempos de viajes en las vías urbanas, las emisiones de gases de efecto invernadero y el consumo de combustible, mediante el diseño y uso de nuevos algoritmos bioinspirados. Estos algoritmos se utilizan para la optimización de escenarios realistas, cuyo trazado urbano se obtiene desde OpenStreetMap, y que son luego evaluados en el microsimulador SUMO. Primero se describen las bases científicas y tecnológicas, incluyendo la definición y estado del arte de los problemas a abordar, las metaheurísticas que se utilizarán durante el desarrollo de los experimentos, así como las correspondientes validaciones estadísticas. A continuación se describen los simuladores de movilidad como principal herramienta para construir y evaluar los escenarios urbanos. Por último se presenta una propuesta para generar tráfico vehicular realista a partir de datos de sensores que cuentan el número de vehículos en la ciudad, utilizando herramientas incluidas en SUMO combinadas con algoritmos evolutivos. En la segunda parte se modelan y resuelven problemas de movilidad inteligente utilizando las nuevas arquitecturas Red Swarm y Green Swarm para sugerir nuevas rutas a los vehículos utilizando nodos con conectividad Wi-Fi. Red Swarm se centra en la reducción de tiempos de viajes evitando la congestión de las calles, mientras que Green Swarm está enfocado en la reducción de emisiones y consumo de combustible. Luego se propone la arquitectura Yellow Swarm que utiliza una serie de paneles LED para indicar desvíos que los vehículos pueden seguir en lugar de nodos Wi-Fi haciendo esta propuesta más accesible. Además se propone un método para genera rutas alternativas para los navegadores GPS de modo que se aprovechen mejor las calles secundarias de las ciudades, reduciendo los atascos

A taxonomy for planning and designing smart mobility services

The development of smart mobility initiatives requires specialized and contextualized policies addressing the needs and interests of many stakeholders involved. Since the development of such policies is challenging, there is a need to learn from the experience of many cities around the world offering efficient and successfully adopted smart mobility services. However, in practice, the information provided about such initiatives is shallow and unstructured. To address this issue, we study the state of the art in mobility services, reviewing scientific publications and 42 smart mobility services delivered by nine smart cities around the world, and we propose a taxonomy for planning and designing smart mobility services. The taxonomy provides a common vocabulary to discuss and share information about such services. It comprises eight dimensions: type of services, maturity level, users, applied technologies, delivery channels, benefits, beneficiaries, and common functionality. The contribution of the proposed taxonomy is to serve as a tool for guiding policy makers by identifying a spectrum of mobility services that can be provided, to whom, what technologies can be used to deliver them, and what is the delivered public value so to justify their implementation. In addition, the taxonomy can also assist researchers in further developing the domain. By identifying common functionality, it could also help Information Technology (IT) teams in building and maintaining smart mobility services. Finally, we further discuss usage scenarios of the taxonomy by policy makers, IT staff and researchers.NORTE-01-0145-FEDER-000037, supported by Norte Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (EFDR). The first author is also supported by the Portuguese funding agency, FCT, under grant PD/BD/52238/201

Natural computing for vehicular networks

La presente tesis aborda el diseño inteligente de soluciones para el despliegue de redes vehiculares ad-hoc (vehicular ad hoc networks, VANETs). Estas son redes de comunicación inalámbrica formada principalmente por vehículos y elementos de infraestructura vial. Las VANETs ofrecen la oportunidad para desarrollar aplicaciones revolucionarias en el ámbito de la seguridad y eficiencia vial. Al ser un dominio tan novedoso, existe una serie de cuestiones abiertas, como el diseño de la infraestructura de estaciones base necesaria y el encaminamiento (routing) y difusión (broadcasting) de paquetes de datos, que todavía no han podido resolverse empleando estrategias clásicas. Es por tanto necesario crear y estudiar nuevas técnicas que permitan de forma eficiente, eficaz, robusta y flexible resolver dichos problemas. Este trabajo de tesis doctoral propone el uso de computación inspirada en la naturaleza o Computación Natural (CN) para tratar algunos de los problemas más importantes en el ámbito de las VANETs, porque representan una serie de algoritmos versátiles, flexibles y eficientes para resolver problemas complejos. Además de resolver los problemas VANET en los que nos enfocamos, se han realizado avances en el uso de estas técnicas para que traten estos problemas de forma más eficiente y eficaz. Por último, se han llevado a cabo pruebas reales de concepto empleando vehículos y dispositivos de comunicación reales en la ciudad de Málaga (España). La tesis se ha estructurado en cuatro grandes fases. En la primera fase, se han estudiado los principales fundamentos en los que se basa esta tesis. Para ello se hizo un estudio exhaustivo sobre las tecnologías que emplean las redes vehiculares, para así, identificar sus principales debilidades. A su vez, se ha profundizado en el análisis de la CN como herramienta eficiente para resolver problemas de optimización complejos, y de cómo utilizarla en la resolución de los problemas en VANETs. En la segunda fase, se han abordado cuatro problemas de optimización en redes vehiculares: la transferencia de archivos, el encaminamiento (routing) de paquetes, la difusión (broadcasting) de mensajes y el diseño de la infraestructura de estaciones base necesaria para desplegar redes vehiculares. Para la resolución de dichos problemas se han propuesto diferentes algoritmos CN que se clasifican en algoritmos evolutivos (evolutionary algorithms, EAs), métodos de inteligencia de enjambre (swarm intelligence, SI) y enfriamiento simulado (simulated annealing, SA). Los resultados obtenidos han proporcionado protocolos de han mejorado de forma significativa las comunicaciones en VANETs. En la tercera y última fase, se han realizado experimentos empleando vehículos reales circulando por las carreteras de Málaga y que se comunicaban entre sí. El principal objetivo de estas pruebas ha sido el validar las mejoras que presentan los protocolos que se han optimizado empleando CN. Los resultados obtenidos de las fases segunda y tercera confirman la hipótesis de trabajo, que la CN es una herramienta eficiente para tratar el diseño inteligente en redes vehiculares

Recent Developments on Mobile Ad-Hoc Networks and Vehicular Ad-Hoc Networks

This book presents collective works published in the recent Special Issue (SI) entitled "Recent Developments on Mobile Ad-Hoc Networks and Vehicular Ad-Hoc Networks”. These works expose the readership to the latest solutions and techniques for MANETs and VANETs. They cover interesting topics such as power-aware optimization solutions for MANETs, data dissemination in VANETs, adaptive multi-hop broadcast schemes for VANETs, multi-metric routing protocols for VANETs, and incentive mechanisms to encourage the distribution of information in VANETs. The book demonstrates pioneering work in these fields, investigates novel solutions and methods, and discusses future trends in these field

The urban real-time traffic control (URTC) system : a study of designing the controller and its simulation

The growth of the number of automobiles on the roads in China has put higher demands on the traffic control system that needs to efficiently reduce the level of congestion occurrence, which increases travel delay, fuel consumption, and air pollution. The traffic control system, urban real-time traffic control system based on multi-agent (MA-URTC) is presented in this thesis. According to the present situation and the traffic's future development in China, the researches on intelligent traffic control strategy and simulation based on agent lays a foundation for the realization of the system. The thesis is organized as follows: The first part focuses on the intersection' real-time signal control strategy. It contains the limitations of current traffic control systems, application of artificial intelligence in the research, how to bring the dynamic traffic flow forecast into effect by combining the neural network with the genetic arithmetic, and traffic signal real-time control strategy based on fuzzy control. The author uses sorne simple simulation results to testify its superiority. We adopt the latest agent technology in designing the logical structure of the MA-URTC system. By exchanging traffic flows information among the relative agents, MA-URTC provides a new concept in urban traffic control. With a global coordination and cooperation on autonomy-based view of the traffic in cities, MA-URTC anticipates the congestion and control traffic flows. It is designed to support the real-time dynamic selection of intelligent traffic control strategy and the real-time communication requirements, together with a sufficient level of fault-tolerance. Due to the complexity and levity of urban traffic, none strategy can be universally applicable. The agent can independently choose the best scheme according to the real-time situation. To develop an advanced traffic simulation system it can be helpful for us to find the best scheme and the best switch-point of different schemes. Thus we can better deal with the different real-time traffic situations. The second part discusses the architecture and function of the intelligent traffic control simulation based on agent. Meanwhile the author discusses the design model of the vehicle-agent, road agent in traffic network and the intersection-agent so that we can better simulate the real-time environment. The vehicle-agent carries out the intelligent simulation based on the characteristics of the drivers in the actual traffic condition to avoid the disadvantage of the traditional traffic simulation system, simple-functioned algorithm of the vehicles model and unfeasible forecasting hypothesis. It improves the practicability of the whole simulation system greatly. The road agent's significance lies in its guidance of the traffic participants. It avoids the urban traffic control that depends on only the traffic signal control at intersection. It gives the traffic participants the most comfortable and direct guidance in traveling. It can also make a real-time and dynamic adjustment on the urban traffic flow, thus greatly lighten the pressure of signal control in intersection area. To sorne extent, the road agent is equal to the pre-caution mechanism. In the future, the construction of urban roads tends to be more intelligent. Therefore, the research on road agent is very important. All kinds of agents in MA-URTC are interconnected through a computer network. In the end, the author discusses the direction of future research. As the whole system is a multi-agent system, the intersection, the road and the vehicle belongs to multi-agent system respectively. So the emphasis should be put on the structure design and communication of all kinds of traffic agents in the system. Meanwhile, as an open and flexible real-time traffic control system, it is also concerned with how to collaborate with other related systems effectively, how to conform the resources and how to make the traffic participants anywhere throughout the city be in the best traffic guidance at all times and places. To actualize the genuine ITS will be our final goal. \ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : Artificial Intelligence, Computer simulation, Fuzzy control, Genetic Algorithm, Intelligent traffic control, ITS, Multi-agent, Neural Network, Real-time

Exploring the forecasting approach for road accidents: Analytical measures with hybrid machine learning

International audienceUrban traffic forecasting models generally follow either a Gaussian Mixture Model (GMM) or a Support Vector Classifier (SVC) to estimate the features of potential road accidents. Although SVC can provide good performances with less data than GMM, it incurs a higher computational cost. This paper proposes a novel framework that combines the descriptive strength of the Gaussian Mixture Model with the high-performance classification capabilities of the Support Vector Classifier. A new approach is presented that uses the mean vectors obtained from the GMM model as input to the SVC. Experimental results show that the approach compares very favorably with baseline statistical methods

A genetic programming system with an epigenetic mechanism for traffic signal control

Traffic congestion is an increasing problem in most cities around the world. It impacts businesses as well as commuters, small cities and large ones in developing as well as developed economies. One approach to decrease urban traffic congestion is to optimize the traffic signal behaviour in order to be adaptive to changes in the traffic conditions. From the perspective of intelligent transportation systems, this optimization problem is called the traffic signal control problem and is considered a large combinatorial problem with high complexity and uncertainty. A novel approach to the traffic signal control problem is proposed in this thesis. The approach includes a new mechanism for Genetic Programming inspired by Epigenetics. Epigenetic mechanisms play an important role in biological processes such as phenotype differentiation, memory consolidation within generations and environmentally induced epigenetic modification of behaviour. These properties lead us to consider the implementation of epigenetic mechanisms as a way to improve the performance of Evolutionary Algorithms in solution to real-world problems with dynamic environmental changes, such as the traffic control signal problem. The epigenetic mechanism proposed was evaluated in four traffic scenarios with different properties and traffic conditions using two microscopic simulators. The results of these experiments indicate that Genetic Programming was able to generate competitive actuated traffic signal controllers for all the scenarios tested. Furthermore, the use of the epigenetic mechanism improved the performance of Genetic Programming in all the scenarios. The evolved controllers adapt to modifications in the traffic density and require less monitoring and less human interaction than other solutions because they dynamically adjust the signal behaviour depending on the local traffic conditions at each intersection