Detection and Tracking of Motorcycles in Congested Urban Environments Using Deep Learning and Markov Decision Processes

Part of the Lecture Notes in Computer Science book series (LNCS, volume 11524) pp 139-148 Also part of the Image Processing, Computer Vision, Pattern Recognition, and Graphics book sub series (LNIP, volume 11524)Also part of the Image Processing, Computer Vision, Pattern Recognition, and Graphics book sub series (LNIP, volume 11524)This research describes “EspiNet”, a Deep Learning Convolutional Neural Network model, in conjunction with a Markov Decision Process (MDP) tracker for detection and tracking of occluded motorcycles in urban environments. The model is trained and evaluated, using a new public dataset with up to 10,000 annotated images, created for this research, and captured in real urban traffic scenes. Images were captured using a moving camera mounted in a drone, where more than 60% of the motorcycles are affected by occlusions. The network design involves many tests, where a promising result of 88.84% in average precision (AP) is achieved, despite the considerable number of occluded vehicles, the movement of the camera and the low angle used for capture. The model predictions are used as input to an MDP tracker, reaching results up to 85.2% in Multiple Object Tracking Accuracy (MOTA). The proposed network architecture outperforms state of the art YOLO (You Look Only Once) v3.0 and Faster R-CNN (VGG16 based) detection models, producing also better tracking results in comparison with the use of the other two models as detector base for the MDP tracker.This work was partially supported by COLCIENCIAS project: Reduccion de Emisiones Vehiculares Mediante el Modelado y Gestion Optima de Trafico en Areas Metropolitanas - Caso Medellin - Area Metropolitana del Valle de Aburra, codigo 111874558167, CT 049-2017. Universidad Nacional de Colombia. Proyecto HERMES 25374. The authors gratefully acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research

Detection of Motorcycles in Urban Traffic Using Video Analysis: A Review

Motorcycles are Vulnerable Road Users (VRU) and as such, in addition to bicycles and pedestrians, they are the traffic actors most affected by accidents in urban areas. Automatic video processing for urban surveillance cameras has the potential to effectively detect and track these road users. The present review focuses on algorithms used for detection and tracking of motorcycles, using the surveillance infrastructure provided by CCTV cameras. Given the importance of results achieved by Deep Learning theory in the field of computer vision, the use of such techniques for detection and tracking of motorcycles is also reviewed. The paper ends by describing the performance measures generally used, publicly available datasets (introducing the Urban Motorbike Dataset (UMD) with quantitative evaluation results for different detectors), discussing the challenges ahead and presenting a set of conclusions with proposed future work in this evolving area

Perspective Chapter: Future Perspectives of Intelligent Autonomous Vehicles

The chapter explains the effects of intelligent autonomous vehicles from future perspectives. The chapter gives readers an overview of the future intelligent autonomous vehicles and promotes the development potential on intelligent. To be specific, the chapter first gives the readers an overview of the development of autonomous vehicles. Then, the chapter introduces the potential of intelligent autonomous vehicles, key technologies that are needed for future intelligent autonomous vehicles, and how intelligent autonomous vehicles affect the future. Finally, the chapter discusses barriers in intelligent autonomous vehicles development. The chapter will be contributed as a start point for people who want to keep working on intelligent autonomous vehicles and help them understand the general condition of future intelligent autonomous vehicles

Multiple Object Tracking in Urban Traffic Scenes

RÉSUMÉ:Le suivi multiobjets (MOT) est un domaine très étudié qui a évolué et changé beaucoup durant les années grâce à ses plusieurs applications potentielles pour améliorer notre qualité de vie. Dans notre projet de recherche, spécifiquement, nous sommes intéressés par le MOT dans les scènes de trafic urbain pour extraire précisément les trajectoires des usagers de la route, afin d’améliorer les systèmes de circulation routière desquels nous bénéficions tous.Notre première contribution est l’introduction d’informations sur les étiquettes de classe dans l’ensemble des caractéristiques qui décrivent les objets pour les associer sur différents trames, afin de bien capturer leur mouvement sous forme de trajectoires dans un environnement réel.Nous capitalisons sur les informations provenant d’un détecteur basé sur l’apprentissage profond qui est utilisé pour l’extraction des objets d’intérêt avant la procédure de suivi, carnous avons été intrigués par leurs popularités croissantes et les bonnes performances qu’ils obtiennent. Cependant, malgré leur potentiel prometteur dans la littérature, nous avons constaté que les résultats étaient décevants dans nos expériences. La qualité des détections,telle que postulée, affecte grandement la qualité des trajectoires finales. Néanmoins, nous avons observé que les informations des étiquettes de classe, ainsi que son score de confiance, sont très utiles pour notre application, où il y a un nombre élevé de variabilité pour les types d’usagers de la route.Ensuite, nous avons concentré nos efforts sur la fusion des entrées de deux sources différentes afin d’obtenir un ensemble d’objets en entrée avec un niveau de précision satisfaisant pour procéder à l’étape de suivi. À ce stade, nous avons travaillé sur l’intégration des boîtes englobantes à partir d’un détecteur multi-classes par apprentissage et d’une méthode basée sur la soustraction d’arrière-plan pour résoudre les problèmes tels que la fragmentation et les représentations redondantes du même objet.---------- ABSTRACT:Multiple object tracking (MOT) is an intensively researched area that have evolved and undergone much innovation throughout the years due to its potential in a lot of applications to improve our quality of life. In our research project, specifically, we are interested in applying MOT in urban traffic scenes to portray an accurate representation of the road user trajectories for the eventual improvements of road traffic systems that affect people from all walks of life. Our first contribution is the introduction of class label information as part of the features that describe the targets and for associating them across frames to capture their motion into trajectories in real environment. We capitalize on that information from a deep learning detector that is used for extraction of objects of interest prior to the tracking procedure, since we were intrigued by their growing popularity and reported good performances. However,despite their promising potential in the literature, we found that the results were disappointing in our experiments. The quality of extracted input, as postulated, critically affects the quality of the final trajectories obtained as tracking output. Nevertheless, we observed that the class label information, along with its confidence score, is invaluable for our application of urban traffic settings where there are a high number of variability in terms of types of road users. Next, we focused our effort on fusing inputs from two different sources in order to obtain a set of objects with a satisfactory level of accuracy to proceed with the tracking stage. At this point, we worked on the integration of the bounding boxes from a learned multi-class object detector and a background subtraction-based method to resolve issues, such as fragmentation and redundant representations of the same object

Risk analysis of autonomous vehicle and its safety impact on mixed traffic stream

In 2016, more than 35,000 people died in traffic crashes, and human error was the reason for 94% of these deaths. Researchers and automobile companies are testing autonomous vehicles in mixed traffic streams to eliminate human error by removing the human driver behind the steering wheel. However, recent autonomous vehicle crashes while testing indicate the necessity for a more thorough risk analysis. The objectives of this study were (1) to perform a risk analysis of autonomous vehicles and (2) to evaluate the safety impact of these vehicles in a mixed traffic stream. The overall research was divided into two phases: (1) risk analysis and (2) simulation of autonomous vehicles. Risk analysis of autonomous vehicles was conducted using the fault tree method. Based on failure probabilities of system components, two fault tree models were developed and combined to predict overall system reliability. It was found that an autonomous vehicle system could fail 158 times per one-million miles of travel due to either malfunction in vehicular components or disruption from infrastructure components. The second phase of this research was the simulation of an autonomous vehicle, where change in crash frequency after autonomous vehicle deployment in a mixed traffic stream was assessed. It was found that average travel time could be reduced by about 50%, and 74% of conflicts, i.e., traffic crashes, could be avoided by replacing 90% of the human drivers with autonomous vehicles

Intelligent Transportation Related Complex Systems and Sensors

Building around innovative services related to different modes of transport and traffic management, intelligent transport systems (ITS) are being widely adopted worldwide to improve the efficiency and safety of the transportation system. They enable users to be better informed and make safer, more coordinated, and smarter decisions on the use of transport networks. Current ITSs are complex systems, made up of several components/sub-systems characterized by time-dependent interactions among themselves. Some examples of these transportation-related complex systems include: road traffic sensors, autonomous/automated cars, smart cities, smart sensors, virtual sensors, traffic control systems, smart roads, logistics systems, smart mobility systems, and many others that are emerging from niche areas. The efficient operation of these complex systems requires: i) efficient solutions to the issues of sensors/actuators used to capture and control the physical parameters of these systems, as well as the quality of data collected from these systems; ii) tackling complexities using simulations and analytical modelling techniques; and iii) applying optimization techniques to improve the performance of these systems. It includes twenty-four papers, which cover scientific concepts, frameworks, architectures and various other ideas on analytics, trends and applications of transportation-related data

Modeling present and future freeway management strategies : variable speed limits, lane-changing and platooning of connected autonomous vehicles

Freeway traffic management is necessary to improve capacity and reduce congestion, especially in metropolitan freeways where the rush period lasts several hours per day. Traffic congestion implies delays and an increase in air pollutant emissions, both with harmful effects to society. Active management strategies imply regulating traffic demand and improving freeway capacity. While both aspects are necessary, the present thesis only addresses the supply side. Part of the research in traffic flow theory is grounded on empirical data. Today, in order to extend our knowledge on traffic dynamics, detailed and high-quality data is needed. To that end, the thesis presents a pioneering data collection campaign, which was developed in a freeway accessing Barcelona. In a Variable Speed Limits (VSL) environment, different speed limits where posted, in order to observe their real and detailed effects on traffic. All the installed surveillance instruments were set to capture data in the highest possible level of detail, including video recordings, from where to count lane-changing maneuvers. With this objective, a semi-automatic method to reliably count lane changes form video recordings was developed and is presented in the thesis. Data analysis proved that the speed limit fulfillment was only relevant in sections with enforcement devices. In these sections, it is confirmed that, the lower the speed limit, the higher the occupancy to achieve a given flow. In contrast, the usually assumed mainline metering effect of low speed limits was not relevant. This might be different in case of stretch enforcement. These findings mean that, on the one hand, VSL strategies aiming to restrict the mainline flow on a freeway by using low speed limits will need to be applied carefully, avoiding conditions as the ones presented here. On the other hand, VSL strategies trying to get the most from the increased vehicle storage capacity of freeways under low speed limits might be rather promising. Results also show that low speed limits increase the speed differences across lanes for moderate demands. This, in turn, also increases the lane changing rates. In contrast, lower speed limits widen the range of flows under uniform lane flow distributions, so that, even for moderate to low demands, the under-utilization of any lane can be avoided. Further analysis of lane-changing activity allowed unveiling that high lane-changing rates prevent achieving the highest flows. This inverse relationship is modeled in the thesis using a stochastic model based on Bayesian inference. This model could be used as a control tool, in order to determine which level of lane-changing activity can be allowed to achieve a desired capacity with some level of reliability. Previous results identify drivers' fulfillment of traffic regulations as a weak point in order to maximize the benefits of current management strategies, like VSL or lane-changing control. This is likely to change in the near future with the irruption of Autonomous Vehicles (AV) in freeways. V2X communications will allow directly actuating on individual vehicles with high accuracy. This will open the door to new management strategies based on simultaneous communication to groups of AVs and extremely short reaction times, like platooning, which stands out as a strategy with a huge potential to improve freeway traffic. Strings of AVs traveling at extremely short gaps (i.e. platoons) allow achieving higher capacities and lower energy consumption rates. In this context, the thesis presents a parsimonious macroscopic model for AVs platooning in mixed traffic (i.e. platoons of AVs travelling together with human driven vehicles). The model allows determining the average platoon length and reproducing the overall traffic dynamics leading to higher capacities. Results prove that with a 50% penetration rate of AVs in the lane, capacity could reach 3400 veh/h/lane under a cooperative platooning strategy.Per tal de millorar la capacitat i reduir la congestió a les autopistes cal gestionar el trànsit de manera activa. Les estratègies de gestió activa del trànsit són d’especial importància en autopistes metropolitanes. La congestió provoca retards i un increment del consum de combustible que va lligat a unes majors emissions de gasos contaminants, tots amb efectes perniciosos per la societat. La gestió activa del transit requereix regular la demanda i millorar la capacitat de la via. Encara que tots dos aspectes son necessaris, la present tesis només analitza la gestió de l’oferta. Part de la recerca en l’anàlisi i la teoria del trànsit es basa en dades empíriques. Per satisfer el requeriment de dades detallades i d’alta qualitat, aquesta tesis presenta una campanya pionera de recol·lecció de dades. Les dades es van recollir a l’autopista B-23 d’accés a Barcelona. Tots els instruments de mesura es van configurar per tal de registrar les dades amb el major nivell de detall possible, incloent les càmeres de videovigilància, d’on es varen extreure els comptatges de canvi de carril. Amb aquest objectiu, es va desenvolupar una metodologia semiautomàtica per comptar canvis de carril a partir de gravacions de trànsit, que es presenta en el cos de la tesi. L’anàlisi de les dades obtingudes ha demostrat que el compliment dels límits de velocitat només resulta rellevant en aquelles seccions que compten amb un radar. És en aquestes seccions on s’ha confirmat que com menor és el límit de velocitat, major es l’ocupació per a un flux donat. Per contra, la hipòtesi habitual de que uns límits de velocitat baixos produeixen una restricció del flux no es va observar de forma rellevant. Aquest comportament podria esser diferent en el cas d’implantar un radar de tram. Els resultats obtinguts també mostren com les diferències de velocitats entre carrils s’incrementen per a límits de velocitat baixos i en condicions de demanda moderada. Això, alhora, incrementa el nombre de canvis de carril. Per contra, els límits de velocitat baixos contribueixen a una distribució de flux més uniforme entre carrils, de forma que es pot evitar la infrautilització de carrils. L’anàlisi més detallat de l’activitat de canvi de carril demostra que una taxa elevada de canvis de carril impedeix assolir fluxos grans de circulació. En la tesi, aquesta relació inversa entre la taxa de canvis de carril i el flux màxim de trànsit a l’autopista s’ha modelat de forma estocàstica utilitzant un model basat en la inferència Bayesiana. Aquest model es pot utilitzar com una eina de control, per tal de determinar quina taxa de canvi de carril es pot permetre si es vol assolir una capacitat determinada amb una determinada probabilitat de compliment. En vista dels resultats previs, la falta de compliment de les normes de trànsit per part dels conductors s’identifica com un punt dèbil a l’hora de maximitzar els beneficis de les actuals estratègies de gestió del transit. Això probablement canviarà en el futur pròxim amb la irrupció dels Vehicles Autònoms (VA) a les autopistes. Els sistemes de comunicació V2X permetran actuar individualment sobre cada vehicle amb una gran precisió. Això obrirà la porta a noves estratègies de gestió, basades en la comunicació simultània entre diferents grups de VA i en temps de reacció extremadament curts, com per exemple és el “platooning”, que destaca pel seu gran potencial per millorar el trànsit en autopista. Els “platons” son cadenes de VA viatjant amb uns espaiaments extremadament curts que permeten assolir capacitats mes elevades i un menor consum energètic. En aquest context, la tesi presenta un model macroscòpic parsimoniós per a “platons” de VA en condicions de transit mixt, és a dir, compartint la infraestructura amb vehicles tradicionals. El model permet determinar la longitud mitjana del “platons” i reproduir el trànsit global dinàmiques que condueixen a majors capacitats. Els resultats demostren que amb un 50% la velocitat de penetració dels AV al carril, la capacitat podria arribar als 3.400 vehicles / h / carril sota una estratègia cooperativa de “platooning