Real-time detection and tracking of multiple objects with partial decoding in H.264/AVC bitstream domain

In this paper, we show that we can apply probabilistic spatiotemporal macroblock filtering (PSMF) and partial decoding processes to effectively detect and track multiple objects in real time in H.264|AVC bitstreams with stationary background. Our contribution is that our method cannot only show fast processing time but also handle multiple moving objects that are articulated, changing in size or internally have monotonous color, even though they contain a chaotic set of non-homogeneous motion vectors inside. In addition, our partial decoding process for H.264|AVC bitstreams enables to improve the accuracy of object trajectories and overcome long occlusion by using extracted color information.Comment: SPIE Real-Time Image and Video Processing Conference 200

Moving object detection for automobiles by the shared use of H.264/AVC motion vectors : innovation report.

Cost is one of the problems for wider adoption of Advanced Driver Assistance Systems (ADAS) in China. The objective of this research project is to develop a low-cost ADAS by the shared use of motion vectors (MVs) from a H.264/AVC video encoder that was originally designed for video recording only. There were few studies on the use of MVs from video encoders on a moving platform for moving object detection. The main contribution of this research is the novel algorithm proposed to address the problems of moving object detection when MVs from a H.264/AVC encoder are used. It is suitable for mass-produced in-vehicle devices as it combines with MV based moving object detection in order to reduce the cost and complexity of the system, and provides the recording function by default without extra cost. The estimated cost of the proposed system is 50% lower than that making use of the optical flow approach. To reduce the area of region of interest and to account for the real-time computation requirement, a new block based region growth algorithm is used for the road region detection. To account for the small amplitude and limited precision of H.264/AVC MVs on relatively slow moving objects, the detection task separates the region of interest into relatively fast and relatively slow speed regions by examining the amplitude of MVs, the position of focus of expansion and the result of road region detection. Relatively slow moving objects are detected and tracked by the use of generic horizontal and vertical contours of rear-view vehicles. This method has addressed the problem of H.264/AVC encoders that possess limited precision and erroneous motion vectors for relatively slow moving objects and regions near the focus of expansion. Relatively fast moving objects are detected by a two-stage approach. It includes a Hypothesis Generation (HG) and a Hypothesis Verification (HV) stage. This approach addresses the problem that the H.264/AVC MVs are generated for coding efficiency rather than for minimising motion error of objects. The HG stage will report a potential moving object based on clustering the planar parallax residuals satisfying the constraints set out in the algorithm. The HV will verify the existence of the moving object based on the temporal consistency of its displacement in successive frames. The test results show that the vehicle detection rate higher than 90% which is on a par to methods proposed by other authors, and the computation cost is low enough to achieve the real-time performance requirement. An invention patent, one international journal paper and two international conference papers have been either published or accepted, showing the originality of the work in this project. One international journal paper is also under preparation

Design, Control, and Evaluation of a Human-Inspired Robotic Eye

Schulz S. Design, Control, and Evaluation of a Human-Inspired Robotic Eye. Bielefeld: Universität Bielefeld; 2020.The field of human-robot interaction deals with robotic systems that involve humans and robots closely interacting with each other. With these systems getting more complex, users can be easily overburdened by the operation and can fail to infer the internal state of the system or its ”intentions”. A social robot, replicating the human eye region with its familiar features and movement patterns, that are the result of years of evolution, can counter this. However, the replication of these patterns requires hard- and software that is able to compete with the human characteristics and performance. Comparing previous systems found in literature with the human capabili- ties reveal a mismatch in this regard. Even though individual systems solve single aspects, the successful combination into a complete system remains an open challenge. In contrast to previous work, this thesis targets to close this gap by viewing the system as a whole — optimizing the hard- and software, while focusing on the replication of the human model right from the beginning. This work ultimately provides a set of interlocking building blocks that, taken together, form a complete end-to-end solution for the de- sign, control, and evaluation of a human-inspired robotic eye. Based on the study of the human eye, the key driving factors are identified as the success- ful combination of aesthetic appeal, sensory capabilities, performance, and functionality. Two hardware prototypes, each based on a different actua- tion scheme, have been developed in this context. Furthermore, both hard- ware prototypes are evaluated against each other, a previous prototype, and the human by comparing objective numbers obtained by real-world mea- surements of the real hardware. In addition, a human-inspired and model- driven control framework is developed out, again, following the predefined criteria and requirements. The quality and human-likeness of the motion, generated by this model, is evaluated by means of a user study. This frame- work not only allows the replication of human-like motion on the specific eye prototype presented in this thesis, but also promotes the porting and adaption to less equipped humanoid robotic heads. Unlike previous systems found in literature, the presented approach provides a scaling and limiting function that allows intuitive adjustments of the control model, which can be used to reduce the requirements set on the target platform. Even though a reduction of the overall velocities and accelerations will result in a slower motion execution, the human characteristics and the overall composition of the interlocked motion patterns remain unchanged

Modeling present and future freeway management strategies : variable speed limits, lane-changing and platooning of connected autonomous vehicles

Freeway traffic management is necessary to improve capacity and reduce congestion, especially in metropolitan freeways where the rush period lasts several hours per day. Traffic congestion implies delays and an increase in air pollutant emissions, both with harmful effects to society. Active management strategies imply regulating traffic demand and improving freeway capacity. While both aspects are necessary, the present thesis only addresses the supply side. Part of the research in traffic flow theory is grounded on empirical data. Today, in order to extend our knowledge on traffic dynamics, detailed and high-quality data is needed. To that end, the thesis presents a pioneering data collection campaign, which was developed in a freeway accessing Barcelona. In a Variable Speed Limits (VSL) environment, different speed limits where posted, in order to observe their real and detailed effects on traffic. All the installed surveillance instruments were set to capture data in the highest possible level of detail, including video recordings, from where to count lane-changing maneuvers. With this objective, a semi-automatic method to reliably count lane changes form video recordings was developed and is presented in the thesis. Data analysis proved that the speed limit fulfillment was only relevant in sections with enforcement devices. In these sections, it is confirmed that, the lower the speed limit, the higher the occupancy to achieve a given flow. In contrast, the usually assumed mainline metering effect of low speed limits was not relevant. This might be different in case of stretch enforcement. These findings mean that, on the one hand, VSL strategies aiming to restrict the mainline flow on a freeway by using low speed limits will need to be applied carefully, avoiding conditions as the ones presented here. On the other hand, VSL strategies trying to get the most from the increased vehicle storage capacity of freeways under low speed limits might be rather promising. Results also show that low speed limits increase the speed differences across lanes for moderate demands. This, in turn, also increases the lane changing rates. In contrast, lower speed limits widen the range of flows under uniform lane flow distributions, so that, even for moderate to low demands, the under-utilization of any lane can be avoided. Further analysis of lane-changing activity allowed unveiling that high lane-changing rates prevent achieving the highest flows. This inverse relationship is modeled in the thesis using a stochastic model based on Bayesian inference. This model could be used as a control tool, in order to determine which level of lane-changing activity can be allowed to achieve a desired capacity with some level of reliability. Previous results identify drivers' fulfillment of traffic regulations as a weak point in order to maximize the benefits of current management strategies, like VSL or lane-changing control. This is likely to change in the near future with the irruption of Autonomous Vehicles (AV) in freeways. V2X communications will allow directly actuating on individual vehicles with high accuracy. This will open the door to new management strategies based on simultaneous communication to groups of AVs and extremely short reaction times, like platooning, which stands out as a strategy with a huge potential to improve freeway traffic. Strings of AVs traveling at extremely short gaps (i.e. platoons) allow achieving higher capacities and lower energy consumption rates. In this context, the thesis presents a parsimonious macroscopic model for AVs platooning in mixed traffic (i.e. platoons of AVs travelling together with human driven vehicles). The model allows determining the average platoon length and reproducing the overall traffic dynamics leading to higher capacities. Results prove that with a 50% penetration rate of AVs in the lane, capacity could reach 3400 veh/h/lane under a cooperative platooning strategy.Per tal de millorar la capacitat i reduir la congestió a les autopistes cal gestionar el trànsit de manera activa. Les estratègies de gestió activa del trànsit són d’especial importància en autopistes metropolitanes. La congestió provoca retards i un increment del consum de combustible que va lligat a unes majors emissions de gasos contaminants, tots amb efectes perniciosos per la societat. La gestió activa del transit requereix regular la demanda i millorar la capacitat de la via. Encara que tots dos aspectes son necessaris, la present tesis només analitza la gestió de l’oferta. Part de la recerca en l’anàlisi i la teoria del trànsit es basa en dades empíriques. Per satisfer el requeriment de dades detallades i d’alta qualitat, aquesta tesis presenta una campanya pionera de recol·lecció de dades. Les dades es van recollir a l’autopista B-23 d’accés a Barcelona. Tots els instruments de mesura es van configurar per tal de registrar les dades amb el major nivell de detall possible, incloent les càmeres de videovigilància, d’on es varen extreure els comptatges de canvi de carril. Amb aquest objectiu, es va desenvolupar una metodologia semiautomàtica per comptar canvis de carril a partir de gravacions de trànsit, que es presenta en el cos de la tesi. L’anàlisi de les dades obtingudes ha demostrat que el compliment dels límits de velocitat només resulta rellevant en aquelles seccions que compten amb un radar. És en aquestes seccions on s’ha confirmat que com menor és el límit de velocitat, major es l’ocupació per a un flux donat. Per contra, la hipòtesi habitual de que uns límits de velocitat baixos produeixen una restricció del flux no es va observar de forma rellevant. Aquest comportament podria esser diferent en el cas d’implantar un radar de tram. Els resultats obtinguts també mostren com les diferències de velocitats entre carrils s’incrementen per a límits de velocitat baixos i en condicions de demanda moderada. Això, alhora, incrementa el nombre de canvis de carril. Per contra, els límits de velocitat baixos contribueixen a una distribució de flux més uniforme entre carrils, de forma que es pot evitar la infrautilització de carrils. L’anàlisi més detallat de l’activitat de canvi de carril demostra que una taxa elevada de canvis de carril impedeix assolir fluxos grans de circulació. En la tesi, aquesta relació inversa entre la taxa de canvis de carril i el flux màxim de trànsit a l’autopista s’ha modelat de forma estocàstica utilitzant un model basat en la inferència Bayesiana. Aquest model es pot utilitzar com una eina de control, per tal de determinar quina taxa de canvi de carril es pot permetre si es vol assolir una capacitat determinada amb una determinada probabilitat de compliment. En vista dels resultats previs, la falta de compliment de les normes de trànsit per part dels conductors s’identifica com un punt dèbil a l’hora de maximitzar els beneficis de les actuals estratègies de gestió del transit. Això probablement canviarà en el futur pròxim amb la irrupció dels Vehicles Autònoms (VA) a les autopistes. Els sistemes de comunicació V2X permetran actuar individualment sobre cada vehicle amb una gran precisió. Això obrirà la porta a noves estratègies de gestió, basades en la comunicació simultània entre diferents grups de VA i en temps de reacció extremadament curts, com per exemple és el “platooning”, que destaca pel seu gran potencial per millorar el trànsit en autopista. Els “platons” son cadenes de VA viatjant amb uns espaiaments extremadament curts que permeten assolir capacitats mes elevades i un menor consum energètic. En aquest context, la tesi presenta un model macroscòpic parsimoniós per a “platons” de VA en condicions de transit mixt, és a dir, compartint la infraestructura amb vehicles tradicionals. El model permet determinar la longitud mitjana del “platons” i reproduir el trànsit global dinàmiques que condueixen a majors capacitats. Els resultats demostren que amb un 50% la velocitat de penetració dels AV al carril, la capacitat podria arribar als 3.400 vehicles / h / carril sota una estratègia cooperativa de “platooning

Modeling present and future freeway management strategies : variable speed limits, lane-changing and platooning of connected autonomous vehicles

Premi Extraordinari de Doctorat, promoció 2018-2019. Àmbit d’Enginyeria Civil i AmbientalFreeway traffic management is necessary to improve capacity and reduce congestion, especially in metropolitan freeways where the rush period lasts several hours per day. Traffic congestion implies delays and an increase in air pollutant emissions, both with harmful effects to society. Active management strategies imply regulating traffic demand and improving freeway capacity. While both aspects are necessary, the present thesis only addresses the supply side. Part of the research in traffic flow theory is grounded on empirical data. Today, in order to extend our knowledge on traffic dynamics, detailed and high-quality data is needed. To that end, the thesis presents a pioneering data collection campaign, which was developed in a freeway accessing Barcelona. In a Variable Speed Limits (VSL) environment, different speed limits where posted, in order to observe their real and detailed effects on traffic. All the installed surveillance instruments were set to capture data in the highest possible level of detail, including video recordings, from where to count lane-changing maneuvers. With this objective, a semi-automatic method to reliably count lane changes form video recordings was developed and is presented in the thesis. Data analysis proved that the speed limit fulfillment was only relevant in sections with enforcement devices. In these sections, it is confirmed that, the lower the speed limit, the higher the occupancy to achieve a given flow. In contrast, the usually assumed mainline metering effect of low speed limits was not relevant. This might be different in case of stretch enforcement. These findings mean that, on the one hand, VSL strategies aiming to restrict the mainline flow on a freeway by using low speed limits will need to be applied carefully, avoiding conditions as the ones presented here. On the other hand, VSL strategies trying to get the most from the increased vehicle storage capacity of freeways under low speed limits might be rather promising. Results also show that low speed limits increase the speed differences across lanes for moderate demands. This, in turn, also increases the lane changing rates. In contrast, lower speed limits widen the range of flows under uniform lane flow distributions, so that, even for moderate to low demands, the under-utilization of any lane can be avoided. Further analysis of lane-changing activity allowed unveiling that high lane-changing rates prevent achieving the highest flows. This inverse relationship is modeled in the thesis using a stochastic model based on Bayesian inference. This model could be used as a control tool, in order to determine which level of lane-changing activity can be allowed to achieve a desired capacity with some level of reliability. Previous results identify drivers' fulfillment of traffic regulations as a weak point in order to maximize the benefits of current management strategies, like VSL or lane-changing control. This is likely to change in the near future with the irruption of Autonomous Vehicles (AV) in freeways. V2X communications will allow directly actuating on individual vehicles with high accuracy. This will open the door to new management strategies based on simultaneous communication to groups of AVs and extremely short reaction times, like platooning, which stands out as a strategy with a huge potential to improve freeway traffic. Strings of AVs traveling at extremely short gaps (i.e. platoons) allow achieving higher capacities and lower energy consumption rates. In this context, the thesis presents a parsimonious macroscopic model for AVs platooning in mixed traffic (i.e. platoons of AVs travelling together with human driven vehicles). The model allows determining the average platoon length and reproducing the overall traffic dynamics leading to higher capacities. Results prove that with a 50% penetration rate of AVs in the lane, capacity could reach 3400 veh/h/lane under a cooperative platooning strategy.Per tal de millorar la capacitat i reduir la congestió a les autopistes cal gestionar el trànsit de manera activa. Les estratègies de gestió activa del trànsit són d’especial importància en autopistes metropolitanes. La congestió provoca retards i un increment del consum de combustible que va lligat a unes majors emissions de gasos contaminants, tots amb efectes perniciosos per la societat. La gestió activa del transit requereix regular la demanda i millorar la capacitat de la via. Encara que tots dos aspectes son necessaris, la present tesis només analitza la gestió de l’oferta. Part de la recerca en l’anàlisi i la teoria del trànsit es basa en dades empíriques. Per satisfer el requeriment de dades detallades i d’alta qualitat, aquesta tesis presenta una campanya pionera de recol·lecció de dades. Les dades es van recollir a l’autopista B-23 d’accés a Barcelona. Tots els instruments de mesura es van configurar per tal de registrar les dades amb el major nivell de detall possible, incloent les càmeres de videovigilància, d’on es varen extreure els comptatges de canvi de carril. Amb aquest objectiu, es va desenvolupar una metodologia semiautomàtica per comptar canvis de carril a partir de gravacions de trànsit, que es presenta en el cos de la tesi. L’anàlisi de les dades obtingudes ha demostrat que el compliment dels límits de velocitat només resulta rellevant en aquelles seccions que compten amb un radar. És en aquestes seccions on s’ha confirmat que com menor és el límit de velocitat, major es l’ocupació per a un flux donat. Per contra, la hipòtesi habitual de que uns límits de velocitat baixos produeixen una restricció del flux no es va observar de forma rellevant. Aquest comportament podria esser diferent en el cas d’implantar un radar de tram. Els resultats obtinguts també mostren com les diferències de velocitats entre carrils s’incrementen per a límits de velocitat baixos i en condicions de demanda moderada. Això, alhora, incrementa el nombre de canvis de carril. Per contra, els límits de velocitat baixos contribueixen a una distribució de flux més uniforme entre carrils, de forma que es pot evitar la infrautilització de carrils. L’anàlisi més detallat de l’activitat de canvi de carril demostra que una taxa elevada de canvis de carril impedeix assolir fluxos grans de circulació. En la tesi, aquesta relació inversa entre la taxa de canvis de carril i el flux màxim de trànsit a l’autopista s’ha modelat de forma estocàstica utilitzant un model basat en la inferència Bayesiana. Aquest model es pot utilitzar com una eina de control, per tal de determinar quina taxa de canvi de carril es pot permetre si es vol assolir una capacitat determinada amb una determinada probabilitat de compliment. En vista dels resultats previs, la falta de compliment de les normes de trànsit per part dels conductors s’identifica com un punt dèbil a l’hora de maximitzar els beneficis de les actuals estratègies de gestió del transit. Això probablement canviarà en el futur pròxim amb la irrupció dels Vehicles Autònoms (VA) a les autopistes. Els sistemes de comunicació V2X permetran actuar individualment sobre cada vehicle amb una gran precisió. Això obrirà la porta a noves estratègies de gestió, basades en la comunicació simultània entre diferents grups de VA i en temps de reacció extremadament curts, com per exemple és el “platooning”, que destaca pel seu gran potencial per millorar el trànsit en autopista. Els “platons” son cadenes de VA viatjant amb uns espaiaments extremadament curts que permeten assolir capacitats mes elevades i un menor consum energètic. En aquest context, la tesi presenta un model macroscòpic parsimoniós per a “platons” de VA en condicions de transit mixt, és a dir, compartint la infraestructura amb vehicles tradicionals. El model permet determinar la longitud mitjana del “platons” i reproduir el trànsit global dinàmiques que condueixen a majors capacitats. Els resultats demostren que amb un 50% la velocitat de penetració dels AV al carril, la capacitat podria arribar als 3.400 vehicles / h / carril sota una estratègia cooperativa de “platooning”Award-winningPostprint (published version

A right hemisphere advantage for processing blurred faces

No description supplie