Context Exploitation in Data Fusion

Complex and dynamic environments constitute a challenge for existing tracking algorithms. For this reason, modern solutions are trying to utilize any available information which could help to constrain, improve or explain the measurements. So called Context Information (CI) is understood as information that surrounds an element of interest, whose knowledge may help understanding the (estimated) situation and also in reacting to that situation. However, context discovery and exploitation are still largely unexplored research topics. Until now, the context has been extensively exploited as a parameter in system and measurement models which led to the development of numerous approaches for the linear or non-linear constrained estimation and target tracking. More specifically, the spatial or static context is the most common source of the ambient information, i.e. features, utilized for recursive enhancement of the state variables either in the prediction or the measurement update of the filters. In the case of multiple model estimators, context can not only be related to the state but also to a certain mode of the filter. Common practice for multiple model scenarios is to represent states and context as a joint distribution of Gaussian mixtures. These approaches are commonly referred as the join tracking and classification. Alternatively, the usefulness of context was also demonstrated in aiding the measurement data association. Process of formulating a hypothesis, which assigns a particular measurement to the track, is traditionally governed by the empirical knowledge of the noise characteristics of sensors and operating environment, i.e. probability of detection, false alarm, clutter noise, which can be further enhanced by conditioning on context. We believe that interactions between the environment and the object could be classified into actions, activities and intents, and formed into structured graphs with contextual links translated into arcs. By learning the environment model we will be able to make prediction on the target\u2019s future actions based on its past observation. Probability of target future action could be utilized in the fusion process to adjust tracker confidence on measurements. By incorporating contextual knowledge of the environment, in the form of a likelihood function, in the filter measurement update step, we have been able to reduce uncertainties of the tracking solution and improve the consistency of the track. The promising results demonstrate that the fusion of CI brings a significant performance improvement in comparison to the regular tracking approaches

Contextual information aided target tracking and path planning for autonomous ground vehicles

Recently, autonomous vehicles have received worldwide attentions from academic research, automotive industry and the general public. In order to achieve a higher level of automation, one of the most fundamental requirements of autonomous vehicles is the capability to respond to internal and external changes in a safe, timely and appropriate manner. Situational awareness and decision making are two crucial enabling technologies for safe operation of autonomous vehicles. This thesis presents a solution for improving the automation level of autonomous vehicles in both situational awareness and decision making aspects by utilising additional domain knowledge such as constraints and influence on a moving object caused by environment and interaction between different moving objects. This includes two specific sub-systems, model based target tracking in environmental perception module and motion planning in path planning module. In the first part, a rigorous Bayesian framework is developed for pooling road constraint information and sensor measurement data of a ground vehicle to provide better situational awareness. Consequently, a new multiple targets tracking (MTT) strategy is proposed for solving target tracking problems with nonlinear dynamic systems and additional state constraints. Besides road constraint information, a vehicle movement is generally affected by its surrounding environment known as interaction information. A novel dynamic modelling approach is then proposed by considering the interaction information as virtual force which is constructed by involving the target state, desired dynamics and interaction information. The proposed modelling approach is then accommodated in the proposed MTT strategy for incorporating different types of domain knowledge in a comprehensive manner. In the second part, a new path planning strategy for autonomous vehicles operating in partially known dynamic environment is suggested. The proposed MTT technique is utilized to provide accurate on-board tracking information with associated level of uncertainty. Based on the tracking information, a path planning strategy is developed to generate collision free paths by not only predicting the future states of the moving objects but also taking into account the propagation of the associated estimation uncertainty within a given horizon. To cope with a dynamic and uncertain road environment, the strategy is implemented in a receding horizon fashion

Multi Sensor Multi Target Perception and Tracking for Informed Decisions in Public Road Scenarios

Multi-target tracking in public traffic calls for a tracking system with automated track initiation and termination facilities in a randomly evolving driving environment. Besides, the key problem of data association needs to be handled effectively considering the limitations in the computational resources on-board an autonomous car. The challenge of the tracking problem is further evident in the use of high-resolution automotive sensors which return multiple detections per object. Furthermore, it is customary to use multiple sensors that cover different and/or over-lapping Field of View and fuse sensor detections to provide robust and reliable tracking. As a consequence, in high-resolution multi-sensor settings, the data association uncertainty, and the corresponding tracking complexity increases pointing to a systematic approach to handle and process sensor detections. In this work, we present a multi-target tracking system that addresses target birth/initiation and death/termination processes with automatic track management features. These tracking functionalities can help facilitate perception during common events in public traffic as participants (suddenly) change lanes, navigate intersections, overtake and/or brake in emergencies, etc. Various tracking approaches including the ones based on joint integrated probability data association (JIPDA) filter, Linear Multi-target Integrated Probabilistic Data Association (LMIPDA) Filter, and their multi-detection variants are adapted to specifically include algorithms that handle track initiation and termination, clutter density estimation and track management. The utility of the filtering module is further elaborated by integrating it into a trajectory tracking problem based on model predictive control. To cope with tracking complexity in the case of multiple high-resolution sensors, we propose a hybrid scheme that combines the approaches of data clustering at the local sensor and multiple detections tracking schemes at the fusion layer. We implement a track-to-track fusion scheme that de-correlates local (sensor) tracks to avoid double counting and apply a measurement partitioning scheme to re-purpose the LMIPDA tracking algorithm to multi-detection cases. In addition to the measurement partitioning approach, a joint extent and kinematic state estimation scheme are integrated into the LMIPDA approach to facilitate perception and tracking of an individual as well as group targets as applied to multi-lane public traffic. We formulate the tracking problem as a two hierarchical layer. This arrangement enhances the multi-target tracking performance in situations including but not limited to target initialization(birth process), target occlusion, missed detections, unresolved measurement, target maneuver, etc. Also, target groups expose complex individual target interactions to help in situation assessment which is challenging to capture otherwise. The simulation studies are complemented by experimental studies performed on single and multiple (group) targets. Target detections are collected from a high-resolution radar at a frequency of 20Hz; whereas RTK-GPS data is made available as ground truth for one of the target vehicle\u27s trajectory

Performance of Sensor Fusion for Vehicular Applications

Sensor fusion is a key system in Advanced Driver Assistance Systems, ADAS. The perfor-mance of the sensor fusion depends on many factors such as the sensors used, the kinematicmodel used in the Extended Kalman Filter, EKF, the motion of the vehicles, the type ofroad, the density of vehicles, and the gating methods. The interactions between parametersand the extent to which individual parameters contribute to the overall accuracy of a sensorfusion system can be difficult to assess.In this study, a full-factorial experimental evaluation of a sensor fusion system basedon a real vehicle was performed. The experimental results for different driving scenariosand parameters are discussed and the factors that make the most impact are identified.The performance of sensor fusion depends on many factors such as the sensors used, thekinematic model used in the Extended Kalman Filter (EKF) motion of the vehicles, type ofroad, density of vehicles, and gating methods.This study identified that the distance between the vehicles has the largest impact on theestimation error because the vision sensor performs poorly with increased distance. In addi-tion, it was identified that the kinematic models had no significant impact on the estimation.Last but not least, the ellipsoid gates performed better than rectangular gates.In addition, we propose a new gating algorithm called an angular gate. This algorithmis based on the observation that the data for each target lies in the direction of that target.Therefore, the angle and the range can be used for setting up a two-level gating approachthat is both more intuitive and computationally faster than ellipsoid gates. The angulargates can achieve a speedup factor of up to 2.27 compared to ellipsoid gates.Furthermore, we provide time complexity analysis of angular gates, ellipsoid gates, andrectangular gates demonstrating the theoretical reasons why angular gates perform better.Last, we evaluated the performance of the Munkres algorithm using a full factorial designand identified that narrower gates can speedup the running time of the Munkres algorithmand, surprisingly, even improve the RMSE in some cases.The low target maneuvering index of vehicular systems was identified as the reason whythe kinematic models do not have an impact on the estimation. This finding supports the useof simpler and computationally inexpensive filters instead of complex Interacting MultipleModel filters. The angular gates also improve the computational efficiency of the overallsensor fusion system making them suitable for vehicular application as well as for embeddedsystems and robotics

Context-based Information Fusion: A survey and discussion

This survey aims to provide a comprehensive status of recent and current research on context-based Information Fusion (IF) systems, tracing back the roots of the original thinking behind the development of the concept of \u201ccontext\u201d. It shows how its fortune in the distributed computing world eventually permeated in the world of IF, discussing the current strategies and techniques, and hinting possible future trends. IF processes can represent context at different levels (structural and physical constraints of the scenario, a priori known operational rules between entities and environment, dynamic relationships modelled to interpret the system output, etc.). In addition to the survey, several novel context exploitation dynamics and architectural aspects peculiar to the fusion domain are presented and discussed

Multiple-Object Estimation Techniques for Challenging Scenarios

A series of methods for solving the multi-object estimation problem in the context sequential Bayesian inference is presented. These methods concentrate on dealing with challenging scenarios of multiple target tracking, involving fundamental problems of nonlinearity and non-Gaussianity of processes, high state dimensionality, high number of targets, statistical dependence between target states, and degenerate cases of low signal-to-noise ratio, high uncertainty, lowly observable states or uninformative observations. These difficulties pose obstacles to most practical multi-object inference problems, lying at the heart of the shortcomings reported for state-of-the-art methods, and so elicit novel treatments to enable tackling a broader class of real problems. The novel algorithms offered as solutions in this dissertation address such challenges by acting on the root causes of the associated problems. Often this involves essential dilemmas commonly manifested in Statistics and Decision Theory, such as trading off estimation accuracy with algorithm complexity, soft versus hard decision, generality versus tractability, conciseness versus interpretativeness etc. All proposed algorithms constitute stochastic filters, each of which is formulated to address specific aspects of the challenges at hand while offering tools to achieve judicious compromises in the aforementioned dilemmas. Two of the filters address the weight degeneracy observed in sequential Monte Carlo filters, particularly for nonlinear processes. One of these filters is designed for nonlinear non-Gaussian high-dimensional problems, delivering representativeness of the uncertainty in high-dimensional states while mitigating part of the inaccuracies that arise from the curse of dimensionality. This filter is shown to cope well with scenarios of multimodality, high state uncertainty, uninformative observations and high number of false alarms. A multi-object filter deals with the problem of considering dependencies between target states in a way that is scalable to a large number of targets, by resorting to probabilistic graphical structures. Another multi-object filter treats the problem of reducing the computational complexity of a state-of-the-art cardinalized filter to deal with a large number of targets, without compromising accuracy significantly. Finally, a framework for associating measurements across observation sessions for scenarios of low state observability is proposed, with application to an important Space Surveillance task: cataloging of space debris in the geosynchronous/geostationary belt. The devised methods treat the considered challenges by bringing about rather general questions, and provide not only principled solutions but also analyzes the essence of the investigated problems, extrapolating the implemented techniques to a wider spectrum of similar problems in Signal Processing

Fusion of Data from Heterogeneous Sensors with Distributed Fields of View and Situation Evaluation for Advanced Driver Assistance Systems

In order to develop a driver assistance system for pedestrian protection, pedestrians in the environment of a truck are detected by radars and a camera and are tracked across distributed fields of view using a Joint Integrated Probabilistic Data Association filter. A robust approach for prediction of the system vehicles trajectory is presented. It serves the computation of a probabilistic collision risk based on reachable sets where different sources of uncertainty are taken into account

Adaptivne tehnike u sistemima za praćenje pokretnih ciljeva

The most critical and challenging task in the algorithms of multiple target tracking in the presence of false observations is the correct assignment of measurements to tracks the so-called data association task. That is the core component of all target tracking systems. Regardless of the particular method used, the efficiency of any target tracking system depends on the understanding of the background or clutters “certain parameters that describe the environment”, and the parameters that describe the detection properties of the objects. The character of these parameters is statistical, and not only they are usually unknown in practice, and they are also time-invariant. Moreover, the statistics that describe the environment are spatially dependent. The most important among these are the probability of target detection and the density of false alarm. These parameters are usually unknown as well as variable, and even though there are many algorithms for estimation of these parameters, the usefulness of these estimates is quite limited. Successful implementation of any target tracking system depends on the precise knowledge of the statistical quantities such as the probability of target detection and density of false alarm. This thesis proposes one approach for estimating the time-varying probability of detection of each tracked object individually and the density of false alarm in the immediate vicinity of the current position of an object. The proposed approach is based on the generalized maximum likelihood (GML) approach, assuming the tracking of a single target. To reduce the numerical complexity, the proposed technique reduces the number of the formulated hypotheses based on the calculation of their likelihood. The obtained estimators have a very simple form, but as shown, this simplicity comes with a significant bias, which is present in most similar techniques, and relatively large variance of the estimators. The research presented in the thesis coped with these two problems and resulted in an algorithm with significantly reduced bias and error variances. This thesis also analyses the influences of the unknown measurement noise covariance on an estimation of the probability of target detection and density of false alarm and proposes an improvement in the case of noise covariance matrix uncertainty. The thesis presents the applicability and constraints of the proposed solution. The results are illustrated by simulations and present a fair analysis of the proposed algorithm. Finally, the ideas for further improvement of the method are given.Vrlo izazovan i kritičan zadatak u algoritmima praćenja pokretnih ciljeva uz prisustvo lažnih alarma jeste pravilna asocijacija pristiglih opservacija takozvanim tragovima. To je osnovni i verovatno najvažniji deo svakog sistema za praćenje više pokretnih ciljeva. Bez obzira na to koja se metoda pridruživanja podataka koristi, efikasnost bilo kog takvog sistema itekako zavisi od poznavanja statističkih parametara koji karakterišu okruženje i parametara koji karakterišu ponašanje praćenih objekata, u smislu njihove detektibilnosti. Nažalost, u praksi, ovi podaci nikada nisu poznati, i gore od toga, vremenski su promenljivi, a parametri prisustva takozvanih lažnih alarma sui prostorno zavisni. Najvažniji od tih parametara su verovatnoća detekcije cilja i gustina lažnog alarma. Sama činjenica da postoje različiti pristupi za estimaciju ovih parametara govori, kako o njihovom značaju, tako i o kompleksnosti procedura za njihovu estimaciju. Lako se pokazuje da uspešna primena bilo kog algoritma za praćenje itekako zavisi od kvaliteta i nivoa neodređenosti u poznavanju ovih statističkih parametara kakvi su verovatnoća detekcije cilja i gustina lažnih alarma. U ovoj doktorskoj disertaciji je predložen novi pristup za procenu vremenski promenljive verovatnoće detekcije ciljeva kao i gustine lažnog alarma ali u naposrednom okruženju objekta koji se prati. Predloženi pristup je zasnovan na dobro poznatom metodu maksimalne verodostojnosti, pri čemu je pretpostavljeno da se u prostoru od interesa nalazi samo jedan pokretni objekat. Kako bi se minimizovala numerička složenost predloženog algoritma, minimizovan je i broj hipoteza za koje se računaju odgovarajuće verodostojnosti. Dobijeni estimatori imaju vrlo jednostavnu formu. Međutim, kao što se i očekivalo, statističke osobine dobijenih estimatora su vrlo slične onim estimatorima koji su dostupni u literature. Naime, pokazalo se da izvedeni estimatori imaju značajan pomeraj u proceni kao i nedopustivo veliku varijansu. Zato je posebna pažnja u disertaciji posvećena postupcima za eliminaciju pomeraja i smenjenje varijanse. Pokazano je da se uz minimalno povećanje numeričke složenosti algoritma značajno popravljaju njegove statističke performanse. U ovoj doktorskoj disertaciji je takođe razmatran uticaj nepoznavanja statistika mernog šuma na kvalitet estimatora verovatnoće detekcije ciljeva i gustine lažnih alarma. Pokazano je da ova neodređenost može značajno da degradira kvalitet celokupnog postupka, tako da je predložena dodatna adaptacija koja u kontekstu primenjenog Kalmanovog filtra estimira kovarijacionu matricu mernog šuma. Konačno, u tezi su ilustrovani primenjivost kao i ograničenja predloženog rešenja. Svi zaključci i pretpostavke su potkrepljeni iscrpnim simulacijama koje su kroz Monte Carlo simulacije sa više od 20.000 ponavljanja uspevale da potisnu uticaj nesavršenosti generatora slučajnih brojeva. Na kraju teze su date i ideje za dalje unapređenje predložene metode