Multi Sensor Multi Target Perception and Tracking for Informed Decisions in Public Road Scenarios

Multi-target tracking in public traffic calls for a tracking system with automated track initiation and termination facilities in a randomly evolving driving environment. Besides, the key problem of data association needs to be handled effectively considering the limitations in the computational resources on-board an autonomous car. The challenge of the tracking problem is further evident in the use of high-resolution automotive sensors which return multiple detections per object. Furthermore, it is customary to use multiple sensors that cover different and/or over-lapping Field of View and fuse sensor detections to provide robust and reliable tracking. As a consequence, in high-resolution multi-sensor settings, the data association uncertainty, and the corresponding tracking complexity increases pointing to a systematic approach to handle and process sensor detections. In this work, we present a multi-target tracking system that addresses target birth/initiation and death/termination processes with automatic track management features. These tracking functionalities can help facilitate perception during common events in public traffic as participants (suddenly) change lanes, navigate intersections, overtake and/or brake in emergencies, etc. Various tracking approaches including the ones based on joint integrated probability data association (JIPDA) filter, Linear Multi-target Integrated Probabilistic Data Association (LMIPDA) Filter, and their multi-detection variants are adapted to specifically include algorithms that handle track initiation and termination, clutter density estimation and track management. The utility of the filtering module is further elaborated by integrating it into a trajectory tracking problem based on model predictive control. To cope with tracking complexity in the case of multiple high-resolution sensors, we propose a hybrid scheme that combines the approaches of data clustering at the local sensor and multiple detections tracking schemes at the fusion layer. We implement a track-to-track fusion scheme that de-correlates local (sensor) tracks to avoid double counting and apply a measurement partitioning scheme to re-purpose the LMIPDA tracking algorithm to multi-detection cases. In addition to the measurement partitioning approach, a joint extent and kinematic state estimation scheme are integrated into the LMIPDA approach to facilitate perception and tracking of an individual as well as group targets as applied to multi-lane public traffic. We formulate the tracking problem as a two hierarchical layer. This arrangement enhances the multi-target tracking performance in situations including but not limited to target initialization(birth process), target occlusion, missed detections, unresolved measurement, target maneuver, etc. Also, target groups expose complex individual target interactions to help in situation assessment which is challenging to capture otherwise. The simulation studies are complemented by experimental studies performed on single and multiple (group) targets. Target detections are collected from a high-resolution radar at a frequency of 20Hz; whereas RTK-GPS data is made available as ground truth for one of the target vehicle\u27s trajectory

Best Linear Unbiased Estimation Fusion with Constraints

Estimation fusion, or data fusion for estimation, is the problem of how to best utilize useful information contained in multiple data sets for the purpose of estimating an unknown quantity — a parameter or a process. Estimation fusion with constraints gives rise to challenging theoretical problems given the observations from multiple geometrically dispersed sensors: Under dimensionality constraints, how to preprocess data at each local sensor to achieve the best estimation accuracy at the fusion center? Under communication bandwidth constraints, how to quantize local sensor data to minimize the estimation error at the fusion center? Under constraints on storage, how to optimally update state estimates at the fusion center with out-of-sequence measurements? Under constraints on storage, how to apply the out-of-sequence measurements (OOSM) update algorithm to multi-sensor multi-target tracking in clutter? The present work is devoted to the above topics by applying the best linear unbiased estimation (BLUE) fusion. We propose optimal data compression by reducing sensor data from a higher dimension to a lower dimension with minimal or no performance loss at the fusion center. For single-sensor and some particular multiple-sensor systems, we obtain the explicit optimal compression rule. For a multisensor system with a general dimensionality requirement, we propose the Gauss-Seidel iterative algorithm to search for the optimal compression rule. Another way to accomplish sensor data compression is to find an optimal sensor quantizer. Using BLUE fusion rules, we develop optimal sensor data quantization schemes according to the bit rate constraints in communication between each sensor and the fusion center. For a dynamic system, how to perform the state estimation and sensor quantization update simultaneously is also established, along with a closed form of a recursion for a linear system with additive white Gaussian noise. A globally optimal OOSM update algorithm and a constrained optimal update algorithm are derived to solve one-lag as well as multi-lag OOSM update problems. In order to extend the OOSM update algorithms to multisensor multitarget tracking in clutter, we also study the performance of OOSM update associated with the Probabilistic Data Association (PDA) algorithm

Research Article An Improved Data Correlation Algorithm for Multi-passive-sensor Tracking System

Abstract: For improving the performance of the optimal assignment problem of data correlation of multi-passivesensor system, an improved optimal assignment algorithm based on multi-source information fusion is put forward. The new algorithm takes advantage of the optimal solution and a certain number of near-optimal solutions of the traditional optimal assignment problem to construct a set of effective multi-tuple of measurement and constructs correlation probability fusing multi-source information between above effective multi-tuple of measurement and target track by using combination rule of D-S evidence theory. The result of simulation experiments shows that, compared with the traditional optimal assignment algorithm, the new algorithm not only improves the accuracy of multi-target tracking in different degrees but also saves a lot of time. So it is an effective data correlation algorithm for multi-passive-sensor system

Target detection, tracking, and localization using multi-spectral image fusion and RF Doppler differentials

It is critical for defense and security applications to have a high probability of detection and low false alarm rate while operating over a wide variety of conditions. Sensor fusion, which is the the process of combining data from two or more sensors, has been utilized to improve the performance of a system by exploiting the strengths of each sensor. This dissertation presents algorithms to fuse multi-sensor data that improves system performance by increasing detection rates, lowering false alarms, and improving track performance. Furthermore, this dissertation presents a framework for comparing algorithm error for image registration which is a critical pre-processing step for multi-spectral image fusion. First, I present an algorithm to improve detection and tracking performance for moving targets in a cluttered urban environment by fusing foreground maps from multi-spectral imagery. Most research in image fusion consider visible and long-wave infrared bands; I examine these bands along with near infrared and mid-wave infrared. To localize and track a particular target of interest, I present an algorithm to fuse output from the multi-spectral image tracker with a constellation of RF sensors measuring a specific cellular emanation. The fusion algorithm matches the Doppler differential from the RF sensors with the theoretical Doppler Differential of the video tracker output by selecting the sensor pair that minimizes the absolute difference or root-mean-square difference. Finally, a framework to quantify shift-estimation error for both area- and feature-based algorithms is presented. By exploiting synthetically generated visible and long-wave infrared imagery, error metrics are computed and compared for a number of area- and feature-based shift estimation algorithms. A number of key results are presented in this dissertation. The multi-spectral image tracker improves the location accuracy of the algorithm while improving the detection rate and lowering false alarms for most spectral bands. All 12 moving targets were tracked through the video sequence with only one lost track that was later recovered. Targets from the multi-spectral tracking algorithm were correctly associated with their corresponding cellular emanation for all targets at lower measurement uncertainty using the root-mean-square difference while also having a high confidence ratio for selecting the true target from background targets. For the area-based algorithms and the synthetic air-field image pair, the DFT and ECC algorithms produces sub-pixel shift-estimation error in regions such as shadows and high contrast painted line regions. The edge orientation feature descriptors increase the number of sub-field estimates while improving the shift-estimation error compared to the Lowe descriptor

다중 보행자 인지를 위한 센서 융합 알고리즘 개발

학위논문(석사)--서울대학교 대학원 :공과대학 기계항공공학부,2019. 8. 이경수.환경 센서를 이용하여 보행자를 인지하고 추적하는 알고리즘은 안전한 도심 자율주행을 위해 가장 중요한 기술 중 하나이다. 본 논문은 상업용 비전 센서, 라이다 센서, 그리고 디지털 지도 정보를 융합해 보행자를 추적하는 새로운 알고리즘을 제시한다. 상업용 비전 센서는 보행자를 효과적으로 탐지하는 반면 라이다 센서는 거리를 정확하게 측정한다. 본 시스템은 상업용 비전 센서를 이용해 보행자를 탐지하며, 라이다 센서를 이용하여 상태 추정 성능을 향상시켰다. 또한 디지털 지도를 이용해 라이다 센서의 관심 영역을 설정하였다. 탐지 결과는 서울대학교 캠퍼스에서 약 4600프레임 주행 데이터로, 추정의 정확성은 주행 실험을 통해 검증하여 복잡한 도심 주행 상황에서도 본 알고리즘이 유용함을 검증하였다.Pedestrian detection and tracking algorithm using environmental sensors is one of the most fundamental technology for safe urban autonomous driving. This paper presents a novel sensor fusion algorithm for multi pedestrian tracking using commercial vision sensor, LiDAR sensor, and digital HD map. The commercial vision sensor effectively detects pedestrian, whereas LiDAR sensor accurately measures a distance. Our system uses commercial vision sensor as detector and utilize LiDAR sensor to enhance estimation. In addition, digital HD map is utilized to properly define Region of Interest (ROI) of LiDAR sensor point cloud data. The detection performance is validated by about 4600 frames of SNU campus driving data and estimation accuracy is calculated through driving experiment. The proposed algorithm can be utilized for autonomous driving vehicles in various urban driving situationChapter 1 Introduction……………………………………………………1 1.1 Motivation…………………………………………………………1 1.2 Previous Research………………………………………………3 1.3 Contributions……………………………………………………4 1.4 Thesis Outline ……………………………………………………5 Chapter 2 System Architecture ……………………………………………6 2.1 Vehicle Sensor Configuration……………………………………6 2.2 Fusion Architecture………………………………………………8 Chapter 3 Vision Track Management & Filtering………………………9 3.1 Filtering for Target Tracking……………………………………10 3.1.1 Process Model……………………………………………10 3.1.2 Measurement model……………………………………13 3.2 Data Association…………………………………………………14 Chapter 4 Vision Guided LiDAR Track Management & Filtering……15 4.1 Cluster Validation…………………………………………………17 4.2 Filtering for Target Tracking……………………………………18 4.2.1 Process Model…………………………………………18 4.2.2 Measurement model…………………………………18 4.3 Track Management Rule………………………………………19 Chapter 5 Fusion Method…………………………………………………20 5.1 Track Association…………………………………………………20 5.2 State Fusion…………………………………………………………21 Chapter 6 Experimental Result……………………………………………22 6.1 Track Initializing and Association Probability along Longitudinal distance ……………………………………………………………………………………23 6.2 Detection & Association Rate in SNU Campus Driving Data…25 6.3 Error of States……………………………………………………………26 Chapter 7 Conclusion ………………………………………………………28 Bibliography……………………………………………………………………29 국문 초록…………………………………………………………………………32Maste

A scheme on indoor tracking of ship dynamic positioning based on distributed multi-sensor data fusion

Investigating the model ship dynamic positioning system by simulating the actual sea conditions in the laboratory can not only avoid the risks caused by the directly experiments on a true ship, but also reduce the costs. With the purpose of realizing the high accuracy control of the dynamic positioning, besides a high accuracy mathematical model of the ship, an important condition is that the position information provided by the position detection system must be accurate, reliable and continuous. The global positioning system (GPS) signal is restricted when the model ship dynamic positioning system is set indoors. This paper describes a novel scheme for ship target tracking based on the multi-sensor data fusion techniques. To improve the accuracy of indoor positioning and ship target tracking, the characteristics of many sensors are systematically analyzed, such as radar, difference global positioning system (DGPS) and ultrasonic sensors. Other important factors, including the indoor temperature, position and environment, are also taken into account to further optimize the performance. Combining the Kalman filter method, the time alignment method, the coordinate transformation method and the optimal fusion criterion method, the core algorithm of our framework employs the track correlation as the performance index of the optimal fusion. The experimental results indicate that our method outperforms the methods based on a single ultrasonic sensor. The maximum error between the estimated location and the real location is only 1.32 cm, which meets the standard for engineering applications

A Dempster-Shafer Method for Multi-Sensor Fusion

The Dempster-Shafer Theory, a generalization of the Bayesian theory, is based on the idea of belief and as such can handle ignorance. When all of the required information is available, many data fusion methods provide a solid approach. Yet, most do not have a good way of dealing with ignorance. In the absence of information, these methods must then make assumptions about the sensor data. However, the real data may not fit well within the assumed model. Consequently, the results are often unsatisfactory and inconsistent. The Dempster-Shafer Theory is not hindered by incomplete models or by the lack of prior information. Evidence is assigned based solely on what is known, and nothing is assumed. Hence, it can provide a fast and accurate means for multi-sensor fusion with ignorance. In this research, we apply the Dempster-Shafer Theory in target tracking and in gait analysis. We also discuss the Dempster-Shafer framework for fusing data from a Global Positioning System (GPS) and an Inertial Measurement Unit (IMU) sensor unit for precise local navigation. Within this application, we present solutions where GPS outages occur

Bibliographic Review on Distributed Kalman Filtering

In recent years, a compelling need has arisen to understand the effects of distributed information structures on estimation and filtering. In this paper, a bibliographical review on distributed Kalman filtering (DKF) is provided.\ud The paper contains a classification of different approaches and methods involved to DKF. The applications of DKF are also discussed and explained separately. A comparison of different approaches is briefly carried out. Focuses on the contemporary research are also addressed with emphasis on the practical applications of the techniques. An exhaustive list of publications, linked directly or indirectly to DKF in the open literature, is compiled to provide an overall picture of different developing aspects of this area

Using Bayesian Programming for Multisensor Multi-Target Tracking in Automative Applications

A prerequisite to the design of future Advanced Driver Assistance Systems for cars is a sensing system providing all the information required for high-level driving assistance tasks. Carsense is a European project whose purpose is to develop such a new sensing system. It will combine different sensors (laser, radar and video) and will rely on the fusion of the information coming from these sensors in order to achieve better accuracy, robustness and an increase of the information content. This paper demonstrates the interest of using probabilistic reasoning techniques to address this challenging multi-sensor data fusion problem. The approach used is called Bayesian Programming. It is a general approach based on an implementation of the Bayesian theory. It was introduced rst to design robot control programs but its scope of application is much broader and it can be used whenever one has to deal with problems involving uncertain or incomplete knowledge