Multi-camera Realtime 3D Tracking of Multiple Flying Animals

Automated tracking of animal movement allows analyses that would not otherwise be possible by providing great quantities of data. The additional capability of tracking in realtime - with minimal latency - opens up the experimental possibility of manipulating sensory feedback, thus allowing detailed explorations of the neural basis for control of behavior. Here we describe a new system capable of tracking the position and body orientation of animals such as flies and birds. The system operates with less than 40 msec latency and can track multiple animals simultaneously. To achieve these results, a multi target tracking algorithm was developed based on the Extended Kalman Filter and the Nearest Neighbor Standard Filter data association algorithm. In one implementation, an eleven camera system is capable of tracking three flies simultaneously at 60 frames per second using a gigabit network of nine standard Intel Pentium 4 and Core 2 Duo computers. This manuscript presents the rationale and details of the algorithms employed and shows three implementations of the system. An experiment was performed using the tracking system to measure the effect of visual contrast on the flight speed of Drosophila melanogaster. At low contrasts, speed is more variable and faster on average than at high contrasts. Thus, the system is already a useful tool to study the neurobiology and behavior of freely flying animals. If combined with other techniques, such as `virtual reality'-type computer graphics or genetic manipulation, the tracking system would offer a powerful new way to investigate the biology of flying animals.Comment: pdfTeX using libpoppler 3.141592-1.40.3-2.2 (Web2C 7.5.6), 18 pages with 9 figure

Trajectory generation for lane-change maneuver of autonomous vehicles

Lane-change maneuver is one of the most thoroughly investigated automatic driving operations that can be used by an autonomous self-driving vehicle as a primitive for performing more complex operations like merging, entering/exiting highways or overtaking another vehicle. This thesis focuses on two coherent problems that are associated with the trajectory generation for lane-change maneuvers of autonomous vehicles in a highway scenario: (i) an effective velocity estimation of neighboring vehicles under different road scenarios involving linear and curvilinear motion of the vehicles, and (ii) trajectory generation based on the estimated velocities of neighboring vehicles for safe operation of self-driving cars during lane-change maneuvers. ^ We first propose a two-stage, interactive-multiple-model-based estimator to perform multi-target tracking of neighboring vehicles in a lane-changing scenario. The first stage deals with an adaptive window based turn-rate estimation for tracking maneuvering target vehicles using Kalman filter. In the second stage, variable-structure models with updated estimated turn-rate are utilized to perform data association followed by velocity estimation. Based on the estimated velocities of neighboring vehicles, piecewise Bezier-curve-based methods that minimize the safety/collision risk involved and maximize the comfort ride have been developed for the generation of desired trajectory for lane-change maneuvers. The proposed velocity-estimation and trajectory-generation algorithms have been validated experimentally using Pioneer3- DX mobile robots in a simulated lane-change environment as well as validated by computer simulations

Extended Object Tracking: Introduction, Overview and Applications

This article provides an elaborate overview of current research in extended object tracking. We provide a clear definition of the extended object tracking problem and discuss its delimitation to other types of object tracking. Next, different aspects of extended object modelling are extensively discussed. Subsequently, we give a tutorial introduction to two basic and well used extended object tracking approaches - the random matrix approach and the Kalman filter-based approach for star-convex shapes. The next part treats the tracking of multiple extended objects and elaborates how the large number of feasible association hypotheses can be tackled using both Random Finite Set (RFS) and Non-RFS multi-object trackers. The article concludes with a summary of current applications, where four example applications involving camera, X-band radar, light detection and ranging (lidar), red-green-blue-depth (RGB-D) sensors are highlighted.Comment: 30 pages, 19 figure

Study of Multi-Modal and Non-Gaussian Probability Density Functions in Target Tracking with Applications to Dim Target Tracking

The majority of deployed target tracking systems use some variant of the Kalman filter for their state estimation algorithm. In order for a Kalman filter to be optimal, the measurement and state equations must be linear and the process and measurement noises must be Gaussian random variables (or vectors). One problem arises when the state or measurement function becomes a multi-modal Gaussian mixture. This typically occurs with the interactive multiple model (IMM) technique and its derivatives and also with probabilistic and joint probabilistic data association (PDA/JPDA) algorithms. Another common problem in target tracking is that the target\u27s signal-to-noise ratio (SNR) at the sensor is often low. This situation is often referred to as the dim target tracking or track-before-detect (TBD) scenario. When this occurs, the probability density function (PDF) of the measurement likelihood function becomes non-Gaussian and often has a Rayleigh or Ricean distribution. In this case, a Kalman filter variant may also perform poorly. The common solution to both of these problems is the particle filter (PF). A key drawback of PF algorithms, however, is that they are computationally expensive. This dissertation, thus, concentrates on developing PF algorithms that provide comparable performance to conventional PFs but at lower particle costs and presents the following four research efforts. 1. A multirate multiple model particle filter (MRMMPF) is presented in Section-3. The MRMMPF tracks a single, high signal-to-noise-ratio, maneuvering target in clutter. It coherently accumulates measurement information over multiple scans via discrete wavelet transforms (DWT) and multirate processing. This provides the MRMMPF with a much stronger data association capability than is possible with a single scan algorithm. In addition, its particle filter nature allows it to better handle multiple modes that arise from multiple target motion models. Consequently, the MRMMPF provides substantially better root-mean-square error (RMSE) tracking performance than either a full-rate or multirate Kalman filter tracker or full-rate multiple model particle filter (MMPF) with a same particle count. 2. A full-rate multiple model particle filter for track-before-detect (MMPF-TBD) and a multirate multiple model particle filter for track-before-detect (MRMMPF-TBD) are presented in Section-4. These algorithms extend the areas mentioned above and track low SNR targets which perform small maneuvers. The MRMMPF-TBD and MMPF-TBD both use a combined probabilistic data association (PDA) and maximum likelihood (ML) approach. The MRMMPF-TBD provides equivalent RMSE performance at substantially lower particle counts than a full-rate MMPF-TBD. In addition, the MRMMPF-TBD tracks very dim constant velocity targets that the MMPF-TBD cannot. 3. An extended spatial domain multiresolutional particle filter (E-SD-MRES-PF) is developed in Section-5. The E-SD-MRES-PF modifies and extends a recently developed spatial domain multiresolutional particle filter prototype. The prototype SD-MRES-PF was only demonstrated for one update cycle. In contrast, E-SD-MRES-PF functions over multiple update cycles and provides comparable RMSE performance at a reduced particle cost under a variety of PDF scenarios. 4. Two variants of a single-target Gaussian mixture model particle filter (GMMPF) are presented in Section-6. The GMMPF models the particle cloud as a Gaussian finite mixture model (FMM). MATLAB simulations show that the GMMPF provides performance comparable to a particle filter but at a lower particle cost

Variational Bayesian Expectation Maximization for Radar Map Estimation

For self-localization, a detailed and reliable map of the environment can be used to relate sensor data to static features with known locations. This paper presents a method for construction of detailed radar maps that describe the expected intensity of detections. Specifically, the measurements are modelled by an inhomogeneous Poisson process with a spatial intensity function given by the sum of a constant clutter level and an unnormalized Gaussian mixture. A substantial difficulty with radar mapping is the presence of data association uncertainties, i.e., the unknown associations between measurements and landmarks. In this paper, the association variables are introduced as hidden variables in a variational Bayesian expectation maximization (VBEM) framework, resulting in a computationally efficient mapping algorithm that enables a joint estimation of the number of landmarks and their parameters

Context Exploitation in Data Fusion

Complex and dynamic environments constitute a challenge for existing tracking algorithms. For this reason, modern solutions are trying to utilize any available information which could help to constrain, improve or explain the measurements. So called Context Information (CI) is understood as information that surrounds an element of interest, whose knowledge may help understanding the (estimated) situation and also in reacting to that situation. However, context discovery and exploitation are still largely unexplored research topics. Until now, the context has been extensively exploited as a parameter in system and measurement models which led to the development of numerous approaches for the linear or non-linear constrained estimation and target tracking. More specifically, the spatial or static context is the most common source of the ambient information, i.e. features, utilized for recursive enhancement of the state variables either in the prediction or the measurement update of the filters. In the case of multiple model estimators, context can not only be related to the state but also to a certain mode of the filter. Common practice for multiple model scenarios is to represent states and context as a joint distribution of Gaussian mixtures. These approaches are commonly referred as the join tracking and classification. Alternatively, the usefulness of context was also demonstrated in aiding the measurement data association. Process of formulating a hypothesis, which assigns a particular measurement to the track, is traditionally governed by the empirical knowledge of the noise characteristics of sensors and operating environment, i.e. probability of detection, false alarm, clutter noise, which can be further enhanced by conditioning on context. We believe that interactions between the environment and the object could be classified into actions, activities and intents, and formed into structured graphs with contextual links translated into arcs. By learning the environment model we will be able to make prediction on the target\u2019s future actions based on its past observation. Probability of target future action could be utilized in the fusion process to adjust tracker confidence on measurements. By incorporating contextual knowledge of the environment, in the form of a likelihood function, in the filter measurement update step, we have been able to reduce uncertainties of the tracking solution and improve the consistency of the track. The promising results demonstrate that the fusion of CI brings a significant performance improvement in comparison to the regular tracking approaches

Performance evaluation of track association and maintenance for a MFPAR with doppler velocity measurements

This study investigates the effects of incorporating Doppler velocity measurements directly into track association and maintenance parts for single and multiple target tracking unit in a multi function phased array radar (MFPAR). Since Doppler velocity is the major discriminant of clutter from a desired target, the measurement set has been expanded from range, azimuth and elevation angles to include Doppler velocity measurements. We have developed data association and maintenance part of a well known tracking method, Interacting Multiple Model Probabilistic Data Association

Decentralized kalman filter approach for multi-sensor multi-target tracking problems

06.03.2018 tarihli ve 30352 sayılı Resmi Gazetede yayımlanan “Yükseköğretim Kanunu İle Bazı Kanun Ve Kanun Hükmünde Kararnamelerde Değişiklik Yapılması Hakkında Kanun” ile 18.06.2018 tarihli “Lisansüstü Tezlerin Elektronik Ortamda Toplanması, Düzenlenmesi ve Erişime Açılmasına İlişkin Yönerge” gereğince tam metin erişime açılmıştır.Doğru pozisyon ve hedeflerin sayısı hava trafik kontrol ve füze savunması için çok önemli bilgilerdir. Bu çalışma, çoklu sensorlü çoklu hedef takibi sistemlerindeki veri füzyonu ve durum tahmini problemlerı için dağıtık Kalman Filtreleme Algoritması sunmaktadır. Problem, radar olarak her biri kendi veri işleme birimine sahip aktif sensörlerin hedef alanını gözlemlemesini esas almaktadır. Bu durumda her bir sistemin iz sayısı olacaktır. Çalışmada önerilen dağıtık Kalman Filtresi, başta füze sistemleri olmak üzere savunma sistemlerinde hareketli hedeflerin farklı sensörlerle izlerini kestirmek ve farklı hedefleri ayrıd etmek için kullanmaktır. Önerilen teknik, çoklu sensör sisteminden gelen verileri işleyen iki aşamalı veri işleme yaklaşımını içermektedir. İlk aşamada, her yerel işlemci kendi verilerini ve standart Kalman filtresi ise en iyi kestirimi yapmak için kullanılmaktadır. Sonraki aşamada bu kestirimler en iyi küresel bir kestirimi yapmak amacıyla dağıtık işlem modunda elde edilir. Bu çalışmada iki radar sistemi iki yerel Kalman filtresi ile uçakların pozisyonunu kestirmek amacıyla kullanılmakta, ardından bu kestirimler merkez işlemciye iletilmektedir. Merkez işlemci doğrulama maksadıyla bu bilgileri birleştirip küresel bir kestirim üretmektedir. Önerilen model uygulama olarak dört senaryo üzerinde test edildi. İlk senaryoda, tek bir hedef iki sensor tarafından izlenirken, ikincisinde, iki hedeften oluşan uzay herhangi bir sensor tarafından izlenmekte, üçüncüsünde, iki hedefin de herhangi bir sensor tarafından aynı anda izlenmesi, son olarak ise iki sensörden her birinin toplam üç hedeften herhangi ikisini izlediği senaryo göz önüne alınmıştır. Önerilen tekniğin performansı hata kovaryans matrisi kullanılarak değerlendirildi ve yüksek doğruluk ve optimal kestirim elde edildi. Uygulama sonuçları önerilen tekniğin yeteneğinin, yerel sensörlerce belirlenen ortak hedeflerin merkezi sistem tarafından ayırd edilebildiğini göstermiştir.For air traffic control and missile defense, the accurate position and the numbers of targets are the most important information needed. This thesis presents a decentralized kalman filtering algorithm (DKF) for data fusion and state estimation problems in multi-sensor multi-target tracking system. The problem arises when several sensors carry out surveillance over a certain area and each sensor has its own data processing system. In this situation, each system has a number of tracks. The DKF is used to estimate and separate the tracks from different sensors represent the targets, when the ability to track targets is essential in missile defense. The proposed technique is a two stage data processing technique which processes data from multi sensor system. In the first stage, each local processor uses its own data to make the best local estimation using standard kalman filter and then these estimations are then obtained in parallel processing mode to make best global estimation. In this work, two radar systems are used as sensors with two local Kalman filters to estimate the position of an aircraft and then they transmit these estimations to a central processor, which combines this information to produce a global estimation. The proposed model is tested on four scenarios, firstly, when there is one target and the two sensors are tracking the same target, secondly, when there are two targets and any sensor is tracking one of them, thirdly, when there are two targets and any sensor is tracking both of them and finally, when two sensors are used to track three targets and any sensor tracks any two of them. The performance of the proposed technique is evaluated using measures such as the error covariance matrix and it gave high accuracy and optimal estimation. The experimental results showed that the proposed method has the ability to separate the joint targets detected by the local sensors