Adaptive object segmentation and tracking

Efficient tracking of deformable objects moving with variable velocities is an important current research problem. In this thesis a robust tracking model is proposed for the automatic detection, recognition and tracking of target objects which are subject to variable orientations and velocities and are viewed under variable ambient lighting conditions. The tracking model can be applied to efficiently track fast moving vehicles and other objects in various complex scenarios. The tracking model is evaluated on both colour visible band and infra-red band video sequences acquired from the air by the Sussex police helicopter and other collaborators. The observations made validate the improved performance of the model over existing methods. The thesis is divided in three major sections. The first section details the development of an enhanced active contour for object segmentation. The second section describes an implementation of a global active contour orientation model. The third section describes the tracking model and assesses it performance on the aerial video sequences. In the first part of the thesis an enhanced active contour snake model using the difference of Gaussian (DoG) filter is reported and discussed in detail. An acquisition method based on the enhanced active contour method developed that can assist the proposed tracking system is tested. The active contour model is further enhanced by the use of a disambiguation framework designed to assist multiple object segmentation which is used to demonstrate that the enhanced active contour model can be used for robust multiple object segmentation and tracking. The active contour model developed not only facilitates the efficient update of the tracking filter but also decreases the latency involved in tracking targets in real-time. As far as computational effort is concerned, the active contour model presented improves the computational cost by 85% compared to existing active contour models. The second part of the thesis introduces the global active contour orientation (GACO) technique for statistical measurement of contoured object orientation. It is an overall object orientation measurement method which uses the proposed active contour model along with statistical measurement techniques. The use of the GACO technique, incorporating the active contour model, to measure object orientation angle is discussed in detail. A real-time door surveillance application based on the GACO technique is developed and evaluated on the i-LIDS door surveillance dataset provided by the UK Home Office. The performance results demonstrate the use of GACO to evaluate the door surveillance dataset gives a success rate of 92%. Finally, a combined approach involving the proposed active contour model and an optimal trade-off maximum average correlation height (OT-MACH) filter for tracking is presented. The implementation of methods for controlling the area of support of the OT-MACH filter is discussed in detail. The proposed active contour method as the area of support for the OT-MACH filter is shown to significantly improve the performance of the OT-MACH filter's ability to track vehicles moving within highly cluttered visible and infra-red band video sequence

Comparison between gaze and moving objects in videos for smooth pursuit eye movement evaluation

When viewing moving objects in videos the movement of the eyes is called smooth pursuit. For evaluating the relationship of eye tracking data to the moving objects, the objects in the videos need to be detected and tracked. In the first part of this thesis, a method for detecting and tracking of moving objects in videos is developed. The method mainly consists of a modified version of the Gaussian mixture model, The Tracking feature point method, a modified version of the Mean shift algorithm, Matlabs function bwlabel and a set of new developed methods. The performance of the method is highest when the background is static and the objects differ in colour from the background. The false detection rate increases, when the video environment becomes more dynamic and complex. In the second part of this thesis the distance between the point of gaze and the moving objects centre point is calculated. The eyes may not always follow the centre position of an object, but rather some other part of the object. Therefore, the method gives more satisfactory result when the objects are small.Utvärdering av smooth pursuit-rörelser. En jämförelse mellan ögonrörelser och rörliga objekt i videosekvenser Populärvetenskaplig sammanfattning av examensarbetet: Andrea Åkerström Ett forskningsområde som har vuxit mycket de senaste åren är ”eye tracking”: en teknik för att undersöka ögonrörelser. Tekniken har visat sig intressant för studier inom exempelvis visuella system, i psykologi och i interaktioner mellan datorer och människor. Ett eye tracking system mäter ögonens rörelser så att de punkterna ögat tittar på kan bli estimerade. Tidigare har de flesta studier inom eye tracking baserats på bilder, men på senare tid har även intresset för att studera filmsekvenser vuxit. Den typ av rörelse som ögat utför när det följer ett rörligt objekt kallas för smooth pursuitrörelse. En av svårigheterna med att utvärdera relationen mellan eye tracking-data och rörliga objekten i filmer är att objekten, antingen manuellt mäts ut eller att ett intelligent system utvecklas för en automatisk utvärdering. Det som gör processen att detektera och följa rörliga objekt i filmer komplex är att olika videosekvenser kan ha många olika typer av svåra videoscenarion som metoden måste klara av. Till exempel kan bakgrunden i en video vara dynamisk, det kan finnas störningar som regn eller snö, eller kan problemet vara att kameran skakar eller rör sig. Syftet med detta arbete består av två delar. Den först delen, som också har varit den största, har varit att utveckla en metod som kan detektera och följa rörliga objekt i olika typer av videosekvenser, baserad på metoder från tidigare forskning. Den andra delen har varit att försöka utveckla en automatisk utvärdering av ögonrörelsen smooth persuit, genom att använda de detekterade och följda objekten i videosekvenserna tillsammans med redan existerande ögondata. För att utveckla den metod har olika metoder från tidigare forskning kombinerat. Alla metoder som har utvecklas i detta område har olika för och nackdelar och fungerade bättre eller sämre för olika typer av videoscenarion. Målet för metoden i detta arbete har varit att hitta en kombination av olika metoder som, genom att kompensera varandras för- och nackdelar, kan ge en så bra detektering som möjligt för olika typer av filmsekvenser. Min metod är till största del uppbyggd av tre metoder: En modifierad version av Guasssian Mixture Model, Tracking Feature Point och en modifierad version av Mean Shift Algorithmen. Guassian Mixture Model-metoden används för att detekterar pixlar i filmen som tillhör objekt som är i rörelse. Metoden tar fram dynamiska modeller av bakgrunden i filmen och detekterar pixlar som skiljer sig från backgrundsmodellerna. Detta är en väl använd metod som kan hantera komplexa bakgrunder med periodiskt brus, men den ger samtidigt ofta upphov till felaktiga detektioner och den kan inte hantera kamerarörelser. För att hantera kamerarörelser används Tracking Feature Point-metoden och på så sätt kompenseras denna brist hos Guassian Mixture Modell-metoden. Tracking Feature Point tar fram ”feature points” ut videobilder och med hjälp av dem kan metoden estimera kameraförflyttningar. Denna metod räknar dock endast ut de förflyttningar som kameran gör, men den tar inte hänsyn till om kameran roterar. Mean Shift Algoritm är en metod som används för att räkna ut det rörliga objektets nya position i en efterföljande bild. För mitt arbete har endast delar av denna metod används till att bestämma vilka detektioner av objekt i de olika bilderna som representerar samma objekt. Genom att ta fram modeller för objekten i varje bild, vilka sedan jämförs, kan metoden bestämma vilka objekt som kan klassas som samma objekt. Den metod som har utvecklat i detta arbete gav bäst resultat när bakgrunden var statisk och objektets färg skiljde sig från bakgrunden. När bakgrunden blir mer dynamisk och komplex ökade mängden falska detektioner och för vissa videosekvenser misslyckas metoden att detektera hela objekten. Den andra delen av detta arbetes syfte var att använda resultatet från metoden för att utvärdera eye tracking-data. Den automatiska utvärderingen av ögonrörelsen smooth pursuit ger ett mått på hur bra ögat kan följa objekt som rör sig. För att utföra detta mäts avståndet mellan den punkt som ögat tittar på och det detekterade objektets centrum. Den automatiskt utvärderingen av smooth pursuit-rörelsen gav bäst resultat när objekten var små. För större objekt följer ögat inte nödvändigtvis objektets mittenpunkt utan istället någon annan del av objektet och metoden kan därför i dessa fall ge ett missvisande resultat. Detta arbete har inte resulterat i en färdig metod utan det finns många områden för förbättringar. Exempelvis skulle en estimering av kamerans rotationer förbättra resultaten. Utvärderingen av hur väl ögat följer rörliga objekt kan även utvecklas mer, genom att konturerna av objekten beräknades. På detta sätt skulle även avståndet mellan punkterna ögat tittar på och objektets area kunnat bestämmas. Både eye tracking och att detektera och följa rörliga objekt i filmer är idag aktiva forskningsområden och det finns alltså fortfarande mycket att utveckla i dessa områden. Syfte med detta arbete har varit att försöka utveckla en mer generell metod som kan fungera för olika typer av filmsekvenser

Novel Texture-based Probabilistic Object Recognition and Tracking Techniques for Food Intake Analysis and Traffic Monitoring

More complex image understanding algorithms are increasingly practical in a host of emerging applications. Object tracking has value in surveillance and data farming; and object recognition has applications in surveillance, data management, and industrial automation. In this work we introduce an object recognition application in automated nutritional intake analysis and a tracking application intended for surveillance in low quality videos. Automated food recognition is useful for personal health applications as well as nutritional studies used to improve public health or inform lawmakers. We introduce a complete, end-to-end system for automated food intake measurement. Images taken by a digital camera are analyzed, plates and food are located, food type is determined by neural network, distance and angle of food is determined and 3D volume estimated, the results are cross referenced with a nutritional database, and before and after meal photos are compared to determine nutritional intake. We compare against contemporary systems and provide detailed experimental results of our system\u27s performance. Our tracking systems consider the problem of car and human tracking on potentially very low quality surveillance videos, from fixed camera or high flying \acrfull{uav}. Our agile framework switches among different simple trackers to find the most applicable tracker based on the object and video properties. Our MAPTrack is an evolution of the agile tracker that uses soft switching to optimize between multiple pertinent trackers, and tracks objects based on motion, appearance, and positional data. In both cases we provide comparisons against trackers intended for similar applications i.e., trackers that stress robustness in bad conditions, with competitive results

Spatial Pyramid Context-Aware Moving Object Detection and Tracking for Full Motion Video and Wide Aerial Motion Imagery

A robust and fast automatic moving object detection and tracking system is essential to characterize target object and extract spatial and temporal information for different functionalities including video surveillance systems, urban traffic monitoring and navigation, robotic. In this dissertation, I present a collaborative Spatial Pyramid Context-aware moving object detection and Tracking system. The proposed visual tracker is composed of one master tracker that usually relies on visual object features and two auxiliary trackers based on object temporal motion information that will be called dynamically to assist master tracker. SPCT utilizes image spatial context at different level to make the video tracking system resistant to occlusion, background noise and improve target localization accuracy and robustness. We chose a pre-selected seven-channel complementary features including RGB color, intensity and spatial pyramid of HoG to encode object color, shape and spatial layout information. We exploit integral histogram as building block to meet the demands of real-time performance. A novel fast algorithm is presented to accurately evaluate spatially weighted local histograms in constant time complexity using an extension of the integral histogram method. Different techniques are explored to efficiently compute integral histogram on GPU architecture and applied for fast spatio-temporal median computations and 3D face reconstruction texturing. We proposed a multi-component framework based on semantic fusion of motion information with projected building footprint map to significantly reduce the false alarm rate in urban scenes with many tall structures. The experiments on extensive VOTC2016 benchmark dataset and aerial video confirm that combining complementary tracking cues in an intelligent fusion framework enables persistent tracking for Full Motion Video and Wide Aerial Motion Imagery.Comment: PhD Dissertation (162 pages

Autocalibrating vision guided navigation of unmanned air vehicles via tactical monocular cameras in GPS denied environments

This thesis presents a novel robotic navigation strategy by using a conventional tactical monocular camera, proving the feasibility of using a monocular camera as the sole proximity sensing, object avoidance, mapping, and path-planning mechanism to fly and navigate small to medium scale unmanned rotary-wing aircraft in an autonomous manner. The range measurement strategy is scalable, self-calibrating, indoor-outdoor capable, and has been biologically inspired by the key adaptive mechanisms for depth perception and pattern recognition found in humans and intelligent animals (particularly bats), designed to assume operations in previously unknown, GPS-denied environments. It proposes novel electronics, aircraft, aircraft systems, systems, and procedures and algorithms that come together to form airborne systems which measure absolute ranges from a monocular camera via passive photometry, mimicking that of a human-pilot like judgement. The research is intended to bridge the gap between practical GPS coverage and precision localization and mapping problem in a small aircraft. In the context of this study, several robotic platforms, airborne and ground alike, have been developed, some of which have been integrated in real-life field trials, for experimental validation. Albeit the emphasis on miniature robotic aircraft this research has been tested and found compatible with tactical vests and helmets, and it can be used to augment the reliability of many other types of proximity sensors

Motion-based Segmentation and Classification of Video Objects

In this thesis novel algorithms for the segmentation and classification of video objects are developed. The segmentation procedure is based on motion and is able to extract moving objects acquired by either a static or a moving camera. The classification of those objects is performed by matching their outlines gathered from a number of consecutive frames of the video with preprocessed views of prototypical objects stored in a database. This thesis contributes to four areas of image processing and computer vision: motion analysis, implicit active contour models, motion-based segmentation, and object classification. In detail, in the field of motion analysis, the tensor-based motion estimation approach is extended by a non-maximum suppression scheme, which improves the identification of relevant image structures significantly. In order to analyze videos that contain large image displacements, a feature-based motion estimation method is developed. In addition, to include camera operations into the segmentation process, a robust camera motion estimator based on least trimmed squares regression is presented. In the area of implicit active contour models, a model that unifies geometric and geodesic active contours is developed. For this model an efficient numerical implementation based on a new narrow-band method and a semi-implicit discretization is provided. Compared to standard algorithms these optimizations reduce the computational complexity significantly. Integrating the results of the motion analysis into the fast active contour implementation, novel algorithms for motion-based segmentation are developed. In the field of object classification, a shape-based classification approach is extended and adapted to image sequence processing. Finally, a system for video object classification is derived by combining the proposed motion-based segmentation algorithms with the shape-based classification approach

Biological, simulation, and robotic studies to discover principles of swimming within granular media

The locomotion of organisms whether by running, flying, or swimming is the result of multiple degree-of-freedom nervous and musculoskeletal systems interacting with an environment that often flows and deforms in response to movement. A major challenge in biology is to understand the locomotion of organisms that crawl or burrow within terrestrial substrates like sand, soil, and muddy sediments that display both solid and fluid-like behavior. In such materials, validated theories such as the Navier-Stokes equations for fluids do not exist, and visualization techniques (such as particle image velocimetry in fluids) are nearly nonexistent. In this dissertation we integrated biological experiment, numerical simulation, and a physical robot model to reveal principles of undulatory locomotion in granular media. First, we used high speed x-ray imaging techniques to reveal how a desert dwelling lizard, the sandfish, swims within dry granular media without limb use by propagating a single period sinusoidal traveling wave along its body, resulting in a wave efficiency, the ratio of its average forward speed to wave speed, of approximately 0.5. The wave efficiency was independent of the media preparation (loosely and tightly packed). We compared this observation against two complementary modeling approaches: a numerical model of the sandfish coupled to a discrete particle simulation of the granular medium, and an undulatory robot which was designed to swim within granular media. We used these mechanical models to vary the ratio of undulation amplitude (A) to wavelength (λ) and demonstrated that an optimal condition for sand-swimming exists which results from competition between A and λ. The animal simulation and robot model, predicted that for a single period sinusoidal wave, maximal speed occurs for A/ λ = 0.2, the same kinematics used by the sandfish. Inspired by the tapered head shape of the sandfish lizard, we showed that the lift forces and hence vertical position of the robot as it moves forward within granular media can be varied by designing an appropriate head shape and controlling its angle of attack, in a similar way to flaps or wings moving in fluids. These results support the biological hypotheses which propose that morphological adaptations of desert dwelling organisms aid in their subsurface locomotion. This work also demonstrates that the discovery of biological principles of high performance locomotion within sand can help create the next generation of biophysically inspired robots that could explore potentially hazardous complex flowing environments.PhDCommittee Chair: Daniel I. Goldman; Committee Member: Hang Lu; Committee Member: Jeanette Yen; Committee Member: Shella Keilholz; Committee Member: Young-Hui Chan

Real-Time, Multiple Pan/Tilt/Zoom Computer Vision Tracking and 3D Positioning System for Unmanned Aerial System Metrology

The study of structural characteristics of Unmanned Aerial Systems (UASs) continues to be an important field of research for developing state of the art nano/micro systems. Development of a metrology system using computer vision (CV) tracking and 3D point extraction would provide an avenue for making these theoretical developments. This work provides a portable, scalable system capable of real-time tracking, zooming, and 3D position estimation of a UAS using multiple cameras. Current state-of-the-art photogrammetry systems use retro-reflective markers or single point lasers to obtain object poses and/or positions over time. Using a CV pan/tilt/zoom (PTZ) system has the potential to circumvent their limitations. The system developed in this paper exploits parallel-processing and the GPU for CV-tracking, using optical flow and known camera motion, in order to capture a moving object using two PTU cameras. The parallel-processing technique developed in this work is versatile, allowing the ability to test other CV methods with a PTZ system using known camera motion. Utilizing known camera poses, the object\u27s 3D position is estimated and focal lengths are estimated for filling the image to a desired amount. This system is tested against truth data obtained using an industrial system

Feature-based detection and tracking of individuals in dense crowds

Ph.DDOCTOR OF PHILOSOPH