Detection and segmentation of moving objects in video using optical vector flow estimation

The objective of this thesis is to detect and identify moving objects in a video sequence. The currently available techniques for motion estimation can be broadly categorized into two main classes: block matching methods and optical flow methods.This thesis investigates the different motion estimation algorithms used for video processing applications. Among the available motion estimation methods, the Lucas Kanade Optical Flow Algorithm has been used in this thesis for detection of moving objects in a video sequence. Derivatives of image brightness with respect to x-direction, y-direction and time t are calculated to solve the Optical Flow Constraint Equation. The algorithm produces results in the form of horizontal and vertical components of optical flow velocity, u and v respectively. This optical flow velocity is measured in the form of vectors and has been used to segment the moving objects from the video sequence. The algorithm has been applied to different sets of synthetic and real video sequences.This method has been modified to include parameters such as neighborhood size and Gaussian pyramid filtering which improve the motion estimation process. The concept of Gaussian pyramids has been used to simplify the complex video sequences and the optical flow algorithm has been applied to different levels of pyramids. The estimated motion derived from the difference in the optical flow vectors for moving objects and stationary background has been used to segment the moving objects in the video sequences. A combination of erosion and dilation techniques is then used to improve the quality of already segmented content.The Lucas Kanade Optical Flow Algorithm along with other considered parameters produces encouraging motion estimation and segmentation results. The consistency of the algorithm has been tested by the usage of different types of motion and video sequences. Other contributions of this thesis also include a comparative analysis of the optical flow algorithm with other existing motion estimation and segmentation techniques. The comparison shows that there is need to achieve a balance between accuracy and computational speed for the implementation of any motion estimation algorithm in real time for video surveillance

Real-time Visual Flow Algorithms for Robotic Applications

Vision offers important sensor cues to modern robotic platforms. Applications such as control of aerial vehicles, visual servoing, simultaneous localization and mapping, navigation and more recently, learning, are examples where visual information is fundamental to accomplish tasks. However, the use of computer vision algorithms carries the computational cost of extracting useful information from the stream of raw pixel data. The most sophisticated algorithms use complex mathematical formulations leading typically to computationally expensive, and consequently, slow implementations. Even with modern computing resources, high-speed and high-resolution video feed can only be used for basic image processing operations. For a vision algorithm to be integrated on a robotic system, the output of the algorithm should be provided in real time, that is, at least at the same frequency as the control logic of the robot. With robotic vehicles becoming more dynamic and ubiquitous, this places higher requirements to the vision processing pipeline. This thesis addresses the problem of estimating dense visual flow information in real time. The contributions of this work are threefold. First, it introduces a new filtering algorithm for the estimation of dense optical flow at frame rates as fast as 800 Hz for 640x480 image resolution. The algorithm follows a update-prediction architecture to estimate dense optical flow fields incrementally over time. A fundamental component of the algorithm is the modeling of the spatio-temporal evolution of the optical flow field by means of partial differential equations. Numerical predictors can implement such PDEs to propagate current estimation of flow forward in time. Experimental validation of the algorithm is provided using high-speed ground truth image dataset as well as real-life video data at 300 Hz. The second contribution is a new type of visual flow named structure flow. Mathematically, structure flow is the three-dimensional scene flow scaled by the inverse depth at each pixel in the image. Intuitively, it is the complete velocity field associated with image motion, including both optical flow and scale-change or apparent divergence of the image. Analogously to optic flow, structure flow provides a robotic vehicle with perception of the motion of the environment as seen by the camera. However, structure flow encodes the full 3D image motion of the scene whereas optic flow only encodes the component on the image plane. An algorithm to estimate structure flow from image and depth measurements is proposed based on the same filtering idea used to estimate optical flow. The final contribution is the spherepix data structure for processing spherical images. This data structure is the numerical back-end used for the real-time implementation of the structure flow filter. It consists of a set of overlapping patches covering the surface of the sphere. Each individual patch approximately holds properties such as orthogonality and equidistance of points, thus allowing efficient implementations of low-level classical 2D convolution based image processing routines such as Gaussian filters and numerical derivatives. These algorithms are implemented on GPU hardware and can be integrated to future Robotic Embedded Vision systems to provide fast visual information to robotic vehicles

Reliable camera motion estimation from compressed MPEG videos using machine learning approach

As an important feature in characterizing video content, camera motion has been widely applied in various multimedia and computer vision applications. A novel method for fast and reliable estimation of camera motion from MPEG videos is proposed, using support vector machine for estimation in a regression model trained on a synthesized sequence. Experiments conducted on real sequences show that the proposed method yields much improved results in estimating camera motions while the difficulty in selecting valid macroblocks and motion vectors is skipped

Real-time model-based video stabilization for microaerial vehicles

The emerging branch of micro aerial vehicles (MAVs) has attracted a great interest for their indoor navigation capabilities, but they require a high quality video for tele-operated or autonomous tasks. A common problem of on-board video quality is the effect of undesired movements, so different approaches solve it with both mechanical stabilizers or video stabilizer software. Very few video stabilizer algorithms in the literature can be applied in real-time but they do not discriminate at all between intentional movements of the tele-operator and undesired ones. In this paper, a novel technique is introduced for real-time video stabilization with low computational cost, without generating false movements or decreasing the performance of the stabilized video sequence. Our proposal uses a combination of geometric transformations and outliers rejection to obtain a robust inter-frame motion estimation, and a Kalman filter based on an ANN learned model of the MAV that includes the control action for motion intention estimation.Peer ReviewedPostprint (author's final draft

Robust Techniques for Feature-based Image Mosaicing

Since the last few decades, image mosaicing in real time applications has been a challenging field for image processing experts. It has wide applications in the field of video conferencing, 3D image reconstruction, satellite imaging and several medical as well as computer vision fields. It can also be used for mosaic-based localization, motion detection & tracking, augmented reality, resolution enhancement, generating large FOV etc. In this research work, feature based image mosaicing technique using image fusion have been proposed. The image mosaicing algorithms can be categorized into two broad horizons. The first is the direct method and the second one is based on image features. The direct methods need an ambient initialization whereas, Feature based methods does not require initialization during registration. The feature-based techniques are primarily followed by the four steps: feature detection, feature matching, transformation model estimation, image resampling and transformation. SIFT and SURF are such algorithms which are based on the feature detection for the accomplishment of image mosaicing, but both the algorithms has their own limitations as well as advantages according to the applications concerned. The proposed method employs this two feature based image mosaicing techniques to generate an output image that works out the limitations of the both in terms of image quality The developed robust algorithm takes care of the combined effect of rotation, illumination, noise variation and other minor variation. Initially, the input images are stitched together using the popular stitching algorithms i.e. Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF). To extract the best features from the stitching results, the blending process is done by means of Discrete Wavelet Transform (DWT) using the maximum selection rule for both approximate as well as detail-components

New image processing tools for structural dynamic monitoring

This paper presents an introduction to structural damage assessment using image processing on real data (non ideal conditions). Our contribution is much more a groundwork than a classical experimental validation. After measuring the bridge dynamic parameter on a small resolution video, we conjointly present advantages and limitations of our method. Finally we introduce several "computer vision" based rules and focus on the technical ability to detect damage using camera and video motion estimation