Model based estimation of image depth and displacement

Passive depth and displacement map determinations have become an important part of computer vision processing. Applications that make use of this type of information include autonomous navigation, robotic assembly, image sequence compression, structure identification, and 3-D motion estimation. With the reliance of such systems on visual image characteristics, a need to overcome image degradations, such as random image-capture noise, motion, and quantization effects, is clearly necessary. Many depth and displacement estimation algorithms also introduce additional distortions due to the gradient operations performed on the noisy intensity images. These degradations can limit the accuracy and reliability of the displacement or depth information extracted from such sequences. Recognizing the previously stated conditions, a new method to model and estimate a restored depth or displacement field is presented. Once a model has been established, the field can be filtered using currently established multidimensional algorithms. In particular, the reduced order model Kalman filter (ROMKF), which has been shown to be an effective tool in the reduction of image intensity distortions, was applied to the computed displacement fields. Results of the application of this model show significant improvements on the restored field. Previous attempts at restoring the depth or displacement fields assumed homogeneous characteristics which resulted in the smoothing of discontinuities. In these situations, edges were lost. An adaptive model parameter selection method is provided that maintains sharp edge boundaries in the restored field. This has been successfully applied to images representative of robotic scenarios. In order to accommodate image sequences, the standard 2-D ROMKF model is extended into 3-D by the incorporation of a deterministic component based on previously restored fields. The inclusion of past depth and displacement fields allows a means of incorporating the temporal information into the restoration process. A summary on the conditions that indicate which type of filtering should be applied to a field is provided

Aportació als mètodes de seguiment tridimensional d'objectes d'alta velocitat d'operació mitjançant l'estereovisió

Postprint (published version

A hypercolumn based stereo vision model.

by Lam Shu Sun.Thesis (M.Phil.)--Chinese University of Hong Kong, 1993.Includes bibliographical references (leaves [91]-94).Chapter Chapter1 --- Introduction: Binocular Depth Visual Perception of Human --- p.1Chapter 1.1 --- Introduction --- p.1Chapter 1.2 --- The visual pathway --- p.2Chapter 1.3 --- The retina --- p.3Chapter 1.4 --- The ganglion cells --- p.5Chapter 1.5 --- The lateral geniculate nucleus --- p.7Chapter 1.6 --- The visual cortex --- p.8Chapter 1.6.1 --- The cortical cells --- p.8Chapter 1.6.2 --- The organization of the visual cortex --- p.9Chapter 1.7 --- Stereopsis --- p.11Chapter 1.7.1 --- Corresponding retinal points --- p.12Chapter 1.7.2 --- Binocular fusion --- p.14Chapter 1.7.3 --- The binocular depth cells --- p.14Chapter 1.8 --- Conclusion of chapter 1 --- p.15Chapter Chapter2 --- Computational Stereo Vision --- p.15Chapter 2.1 --- Stereo image geometry --- p.16Chapter 2.1.1 --- The crossed-looking geometry --- p.17Chapter 2.1.2 --- The parallel optical axes geometry --- p.19Chapter 2.2 --- The false targets problem --- p.20Chapter 2.3 --- Feature selection --- p.21Chapter 2.3.1 --- Zero-crossing method --- p.21Chapter 2.3.2 --- A network model for ganglion cell --- p.24Chapter 2.4 --- The constraints of matching --- p.28Chapter 2.5 --- Correspondence techniques --- p.29Chapter 2.6 --- Conclusion of chapter 2 --- p.29Chapter Chapter3 --- A Hypercolumn Based Stereo Vision Model --- p.30Chapter 3.1 --- A visual model for stereo vision --- p.30Chapter 3.2 --- The model of PSVM (A Computerized Visual Model) --- p.32Chapter 3.3 --- Local orientated line extraction (Stage 1 of PSVM) --- p.34Chapter 3.3.1 --- Orientated line detection network --- p.35Chapter 3.3.2 --- On-type orientated lines and off-type orientated lines --- p.37Chapter 3.4 --- Local line matching (Stage 2 of PSVM) --- p.38Chapter 3.4.1 --- Structure of hypercolumn in PSVM --- p.39Chapter 3.4.2 --- Line length discrimination model (Part of stage 2 of PSVM) --- p.41Chapter 3.4.3 --- Orientation-length detector --- p.42Chapter 3.4.4 --- Line length selection --- p.45Chapter 3.4.5 --- The matching model --- p.46Chapter 3.4.6 --- Fusional area in PSVM --- p.48Chapter 3.4.7 --- Matching mechanism --- p.49Chapter 3.4.8 --- Disparity detection --- p.50Chapter 3.5 --- Disparity integrations (Stage 3 of PSVM) --- p.53Chapter 3.5.1 --- The voter network --- p.54Chapter 3.5.2 --- The redistributor network --- p.55Chapter 3.6 --- Conculsion of chpater 3 --- p.57Chapter Chapter4 --- Implementation and Analysis --- p.58Chapter 4.1 --- The imaging geometry of PSVM --- p.58Chapter 4.2 --- Input --- p.59Chapter 4.3 --- The hypercolumn construction --- p.59Chapter 4.4 --- Analysis of matching mechanism in PSVM --- p.59Chapter 4.4.1 --- Fusional condition --- p.61Chapter 4.4.2 --- Disparity detection --- p.61Chapter 4.5. --- Matching rules in PSVM --- p.63Chapter 4.5.1 --- The ordering constraint --- p.63Chapter 4.5.2 --- The uniqueness constraint --- p.64Chapter 4.5.3 --- The figural continuity constraint --- p.64Chapter 4.5.4 --- The smoothness assumption --- p.65Chapter 4.6. --- Use multi-lengths of oriented line to solve the occlusion problem --- p.66Chapter 4.7 --- Performance of PSVM --- p.67Chapter 4.7.1 --- Artificial scene --- p.67Chapter 4.7.2 --- Natural images --- p.71Chapter 4.8 --- Discussion --- p.83Chapter 4.9 --- Overall conclusion --- p.83Appendix: Illustration example --- p.85References --- p.9

A Cooperative algorithm for stereo disparity computation.

by Or Siu Hang.Thesis (M.Phil.)--Chinese University of Hong Kong, 1991.Bibliography: leaves [102]-[105].Acknowledgements --- p.VChapter Chapter 1 --- IntroductionChapter 1.1 --- The problem --- p.1Chapter 1.1.1 --- The correspondence problem --- p.5Chapter 1.1.2 --- The problem of surface reconstruction --- p.6Chapter 1.2 --- Our goal --- p.8Chapter 1.3 --- Previous works --- p.8Chapter 1.3.1 --- Constraints on matching --- p.10Chapter 1.3.2 --- Interpolation of disparity surfaces --- p.12Chapter Chapter 2 --- Preprocessing of imagesChapter 2.1 --- Which operator to use --- p.14Chapter 2.2 --- Directional zero-crossing --- p.14Chapter 2.3 --- Laplacian of Gaussian --- p.16Chapter 2.3.1 --- Theoretical background of the Laplacian of Gaussian --- p.18Chapter 2.3.2 --- Implementation of the operator --- p.21Chapter Chapter 3 --- Disparity Layers GenerationChapter 3.1 --- Geometrical constraint --- p.23Chapter 3.2 --- Basic idea of disparity layer --- p.26Chapter 3.3 --- Consideration in matching --- p.28Chapter 3.4 --- effect of vertical misalignment of sensor --- p.37Chapter 3.5 --- Final approach --- p.39Chapter Chapter 4 --- Disparity combinationChapter 4.1 --- Ambiguous match from different layers --- p.52Chapter 4.2 --- Our approach --- p.54Chapter Chapter 5 --- Generation of dense disparity mapChapter 5.1 --- Introduction --- p.58Chapter 5.2 --- Cooperative computation --- p.58Chapter 5.2.1 --- Formulation of oscillation algorithm --- p.59Chapter 5.3 --- Interpolation by Gradient descent method --- p.69Chapter 5.3.1 --- Formulation of constraints --- p.70Chapter 5.3.2 --- Gradient projection interpolation algorithm --- p.72Chapter 5.3.3 --- Implementation of the algorithm --- p.78Chapter Chapter 6 --- Conclusion --- p.89ReferenceAppendix (Dynamical behavior of the cooperative algorithm

Stereo vision based on compressed feature correlation and graph cut

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2005.Includes bibliographical references (p. 131-145).This dissertation has developed a fast and robust algorithm to solve the dense correspondence problem with a good performance in untextured regions by merging Sparse Array Correlation from the computational fluids community into graph cut from the computer vision community. The proposed methodology consists of two independent modules. The first module is named Compressed Feature Correlation which is originated from Particle Image Velocimetry (PIV). The algorithm uses an image compression scheme that retains pixel values in high-intensity gradient areas while eliminating pixels with little correlation information in smooth surface regions resulting in a highly reduced image datasets. In addition, by utilizing an error correlation function, pixel comparisons are made through single integer calculations eliminating time consuming multiplication and floating point arithmetic. Unlike the traditional fixed window sorting scheme, adaptive correlation window positioning is implemented by dynamically placing strong features at the center of each correlation window. A confidence measure is developed to validate correlation outputs. The sparse depth map generated by this ultra-fast Compressed Feature Correlation may either serve as inputs to global methods or be interpolated into dense depth map when object boundaries are clearly defined. The second module enables a modified graph cut algorithm with an improved energy model that accepts prior information by fixing data energy penalties. The image pixels with known disparity values stabilize and speed up global optimization. As a result less iterations are necessary and sensitivity to parameters is reduced.(cont.) An efficient hybrid approach is implemented based on the above two modules. By coupling a simpler and much less expensive algorithm, Compressed Feature Correlation, with a more expensive algorithm, graph cut, the computational expense of the hybrid calculation is one third of performing the entire calculation using the more expensive of the two algorithms, while accuracy and robustness are improved at the same time. Qualitative and quantitative results on both simulated disparities and real stereo images are presented.by Sheng Sarah Tan.Ph.D

The systematic development of a machine vision based milking robot

Agriculture involves unique interactions between man, machines, and various elements from nature. Therefore the implementation of advanced technology in agriculture holds different challenges than in other sectors of the economy. This dissertation stems from research into the application of advanced technology in dairying - focusing on the systematic analysis and synthesis of concepts for a robotic milking machine for cows. The main subsystems of the milking robot are identified as a machine perception subsystem and a mechanical manipulator subsystem. The machine perception subsystem consists of one or more sensors and a signal processor; while the manipulator subsystem typically consists of a robot arm; a robot hand; actuators; and a controller. After the evaluation of different sensor concepts in terms of a defined set of technical performance requirements, television cameras are chosen as a suitable sensor concept for a milking robot. Therefore the signal processor is only concerned with image processing techniques. The primary task of the milking robot's image processor is to derive a computerized description of the spatial positions of the endpoints of a cow's four teats, in terms of a pre-defined frame of reference (called the word coordinates ). This process is called scene description ; and based on extensive experimental results, three-dimensional scene description - making use of a stereo-vision set-up - is shown to be feasible for application as part of a milking robot. Different processes are involved in stereo machine vision - such as data reduction, with the minimum loss of Image information (for which the Sobel edge enhancement operator is used); the accurate localisation of target objects in the two stereo images (for which the parabolic Hough transform is used); and correlation of features in the two stereo images. These aspects are all addressed for the milking robot, by means of concept analysis, trade-oft, and experimental verification. From a trade-off, based on a set of performance requirements for the manipulator subsystem, a cartesian robot arm is chosen as a suitable configuration for the milking robot; while sealed direct current servo motors are chosen as a suitable actuator concept. A robot arm and its actuators are designed by means of computer-aided design techniques; and computer simulation results are presented for the dynamic response of the arm and its actuators. A suitable robot hand is also designed - based on systematic trade-oft for different parts of a robot hand. From an analysis of the desired controller functions, and of different control concepts, it is concluded that a positional controller, making use of on-line obstruction avoidance, is required for the milking robot. Because this research project involved systematic concept exploration, there are still some details to be sorted out in a follow-up development phase. The basic principles of a machine vision based milking robot are however established; and the work in this dissertation represents a suitable baseline for further development