3D image analysis for pedestrian detection

A method for solving the dense disparity stereo correspondence problem is presented in this paper. This technique is designed specifically for pedestrian detection type applications. A new Ground Control Points (GCPs) scheme is introduced, using groundplane homography information to determine regions in which good GCPs are likely to occur. The method also introduces a dynamic disparity limit constraint to further improve GCP selection and dense disparity generation. The technique is applied to a real world pedestrian detection scenario with a background modeling system based on disparity and edges

NOVEL DENSE STEREO ALGORITHMS FOR HIGH-QUALITY DEPTH ESTIMATION FROM IMAGES

This dissertation addresses the problem of inferring scene depth information from a collection of calibrated images taken from different viewpoints via stereo matching. Although it has been heavily investigated for decades, depth from stereo remains a long-standing challenge and popular research topic for several reasons. First of all, in order to be of practical use for many real-time applications such as autonomous driving, accurate depth estimation in real-time is of great importance and one of the core challenges in stereo. Second, for applications such as 3D reconstruction and view synthesis, high-quality depth estimation is crucial to achieve photo realistic results. However, due to the matching ambiguities, accurate dense depth estimates are difficult to achieve. Last but not least, most stereo algorithms rely on identification of corresponding points among images and only work effectively when scenes are Lambertian. For non-Lambertian surfaces, the brightness constancy assumption is no longer valid. This dissertation contributes three novel stereo algorithms that are motivated by the specific requirements and limitations imposed by different applications. In addressing high speed depth estimation from images, we present a stereo algorithm that achieves high quality results while maintaining real-time performance. We introduce an adaptive aggregation step in a dynamic-programming framework. Matching costs are aggregated in the vertical direction using a computationally expensive weighting scheme based on color and distance proximity. We utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this process over two orders of magnitude. In addressing high accuracy depth estimation, we present a stereo model that makes use of constraints from points with known depths - the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel regularization prior is naturally integrated into a global inference framework in a principled way using the Bayes rule. Our probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate information from various sensors. In addressing non-Lambertian reflectance, we introduce a new invariant for stereo correspondence which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs). This invariant can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies

Development Of Double Stage Filter (DSF) On Stereo Matching Algorithm For 3D Computer Vision Applications

In the field of stereo vision, some of existing stereo matching algorithms are designed with less accuracy of algorithm. Thus, a new hybrid algorithm with higher accuracy of computation is developed in this project. This thesis will present the design, development and analysis of performance on a developed Double Stage Filter (DSF) algorithm and other existing stereo matching algorithms. DSF algorithm is a hybrid stereo matching algorithm which divided into two phases. Phase 1 is consists of the part on Sum of Absolute Differences from basic block matching and the part of Scanline Optimization (SO) from dynamic programming approches while phase 2 includes segmentation, merging and basic median filter process. The main feature of DSF algorithm is mainly on the phase 2 or as post-processing in which to remove the unwanted aspects like random noises and horizontal streaks, which is obtained from the raw disparity depth map on the step of optimization. In order to remove the unwanted aspects, two stages filtering process are needed along with the developed approaches in the phase 2 of DSF algorithm. There are two categorized evaluations done on the disparity maps obtained by the algorithms : objective evaluation and subjective evaluation. The objective evaluation includes the evaluation system of Middlebury Stereo Vision website page, computation analysis and traditional methods of Mean Square Errors (MSE), Peak to Signal Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM). Besides, for subjective evaluation, the datasets are captured from LNC IP camera and the results obtained by the selected algorithms are evaluated by human's eyes perception. Based on the results of evaluations, the results obtained by DSF is better than the algorithms, basic block matching and dynamic programming

Enhanced Image View Synthesis Using Multistage Hybrid Median Filter For Stereo Images

Disparity depth map estimation of stereo matching algorithm is one of the most active research topics in computer vision.In the field of image processing,many existing stereo matching algorithms to obtain disparity depth map are developed and designed with low accuracy.To improve the accuracy of disparity depth map is quite challenging and difficult especially with uncontrolled dynamic environment.The accuracy is affected by many unwanted aspects including random noises,horizontal streaks,low texture,depth map non-edge preserving, occlusion,and depth discontinuities.Thus,this research proposed a new robust method of hybrid stereo matching algorithm with significant accuracy of computation.The thesis will present in detail the development,design, and analysis of performance on Multistage Hybrid Median Filter (MHMF).There are two main parts involved in our developed method which combined in two main stages.Stage 1 consists of the Sum of Absolute Differences (SAD) from Basic Block Matching (BBM) algorithm and the part of Scanline Optimization (SO) from Dynamic Programming (DP) algorithm.While,Stage 2 is the main core of our MHMF as a post-processing step which included segmentation,merging, and hybrid median filtering.The significant feature of the post-processing step is on its ability to handle efficiently the unwanted aspects obtained from the raw disparity depth map on the step of optimization.In order to remove and overcome the challenges unwanted aspects, the proposed MHMF has three stages of filtering process along with the developed approaches in Stage 2 of MHMF algorithm.There are two categories of evaluation performed on the obtained disparity depth map: subjective evaluation and objective evaluation.The objective evaluation involves the evaluation on Middlebury Stereo Vision system and evaluation using traditional methods such as Mean Square Errors (MSE),Peak to Signal Noise Ratio (PSNR) and Structural Similarity Index Metric (SSIM).Based on the results of the standard benchmarking datasets from Middlebury,the proposed algorithm is able to reduce errors of non-occluded and all errors respectively.While,the subjective evaluation is done for datasets captured from MV BLUE FOX camera using human's eyes perception.Based on the results,the proposed MHMF is able to obtain accurate results, specifically 69% and 71% of non-occluded and all errors for disparity depth map, and it outperformed some of the existing methods in the literature such as BBM and DP algorithms

Disparity-compensated view synthesis for s3D content correction

International audienceThe production of stereoscopic 3D HD content is considerably increasing and experience in 2-view acquisition is in progress. High quality material to the audience is required but not always ensured, and correction of the stereo views may be required. This is done via disparity-compensated view synthesis. A robust method has been developed dealing with these acquisition problems that introduce discomfort (e.g hyperdivergence and hyperconvergence...) as well as those ones that may disrupt the correction itself (vertical disparity, color difference between views...). The method has three phases: a preprocessing in order to correct the stereo images and estimate features (e.g. disparity range...) over the sequence. The second (main) phase proceeds then to disparity estimation and view synthesis. Dual disparity estimation based on robust block-matching, discontinuity-preserving filtering, consistency and occlusion handling has been developed. Accurate view synthesis is carried out through disparity compensation. Disparity assessment has been introduced in order to detect and quantify errors. A post-processing deals with these errors as a fallback mode. The paper focuses on disparity estimation and view synthesis of HD images. Quality assessment of synthesized views on a large set of HD video data has proved the effectiveness of our method

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

A high speed Tri-Vision system for automotive applications

Purpose: Cameras are excellent ways of non-invasively monitoring the interior and exterior of vehicles. In particular, high speed stereovision and multivision systems are important for transport applications such as driver eye tracking or collision avoidance. This paper addresses the synchronisation problem which arises when multivision camera systems are used to capture the high speed motion common in such applications. Methods: An experimental, high-speed tri-vision camera system intended for real-time driver eye-blink and saccade measurement was designed, developed, implemented and tested using prototype, ultra-high dynamic range, automotive-grade image sensors specifically developed by E2V (formerly Atmel) Grenoble SA as part of the European FP6 project – sensation (advanced sensor development for attention stress, vigilance and sleep/wakefulness monitoring). Results : The developed system can sustain frame rates of 59.8 Hz at the full stereovision resolution of 1280 × 480 but this can reach 750 Hz when a 10 k pixel Region of Interest (ROI) is used, with a maximum global shutter speed of 1/48000 s and a shutter efficiency of 99.7%. The data can be reliably transmitted uncompressed over standard copper Camera-Link® cables over 5 metres. The synchronisation error between the left and right stereo images is less than 100 ps and this has been verified both electrically and optically. Synchronisation is automatically established at boot-up and maintained during resolution changes. A third camera in the set can be configured independently. The dynamic range of the 10bit sensors exceeds 123 dB with a spectral sensitivity extending well into the infra-red range. Conclusion: The system was subjected to a comprehensive testing protocol, which confirms that the salient requirements for the driver monitoring application are adequately met and in some respects, exceeded. The synchronisation technique presented may also benefit several other automotive stereovision applications including near and far-field obstacle detection and collision avoidance, road condition monitoring and others.Partially funded by the EU FP6 through the IST-507231 SENSATION project.peer-reviewe