Disparity refinement process based on RANSAC plane fitting for machine vision applications

This paper presents a new disparity map refinement process for stereo matching algorithm and the refinement stage that will be implemented by partitioning the place or mask image and re-projected to the preliminary disparity images. This process is to refine the noise and sparse of initial disparity map from weakly textured. The plane fitting algorithm is using Random Sample Consensus. Two well-known stereo matching algorithms have been tested on this framework with different filtering techniques applied at disparity refinement stage. The framework is evaluated on three Middlebury datasets. The experimental results show that the proposed framework produces better-quality and more accurate than normal flow state-of-the-art stereo matching algorithms. The performance evaluations are based on standard image quality metrics i.e. structural similarity index measure, peak signal-to-noise ratio and mean square error.Keywords: computer vision; disparity refinement; image segmentation; RANSAC; stereo.

ACCURATE AND FAST STEREO VISION

Stereo vision from short-baseline image pairs is one of the most active research fields in computer vision. The estimation of dense disparity maps from stereo image pairs is still a challenging task and there is further space for improving accuracy, minimizing the computational cost and handling more efficiently outliers, low-textured areas, repeated textures, disparity discontinuities and light variations. This PhD thesis presents two novel methodologies relating to stereo vision from short-baseline image pairs: I. The first methodology combines three different cost metrics, defined using colour, the CENSUS transform and SIFT (Scale Invariant Feature Transform) coefficients. The selected cost metrics are aggregated based on an adaptive weights approach, in order to calculate their corresponding cost volumes. The resulting cost volumes are merged into a combined one, following a novel two-phase strategy, which is further refined by exploiting semi-global optimization. A mean-shift segmentation-driven approach is exploited to deal with outliers in the disparity maps. Additionally, low-textured areas are handled using disparity histogram analysis, which allows for reliable disparity plane fitting on these areas. II. The second methodology relies on content-based guided image filtering and weighted semi-global optimization. Initially, the approach uses a pixel-based cost term that combines gradient, Gabor-Feature and colour information. The pixel-based matching costs are filtered by applying guided image filtering, which relies on support windows of two different sizes. In this way, two filtered costs are estimated for each pixel. Among the two filtered costs, the one that will be finally assigned to each pixel, depends on the local image content around this pixel. The filtered cost volume is further refined by exploiting weighted semi-global optimization, which improves the disparity accuracy. The handling of the occluded areas is enhanced by incorporating a straightforward and time efficient scheme. The evaluation results show that both methodologies are very accurate, since they handle efficiently low-textured/occluded areas and disparity discontinuities. Additionally, the second approach has very low computational complexity. Except for the aforementioned two methodologies that use as input short-baseline image pairs, this PhD thesis presents a novel methodology for generating 3D point clouds of good accuracy from wide-baseline stereo pairs

A Disparity Map Refinement To Enhance Weakly-textured Urban Environment Data

This paper presents an approach to refine noisy and sparse disparity maps from weakly-textured urban environments, enhancing their applicability in perception algorithms applied to autonomous vehicles urban navigation. Typically, the disparity maps are constructed by stereo matching techniques based on some image correlation algorithm. However, in urban environments with low texture variance elements, like asphalt pavements and shadows, the images' pixels are hard to match, which result in sparse and noisy disparity maps. In this work, the disparity map refinement will be performed by segmenting the reference image of the stereo system with a combination of filters and the Watershed transform to fit the formed clusters in planes with a RANSAC approach. The refined disparity map was processed with the KITTI flow benchmark achieving improvements in the final average error and data density. © 2013 IEEE.McBride, J.R., Ivan, J.C., Rhode, D.S., Rupp, J.D., Rupp, M.Y., Higgins, J.D., Turner, D.D., Eustice, R.M., A perspective on emerging automotive safety applications, derived from lessons learned through participation in the darpa grand challenges (2008) Journal of Field Robotics, 25, pp. 808-840. , OctoberVitor, G.B., Lima, D.A., Victorino, A.C., Ferreira, J.V., A 2d/3d vision based approach applied to road detection in urban environments (2013) Intelligent Vehicles Symposium (IV), 2013 IEEE, pp. 952-957Chen, Y.-L., Sundareswaran, V., Anderson, C., Broggi, A., Grisleri, P., Porta, P., Zani, P., Beck, J., Terramax: Team oshkosh urban robot (2009) The DARPA Urban Challenge, Ser. Springer Tracts in Advanced Robotics, 56, pp. 595-622. , http://dx.doi.org/10.1007/978-3-642-03991-114, M. Buehler, K. Iagnemma, and S. Singh, Eds. Springer Berlin HeidelbergKolmogorov, V., Zabih, R., Computing visual correspondence with occlusions using graph cuts (2001) Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on, 2, pp. 508-515. , vol.2Hirschmuller, H., Accurate and efficient stereo processing by semiglobal matching and mutual information (2005) Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, 2, pp. 807-814. , vol. 2Banz, C., Hesselbarth, S., Flatt, H., Blume, H., Pirsch, P., Real-time stereo vision system using semi-global matching disparity estimation: Architecture and fpga-implementation (2010) Embedded Computer Sys-Tems (SAMOS), 2010 International Conference on, pp. 93-101Gupta, R., Cho, S.Y., A color-based approach for disparity refinement (2010) Control Automation Robotics Vision (ICARCV), 2010 11th International Conference on, pp. 664-667. , decXu, L., Au, O., Sun, W., Li, Y., Li, J., Hybrid plane fitting for depth estimation (2012) Signal Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012 Asia-Pacific, pp. 1-4. , decGeiger, A., Lenz, P., Urtasun, R., Are we ready for autonomous driving? the kitti vision benchmark suite (2012) Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 3354-3361Labayrade, R., Aubert, D., Tarel, J.P., Real time obstacle detection in stereovision on non flat road geometry through "V-disparity" representation (2002) Proceedings of the IEEE Symposium on Intelligent Vehicles, 2, pp. 646-651Hajebi, K., Zelek, J., Structure from infrared stereo images (2008) Computer and Robot Vision, 2008. CRV '08. Canadian Conference on, pp. 105-112. , maySoquet, N., Aubert, D., Hautiere, N., Road segmentation supervised by an extended v-disparity algorithm for autonomous navigation (2007) Proceedings of the IEEE Symposium on Intelligent Vehicles, pp. 160-165Dornaika, F., Alvarez, J., Sappa, A., Lopez, A., A new framework for stereo sensor pose through road segmentation and registration (2011) Intelligent Transportation Systems, IEEE Transactions on, 12 (4), pp. 954-966. , decTeutsch, M., Heger, T., Schamm, T., Zollner, J., 3d-segmentation of traffic environments with u/v-disparity supported by radar-given masterpoints (2010) Intelligent Vehicles Symposium (IV), pp. 787-792. , 2010 IEEE, juneHiroshi, T.K., Kano, H., Kimura, S., Yoshida, A., Oda, K., Development of a video-rate stereo machine (1995) IROS '95: Proceedings of the International Conference on Intelligent Robots and Systems-Volume 3, p. 3095. , Washington, DC, USA: IEEE Computer SocietyAudigier, R., De Alencar, L.R., Relationships between some watershed definitions and their tie-zone transforms (2010) Image Vision Comput., 28 (10), pp. 1472-1482. , http://dx.doi.org/10.1016/j.imavis.2009.11.002, OctDougherty, E.R., Lotufo, R.A., (2003) Hands-on Morphological Image Processing (SPIE Tutorial Texts in Optical Engineering Vol. TT59), , SPIE Publications, JulMeijster, A., Wilkinson, M.H.F., A comparison of algorithms for connected set openings and closings (2002) Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24 (4), pp. 484-494Vincent, L., Morphological grayscale reconstruction in image analysis: Applications and efficient algorithms Image Processing, IEEE Transactions on, 2 (2), pp. 176-201. , AprRoerdink, J.B.T.M., Meijster, A., (2001) The Watershed Transform: Definitions, Algorithms and Parallelization StrategiesKorbes, A., Vitor, G., De Alencar, L.R., Ferreira, J., Advances on watershed processing on gpu architecture (2011) Mathematical Morphology and Its Applications to Image and Signal Processing, Ser. Lecture Notes in Computer Science, 6671, pp. 260-271. , P. Soille, M. Pesaresi, and G. Ouzounis, Eds. Springer Berlin / Heidelber