Coleman, SA

Gardiner, B

English

Research to date within the area of range image processing has highlighted the many advantages of using range data to represent scenes in a 3-D manner. However, given the large volume of data associated with range images, processing and extracting relevant information from these images has presented a challenge. Due to the irregular distribution of data in many range image types, a resampling process is required to map the irregular data to a regular grid structure, providing an opportunity to resample directly to a uniform hexagonal framework. This paper therefore proposes a novel framework for range image feature extraction, which utilises the efficiency and accuracy of the hexagonal pixel- based framework

Ulster University's Research Portal

 Range Image Feature Extraction using  a Hexagonal Pixel-based Framework  Bryan Gardiner Ulster University Northland Road, Londonderry, BT48 7JL b.gardiner@ulster.ac.uk Sonya Coleman Ulster University Northland Road, Londonderry, BT48 7JL Sa.coleman@ulster.ac.uk Abstract Research to date within the area of range image processing has highlighted the many advantages of using range data to represent scenes in a 3-D manner. However, given the large volume of data associated with range images, processing and extracting relevant information from these images has presented a challenge. Due to the irregular distribution of data in many range image types, a resampling process is required to map the irregular data to a regular grid structure, providing an opportunity to resample directly to a uniform hexagonal framework. This paper therefore proposes a novel framework for range image feature extraction, which utilises the efficiency and accuracy of the hexagonal pixel-based framework.  Keywords: Hexagonal Framework, Range Images, Feature Extraction. 1      Introduction  Due to the recent development of low-cost RGB-D sensors such as the Microsoft Kinect, the use of range images have become prominent in computer vision techniques, providing an approach to obtain reliable descriptions of 3-D scenes. A range image may be described as a 2-D image that contains distance measurements from a selected reference point or plane to surface points of objects within a scene [16]. Range images provide additional information over conventional intensity images allowing more information about the scene to be captured [3]. It should be noted a range image contains information about only the visible surfaces of the objects, and not their hidden surfaces, and hence is often referred to as 2 21 -D information [16].  Range cameras can operate according to a number of different techniques, for example, laser scanning [4], stereovision [15], pattern projection [18] and time-of-flight lasers [12]. All of these range finders have their advantages and disadvantages depending on their application, where some example application areas include object recognition [13], surface reconstruction [17], surveillance [5], robot navigation [19], etc. Many of these application areas are dependent on the completion of feature extraction on range images, and the process of acquiring these features is significantly different from that when using conventional intensity images. There are two factors to consider when processing range images: the irregularity of the distribution of the range data [2], and the types of features that are acquired from range data feature extraction. Depending on the hardware sensor used to capture the range data, the acquired data may be distributed regularly or irregularly; this should be considered before processing the image. Unlike intensity images, where edges can be found by significant changes in grey level values and hence may be modeled as ramps or steps, range data extracts edges in three different categories: step edges, crease edges and smooth edges.  A recent concept for image representation is the use of hexagonal pixels, introducing the area of hexagonal image processing. As well as the factor of hexagonally structured images mimicking the Proceedings of the 17th Irish Machine Vision and Image Processing conference IMVIP 2015August 26th-28th, 2015, Dublin, Ireland 118 ISBN 978-0-9934207-0-2structure found in the human fovea (a small region within the retina, consisting of a high density of cones shaped and placed in a hexagonal arrangement [9]), hexagonal grids have other advantages over the conventional rectangular grid. Equidistance of all pixel neighbours facilitates the implementation of circular symmetric kernels that is associated with an increase in accuracy when detecting edges, both straight and curved [1], and the improved accuracy of circular and near circular image processing operators has been demonstrated in [6]. Additionally, better spatial sampling efficiency is achieved by the hexagonal structure, leading to improved computational performance. In a hexagonal grid with unit separation of pixel centres, approximately 13% fewer pixels are required to represent the same image resolution as required on a rectangular grid with unit horizontal and vertical separation of pixel centres [14]. In previous work, the authors have shown how the hexagonal framework can be used to improve both efficiency and accuracy with respect to feature extraction on conventional intensity images [8]. This paper progresses this work by developing an approach for range data feature extraction that utilises the advantages acquired from using the hexagonal pixel-based framework. In Section 2 the range image representation using a hexagonal structure is outlined, and Section 3 details the process of applying hexagonal feature extraction operators to range images and determining the features using this approach. Section 4 presents resultant features maps and a conclusion is presented in Section 5. 2      Range Image Representation If we consider a range image to be represented as a spatially irregular sample of values of a continuous function u(x, y)  of depth value, an additional process is required to map the range data to a regular rectangular grid for processing. As resampling is taking place in this process, it provides an opportunity to resample directly onto a regular hexagonal pixel-based structure, utilising the advantages obtained from processing in a hexagonal framework. For regularly distributed range images, for example those captured using an RGB-D sensor, the hexagonal resampling can be completed using the approach of [20] where hexagonal pixels are created through clusters of sub-pixels. We have modified this technique slightly by representing each pixel by a nxn  block of sub-pixels. Each sub-pixel inherits the original pixel intensity, as in [14], in order to create an effect that enables sub-pixel clustering; this limits the loss of image resolution. With this re-sampled hexagonal image, it is possible to represent the range image by using an array of samples of a continuous function u(x, y)  of range data on a domain Ω. Figure 1 (a) represents an image composed of hexagonal pixels with nodes placed in the centre of each pixel, overlaid by the triangular finite element mesh. These nodes are the reference points for the computation of finite element techniques throughout the domain Ω.     02 3416 53032 33343136 354042 43444146 455052 53545156 556062 63646166 651012 13141116 152022 23242126 25 Figure 1 (a).  Hexagonal array of pixels and overlying mesh Figure 1 (b). 1D spiral addressing scheme  As discussed in previous work [8], an efficient approach to addressing hexagonal pixel-based images is via the use of the Spiral Architecture. The addressing of the Spiral Architecture originates at the centre of the hexagonal image and spirals out using one dimensional indexing. This structure facilitates the use of base seven numbering to address each pixel within the image, which permits the grouping of pixels in clusters for efficient pixel access; these clusters are shown in Figure 1 (b). Proceedings of the 17th Irish Machine Vision and Image Processing conference IMVIP 2015August 26th-28th, 2015, Dublin, Ireland 119 ISBN 978-0-9934207-0-23      Feature Extraction Approach  As it is not possible to apply conventional square operators to images represented on a hexagonal grid structure, we have shown in recent work [11] how a finite element based approach can be used to create a hexagonally structured Linear-Gaussian operator (H1) based on the construction of two independent directional derivative operators aligned in the x- and y- directions. The resultant kernels derived from [11] are presented in Figure 2 and will be used for feature extraction on hexagonal range images. Applying the presented edge detection operator to an intensity image will identify edges at the maxima of the gradient output, however the same thresholding does not identify edges in a depth image but instead will identify surfaces [7]. Determining features in hexagonal range images requires that the change point of the surfaces, i.e. slope direction, is identified. This can be achieved by using the first order operator response to identify the different surfaces in the depth image and from this, detecting significant change in gradient output can be determined as an edge point. If we consider the gradient magnitude response at any point in the hexagonal range image as Ñ(Px ), where x denotes the pixel location using the 1-D addressing scheme presented in Section 2, and using a threshold value of T, we can characterize each of the roof edges in the range image using the following set of rules:  For )(),(),()( )6()5()4( sss xxxR PPPP    If TPP Rx ! )()(  then )( xP  is an edge point.  (PR )  is obtained by determining the location of neighbouring pixels in tri-directions from the pixel (Px ) as demonstrated in Figure 3. Spiral addition (+s ) is used to obtain pixel neighbours in a spiral framework as previously described in [10]. Utilising this set of pixel values will ensure adequate comparison of neighbouring edge points when thresholding the obtained feature map.   4      Resultant Feature Maps  To demonstrate the framework described in Section 3, the Linear-Gaussian operator was applied to various depth images that have been resampled to a hexagonal pixel-based image structure. Using the proposed thresholding technique, specifically developed for hexagonal images, edge features were identified within the depth images. An example of an original intensity image, the associated depth image and the resultant feature map acquired from the proposed framework is presented in Figure 4 (a), (b) and (c) respectively. Visual results show merit in using depth information for feature extraction, for example, successful identification of edges of two occluding objects, i.e. the bowl and hat, have been completed using the proposed range image approach, see Figure 4 (c).   5      Conclusion  This paper presents a novel approach to range image feature extraction that utilises the advantages obtained from hexagonal pixel-based images. As resampling is often required to overcome the irregular distribution of range image data, resampling to a hexagonal framework does not require any additional steps, however as a processing framework it offers additional efficiency and accuracy when compared with the conventional rectangular framework. Visual results have shown the benefit of this approach and if used in conjunction with feature extraction on intensity images should provide further accuracy of detecting edge features in 3-D scenes.      Figure 2. x- and y-components of operator H1   )2( sxP )6( sxP )3( sxP )5( sxP )1( sxP  )4( sxP )(xP Figure 3. Neighbouring pixel locations for determining features in hexagonal range image 0-0.147 0.1470.295-0.295-0.147 0.14700.256 0.25600-0.256 -0.256Proceedings of the 17th Irish Machine Vision and Image Processing conference IMVIP 2015August 26th-28th, 2015, Dublin, Ireland 120 ISBN 978-0-9934207-0-2     (a) Original intensity image   (b) Hexagonal depth image  (c) Hexagonal feature map  Figure 4. Hexagonal pixel-based range images 6      References [1] Allen, J. D., “Filter Banks for Images on Hexagonal Grid”, Signal Solutions, 2003. [2] B.K. Lee, K. Yu, M. Pyeon, “Effective reduction of horizontal error in laser scanning information by strip-wise least squares adjustments”, ETRI J., 25 (2), pp. 109–120, 2003. [3] Bellon, O., Silva, L., “New improvements on range image segmentation by edge detection techniques”, Proc. Workshop on Artificial Intelligence and Computer Vision, 2000. [4] Brenner, Claus. "Building reconstruction from images and laser scanning." International Journal of Applied Earth Observation and Geoinformation 6.3 (2005): 187-198. [5] Choi, Wongun, Caroline Pantofaru, and Silvio Savarese. "Detecting and tracking people using an rgb-d camera via multiple detector fusion." Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on. IEEE, 2011. [6] Coleman, S.A., Scotney, B.W., Herron, M.G., “A Systematic Design Procedure for Scalable Near-Circular Laplacian of Gaussian Operators”, 17th International Conference on Pattern Recognition ICPR’04, Vol. 1, pp. 700 – 703, 2004. [7] Coleman, S.A., Suganthan S., and Scotney, B.W., "Gradient operators for feature extraction and characterisation in range images." Pattern Recognition Letters 31.9, pp. 1028-1040, 2010. [8] Coleman, Sonya, Bryan Gardiner, and Bryan Scotney. "Adaptive tri-direction edge detection operators based on the spiral architecture." IEEE International Conference on Image Processing (ICIP), 2010. [9] Curcio, C.A., et al., “Human Photoreceptor Topography”, Journal of Comparative Neurology, Vol. 292, pp. 497-523, 1990. [10] Gardiner, B., “Hexagonal Image Processing,” Ph.D. dissertation, Ulster University, 2010.  [11] Gardiner, Bryan, Sonya Coleman, and Bryan Scotney. "A design procedure for gradient operators on hexagonal images." International Machine Vision and Image Processing Conference, IMVIP 2007.  [12] Ghobadi, Seyed Eghbal, et al. "Detection and classification of moving objects-stereo or time-of-flight images." Computational Intelligence and Security, 2006 International Conference on. Vol. 1. IEEE, 2006. [13] L. Bo, X. Ren, D. Fox, “Unsupervised Feature Learning for RGB-D Based Object Recognition”, In International Symposium on Experimental Robotics, (ISER), June 2012.  [14] Middleton L., Sivaswamy J., Hexagonal Image Processing; A Practical Approach, Springer, 2005. [15] Murray, Don, and James J. Little. "Using real-time stereo vision for mobile robot navigation." Autonomous Robots 8.2 (2000): 161-171. [16] P.J. Besl, “Active, optical range imaging sensors”, Machine Vision Applications, Vol. 1, pp. 127–152, 1988. [17] R.Newcombe, S.Izadi,O.Hilliges,D.Molyneaux,D.Kim,A.Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon, “Kinectfusion: Real-time dense surface mapping and tracking.” IEEE International Symposium on Mixed and Augmented Reality, 2011.  [18] Salvi, Joaquim, Joan Batlle, and E. Mouaddib. "A robust-coded pattern projection for dynamic 3D scene measurement." Pattern Recognition Letters 19.11 (1998): 1055-1065. [19] Sturm, Jürgen, et al. "A benchmark for the evaluation of RGB-D SLAM systems." Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on. IEEE, 2012. [20] Wuthrich, C.A., Stucki, P., “An Algorithm Comparison between Square and Hexagonal Based Grids,” CVGIP: Graphical Models and Image Processing 53, pp. 324-339,1991. Proceedings of the 17th Irish Machine Vision and Image Processing conference IMVIP 2015August 26th-28th, 2015, Dublin, Ireland 121 ISBN 978-0-9934207-0-2

Range Image Feature Extraction using a Hexagonal Pixel-based Framework

https://pure.ulster.ac.uk/ws/files/11509824/IMVIP_2015.pdf

Range Image Feature Extraction using a Hexagonal Pixel-based Framework

Abstract

Similar works

Full text

Available Versions

Ulster University's Research Portal