A conventional approach to image analysis is to perform separately feature extraction at a low level (such as edge detection) and follow this with high level feature extraction to determine structure (e.g. by collecting edge points using the Hough transform. The original image Ray Transform (IRT) demonstrated capability to extract structures at a low level. Here we extend the IRT to add shape specificity that makes it select specific shapes rather than just edges, the new capability is achieved by addition of a single parameter that controls which shape is elected by the extended IRT. The extended approach can then perform low-and high-level feature extraction simultaneously. We show how the IRT process can be extended to focus on chosen shapes such as lines and circles. We confirm the new capability by application of conventional methods for exact shape location. We analyze performance with images from the Caltech-256 dataset and show that the new approach can indeed select chosen shapes. Further research could capitalize on the new extraction ability to extend descriptive capability

Ah Reum, Oh

Nixon, Mark S.

English

Southampton (e-Prints Soton)

On a Shape Adaptive Image Ray Transform  Ah-Reum Oh School of Electronics and Computer Science University of Southampton Southampton, UK aro1g11@ecs.soton.ac.uk Mark S. Nixon School of Electronics and Computer Science University of Southampton Southampton, UK msn@ecs.soton.ac.uk  Abstract— A conventional approach to image analysis is to perform separately feature extraction at a low level (such as edge detection) and follow this with high level feature extraction to determine structure (e.g. by collecting edge points using the Hough transform. The original image Ray Transform (IRT) demonstrated capability to extract structures at a low level. Here we extend the IRT to add shape specificity that makes it select specific shapes rather than just edges; the new capability is achieved by addition of a single parameter that controls which shape is selected by the extended IRT. The extended approach can then perform low-and high-level feature extraction simultaneously. We show how the IRT process can be extended to focus on chosen shapes such as lines and circles. We confirm the new capability by application of conventional methods for exact shape location. We analyze performance with images from the Caltech-256 dataset and show that the new approach can indeed select chosen shapes. Further research could capitalize on the new extraction ability to extend descriptive capability. Keywords- computer vision, feature extraction, shape extraction, Image Ray Transform I. INTRODUCTION The deployment of computer vision has become more important in an increasingly industrialised society, requiring sophisticated methods for feature extraction in image interpretation. The conventional approach in computer vision is to perform feature extraction at a low level, to pre-process an entire image. This is followed by high level feature extraction to determine structure. Physical analogies have been deployed previously times for low level feature extraction. For example, [1] refined anisotropic diffusion [2] to find moving edges using heat flow in the temporal dimension. Force [3] and magnetic [4] fields have also been developed for image segmentation. Apart from these methods, there is a variety of feature extraction techniques using physical properties, such as time [5].  These operators act at low-level, prior to image segmentation.  The Image Ray Transform (IRT) was originally a low-level operator which operates by analogy to light and has been applied as a pre-processing stage for image analysis within ear biometrics and retinal structures [6,7]. The IRT uses an analogy to reflection and refraction of light rays, with materials’ refractive indices calculated from the image information. The analogy actually guides development of the image processing operator; an exact implementation of light modeling is not used. The original IRT is a powerful technique for low level feature extraction and appeared especially suited to the detection of curved objects, as shown in its deployment in detecting ears for biometric purposes.  Our novel extension to the IRT adds a new parameter, a shape factor, to allow edge detection, and object selection at the same time. This is then the first approach to combine low-low level analysis with high level structure. This is illustrated in Fig. 1 (a) which shows a synthetic image of a square and a circle. The original IRT provides the result in Fig.1 (b), showing the edges of the square and of the circle – and the result of applying the Hough transform is to find neither the line nor the circle, precisely. By introducing the new shape factor, the process can be arranged to select circles, as in Fig. 1 (c) – where the circle is correctly detected, or lines, as in Fig. 1 (d) where a line is detected. This is achieved by changing the value of a single parameter – the shape factor.  The Hough Transform [8] can be used to corroborate the quality of the new feature extraction process. Accordingly, adding the shape parameter is a new way to extract features of objects and extends the whole performance of the IRT. Application of the Hough Transform shows that the extended IRT can lead to more general capability, with superior performance, to that of the original IRT.      (a) original image (b) original IRT (c) extended IRT shape factor = 1.5 (d) extended IRT shape factor = 1.9 Figure 1. Illustrating results of the original Image Ray Transform (IRT) and the extended IRT, with the Hough transform results superimposed showing that the circle and the line have been emphasized by different selection of the new shape factor. In this paper, Section II summarises the basic IRT and the new extended IRT. The abilities of the two operators are compared on synthesised and real images, as shown in Section III, with confirming analysis by the Hough Transform, before concluding in Section IV.  II. SHAPE ADAPTIVE IMAGE RAY TRANSFORM A. Basic Image Ray Transform The basic Image Ray Transform (IRT) analogises the progression of light via Snell’s law, in pixel-based images and is unique, natural and efficient. The basic IRT [6] models the propagation of light through image data as a new method of low level feature extraction.  The basic IRT uses Snell’s law within images using reflection and refraction of a light ray as shown on Fig. 2. The amount of reflection and refraction depends on the materials’ refractive indices and the incoming ray inclination   . If two media    and    are different the ray will reflect or refract on the boundary of two media according to the critical angle   .    and the relative equations of Snell’s law are determined by the two media. These are given by following (1),         (    ) ,                                         where    and    represent the angles of reflection and refraction, respectively. Each refractive index is given by following (2),      (    )  (      )                         where      is the maximum refractive index. One of the vital parameters for the IRT is the normals (N) that are used to calculate directions when the ray refracts or reflects. These can be estimated using the Sobel operator (E . After calculating normals for initialising materials’ refractive indices matrix of the whole image, the initial position (x, y) and angle   of each ray are selected at random. The bounds on these selections are        ,          and            , where the width of the image is w and the height of the image is h, respectively. A    describes the direction of the ray. An initial position vector    and direction vector   :     (    )  ,      (          )                         , and those compose the next points    at time  :                                                      Figure 2. Ray Propagation in the Standard and Extended IRT. When a ray meets a boundary where material refractive indices change, we assign the angle of incidence    and n:               ,                                  ) As the ray goes forward from an initial position, position and direction vectors are updated by material indices   ,   and the incidence angle   . Equation (6) describes how to determine the reflection vector    and refraction vector    according to   ,  ,    and    using (1)-(5).       {            if         and                    if         and                      otherwise                                    If        and       the ray reflects with reflection vector   :          (      )                               ( ) when the ray is in the condition        and       it refracts with refracts vector   :              (      )                          ( ) , where       is       √                                        B. Extended Image Ray Transform We propose an extension to the IRT that modifies its principal operations to imbue capability to extract chosen. The extension is derived by quite a simple process. It appears that addition of a single parameter introduces an ability to select between different objects, highlighting one shape – e.g. a circle – in preference to another – e.g. a square. This ability is not shared by other techniques: to extract a circle by curvature, one would need to determine areas of constant curvature and this implies a two-step process. Snakes can be predisposed to be attracted to objects of a different shape, by suitable selection of priors; the Hough transform for circles and evidence gathering for lines differ clearly in the basis equations for describing the shape, rather than in the way the evidence gathering process is defined. As such, we shall show an extension which although simple, appears likely to have significant impact on the generality and use of the image ray transform. The extension is focused on the reflection stage of the IRT since reflection influences emphasis of the edge of objects more effectively than refraction. Equation (10) that can control the amount of reflection is modified by (7) as described in Fig. 2, so it is definitely important to choose a suitable shape factor    for images that are comprised of different objects. The modified version of (7) is given by following, for the same symbols as in section II.A               (      )                          The basic example results according to the variation of shape factor    from zero to two using (10) in synthesised image included only a circle are shown that in Fig. 3.  The process of the IRT is illustrated in Function 1. A ray progresses from an initial point to a final point by the  Normal             Reflected Refracted       Extended IRT Basic IRT       = 0.25   =0.50   =0.75   =1.00       = 1.25   =1.50   =1.75   =2.00 Figure 3. The results of extended IRT according to shape factor variation. (Iteration number : 10000)  Function 1 normals SobelDirection +      RefractiveMatrix(Image, nMax) for    0 to iterationNumber do    Rands  ,width ,     Rands  ,height ,     Rands(0,  )   ,        ,   ,          for i   to masPathIndex do if       and       then               Reflect(  - ,    - ,    - ) else if                  then               Refract(  - ,    - ,    - ) Else     -    - , ,       -    - ,   t = t + 1 End normalise(result) return result IRT without interruption unless the ray meets any boundaries of an image or reaches a specific number, the maximum path index, fixed as a parameter. After the rays are generated by parameterized iteration number the result is normalised for representing to an image, which has gray level from 0 to 255.  III. RESULTS Synthesised images and real images from the Caltech 256 dataset [9] are used to demonstrate the ability of the extended IRT. 10 subsets of the database are used for this experiment and repeated a hundred times in each image to analyse the pattern of the shape factor variation. The results of finding a line and a circle by using the standard Hough Transforms are shown with box plot of the error analysis. The error in estimating the accuracy of extended IRT as a preprocessing stage is given as:      √                                              √                                       when    and    mean the error factor of line and circle, respectively. The values for the subscript m denote the result of Hough Transform and the values of subscript o are the original values used in synthesis.  A. Synthesised images As the number of iterations increases, the extended IRT tends to distinguish objects more clearly from the background in the image as the paths in the background are removed after normalisation. The new shape factor    tends to reduce the amount of reflection so that rays trace better the edge of the image object. If the factor is set between zero and one, the results are unacceptable for extraction of the circle’s boundary while the rays scatter when    exceeds two. However, the result can be acceptable if the new factor is set between one and two. After comparing the results of the basic IRT, which means that    is one, and extended IRT, the object in the image using extended IRT reveals the edges of constituent objects more clearly. As such, the Hough transform can be better used to determine shapes, in this case lines and circles. Fig. 4 shows the effect of choosing different values for the shape factor. Fig. 4 (a) and (b) are the original images containing a circle and a line, and (c-f) show the results of basic IRT and extended IRT with a shape factor        and     . , respectively. Superimposed are the results of applying the Hough Transform for lines and for circles. The result of the shape factor 1.0 is the same as the result of basic IRT and which shows no shape selectivity: one shape appears in the output of the IRT and so the line and the circle are determined at arbitrary positions. When   =1.8 the circle is detected as the size of the circle in the image is closed to original circle while the detected line is tangential to the line. The box plot in Fig. 4 (g) and (h) describes how the error factor changes with the shape factor in the synthesised image. The plot shows the shape factor variation from one to two along the horizontal axis and the error factor from zero along the vertical axis. In the case of finding an ideal circle, the minimum interval of the error is between   =1.7 and 2.0 in Fig. 4 (g), and minimum interval of the error factor in finding a line is between   =1.0 and 1.3 in Fig. 4 (h).   (a) original_circle (b) original_line     (c)   =1.0 (d)   =1.8 (e)   =1.0 (f)   =1.8   (g) The box plot of circle by HT (h) The box plot of line by HT Figure 4. The result examples of basic IRT and extended IRT.      (b)   =1.0 (c)   =1.5 (d)   =1.9    (a) Original image (e)   =1.0 (f)   =1.5 (g)   =1.9   (h) The box plot of line by HT (i) The box plot of circle by HT Figure 5. The result examples of extended IRT and the error factor box plot in a real image.  Since this result is based on the radius of a circle set to 100 (pixels), it can be altered by the radius of a circle that should be found on the Hough Transform stage. Thus, it is important to assign the suitable radius commensurate with the shape type of object in advance for finding specific objects in the image.  B. Real images The result for the extended IRT in real images is more distinct than the result in synthetic images. Fig. 5 (a) is the original image, Fig. 5 (b-d) are the normalized outputs of the IRT with differing values of the shape factor   , showing the emphasis of different shapes. Fig. 5 (e-g) are the results of applying the HT superimposed on the output of the IRT. Although finding a line and a  circle looks effective when    = 1.5, other two images Fig. 5 (e) and Fig. 5 (g) show discrepancy between the result using Hough Transform and visual results (the result using human eyes). Fig. 5 (h) and (i) shows that the shape factor interval for a line and a circle can be separated, where the minimum error factor interval of line is over 1.2 and the minimum error factor interval of circle is under 1.6.  On the other hand, the result in Fig. 6 (b) shows that a circle and a line cannot be found when the shape factor is 1.0. The minimum error factor interval of line and circle are between 1.2 and 1.6 and between 1.4 and 1.6, respectively. It means that there is an appropriate shape factor in each object if the objects are mixed in one image. Another several examples of extended IRT in real      (a) Original image (b)   =1.0 (c)   =1.5 (d)   =1.9   (e) The box plot of line by HT (f) The box plot of circle by HT Figure 6. The result examples of extended IRT and the error factor box plot in a real image.      (a) Original image (b)   =1.0 (c)   =1.6 (d)   =1.9     (e) Original image (f)   =1.0 (g)   =1.6 (h)   =1.9 Figure 7. Examples of detection on real images. images are shown in Fig. 7. The radius for Hough Transform in the result of Fig. 7 (b-d) is set to both circle and the radius Fig. 7 (f-g) is set to the small circle near the bottom of the image. It also shows possibility to search the correct position of the shape by choice of shape factor. We shall continue to refine the analysis of its abilities in real images, doubtless requiring refinement of the techniques’ basis.  IV. CONCLUSIONS The extended Image Ray Transform has improved performance of the original Image Ray Transform for feature extraction in computer vision. Adding a shape factor is beneficial to extend the basic IRT to finding edges of selected shapes. The analyses, using the Hough Transform, of images that include various types of ideal objects verify how the extended IRT can perform on real images and where it can be applied. The shape factor that controls the amount of reflection of light rays is the one of important parameters which can determine the quality of performance in the extended IRT.  Several studies using 3-D volumetric data that describes human body have researched for 3-D biometrics. These researches have used high level feature extraction techniques, however, the extended IRT can help to identify silhouette of body parts for 3-D biometrics. For this reason, further researches are needed to study relevant shape factors and to analyze histogram patterns with various types of objects in real images. If shapes of body part – e.g. an ear, retina and gait – have their own unique histogram patterns, it can be possible to recognise them using the extended IRT in low level feature extraction, thereby extending this new combined approach to feature extraction. REFERENCES [1] C. Direkoglu and M. S. Nixon, “Moving-edge detection via heat flow analogy”, Pattern Recognition Letters, 32(2):270 – 279, 2011. [2] P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629-639,1990. [3] D. J. Hurley, M. S. Nixon and J. N. Carter, “Force field energy functionals for image feature extraction”, Image and Vision Computing, 20:311–317, 2002. [4] S. Beucher and C. Lantuejoul, “Use of watersheds in contour detection”, In Proceedings of Int. Workshop on Image Processing, Real-Time Edge and Motion Detection/Estimation, 1979.  [5] P. Maragos, “PDEs for morphological scale-spaces and eikonal applications”, In A. C. Bovik, Ed., The Image and Video Processing Handbook, chapter 4.16, pp. 587–612. Elsevier Academic Press, 2nd Ed., 2005 [6] A. H. Cummings, M. S. Nixon and J. N. Carter, “A novel ray analogy for enrolment of ear biometrics”, In 4th IEEE BTAS, 2010. [7] A. H. Cummings, M. S. Nixon and J. N. Carter, “The image ray transform for structural feature detection”. Pattern Recognition Letters, 32(15):2053–2060, 2011. [8] P. V. C. Hough, “Method and means for recognizing complex patterns”, U.S. Patent 3.069.654, 1962. [9] G. Griffin, AD. Holub and P. Perona, Caltech-256 Object Category Dataset, Caltech Technical Report, 2006.   

On a shape adaptive image ray transform

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)