PostprintSince the work by Osher and Sethian (1988) on level-sets algorithms for numerical shape evolutions, this technique has been used for a large number of applications in numerous fields. In medical imaging, this numerical technique has been successfully used, for example, in segmentation and cortex unfolding algorithms. The migration from a Lagrangian implementation to a Eulerian one via implicit representations or level-sets brought some of the main advantages of the technique, i.e., topology independence and stability. This migration means also that the evolution is parametrization free. Therefore, the authors do not know exactly how each part of the shape is deforming and the point-wise correspondence is lost. In this note they present a technique to numerically track regions on surfaces that are being deformed using the level-sets method. The basic idea is to represent the region of interest as the intersection of two implicit surfaces and then track its deformation from the deformation of these surfaces. This technique then solves one of the main shortcomings of the very useful level-sets approach. Applications include lesion localization in medical images, region tracking in functional MRI (fMRI) visualization, and geometric surface mapping

Bertalmío, Marcelo

Randall, Gregory

Sapiro, Guillermo

English

LAReferencia - Red Federada de Repositorios Institucionales de Publicaciones Científicas Latinoamericanas

Region Tracking on Surfaces Deforming viaLevel-Sets Methods?Marcelo Bertalmio1, Guillermo Sapiro1, and Gregory Randall21 Electrical and Computer EngineeringUniversity of MinnesotaMinneapolis, MN 55455guille@ece.umn.edu2 I.I.E. Facultad de IngenieriaUniversidad de la RepublicaMontevideo, UruguayAbstract. Since the work by Osher and Sethian on level-sets algorithmsfor numerical shape evolutions, this technique has been used for a largenumber of applications in numerous fields. In medical imaging, this nu-merical technique has been successfully used for example in segmentationand cortex unfolding algorithms. The migration from a Lagrangian im-plementation to an Eulerian one via implicit representations or level-setsbrought some of the main advantages of the technique, mainly, topologyindependence and stability. This migration means also that the evolutionis parametrization free, and therefore we do not know exactly how eachpart of the shape is deforming, and the point-wise correspondence is lost.In this note we present a technique to numerically track regions on sur-faces that are being deformed using the level-sets method. The basic ideais to represent the region of interest as the intersection of two implicitsurfaces, and then track its deformation from the deformation of thesesurfaces. This technique then solves one of the main shortcomings of thevery useful level-sets approach. Applications include lesion localizationin medical images, region tracking in functional MRI visualization, andgeometric surface mapping.Key words: Level-sets, region tracking and correspondence, medical imag-ing, segmentation, visualization, shape deformation.1 IntroductionThe use of level-sets for the numerical implementations of n-dimensional1 shapedeformations became extremely popular following the seminal work of Osher andSethian [17] (see for example [14,18] for some of the applications of this tech-nique and a long list of references). In medical imaging, the technique has beensuccessfully used for example for 2D and 3D segmentation [5,10,12,15,20,21].? A journal version of this paper appears in the May 1999 issue of IEEE Trans. MedicalImaging.1 In this note we consider n ≥ 3.M. Nielsen et al. (Eds.): Scale-Space’99, LNCS 1682, pp. 330–338, 1999.c© Springer-Verlag Berlin Heidelberg 1999Region Tracking on Surfaces Deforming via Level-Sets Methods 331The basic idea is to represent the deformation of an n-dimensional closed sur-face S as the deformation of an n + 1-dimensional function Φ. The surface isrepresented in an implicit form in Φ, for example, via its zero level-set. For-mally, let’s represent the initial surface S(0) as the zero level-set of Φ, i.e.,S(0) ≡ {X ∈ IRn : Φ(X, 0) = 0}. If the surface is deforming according to∂S(t)∂t= βN S , (1)where N S is the unit normal to the surface, then this deformation is representedas the zero level-set of Φ(X, t) : IRn × [0, τ) → IR deforming according to∂Φ(X, t)∂t= β(X, t) ‖ ∇Φ(X, t) ‖, (2)where β(X, t) is computed on the level-sets of Φ(X, t). The formal analysis ofthis algorithm can be found for example in [6,7].The basic idea behind this technique is that we migrate from a Lagrangianimplementation (particles on the surface) to an Eulerian one, i.e., a fix Cartesiancoordinate system. This allows for example to automatically follow changes inthe topology of the deforming surface S, since the topology of the function Φ isfixed. See the mentioned references for more details on the level-sets technique.In a number of applications, it is important not just to know how the wholesurface deforms, but also how some of its regions do. Since the parametrizationis missing, this is not possible in a straightforward level-sets approach. Thisproblem is related to the aperture problem in optical flow computation, and it isalso the reason why the level-sets approach can only deal with parametrizationindependent flows that do not contain tangential velocities. Although tangentialvelocities do not affect the geometry of the deforming shape, they do affect the‘point correspondence’ in the deformation. For example, with a straight level-sets approach, it is not possible to determine where a given point X0 ∈ S(0)is at certain time t. One way to solve this problem is to track isolated pointswith a set of ODE’s, and this was done for example in grid generation andsurface flattening; see [9,18]. This is a possible solution if we are just interestedin tracking a number of isolated points. If we want to track regions for example,then using ‘particles’ brings us back to a ‘Lagrangian formulation’ and some ofthe problems that actually motivated the level-sets approach. For example, whathappens if the region splits during the deformation? What happens if the regionof interest is represented by particles that start to come too close together insome parts of the region and too far from each other in others?In this note we propose an alternative solution to the problem of regiontracking on surface deformations implemented via level-sets.2 The basic idea isto represent the boundary of the region of interest R ∈ S as the intersection2 A different level-set approach for intrinsic motions of generic 3D curves, togetherwith very deep and elegant theoretical results, is introduced in [1]. This approachis difficult to implement numerically, and in some cases not fully appropriate fornumerical 3D curve evolution [16]. A variation of this technique, with very good332 M. Bertalmio, G. Sapiro, and G. Randallof the given surface S and an auxiliary surface Ŝ, both of them given as zerolevel-sets of n + 1-dimensional functions Φ and Φ̂ respectively.3 The tracking ofthe region R is given by tracking the intersection of these two surfaces, that is,by the intersection of the level-sets of Φ and Φ̂. In the rest of this note we givedetails on the technique and present examples.Note that although we use the proposed technique to track regions of intereston deforming surfaces, with the region deformation dictated by the surface defor-mation, the same general approach here presented of simultaneously deformingn hypersurfaces (n ≥ 2) and looking at the intersection of their level-sets canbe used for the numerical implementation of generic geometric deformations ofcurves and surfaces of high co-dimension.42 The algorithmAssume the deformation of the surface S, given by (1), is implemented usingthe level-sets algorithm, i.e., Equation (2). Let R ∈ S be a region we wantto track during this deformation, and ∂R its boundary. Define a new functionΦ̂(X, 0) : IRn → IR (a distance function for example), such that the intersectionof its zero level-set Ŝ with S defines ∂R and then R. In other words,∂R(0) := S(0) ∩ Ŝ(0) = {X ∈ IRn : Φ(X, 0) = Φ̂(X, 0) = 0}.The tracking of R is done by simultaneously deforming Φ and Φ̂. The auxiliaryfunction Φ̂ deforms according to∂Φ̂(X, t)∂t= β̂(X, t) ‖ ∇Φ̂(X, t) ‖, (3)and then Ŝ deforms according to∂Ŝ∂t= β̂N Ŝ . (4)We have then to find the velocity β̂ as a function of β. In order to track theregion of interest, ∂R must have exactly the same geometric velocity both in(2) and (3). The velocity in (2) (or (1)) is given by the problem in hand, and isexperimental results, is introduced in [11]. The Ambrosio-Soner approach and itsvariations deal with intrinsic curve-velocities and do not address the surface-velocityprojection needed for the tracking in this paper.3 The use of multiple level-set functions was used in the past for problems like motionof junctions [13]. Both the problem and its solution are different from the ones inthis paper.4 After this paper was accepted for publication, we became aware of recent work byOsher and colleagues using this general approach mainly to deform curves in 3D andcurves on surfaces [4]. This work also does not deal with the projection of velocitiesas needed for our application.Region Tracking on Surfaces Deforming via Level-Sets Methods 333βN S . Therefore, the velocity in (4) will be the projection of this velocity intothe normal direction N Ŝ (recall that the tangential component of the velocitydoes not affect the geometry of the flow). That is, for (at least) ∂R,β̂ = βN S · N Ŝ .Outside of the region corresponding to R, the velocity β̂ can be any functionthat connects smoothly with the values in ∂R.5This technique, for the moment, requires to find the intersection of the zero-level sets of Φ and Φ̂ at every time step, in order to compute β̂. To avoid this,we choose a particular extension of β̂ outside of ∂R, and simple define β̂ as theprojection of βN S for all the values of X in the domain of Φ and Φ̂.6 Therefore,the auxiliary level-sets flow is given by∂Φ̂∂t(X, t) =(β(X, t)∇Φ(X, t)‖ ∇Φ(X, t) ‖ ·∇Φ̂(X, t)‖ ∇Φ̂(X, t) ‖)‖ ∇Φ̂(X, t) ‖,and the region of interest R(t) is given by the portion of the zero level-sets thatbelongs to Φ(X, t) ∩ Φ̂(X, t):∂R(t) = {X ∈ IRn : Φ(X, t) = Φ̂(X, t) = 0}. (5)For a number of velocities β, short term existence of the solutions to the level-sets flow for Φ̂ (in the viscosity framework) can be obtained from the results ofEvans and Spruck [8].This formulation gives the basic region tracking algorithm. In the next sec-tion, we present some examples.3 Examples and commentsWe now present examples of the proposed technique. We should note that: (a)The numerical implementation of both the flows for Φ and Φ̂ follow the ordinarylevel-sets implementations developed by Osher and Sethian [17]; (b) Recently in-troduced fast techniques like narrow bands, fast-marching [18], or local methods[14], can also be used with the technique here proposed to evolve each one of thesurfaces; (c) In the examples below, we compute a zero-order type of intersectionbetween the implicit surfaces, meaning that we consider part of the intersectionthe full vortex where both surfaces go through (giving a jagged boundary). More5 To avoid the creation of spurious intersections during the deformation of Φ andΦ̂, these functions can be re-initialized every few steps, as frequently done in thelevel-sets approach.6 Note that although S and Ŝ do not occupy the same regions in the n dimensionalspace, their corresponding embedding functions Φ and Φ̂ do have the same domain,making this velocity extension straightforward.334 M. Bertalmio, G. Sapiro, and G. Randallaccurate intersections can be easily computed using sub-divisions as in march-ing cubes. Recapping, the same numerical implementations used for the classicallevel-sets approaches are used to implement the deformation of Φ̂, and findingthe intersection is straightforward from algorithms like marching cubes.Four examples are given in Figure 1 and Figure 2, one per column. In eachexample, the first figure on the top shows the original surface with the markedregions to be tracked (brighter regions), followed by three different time steps ofthe geometric deformation and region tracking.Figure 1 shows two toy examples. We track the painted regions on the sur-faces while they are deforming with a morphing type velocity [2,3]. (β(X, t) issimply the difference between the current surface Φ(X, t) and a desired goal sur-face Φ(X,∞), two separate surfaces and two merged balls respectively, therebymorphing the initial surface toward the desired one [3].) Note how the regionof interest changes topology (splits on the left example and merges on the nextone).Next, Figure 2 presents one of the main applications of this technique. Boththese examples first show, on the top, a portion of the human cortex (white-matter/gray-matter boundary), obtained from MRI and segmented with thetechnique described in [19]. In order to visualize brain activities recorder viafunctional MRI in one of the non-visible folds (sulci), it is necessary to ‘un-fold’ the surface, while tracking the color-coded regions (surface unfolding orflattening has a number of applications in 3D medical imaging beyond fMRIvisualization; see also [9]). In the first of these two examples (left column), thedifferent gray values simply indicate sign of Gaussian curvature on the originalsurface (roughly indicating the sulci), while two arbitrary regions are marked inthe last example (one of them with a big portion hidden inside the fold). Wetrack each one of the colored regions with the technique described in this note.In the first column, β(X, t) = sign(κ1)+sign(κ2)2 min(|κ1|, |κ2|), where κ1 and κ2are the principal curvatures. In the second column, we use a morphing type ve-locity like before [2,3] (in this case, the desired destination shape is a convexsurface). See [9] for additional possible unfolding velocities, including volumeand area preserving ones. The colors on the deforming surfaces then indicate,respectively, the sign of the Gaussian curvature and the two marked regions inthe original surfaces. Note how the surface is unfolded, hidden regions are madevisible, and the tracking of the colored coded regions allow to find the matchingplaces in the original 3D surface representing the cortex. This also allows forexample to quantify, per each single tracked region, possible area/length dis-tortions introduced by the flattening process. In order to track all the markedregions simultaneously in these two examples, we select the zero level-set of Φ̂to intersect the zero level-set of Φ at all these regions. If we have regions withmore than two color codes to track, as will frequently happen in fMRI, we justuse one auxiliary function Φ̂ per color (region).The same technique can be applied to visualize lesions that occur on the‘hidden’ parts of the cortex. After unfolding, the regions become visible, and theregion tracking allows to find their position in the original surface. When usingRegion Tracking on Surfaces Deforming via Level-Sets Methods 335                                                                                                Fig. 1. Two simple examples, one per column, of the algorithm introduced in this note(brighter regions are the ones being tracked), demonstrating possible topological changeson the tracked region.336 M. Bertalmio, G. Sapiro, and G. Randall                                                Fig. 2. Unfolding the cortex, and tracking the marked regions, with a curvature basedflow and a 3D morphing one, left and right columns respectively.Region Tracking on Surfaces Deforming via Level-Sets Methods 337level-sets techniques to deform two given shapes, one toward the other (a 3Dcortex to a canonical cortex for example), this technique can be used to findthe region-to-region correspondence. This technique then solves one of the basicshortcomings of the very useful level-sets approach.AcknowledgmentsGS wants to thank Prof. Brian Wandell from Stanford University for introduc-ing him to the fMRI world and its unfolding issues. This was the motivationbehind the algorithm described in this note. MB wants to thank the I.I.E. andthe Facultad de Ingenieria, Montevideo, Uruguay for their continuous supportthrough the M.Sc. Program and M.Sc. Scholarships. We thank Prof. StanleyOsher from UCLA and Prof. Olivier Faugeras from INRIA and MIT for makingus aware of their related work. Comments from the anonymous reviewers aretruly appreciated. This work was partially supported by NSF-LIS, by the Math,Computer, and Information Sciences Division at ONR, by an ONR Young In-vestigator Award, by the Presidential Early Career Awards for Scientists andEngineers (PECASE), by a National Science Foundation CAREER Award, byCSIC, and by CONICYT.References1. L. Ambrosio and M. Soner, “Level set approach to mean curvature flow in arbitrarycodimension,” Journal of Differential Geometry 43, pp. 693-737, 1996.2. M. Bertalmio, Morphing Active Contours, M.Sc. Thesis, I.I.E., Universidad de laRepublica, Uruguay, June 1998.3. M. Bertalmio, G. Sapiro, and G. Randall, “Morphing active contours,” Proc. IEEE-ICIP, Chicago, October 1998.4. P. Burchard, L. T. Cheng, B. Merriman, and S. Osher, “Level set based motion ofcurves in IR3,” UCLA CAM Report, May 1999.5. V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” InternationalJournal of Computer Vision 22:1, pp. 61-79, 196. Y. G. Chen, Y. Giga, and S. Goto, “Uniqueness and existence of viscosity solutionsof generalized mean curvature flow equations,” Journal of Differential Geometry33, 1991.7. L. C. Evans and J. Spruck, “Motion of level sets by mean curvature, I,” Journal ofDifferential Geometry 33, 1991.8. L. C. Evans and J. Spruck, “Motion of level sets by mean curvature, II,” Trans.Amer. Math. Soc. 330, pp. 321-332, 1992.9. G. Hermosillo, O. Faugeras, and J. Gomes, “Cortex unfolding using level set meth-ods,” INRIA Sophia Antipolis Technical Report 3663, April 1999.10. L. M. Lorigo, O. Faugeras, W. E. L. Grimson, R. Keriven, and R. Kikinis, “Segmen-tation of bone in clinical knee MRI using texture-based geodesic active contours,”Proceedings Medical Image Computing and Computer-Assisted Intervention, MIC-CAI ’98, pp. 1195-1204, Cambridge, MA, Springer, 1998.11. L. Lorigo, O. Faugeras, W.E.L. Grimson, R. Keriven, R. Kikinis, and C-F. Westin,“Co-dimension 2 geodesic active contours for MRA segmentation,” InternationalConference on Information Processing in Medical Imaging, June 1999, forthcoming.338 M. Bertalmio, G. Sapiro, and G. Randall12. R. Malladi, J. A. Sethian and B. C. Vemuri, “Shape modeling with front propaga-tion: A level set approach,” IEEE Trans. on PAMI 17, pp. 158-175, 1995.13. B. Merriman, J. Bence, and S. Osher, “Motion of multiple junctions: A level-setapproach,’ Journal of Computational Physics 112, pp. 334-363, 1994.14. B. Merriman, R. Caflisch, and S. Osher, “Level set methods, with an applicationto modeling the growth of thin films, UCLA CAM Report 98-10, February 1998.15. W. J. Niessen, B. M. Romeny, and M. A. Viergever, “Geodesic deformable modelsfor medical image analysis,” IEEE Trans. Medical Imaging 17, pp. 634-641, 1998.16. S. Osher, Personal communication, March 1999.17. S. Osher and J. Sethian, “Fronts propagating with curvature-dependent speed:Algorithms based on Hamilton-Jacobi formulations,” Journal of ComputationalPhysics 79, pp. 12-49, 1988.18. J. Sethian, Level Set Methods: Evolving Interfaces in Geometry, Fluid Mechanics,Computer Vision and Material Sciences, Cambridge University Press, Cambridge,UK, 1996.19. P. Teo, G. Sapiro, and B. Wandell, “Creating connected representations of corticalgray matter for functional MRI visualization,” IEEE Trans. Medical Imaging 16:06,pp. 852–863, 1997.20. A. Yezzi, S. Kichenassamy, P. Olver, and A. Tannenbaum, “Geometric active con-tours for segmentation of medical imagery,” IEEE Trans. Medical Imaging 16, pp.199-210, 1997.21. X. Zeng, L. H. Staib, R. T. Schultz, and J. S. Duncan, “Segmentation and mea-surement of the cortex from 3D MR Images,” Proceedings Medical Image Computingand Computer-Assisted Intervention, MICCAI ’98, pp. 519-530, Cambridge, MA,Springer, 1998.View publication stats

Region tracking on surfaces deforming via level-sets methods

https://www.colibri.udelar.edu.uy/jspui/bitstream/20.500.12008/20772/1/BSR99a.pdf

Region tracking on surfaces deforming via level-sets methods

Abstract

Similar works

Full text

Available Versions

LAReferencia - Red Federada de Repositorios Institucionales de Publicaciones Científicas Latinoamericanas