This paper proposes a functional that assigns low `energy' to sets of subsets of the image domain consisting of a number of possibly overlapping near-circular regions of approximately a given radius: a `gas of circles'. The model can be used as a prior for object extraction whenever the objects conform to the `gas of circles' geometry, e.g. cells in biological images. Configurations are represented by a multi-layer phase field. Each layer has an associated function, regions being defined by thresholding. Intra-layer interactions assign low energy to configurations consisting of non-overlapping near-circular regions, while overlapping regions are represented in separate layers. Inter-layer interactions penalize overlaps. Here we present a theoretical and experimental analysis of the model

Jermyn, I.

Kato, Z.

Molnar, C.

English

Durham Research Online

A Multi-Layer Phase Field Model for Extracting Multiple Near-CircularObjects∗Csaba Molnar1, Zoltan Kato1, Ian Jermyn21Image Processing and Computer Graphics Department, University of Szeged, P.O. Box 652, 6701 Szeged, Hungary2Department of Mathematical Sciences, Durham University, South Rd, DURHAM DH1 3LE, United Kingdommcsaba@inf.u-szeged.hu, kato@inf.u-szeged.hu, i.h.jermyn@durham.ac.ukAbstractThis paper proposes a functional that assigns low‘energy’ to sets of subsets of the image domain consist-ing of a number of possibly overlapping near-circularregions of approximately a given radius: a ‘gas of cir-cles’. The model can be used as a prior for object ex-traction whenever the objects conform to the ‘gas of cir-cles’ geometry, e.g. cells in biological images. Config-urations are represented by a multi-layer phase field.Each layer has an associated function, regions beingdefined by thresholding. Intra-layer interactions as-sign low energy to configurations consisting of non-overlapping near-circular regions, while overlappingregions are represented in separate layers. Inter-layerinteractions penalize overlaps. Here we present a theo-retical and experimental analysis of the model.1. IntroductionObject extraction is one of the key problems of im-age processing. The problem is simple to state: findthe regions in the image domain occupied by a speci-fied object or objects. It is, however, hard to solve: forall except the simplest images, solution requires objectmodels containing significant shape information in ad-dition to information concerning image properties. Inaddition, in many applications the number of instancesof the object of interest occurring in the image may beunknown a priori, and the objects may interact: theymay cluster together, be regularly spaced, or tend not tooverlap, for example.Most shape modelling methods use one or moretemplate shapes and variations around them to capture∗Partially supported by the grants CNK80370 of NIH & OTKA;the European Union and the European Regional Development Fundwithin the project TÁMOP-4.2.1/B-09/1/KONV-2010-0005.shape uncertainty [1, 9, 8]. These methods are ineffi-cient when the number of objects is unknown or whenobjects interact. Each object requires separate represen-tation and computation, as well as auxiliary variables toachieve geometric invariance. Each pair of interactingobjects requires extra computation. ‘Higher-order ac-tive contour’ (HOAC) models [7] provide an alternativeapproach. HOAC models can describe shape knowl-edge without templates, using long-range dependenciesbetween region boundary points, and are intrinsicallyinvariant. When expressed in terms of phase fields [6],they can be used to model configurations consisting ofmultiple interacting objects at no extra cost.An important subset of object extraction problemsinvolve multiple objects of near-circular shape, e.g. treecrowns in remote sensing images, and cells and otherstructures in biological images, and are thus difficult tosolve using standard shape modelling methods. To ad-dress these problems, Horvath et al. [4, 3] describeda HOAC model favouring subsets of the image domainconsisting of any number of near-circular componentswith approximately a given radius. This ‘gas of circles’(GOC) model was successfully used for the extractionof tree crowns from aerial images. The model suffers,however, from two limitations that render it unsuitablefor many important applications. The first arises fromthe representation: because the configuration space con-sists of subsets of the image domain, as opposed to setsof subsets, touching or overlapping objects cannot berepresented. The second arises from the model: thelong-range interactions that favour near-circular shapesalso create repulsive interactions between nearby ob-jects, meaning that objects in low-energy configurationsare typically separated by a distance comparable to theirsize. In [5], a first attempt was made to overcome theselimitations in a binary Markov random field model.However, the use of binary variables means that the ge-ometric accuracy of the representation is limited, andsolution requires computationally expensive stochasticoptimization.In this paper, we propose a generalization of theGOC model [3] that overcomes all these limitationswhile maintaining computational efficiency: the multi-layer phase field GOC model. This model consists ofmultiple instances of the phase field GOC model, eachinstance being known as a ‘layer’. This makes it pos-sible to represent overlapping objects as subsets on dif-ferent layers, thereby removing the first limitation. Theonly inter-layer interaction is an overlap penalty: thelong-range interaction does not act between differentlayers. As a result, objects in separate layers do not re-pel, thereby removing the second limitation. MAP esti-mates can be computed by minimizing the energy of themodel via gradient descent, which is relatively compu-tationally efficient. With a suitable data likelihood, thenew model can be used for object extraction in the manycases in which the ‘gas of circles’ geometry is relevant.In this paper, we demonstrate its use for the extractionof cells and lipid droplets from biological images.2. Single-layer ‘gas of circles’ phase fieldmodelA phase field model [6] represents a subset R ⊂ Dby a function ϕ : D → R on the image domain D, anda threshold t: R = {x ∈ D : ϕ(x) ≥ t}. The single-layer ‘gas of circles’ phase field energy E is [3]:E(ϕ) =∫D{Df2|∇ϕ|2 + αf(ϕ− ϕ33)+ λf(ϕ44− ϕ22)}− βf2∫∫D×D′∇ϕG(x, x′)∇ϕ′ , (1)where (un)primed functions are evaluated at (x ∈ D)x′ ∈ D′ ≡ D. The interaction function G isG(z) ={12(2− |z|d −1π sin(π(|z|−d)d))if |z| < 2d ,1−H(|z| − d) else ,(2)where d controls the range of interaction and H is theHeaviside step function. This model assigns low energyto subsets of D consisting of a number of near-circularregions of approximately a given radius, separated bydistances comparable to their size [3].3. Multi-layer ‘gas of circles’ phase fieldmodelWe now extend the single-layer model of equa-tion (1) to a multi-layer GOC model. The use of multi-ple layers enables the representation, not just of subsets,Figure 1. Layered phase fieldsbut of sets of subsets of D, because subsets with non-empty intersection can now be represented on separatelayers. As a result, the new model can represent objectsthat touch and overlap in the image.This is not enough on its own because the long-range interaction in equation (1) creates a repulsion be-tween connected components, favouring configurationsin which the objects are separated by a distance com-parable to their size. While appropriate for some prob-lems, e.g. tree crowns in regular plantations [3], it failsfor problems in which objects are touching or overlap-ping, see e.g. Fig. 6. To overcome this limitation, in thenew model the long-range interactions act intra-layerbut not inter-layer. This has two effects. First, the low-energy configurations in each layer are still ‘gas of cir-cles’ configurations, as required. Second, the repulsiveinteraction is eliminated, because repulsively interact-ing regions can ‘escape’ to separate layers. The result isthat overlapping ‘gas of circles’ configurations on sep-arate layers can now be combined without penalty. Toavoid degenerate configurations, in which a given ob-ject is duplicated across all layers, an inter-layer areaoverlap penalty is introduced.To proceed, we redefine the phase field as a multi-component object: ϕ = {ϕi}i∈[1..ℓ] : [1..ℓ] × D → R,ϕ, where ℓ is the number of layers. The total energy Ẽof the new multi-layer model then takes the formẼ(ϕ) =ℓ∑i=1E(ϕi) +κ4∑i̸=j∫D(1 + ϕi)(1 + ϕj) , (3)where E is defined in equation (1), and κ is a new pa-rameter controlling the strength of the overlap penalty.An example of a low-energy configuration is shownin Fig. 1.Note that ‘background’ points, with ϕi ≃ −1, do notgenerate overlap penalty. Note also that if they do notoverlap, objects in range of the repulsive interaction willtend to lie in different layers. If they do overlap, thereis competition between the repulsive interaction and theoverlap penalty. If κ is not too large, they will exist onseparate layers; if κ is large enough, they will exist onthe same layer, perhaps reducing to one object.l = 1 l = 2 l = 3 l = 5κ = 0.02 κ = 0.02 κ = 0.02 κ = 0.02Figure 2. Typical configurations of the prior model(r = 10, negative circle energy). κ = 0 in the top row.3.1 Functional derivative of the energyThe layered phase field energy will be minimizedvia gradient descent, for which we need to compute itsfunctional derivative:δẼ(ϕ)δϕk(x′)=δE(ϕk)δϕk(x′)+κ4δEO(ϕ)δϕk(x′)(4)where EO(ϕ) denotes the overlap energy term fromEq. (3). The first term is simply the functional deriva-tive of E evaluated at the ϕk, and so is known [6, 3].The derivative of the overlap energy isδEO(ϕ)δϕk(x′)= 2∑i ̸=j∫Dδik δ(x, x′)(1 + ϕj(x)) (5a)= 2∑j ̸=k(1 + ϕj(x′)) (5b)= 2(ℓ− 1) + 2∑j ̸=kϕj(x′) . (5c)4. Experimental resultsIn this section, we report the evaluation of the be-haviour of the multi-layer phase field GOC model onsynthetic and real microscope images. For these exper-iments, we used the following phase field data term:Ei(I,R) =∫Ω{γ1∇I · ∇ϕ+ γ2[(I − µin)22σ2inϕ+ +(I − µout)22σ2outϕ−]}, (6)where: ∇ϕ, and ϕ± = (1 ± ϕ)/2 are approximatelythe normal vector to the boundary, and the characteristicfunctions of the region (+) and its complement (−) re-spectively [2]; I : D → R is the image data; µin,out andσin,out are the parameters of pixel-wise Gaussian distri-butions modelling the image in the interior (in) and ex-terior (out) regions, learned from samples; and γ1,2 arepositive weights.Fig. 2 shows some minimum energy configurationsof the prior model. Without loss of generality, we chosed = 10, αf = 0.2795, βf = 0.0911, λf = 0.625 andDf = 0.75 to ensure the negative circle energy neededto have objects in the minimum energy configuration.As expected, κ = 0 yields overlapping objects, whileκ > 0 prevents overlaps. If κ is too high, then either anempty configuration or unstable circles are produced.In the next experiment, we take a closer look at theproperties of the model in the case of two circles withdifferent levels of overlap. We used two layers, circlesof radius 10, and κ values in the range [0, 1]. Fig. 3shows segmentation error versus κ and overlap w. Nearκ = 0.7 there is a clear fall in the segmentation error. Ifw > 10, we need a larger κ to achieve a good segmen-tation.noisy small optimal largeimage κ κ κFigure 3. Segmentation results for noisy images con-taining two overlapping circles. Overlap w: 5 (top);10 (middle); 15 (bottom). Small κ = 0.55; optimalκ = 0.7 (0.8 when w = 15); large κ = 1.Finally, we examine the relationship between thedata weight γ2, overlap penalty κ, and extraction accu-racy on synthetic images with more than two touchingcircles. Fig. 4 shows the ratio of correctly detected cir-cles as a function of κ and γ2. It is clear that there areseveral parameter pairs for which one can achieve a cor-rect segmentation starting from a random initialization.4.1. Biological applicationIn microbiology, one of the main image processingproblems is to extract multiple objects, e.g. cells, lipiddroplets, or other sub-cellular components, that are of-ten near-circular with many overlaps. The light micro-scope techniques used produce noisy, blurred imagesFigure 4. Results on synthetic images. Left: segmen-tation; right: proportion of correctly detected circles asa function of κ and γ2.with low contrast. The results shown in Fig. 5 and Fig. 6show that the proposed model can solve these object ex-traction problems.To initialize the multi-layer phase field, we useda simple thresholding and connected component de-tection, plus random assignment of different layers tonearby initial regions. Typical computation time inMatlab is about 20 seconds for a 200× 100 image with3 layers.Figure 5. Extraction from light microscope imagesof cells having a particular radius.5. ConclusionsThe multi-layer phase field GOC model is capable ofrepresenting and modelling an a priori unknown num-ber of touching or overlapping near-circular objects.Figure 6. Extraction from light microscope imagesof lipid drops having a particular radius.Such problems occur frequently in biomedicine and bi-ology (e.g. cell images). Experiments show that themodel behaves as expected, and that when coupled withan appropriate data model and initialization, can effi-ciently extract such object configurations from syntheticand real images.References[1] D. Cremers, F. Tischhauser, J. Weickert, and C. Schnorr.Diffusion snakes: Introducing statistical shape knowl-edge into the Mumford-Shah functional. InternationalJournal of Computer Vision, 50(3):295–313, 2002.[2] P. Horvath and I. Jermyn. A ’gas of circles’ phase fieldmodel and its application to tree crown extraction. InProc. EUSIPCO, Poznan, Poland, Sep 2007.[3] P. Horvath and I. Jermyn. A new phase field model ofa ’gas of circles’ for tree crown extraction from aerialimages. In Proc. CAIP, pp 702–709, Vienna, Austria,Aug 2007.[4] P. Horvath, I. Jermyn, Z. Kato, and J. Zerubia. A higher-order active contour model of a ‘gas of circles’ and itsapplication to tree crown extraction. Pattern Recognition,42(5):699–709, May 2009.[5] J. Nemeth, Z. Kato, and I. Jermyn. A multi-layer ’gasof circles’ Markov random field model for the extractionof overlapping near-circular objects. In Proc. ACIVS, pp171–182, Ghent, Belgium, Aug 2011.[6] M. Rochery, I. H. Jermyn, and J. Zerubia. Phase fieldmodels and higher-order active contours. In Proc. ICCV,Beijing, China, IEEE, Oct 2005.[7] M. Rochery, I. H. Jermyn, and J. Zerubia. Higher orderactive contours. International Journal of Computer Vi-sion, 69(1):27–42, Aug 2006.[8] C. Russell, D. Metaxas, C. Restif, and P. Torr. Using thePn Potts model with learning methods to segment livecell images. In Proc. ICCV, pp 1–8. IEEE, Oct 2007.[9] A. Srivastava, S. Joshi, W. Mio, and X. Liu. Statisticalshape analysis: Clustering, learning, and testing. IEEETransactions on Pattern Analysis and Machine Intelli-gence, 27(4):590–602, 2005.

A multi-layer phase field model for extracting multiple near-circular objects

https://durham-repository.worktribe.com/file/1152842/1/Accepted%20Conference%20Proceeding

A multi-layer phase field model for extracting multiple near-circular objects

Abstract

Similar works

Full text

Available Versions

Durham Research Online