International audienceMarked point process models have been successfully used in image analysis for feature extraction purposes in high resolution remotely sensed images. The model is usually based on two types of energy terms: a data term, which reflects the configuration's fidelity with respect to the input image, and a prior term, which reflects some knowledge about the objects to be extracted. In this paper, we deal with the problem of elliptical shape extraction. We propose new energy terms for the extraction of boats in harbors, which is a particularly difficult problem and we show results on high resolution optical images

Craciun, Paula

Zerubia, Josiane

INRIA a CCSD electronic archive server

HAL Id: hal-00867585https://hal.inria.fr/hal-00867585Submitted on 30 Sep 2013HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Unsupervised marked point process model for boatextraction in harbors from high resolution opticalremotely sensed imagesPaula Craciun, Josiane ZerubiaTo cite this version:Paula Craciun, Josiane Zerubia. Unsupervised marked point process model for boat extraction inharbors from high resolution optical remotely sensed images. IEEE ICIP - International Conferenceon Image Processing, Sep 2013, Melbourne, Australia. ￿hal-00867585￿UNSUPERVISED MARKED POINT PROCESS MODEL FOR BOAT EXTRACTION INHARBORS FROM HIGH RESOLUTION OPTICAL REMOTELY SENSED IMAGESPaula Craciun, Josiane Zerubia ∗AYIN research group, INRIA Sophia-Antipolis Méditerranée2004 Route des Lucioles, BP 93, 06902 Sophia-Antipolis Cedex (France)e-mail: Firstname.Lastname@inria.frABSTRACTMarked point process models have been successfully used inimage analysis for feature extraction purposes in high reso-lution remotely sensed images. The model is usually basedon two types of energy terms: a data term, which reflects theconfiguration’s fidelity with respect to the input image, and aprior term, which reflects some knowledge about the objectsto be extracted. In this paper, we deal with the problem ofelliptical shape extraction. We propose new energy terms forthe extraction of boats in harbors, which is a particularly dif-ficult problem and we show results on high resolution opticalimages.Index Terms— marked point processes, object extrac-tion, remote sensing, high resolution, optical sensor.1. INTRODUCTIONThe general problem of object extraction in static images ishard to solve, since the system has to distinguish a particularobject class from all the others. Thus, the system must pro-cess a model which has high inter-class and low intra-classvariability. Furthermore, it must deal with situations wherethe object is unevenly illuminated, rotated in the plane, par-tially occluded or blended with the background.In the case of high-resolution images, object-based detec-tion is preferred. In this sense, we devise the use of a MarkedPoint Process (MPP) model [1, 2] for object extraction. Ba-sically, it is a stochastic process which allows for modelingthe scene as a set of unknown number of geometrical shapes.The actual object set can be viewed as a realization of theMPP. The consistency of an object set is given by two typesof energy terms. The first term is called the data term, whichcontrols how well the proposed objects fit to the actual im-age. The second term is called a prior term and reflects priorknowledge about the objects, thus allowing the incorporationof geometrical and spatial constraints into the model. Firstattempts to object extraction using MPPs of circles, proved to∗We thank the French Space Agency (CNES) for providing the data,Saima Ben Hadj from INRIA/I3S for fruitful discussions and Antoine Man-gin from ACRI-ST for useful feedback regarding the model.be very encouraging [3, 4]. The model was then extended tofit other shapes like ellipses and rectangles [5]. Herein, wepropose a modification of the model, in the particular case ofboat detection in harbors, which is a difficult task since theobjects have a particular distribution. The paper is dividedinto two main parts. The first one presents a theoretical studyof the proposed unsupervised method by describing the gen-eral model for object extraction using MPPs. The second partdetails the new model used for boat extraction in harbors. Wedepict and discuss some detection results and propose someperspectives.2. PROPOSED MODEL FOR OBJECT EXTRACTIONWe consider a MPP of ellipses. When seen from above, i.e.satellite images at nadir, the boats tend to have an ellipticalshape. Therefore, ellipses best suit the shape of boats as op-posed to circles or other geometrical shapes. The object spaceW , is a bounded set in R5:W = [0, XM ]× [0, YM ]× [am, aM ]× [bm, bM ]× [0, π]where XM and YM are the width and height of the input im-age, respectively, am and aM are the minimum and the max-imum of the semi-major axis of the ellipse, bm and bM arethe minimum and the maximum of the semi-minor axis of theellipse and ω ∈ [0, π] is the orientation of the ellipse. Theparameterization of the ellipse is shown in Figure 1 .We are interested in a particular family of MPPs, namelythe Gibbs processes [6]. Denoting the observed image withy, the density of the considered MPP is given by:fθ(X = x|y) =1c(θ|y)exp−Uθ(x,y)where c(θ|y) is a normalizing function given by:c(θ|y) =∫Ωexp−Uθ(x,y)µ(dx).Here, θ is a parameter vector which allows the model to beflexible and fit different images. It has to be adjusted accord-ing to the given image. µ(·) is the intensity measure of thereference Poisson process, Uθ(x,y) is called the energy, Ω isthe configuration space and x denotes the object configura-tion.The energy function is divided into two parts: the dataterm Udθ (x,y) which represents the configuration’s fidelitywith respect to the input image and the prior term Upθ (x) thatreflects some knowledge about the objects to be extracted.Using the MAP criterion, the most likely configuration whichallows the extraction of objects corresponds to the global min-imum of the total energy Uθ(x,y):x ∈ Argmaxx∈Ωfθ(X = x/y) = Argminx∈Ω[Uθ(x/y)].Parameter estimation techniques are not presented here, formore details, see [3, 5]. Once the parameter vector θ is deter-mined, a solution to this optimization problem can be foundby means of simulated annealing combined with a samplingalgorithm, e.g. Reversible Jump MCMC [7].Fig. 1. left: Parameterization of an ellipse; right: Representa-tion of the quality function Q(x)2.1. Data energy termTwo main approaches have been previously devised for com-puting the data energy term [3]. The first one, carried out ina Bayesian framework, does not take sufficiently the spatialfeatures of the objects into account. Therefore, a second ap-proach was proposed, the idea behind it being to model thedata energy term locally, for each object in the configuration.Thus, the total data energy term of the configuration x is:Udθd(x,y) = γd∑u∈xUd(u),where γd represents the weight which has to be estimated andθd is the parameter vector.The definition of the local data energy term relies on thecomputation of a distance measure between the distribution ofthe set of pixels belonging to the object u and the set of pixelsbelonging to its border Fρ(u). The local data energy is builtas a qualification of this contrast measure, thus favoring wellpositioned objects (i.e. high contrast between the interior andthe border of the object) while penalizing misplaced ones:Ud(u) = Q(d(u,Fρ(u))d0)where d(u,Fρ(u)) is the Bhattacharya distance [8] betweenthe object u and its boundary Fρ(u) of width ρ and d0 is athreshold for that distance. Q : R+  [−1, 1] is a qualityfunction and attributes a negative value to well placed objects(e.g. those objects u for which d(u,Fρ(u)) is higher thanthe threshold d0) and a positive value to misplaced objects.Q(x) is defined using a cubic root which allows a moderatepenalization when the distance is close to the threshold, asshown in Figure 1:Q(x) ={1− x1/3 if x < 1exp(−x−13 )− 1 if x ≥ 12.2. Prior energy termThis energy term corresponds to a penalization of over-lapping objects, avoiding the detection of the same ob-ject several times. The proposed model uses a hard coreprocess to handle object overlapping. Thus, denoting byA(xi, xj) =Area(xi∩xj)min(Area(xi),Area(xj))the area of intersectionbetween the objects xi and xj , the prior energy can be definedas: Up(x) =∑1<i6=j<n(x) ts(xi, xj), where:ts ={0 if A(xi, xj) < s+∞ otherwisewhere s ∈ [0, 1] corresponds to the amount of overlappingallowed by the model and n(x) is the number of objects in theconfiguration x. Accordingly, all configurations containing atleast two objects that overlap to a higher ratio than specifiedby s are prohibited.3. EXTRACTION MODEL FOR BOATS IN HARBORSAs stated before, boats in harbors tend to have a particular dis-tribution. Previous work has been done in devising a modelfor this difficult case [5]. The energy term has been modified,by considering only a section of the exterior border for com-puting the distance measure as shown in Figure 2, since boatsin harbors tend to be close to each other.Fig. 2. left: Exterior border considered for the computationof the data energy term; right: Alignment constraints on theprior energy termFurthermore, the prior energy term has been modified toincorporate additional constraints. This model implied thatthe objects are very close to each other, they are not shiftedwrt each other and have the same, global orientation. Theseconstraints are graphically represented in Figure 2 (right).One can observe that this model is far too restrictive and isnot applicable for the general case. Therefore, we have triedto relax these constraints in order to better fit the general case.3.1. Modification of the data energy termWe will keep the modification of the data energy term pro-posed in [5] and introduce an additional key element. Con-sidering the entire interior of the objects when computing thecontrast measure might not yield the best results, since thisarea is not always homogeneous. Therefore, we propose tointroduce an additional term, by taking into account the con-trast measure between the interior border of the objects andthe exterior one, as shown in Figure 3 .Fig. 3. Exterior and interior border considered for the com-putation of the new data energy term.The data term, thus becomes:Udθd(u) = Q(d(u,Fρ(u))d0) + γcQ(d(Iρ(u),Fρ(u))d0)where γc is the weight of the contrast measure between theinterior border, Iρ(u), and the exterior border, Fρ(u), of theobject u. Border width, ρ, and contrast threshold, d0, are keptthe same for both parts of Udθd(u) in order to keep the numberof parameters as low as possible.3.2. Modification of the prior energy termConsidering only the non-overlapping constraint does notyield satisfactory results. One can see that indeed objectstend to be locally aligned, where alignment should be inter-preted as neighboring objects sharing the same, or similar,orientation. For this purpose, we define an alignment inter-action between two neighboring ellipses u1 and u2 in thefollowing way:u1 ∼al u2 ⇔ dω(u1, u2) ≤ dωmaxwhere, dω = |ω1 − ω2| is the difference of the orientations ofthe two ellipses u1 and u2 and dωmax is the maximum angleallowed between two neighboring objects. Then, we appenda prior energy that promotes this alignment in the followingway:Ual(u1, u2) = δ$(dω(u1, u2), dωmax)if u1 ∼al u20 otherwisewhere$(x, xmax) = − 1x2max [1+x2max1+x2 −1], for x ≤ xmax isa reward function that favors aligned frames. The prior energyof a configuration x corresponding to the alignment constraintis the sum over all the object pairs of the configuration x:Upal(x) = γal∑1<i<j<n(x)Ual(ui, uj).The weight γal is also a parameter that has to be estimated.A method proposed in [9] for road extraction, is used inorder to locally determine the orientation of the keys using thewater, in a preprocessing step, thus obtaining a set of relevantpixels, i.e. pixels that incorporate data about the orientation.Finally, we search for the local orientation of the objects,i.e. at each creation of an object, we search for a relevantpixel in its neighborhood. If no relevant pixel is found, noassumptions are made on the orientation of the object. If arelevant pixel is found, its position wrt the semi-axes of theellipse is determined. We will favor ellipses perpendicular tothe orientation indicated by the relevant pixel. Thus, the finalprior energy term that refers to alignment becomes:Upalωl(u) =γal∑v∈x Ual(u, v)if relevant pixel is found and||ωu − ωl|| ≤ dωmax , orif no relevant pixel is found0 if a relevant pixel is found and||ωu − ωl|| > dωmaxwhere ωu is the orientation of object u and ωl is the orienta-tion perpendicular to the one retained in the relevant pixel.4. RESULTS AND DISCUSSIONSFigure 4 shows the final results. Figure 4 (a) and (d) representtwo test images of boats in harbors. Figure 4 (b) shows theresult by applying the model presented in [5]. The detectionresults are very good, counting a total number of 523 ellipses,compared to a number of 518 boats counted by an expert. Thecomputations lasted 1h and 38min on a 1.86GHz processoronly for parameter estimation. Figure 4 (c) shows the resultby applying the model we propose in this paper. The detec-tion results are a little bit less accurate than in [5] resultingin a total number of 501 ellipses, but the computation timedecreased to 45min and 3sec for parameter estimation and55min and 21sec for the detection, on a 2.20GHz processor.The strength of the proposed model is visible in Figure4 (f). Where the object extraction of the model presented in[5] fails utterly due to the different orientations of the boats(a) (b) (c)(d) (e) (f)Fig. 4. (a), (d) Image of boats in a harbor c©CNES; (b),(e) Extraction results using the model proposed in [5]; (c),(f) Extractionresults using the proposed model.Boat ImageDetection errorfor old model[5]Detection errorfor new modelFigure 4 (a) ∼ 1% ∼ 10%Figure 4 (d) > 45% < 30%Table 1. Comparison between detection errorsin the image (see Figure 4 (e)), our model proves to be moreefficient, see Table 1. We highlight some problematic areas ingreen, but the overall performance is very good.5. CONCLUSIONS AND FUTURE WORKIn this paper, we have presented a new method for automaticobject extraction based on a marked point process. We ap-plied this method to extracting boats in harbors. We haveshown why previous attempts to solve this problem, resultedinto a very restrictive model. We have, therefore, relaxed theconstraints for such models. This resulted in the modificationof both, the data energy term and the prior term. Future workwill include an automatic detection of the keys in harbors, inorder to better extract the local orientation of the boats.6. REFERENCES[1] C. J. Geyer and J. Møller, “Simulation procedures andlikelihood inference for spatial point processes,” Scandi-navian Journal of Statistics, vol. 21, pp. 359–373, 1994.[2] C. J. Geyer, “Likelihood inference for spatial point pro-cesses,” in Stochastic geometry, likelihood and computa-tion. CRC Press, 1999.[3] F. Chatelain, X. Descombes, and J. Zerubia, “Parameterestimation for a marked point process. application to ob-ject extraction from remote sensing images,” Proc. EMM-CVPR, vol. 5681, pp. 221–234, 2009.[4] G. Perrin, X. Descombes, and J. Zerubia, “A markedpoint process model for tree crown extraction in planta-tions,” Proc. ICIP, vol. 1, pp. 661–664, 2005.[5] S. Ben Hadj, F. Chatelain, X. Descombes, and J. Zerubia,“Parameter estimation for a marked point process withina framework of multidimensional shape extraction fromremote sensing images,” Proc. of ISPRS Conference onPhotogrammetry Computer Vision and Image Analysis,vol. XXXVIII, 2010.[6] D. Stoyan, W. S. Kendall, and J. Mecke, Stochastic Ge-ometry and its Applications, John Wiley and Sons, 1987.[7] P. Green, “Reversible jump Markov Chain MonteCarlo computation and Bayesian model determination,”Biometrika, vol. 82, no. 4, pp. 711–732, 1995.[8] F. Goudail, P. Réfrégier, and G. Delyon, “Bhattacharyyadistance as a contrast parameter for statistical processingof noisy optical images,” Journal of Optical Science ofAmerica A, vol. 21, no. 7, pp. 1231–1240, 2004.[9] Y. Li and R. Briggs, “Automatic extraction of roads fromhigh resolution aerial and satellite images with heavynoise,” Proceedings of the 6th International Conferenceon Geographic Information Systems, 2009.

Unsupervised marked point process model for boat extraction in harbors from high resolution optical remotely sensed images

https://hal.inria.fr/hal-00867585/file/ICIP_13.pdf

Unsupervised marked point process model for boat extraction in harbors from high resolution optical remotely sensed images

Abstract

Similar works

Full text

Available Versions

INRIA a CCSD electronic archive server