We extend to a Engel type structure a cortically inspired model of perceptual completion initially proposed in the Lie group of positions and orientations with a sub-Riemannian metric. According to this model, a given image is lifted in the group and completed by a minimal surface. The main obstacle in extending the model to a higher dimensional group, which can code also curvatures, is the lack of a good definition of codimension 2 minimal surface. We present here this notion, and describe an application to image completion

Citti G.

Giovannardi G.

Ritore M.

Sarti A.

English

Archivio istituzionale della ricerca - Alma Mater Studiorum Università di Bologna

This item was downloaded from IRIS Università di Bologna (https://cris.unibo.it/) When citing, please refer to the published version.       This is the final peer-reviewed accepted manuscript of:  Citti, G., Giovannardi, G., Ritoré, M., Sarti, A. (2021). Submanifolds of Fixed Degree in Graded Manifolds for Perceptual Completion. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2021. Lecture Notes in Computer Science(), vol 12829. Springer, Cham.  The final published version is available online at: https://doi.org/10.1007/978-3-030-80209-7_6.  Rights / License: The terms and conditions for the reuse of this version of the manuscript are specified in the publishing policy. For all terms of use and more information see the publisher's website.    Submanifolds of fixed degree in gradedmanifolds for perceptual completion?G. Citti1, G. Giovannardi2, M. Ritoré2, and A. Sarti31 Department of Mathematics, University of Bologna2 Department of Mathematics, University of Granada3 CAMS, EHESS, ParisAbstract. We extend to a Engel type structure a cortically inspiredmodel of perceptual completion initially proposed in the Lie group ofpositions and orientations with a sub-Riemannian metric. According tothis model, a given image is lifted in the group and completed by aminimal surface. The main obstacle in extending the model to a higherdimensional group, which can code also curvatures, is the lack of a gooddefinition of codimension 2 minimal surface. We present here this notion,and describe an application to image completion.Keywords: perceptual completion· graded structures· fixed degree sur-faces· area formula· degree preservig variations .1 IntroductionMathematical models of the visual cortex expressed in terms of differential ge-ometry were proposed for the first time by Hoffman in [17], Mumford in [23],August Zucker in [2] to quote only a few. Petitot and Tondut in 1999 in [26]described the functional architecture of area V1 by a contact structure, anddescribed the propagation in the cortex by a constrained Lagrangian operator.Only in 2003 Sarti and Citti in [6] and J. Petitot in [24] recognized that thegeometry of the cortex is indeed sub-Riemannian. In [6], the functional archi-tecture of V1 is described as a Lie group with a sub-Riemannian geometry: ifthe visual stimulus is corrupted, it is completed via a sub-Riemannian minimalsurface. A large literature has been provided on sub-Riemannian models bothfor image processing or cortical modelling (we refer to the monograph [25] for alist of references). In section 2 we will present the model [6], and its extensionin the Engel group provided in [25][1].The notion of minimal surface in a sub-Riemannian setting as critical pointsof the first variation of the area functional is well known for co-dimension 1surfaces (see [12], [11] for the area formula, and [9], [7], [16], [18] for the firstvariation). For higher codimension very few results are available: the notion ofarea has been introduced in [10], [20], [19], but the first variation, well-known? Supported by GHAIA, H2020 RICE MCSA project, No 7778222 G. Citti, G. Giovannardi, M. Ritoré, A. Sartifor curves (see [21]), was studied for surfaces only very recently in [4], [5], [13].We will devote section 3 to the description of these results.We conclude this short presentation with an application of this result to thecompletion model in the Engel group, contained in section 4.2 A subriemannian model of the visual cortexThe primary visual cortex is the first part of the brain processing the visualsignal coming from the retina. The receptive profile (RP) ψ(ξ) is the functionthat models the activation of a cortical neuron when a stimulus is applied to apoint ξ = (ξ1, ξ2) of the retinal plane. The hypercolumnar structure organizes thecortical cells of V1 in columns corresponding to different features. As a resultswe will identify cells in the cortex by means of two parameters (x, f), wherex = (x1, x2) is the position of the point, and f a vector of extracted features.We will denote F the set of features, and consequently the cortical space will beidentified with R2 × F . In the presence of a visual stimulus I = I(ξ) the wholehypercolumn fires, giving rise to an outputOF (x, f) =ˆI(ξ)ψ(x,f)(ξ)dξ. (1)It is clear that the same image, filtered with a different family of cells, producesa different output.For every cortical point, the cortical activity, suitably normalized, can beconsidered a probability density. Hence its maximum over the fibre F can beconsidered the most probable value of f , and can be considered the featureidentified by the system (principle of non maxima suppression):|OF (x, fI(x))| = maxf|OF (x, f)|. (2)The output of a family of cells is propagated in the cortical space R2 × F viathe lateral connectivity.2.1 Orientation and curvature selectivityIn [6] the authors considered only simple cells sensible to a direction θ ∈ S1.Hence the set F becomes in this case S1 and the underlying manifold reducesto N := R2 × S1 with a subriemannian metric. The image I is lifted by theprocedure (2) to a graph is this structure. If it is corrupted, it is completed viaa subriemannian minimal surface.The geometric description of the cortex was extended by Citti-Petitot-Sartito a model of orientation and curvature selection, appeared in [25]. In this casethe non maxima suppression process (2) selects a functionΨ : R2 → N2 = R2 × S1 × R, Ψ(x) = (x, fI(x)) = (x, θ(x), k(x)).Submanifolds of fixed degree for perceptual completion 3Level lines of the input I are lifted to integral curves in N of the vector fieldsX1 = cos θ∂∂x1+ sin θ∂∂x2+ k∂∂θ, X2 =∂∂k(3)where x = (x1, x2). The vector fields define an Hörmander type manifold sincethe whole space is spanned at every point by the vectors X1, X2 and theircommutatorsX3 := [X1, X2] = −∂∂θ, X4 := [X1, X3] = − sin θ∂∂x1+ cos θ∂∂x2. (4)These commutation conditions identify the structure of and Engel-type algebra.Integral curves of this model allow better completion of more complex images.In the image below we represent a grouping performed in [1] with an eigenvaluemethod in the sub-Riemannian setting. Precisely for every point xi of the curves,we compute the orientation θi and the curvature ki, and obtain a point pi =(xi, θi, ki) in R2 × S1 × R. Calling d the subriemannian distance induced bythe choice of vector fields, we can compute the affinity matrix A, with entriesaij = d(pi, pj). The eigenvalues of this matrix, reprojected on R2 can be identifiedwith the perceptual units present in the image. In figure 1 (from [1] the samecurve is segmented in the geometry dependent only on orientation, and in thegeometry of orientation and curvature: the second method correctly recovers thelogarithmic spirals.Fig. 1: grouping using orientation (left), or orientation and curvature (right).It could be nice to see if it is possible to extend in this setting also theminimal surface algorithm for image completion [6]. The main obstacle in doingthis was the fact that the notion of area and curvature was not well defined forcodimension two surfaces in a sub-Riemannian metric, and that characterizingadmissible variations presents intrinsic difficulties.3 Graded structuresIn the next section we will present the results obtained in [4], [5], [13] to definethe notion of area of high codimension surfaces and its first variation in thesetting of a graded structure. A graded structure is defined as following4 G. Citti, G. Giovannardi, M. Ritoré, A. SartiDefinition 31. Let N be a smooth manifold and let H1 ⊂ . . . ⊂ Hs be anincreasing filtration of sub-bundles of the tangent bundle TN s.t.X ∈ Hi, Y ∈ Hj ⇒ [X,Y ] ∈ Hi+j and Hsp = TpN for all pWe will say that the fibration H1 ⊂ . . . ⊂ Hs is equiregular if dim(Hj) isconstant in N . For such a manifold, we can define an homogeneous dimensionQ =∑si=1 i(dim(Hi) − dim(Hi−1)). We will say that a basis (X1, . . . , Xn) ofthe tangent plane at every point is an adapted basis if X1, . . . , Xn1 generate H1,X1, . . . , Xn1 , Xn1+1, . . . , Xn2 generate H2, and so on. The presence of a filtrationnaturally allows to define the degree of a vector field and of an m-vector. Inparticular, we say that a vector v ∈ Tp(M) has degree l, and we denote itdeg(v) = l, if v ∈ HlrHl−1. Given m < n, a multi-index J = (j1, · · · , jm), with1 ≤ j1 < . . . < jm ≤ n, and an m-vector field XJ = Xj1 ∧ . . . ∧Xjm we definedeg(XJ) = deg(Xj1) + . . .+ deg(Xjm).If a Riemannian metric g is defined on the graded manifold N we can in-troduce an orthogonal decomposition of the tangent space, which respects thegrading, as follows: K1p := H1p, Ki+1p := (Hip)⊥ ∩ Hi+1p , 1 ≤ i ≤ (s − 1) Sothat TpN = K1p ⊕K2p ⊕ · · · ⊕ Ksp.Example 1. A Carnot manifold with a bracket generating distribution is agraded manifold.The interest of graded manifolds, is that a submanifold of a sub-Riemannianmanifold, is not in general a sub-Riemannian manifold, but submanifolds ofgraded manifolds are graded.3.1 Regular submanifoldsGiven an immersion Φ : M̄ → N, M = Φ(M̄). the manifold M inherits thegraded structure H̃ip = TpM ∩Hip ⊂ · · · ⊂ . The pointwise degree (introduced in[14]) or local homogenous dimension is defined bydegM (p) =s∑i=1idim(H̃i(p)− H̃i−1(p))For submanifolds, it is not possible in general to assume that the local homoge-neous dimension is constant so that we will define the degree of Md := deg(M) = maxp∈MdegM (p).Given a graded manifold N with a Riemannian metric g, we are able tointroduce a notion of area, as a limit of the corresponding riemannian areas. Tobegin with we define a family of Riemannian metrics gr, adapted to the gradingof the manifold as follows:gr|Ki =1ri−1g|Ki , i = 1, . . . , s, for any r > 0Submanifolds of fixed degree for perceptual completion 5Now we assume that M is a submanifold of degree d = deg(M), defined by aparametrization Φ : (M̄, µ)→ N . Also assume that µ is a Riemannian metric onM̄ . For each M ′ ⊂ M̄ we consider the Riemannian area, weighted by its degreerd−m2ˆM ′|E1 ∧ . . . ∧ Em|grdµ(p),where E1, . . . , Em a µ-orthonormal basis of TpM . If the limit as r → 0 exists,we call it the d-area measure. It can be explicitly expressed asAd(M′) =ˆM ′| (E1 ∧ . . . ∧ Em)d |g dµ(p).where (·)d the projection onto the space of m-vectors of degree d:(E1 ∧ . . . ∧ Em)d =∑XJ ,deg(XJ )=d〈E1 ∧ . . . ∧ Em, XJ〉XJThe same formula had been already established by [19].3.2 Admissible variationsIt would be natural to define the curvature as the first variation of the area.However in this case the area functional depends on the degree of the manifold.Hence we need to ensure that the degree does not change during variation, andwe define degree preserving variationsDefinition 32. A smooth map Γ : M̄ × (−ε, ε)→ N is said to be an admissiblevariation of Φ if Γs : M̄ → N , defined by Γs(p̄) := Γ (p̄, s), satisfies(i) Γ0 = Φ,(ii) Γs(M̄) is an immersion of the same degree as Φ(M̄) for small enough s,(iii) Γs(p̄) = Φ(p̄) for p̄ outside a given compact subset of M̄ .We can always choose an adapted frame to a submanifold manifold M .First we choose a tangent basis (E1, · · · , Em), then we complete it to a basis ofthe space Xm+1, · · · , Xn, where Xm+1, · · · , Xm+k have degree less or equal todeg(E1) while Xm+k+1, · · · , Xn have degree bigger than deg(E1).Definition 33. With the previous notation we define the variational vector fieldW asW (p̄) =∂Γ (p̄, 0)∂sIt is always possible to assume that W has no tangential components, so thatit will be represented asW (p̄) =m+k∑i=m+1hiXi +n∑r=m+k+1vrXr = H + V,where H =∑m+ki=m+1 hiXi and V =∑nr=m+k+1 vrXr.Using the fact that if Γ is an admissible variation, which means that thedegree of Γs(M̄) is constant with respect to s, it is possible to prove the following6 G. Citti, G. Giovannardi, M. Ritoré, A. SartiProposition 1. (see [5]) If W is an admissible vector field, then there existmatrices A,B such thatE(V ) +BV +AH = 0, where E is the tangent basis. (5)This property suggests the following definitionDefinition 34. We say that a compactly supported vector field W is admissiblewhen it satisfies the admissibility system E(V ) +BV +AH = 0.3.3 Variation for submanifoldsThe phenomenon of minima which are isolated and do not satisfy any geodesicequation n (abnormal geodesic) was first discovered by Montgomery [22] forgeodesic curves. In 1992 Hsu [15] proved a characterization of integrable vectorfields along curves. We obtained in [4] a partial analogous of the previous resultfor manifolds of dimension bigger than one. Precisely we definedDefinition 35. Φ : M̄ → N is strongly regular at p̄ ∈ M̄ if A(p̄) has full rank,where A is defined in Proposition 1.Theorem 1. [4] Φ : M̄ → (N,H1 ⊂ . . . ⊂ Hs) with a Riemannian metric g.Assume that Φ of degree d is strongly regular at p̄. Then there exists an openneighborhood Up̄ of p̄ such every admissible vector field W with compact supporton Up̄ is integrable.The following properties are satisfiedRemark 1. The admissibility of a vector field is independent of the Riemannianmetric g.Remark 2. All hypersurfaces in a sub-Riemannian manifold are deformable.4 Application to visual perceptionLet us go back to the model of orientation and curvature introduced in Section2.1. The underlying manifold is then N = R2 × S1 ×R. Let us call g the metricwhich makes the vector fields X1, X2, X3, X4 in (3) and (4) an orthonormal basis.Let us consider a submanifold M defined by the parametrizationΦ : R2 ⊃ M̄ → R2 × S1 × R, M = Φ(M̄)In particular the surfaces obtained by non maxima suppression are expressed inthe formΦ(x) = (x, θ(x), κ(x))If we impose the constraint κ = X1(θ) the tangent vectors to M becomeX1 +X1(κ)X2, X4 −X4(θ)X3 +X4(κ)X2, and the area functional reduces toA4(M) =ˆM̄√1 +X1(κ)2 dx. (6)Submanifolds of fixed degree for perceptual completion 7We will denote X̄1 and X̄4 the projection of the vector fields X1, X4 ontoM̄ . Then the admissibility system for a variational vector field W = h2(X2 −X1(κ)X1) + v3X3 is given byX̄1(v3) = −X̄4(θ)v3 −Ah2, where A = (1 + (X̄21 (θ))2)Since rank(A) = 1 we deduce by Theorem 1 that for this type of surfaces eachadmissible vector is integrable, which implies that the minimal surfaces can beobtained via variational methods.This property is quite important, since in [5] the authors proved that themanifold {(0, 0, θ, k)} do not admit degree preserving variation, so that it isisolated. This provides a generalization of the notion of abnormal geodesic.4.1 Implementation and resultsWe directly implement the Euler Lagrangian of the functional (6). Since it isnon linear, we compute a step 0 image with Euclidean Laplacian. After that,we compute at each step, orientation and curvature of level lines of the imageat the previous step, update curvature and orientation via the linearized EulerLagrangian equation and complete the 2D image diffusing along the vector fieldX̄1. We provide here a result in the simplified case with non corrupted pointsin the occluded region, and a preliminary result for the impainting problem.We plan to study the convergence of the algorithm and compare with existingliterature in a forthcoming paper.Fig. 2: Two examples of completion using the proposed algorithm..References1. S. Abbasi-Sureshjani, M. Favali, G. Citti, A. Sarti and B. M. ter Haar Romeny,”Curvature Integration in a 5D Kernel for Extracting Vessel Connections in Reti-nal Images,” in IEEE Trans. Image Processing, 27 (2), 606-621, 2017.2. J. August and S. W. Zucker, The curve indicator random field: Curve organizationvia edge correlation, In Perceptual organization for artificial vision systems, 265–288. Springer, 2000.3. R.M. Bianchini and G. Stefani Graded approximations and controllability alonga trajectory, SIAM J. Control Optim., 28(4), 903–924, 1990.8 G. Citti, G. Giovannardi, M. Ritoré, A. Sarti4. G. Citti, G. Giovannardi, M. Ritoré, Variational formulas for curves of fixed de-gree, preprint (2019), https://arxiv.org/abs/1902.04015.5. G. Citti, G. Giovannardi M. Ritoré, Variational formulas for submanifolds of fixeddegree, preprint (2019), https://arxiv.org/abs/1905.05131.6. G Citti, A Sarti A cortical based model of perceptual completion in the roto-translation space J. of Math. Imag Vis. 24 (3), 307-326, 2006 (announced in Proc.Workshop on Second Order Subelliptic Eq. and Appl., Cortona, 2003/6/15).7. J.-H. Cheng, J.-F. Hwang, Variations of generalized area functionals and p-areaminimizers of bounded variation in the Heisenberg group. Bull. Inst. Math. Acad.Sin. (N.S.), 5(4), 369-412, 2010.8. G. Citti, A. Sarti, editors. Neuromathematics of vision. Lecture Notes in Mor-phogenesis. Springer, Heidelberg, 20149. D. Danielli, N. Garofalo, D. M. Nhieu. Sub-Riemannian calculus on hypersurfacesin Carnot groups. Adv. Math., 215 (1), 292-378, 200010. B. Franchi, R. Serapioni, F. Serra Cassano. Regular submanifolds, graphs andarea formula in Heisenberg groups. Adv. Math., 211(1), 152-203, 200711. B. Franchi, R. Serapioni, F. Serra Cassano, Regular hypersurfaces, intrinsicperimeter and implicit function theorem in Carnot groups, Comm. Anal. Geom.,11(5), 909-944, 2003.12. N. Garofalo, D.-M. Nhieu, Isoperimetric and Sobolev inequalities for Carnot-Carathéodory spaces and the existence of minimal surfaces, Comm. Pure Appl.Math., 49(10), 1081-1144, 1996.13. G. Giovannardi, Higher dimensional holonomy map for ruled submanifolds ingraded manifolds, Anal Geom in Metric Spaces, 8, 68-91, 2020.14. M. Gromov, Carnot-Carathéodory spaces seen from within, In Sub-Riemanniangeometry, 144 of Progr. Math., 79-323, Birkhäuser, Basel, 1996.15. L. Hsu, Calculus of variations via the Griffiths formalism, J. Diff. Geom., 36(3),551-589, 1992.16. R. K. Hladky, S. D. Pauls. Constant mean curvature surfaces in sub-Riemanniangeometry. J. Diff. Geom., 79(1), 111-139, 2008.17. W. Hoffman, Higher visual perception as prolongation of the basic lie transfor-mation group. Mathematical Biosciences, 6, 437 -471, 1970.18. A. Hurtado, M. Ritoré, C. Rosales. Classification of complete stable area-stationary surfaces in the Heisenberg group H1 Adv. Math., 224(2), 561-600 201019. V. Magnani D. Vittone. An intrinsic measure for submanifolds in stratified groups.Journal für die reine und angewandte Mathematik (Crelles J.), 619, 203-232, 2008.20. V. Magnani, J. T. Tyson, D. Vittone. On transversal submanifolds and theirmeasure. J. Anal. Math., 125, 319-351, 2015.21. R. Montgomery. A tour of subriemannian geometries, their geodesics and appli-cations, 91, Mathematical Surveys and Monographs. AMS, 2002.22. R. Montgomery, Abnormal minimizers, SIAM J. Control Optim., 32(6), 1605-1620, 1994.23. D. Mumford, Elastica and computer vision. Algebraic geometry and its applica-tions, 491–506, Springer, 1994.24. J. Petitot, The neurogeometry of pinwheels as a sub-Riemannian contact struc-ture. J Physiol Paris. 97 (2-3), 265-309 2003.25. J. Petitot, Neurogeometrie de la Vision Modeles Mathematiques & Physiques desArchitectures Fonctionnelles 200926. J.Petitot, Y.Tondut, Vers une neurogéométrie. Fibrations corticales, structuresde contact et contours subjectifs modaux Mathématiques et Sciences humaines,Tome 145, 5-101, 1999.

Submanifolds of Fixed Degree in Graded Manifolds for Perceptual Completion

https://cris.unibo.it/bitstream/11585/880683/2/CGRS2.pdf

Submanifolds of Fixed Degree in Graded Manifolds for Perceptual Completion

Abstract

Similar works

Full text

Available Versions

Archivio istituzionale della ricerca - Alma Mater Studiorum Università di Bologna