A semidiscrete version of the Citti-Petitot-Sarti model as a plausible model for anthropomorphic image reconstruction and pattern recognition

In his beautiful book [66], Jean Petitot proposes a sub-Riemannian model for the primary visual cortex of mammals. This model is neurophysiologically justified. Further developments of this theory lead to efficient algorithms for image reconstruction, based upon the consideration of an associated hypoelliptic diffusion. The sub-Riemannian model of Petitot and Citti-Sarti (or certain of its improvements) is a left-invariant structure over the group $SE(2)$ of rototranslations of the plane. Here, we propose a semi-discrete version of this theory, leading to a left-invariant structure over the group $SE(2,N)$, restricting to a finite number of rotations. This apparently very simple group is in fact quite atypical: it is maximally almost periodic, which leads to much simpler harmonic analysis compared to $SE(2).$ Based upon this semi-discrete model, we improve on previous image-reconstruction algorithms and we develop a pattern-recognition theory that leads also to very efficient algorithms in practice.Comment: 123 pages, revised versio

ChebLieNet: Invariant Spectral Graph NNs Turned Equivariant by Riemannian Geometry on Lie Groups

We introduce ChebLieNet, a group-equivariant method on (anisotropic) manifolds. Surfing on the success of graph- and group-based neural networks, we take advantage of the recent developments in the geometric deep learning field to derive a new approach to exploit any anisotropies in data. Via discrete approximations of Lie groups, we develop a graph neural network made of anisotropic convolutional layers (Chebyshev convolutions), spatial pooling and unpooling layers, and global pooling layers. Group equivariance is achieved via equivariant and invariant operators on graphs with anisotropic left-invariant Riemannian distance-based affinities encoded on the edges. Thanks to its simple form, the Riemannian metric can model any anisotropies, both in the spatial and orientation domains. This control on anisotropies of the Riemannian metrics allows to balance equivariance (anisotropic metric) against invariance (isotropic metric) of the graph convolution layers. Hence we open the doors to a better understanding of anisotropic properties. Furthermore, we empirically prove the existence of (data-dependent) sweet spots for anisotropic parameters on CIFAR10. This crucial result is evidence of the benefice we could get by exploiting anisotropic properties in data. We also evaluate the scalability of this approach on STL10 (image data) and ClimateNet (spherical data), showing its remarkable adaptability to diverse tasks.Comment: submitted to NeurIPS'21, https://openreview.net/forum?id=WsfXFxqZXR

Locally Adaptive Frames in the Roto-Translation Group and their Applications in Medical Imaging

Locally adaptive differential frames (gauge frames) are a well-known effective tool in image analysis, used in differential invariants and PDE-flows. However, at complex structures such as crossings or junctions, these frames are not well-defined. Therefore, we generalize the notion of gauge frames on images to gauge frames on data representations $U:\mathbb{R}^{d} \rtimes S^{d-1} \to \mathbb{R}$ defined on the extended space of positions and orientations, which we relate to data on the roto-translation group $SE(d)$, $d=2,3$. This allows to define multiple frames per position, one per orientation. We compute these frames via exponential curve fits in the extended data representations in $SE(d)$. These curve fits minimize first or second order variational problems which are solved by spectral decomposition of, respectively, a structure tensor or Hessian of data on $SE(d)$. We include these gauge frames in differential invariants and crossing preserving PDE-flows acting on extended data representation $U$ and we show their advantage compared to the standard left-invariant frame on $SE(d)$. Applications include crossing-preserving filtering and improved segmentations of the vascular tree in retinal images, and new 3D extensions of coherence-enhancing diffusion via invertible orientation scores

Toward a mathematical theory of perception

A new technique for the modelling of perceptual systems called formal modelling is developed. This technique begins with qualitative observations about the perceptual system, the so-called perceptual symmetries, to obtain through mathematical analysis certain model structures which may then be calibrated by experiment. The analysis proceeds in two different ways depending upon the choice of linear or nonlinear models. For the linear case, the analysis proceeds through the methods of unitary representation theory. It begins with a unitary group representation on the image space and produces what we have called the fundamental structure theorem. For the nonlinear case, the analysis makes essential use of infinite-dimensional manifold theory. It begins with a Lie group action on an image manifold and produces the fundamental structure formula. These techniques will be used to study the brightness perception mechanism of the human visual system. Several visual groups are defined and their corresponding structures for visual system models are obtained. A new transform called the Mandala transform will be deduced from a certain visual group and its implications for image processing will be discussed. Several new phenomena of brightness perception will be presented. New facts about the Mach band illusion along with new adaptation phenomena will be presented. Also a new visual illusion will be presented. A visual model based on the above techniques will be presented. It will also be shown how use of statistical estimation theory can be made in the study of contrast adaptation. Furthermore, a mathematical interpretation of unconscious inference and a simple explanation of the Tolhurst effect without mutual channel inhibition will be given. Finally, image processing algorithms suggested by the model will be used to process a real-world image for enhancement and for "form" and texture extraction

Left-invariant Stochastic Evolution Equations on SE(2) and its Applications to Contour Enhancement and Contour Completion via Invertible Orientation Scores

We provide the explicit solutions of linear, left-invariant, (convection)-diffusion equations and the corresponding resolvent equations on the 2D-Euclidean motion group SE(2). These diffusion equations are forward Kolmogorov equations for stochastic processes for contour enhancement and completion. The solutions are group-convolutions with the corresponding Green's function, which we derive in explicit form. We mainly focus on the Kolmogorov equations for contour enhancement processes which, in contrast to the Kolmogorov equations for contour completion, do not include convection. The Green's functions of these left-invariant partial differential equations coincide with the heat-kernels on SE(2), which we explicitly derive. Then we compute completion distributions on SE(2) which are the product of a forward and a backward resolvent evolved from resp. source and sink distribution on SE(2). On the one hand, the modes of Mumford's direction process for contour completion coincide with elastica curves minimizing $\int \kappa^{2} + \epsilon ds$, related to zero-crossings of 2 left-invariant derivatives of the completion distribution. On the other hand, the completion measure for the contour enhancement concentrates on geodesics minimizing $\int \sqrt{\kappa^{2} + \epsilon} ds$. This motivates a comparison between geodesics and elastica, which are quite similar. However, we derive more practical analytic solutions for the geodesics. The theory is motivated by medical image analysis applications where enhancement of elongated structures in noisy images is required. We use left-invariant (non)-linear evolution processes for automated contour enhancement on invertible orientation scores, obtained from an image by means of a special type of unitary wavelet transform