Moving Frame Net: SE(3)-Equivariant Network for Volumes

Equivariance of neural networks to transformations helps to improve their performance and reduce generalization error in computer vision tasks, as they apply to datasets presenting symmetries (e.g. scalings, rotations, translations). The method of moving frames is classical for deriving operators invariant to the action of a Lie group in a manifold.Recently, a rotation and translation equivariant neural network for image data was proposed based on the moving frames approach. In this paper we significantly improve that approach by reducing the computation of moving frames to only one, at the input stage, instead of repeated computations at each layer. The equivariance of the resulting architecture is proved theoretically and we build a rotation and translation equivariant neural network to process volumes, i.e. signals on the 3D space. Our trained model overperforms the benchmarks in the medical volume classification of most of the tested datasets from MedMNIST3D

Differential invariants for SE(2)-equivariant networks

Symmetry is present in many tasks in computer vision, where the same class of objects can appear transformed, e.g. rotated due to different camera orientations, or scaled due to perspective. The knowledge of such symmetries in data coupled with equivariance of neural networks can improve their generalization to new samples. Differential invariants are equivariant operators computed from the partial derivatives of a function. In this paper we use differential invariants to define equivariant operators that form the layers of an equivariant neural network. Specifically, we derive invariants of the Special Euclidean Group SE(2), composed of rotations and translations, and apply them to construct a SE(2)-equivariant network, called SE(2) Differential Invariants Network (SE2DINNet). The network is subsequently tested in classification tasks which require a degree of equivariance or invariance to rotations. The results compare positively with the state-of-the-art, even though the proposed SE2DINNet has far less parameters than the compared models

Psychophysics, Gestalts and Games

Trabajo aceptado en Citti G., Sarti A. (eds) Neuromathematics of Vision. Lecture Notes in Morphogenesis. Springer, 2014.Many psychophysical studies are dedicated to the evaluation of the human gestalt detection on dot or Gabor patterns, and to model its dependence on the pattern and background parameters. Nevertheless, even for these constrained percepts, psychophysics have not yet reached the challenging prediction stage, where human detection would be quantitatively predicted by a (generic) model. On the other hand, Computer Vision has attempted at defining automatic detection thresholds. This chapter sketches a procedure to confront these two methodologies inspired in gestaltism. Using a computational quantitative version of the non-accidentalness principle, we raise the possibility that the psychophysical and the (older) gestaltist setups, both applicable on dot or Gabor patterns, find a useful complement in a Turing test. In our perceptual Turing test, human performance is compared by the scientist to the detection result given by a computer. This confrontation permits to revive the abandoned method of gestaltic games. We sketch the elaboration of such a game, where the subjects of the experiment are confronted to an alignment detection algorithm, and are invited to draw examples that will fool it. We show that in that way a more precise definition of the alignment gestalt and of its computational formulation seems to emerge. Detection algorithms might also be relevant to more classic psychophysical setups, where they can again play the role of a Turing test. To a visual experiment where subjects were invited to detect alignments in Gabor patterns, we associated a single function measuring the alignment detectability in the form of a number of false alarms (NFA). The first results indicate that the values of the NFA, as a function of all simulation parameters, are highly correlated to the human detection. This fact, that we intend to support by further experiments, might end up confirming that human alignment detection is the result of a single mechanism

Morphologie Mathématique et traitement d'images

International audienceDans ce chapitre nous présentons la Morphologie Mathématique comme une approche non linéaire du traitement d'images, basée sur des critères de forme et de taille. Nous essaierons de montrer son attrait en mettant en avant l'élégance de sa théorie ainsi que la puissance des outils qu'elle permet de construire : fonction distance, filtres, algorithme du Watershed pour la segmentation et autres représentations hiérarchiques, pour les principaux

Saillance et non-accidentalité : la vision humaine comparée à des algorithmes a-contrario

The present dissertation compares the human visual perception to computer vision algorithms based on a mathematical model called a-contrario theory. To this aim, it focuses on two visual tasks that are at the same time easy to model and convenient to test in psychophysical experiments. Both tasks consist in the perceptual grouping of oriented elements, namely Gabor patches. The first one is the detection of alignments and the second one extends to curves, that is to say to more general arrangements of elements in good continuation. In both cases, alignments and curves, psychophysical experiments were set up to collect data on the human visual perception in a masking context.The non-accidentalness principle states that spatial relations are perceptually relevant when their accidental occurrence is unlikely. The a-contrario theory is a formalization of this principle, and is used in computer vision to set detection thresholds accordingly. In this thesis, the a-contrario framework is applied in two practical algorithms designed to detect non-accidental alignments and curves respectively. These algorithms play the part of artificial subjects for our experiments.The experimental data of human subjects is then compared to the detection algorithms on the very same tasks, yielding two main results. First, this procedure shows that the Number of False Alarms (NFA), which is the scalar measure of non-accidentalness in the a-contrario theory, strongly correlates with the detection rates achieved by human subjects on a large variety of stimuli. Secondly,the algorithms' responses match very well the average behavior of human observers.The contribution of this thesis is therefore two-sided. On the one hand, it provides a rigorous validation of the a-contrario theory's relevance to estimate visual thresholds and implement visual tasks in computer vision. On the other hand, it reinforces the importance of the non-accidentalness principle in human vision.Aiming at reproducible research, all the methods are submitted to IPOL journal, including detailed descriptions of the algorithms, commented reference source codes, and online demonstrations for each one.Dans cette thèse, nous comparons la vision humaine à des algorithmes de vision par ordinateur, basés sur un modèle mathématique appelé théorie a-contrario. Pour cela, nous nous concentrons sur deux taches visuelles dont la modélisation d'une part, et l'expérimentation psychophysique d'autre part, sont simples. Celles-ci consistent dans le groupement perceptuel d'éléments orientés appelés patchs de Gabor. Dans la première tache il s'agit de détecter des alignements, et dans la seconde des courbes, soit des configurations plus générales d'éléments en bonne continuation. Dans les deux cas, des expériences psychophysiques ont été menées afin de collecter des données sur la perception visuelle humaine dans un contexte de masquage.Le principe de non-accidentalité désigne le fait que les relations spatiales entre des éléments prennent un sens pour la perception lorsqu'il semble invraisemblable qu'elles soient le fruit du hasard. Ce principe trouve une formalisation dans la théorie a-contrario, qui est utilisée en vision par ordinateur pour déterminer des seuils de détection en accord avec la non-accidentalité. Dans cette thèse, les méthodes a-contrario sont appliquées dans l'implémentation de deux algorithmes conçus pour détecter respectivement des alignements et des courbes non-accidentels. Ces algorithmes ont joué le role de sujets artificiels dans nos expériences.Les données expérimentales des sujets humains ont donc été comparées aux algorithmes sur les memes taches, conduisant à deux principaux résultats. Premièrement, le Nombre de Fausses Alarmes (NFA), qui est la mesure scalaire de non-accidentalité dans la théorie a-contrario, est en forte corrélation avec les taux de détection obtenus par lessujets humains sur un large éventail de stimuli. Deuxièmement, les réponses des algorithmes ressemblent précisément à celles de la moyenne des sujets humains.La contribution de cette thèse est donc double. D'une part, elle valide de façon rigoureuse la pertinence des méthodes a-contrario dans l'estimation de seuils perceptuels, et leur application en vision par ordinateur. D'autre part, elle souligne l'importance du principe de non-accidentalité dans la vision humaine.Dans le but de rendre ces recherches reproductibles, les méthodes décrites dans la thèse ont été publiées dans le journal IPOL. Ces publications fournissent le détail des algorithmes, leur code source commenté, ainsi qu'une démonstration en ligne pour chacun d'eux

Mathematical morphology meets Deep Learning

International audienc

Generation and Detection of Alignments in Gabor Patterns

This paper presents a method to be used in psychophysical experiments to compare directly visual perception to an a contrario algorithm, on a straight patterns detection task. The method is composed of two parts. The first part consists in building a stimulus, namely an array of oriented elements, in which an alignment is present with variable salience. The second part focuses on a detection algorithm, based on the a contrario theory, which is designed to predict which alignment will be considered as the most salient in a given stimulus

Benchmark Smil versus Scikit-Image (Mathematical Morphology features)

Smil is a software library specialized in Mathematical Morphology, created by the Centre de Morphologie Mathématique de l'Ecole des Mines de Paris, while Scikit-Image is a general purpose image library. This document presents a comparative benchmark of these libraries - on processing speed and memory space - on the common features, showing that Smil is much faster than scikit image and less memory consuming. So, as long as Smil has some more advanced features in these domain, it's a good complement to scikit-image for this area