Graph Deep Learning: State of the Art and Challenges

The last half-decade has seen a surge in deep learning research on irregular domains and efforts to extend convolutional neural networks (CNNs) to work on irregularly structured data. The graph has emerged as a particularly useful geometrical object in deep learning, able to represent a variety of irregular domains well. Graphs can represent various complex systems, from molecular structure, to computer and social and traffic networks. Consequent on the extension of CNNs to graphs, a great amount of research has been published that improves the inferential power and computational efficiency of graph- based convolutional neural networks (GCNNs).The research is incipient, however, and our understanding is relatively rudimentary. The majority of GCNNs are designed to operate with certain properties. In this survey we review of the state of graph representation learning from the perspective of deep learning. We consider challenges in graph deep learning that have been neglected in the majority of work, largely because of the numerous theoretical difficulties they present. We identify four major challenges in graph deep learning: dynamic and evolving graphs, learning with edge signals and information, graph estimation, and the generalization of graph models. For each problem we discuss the theoretical and practical issues, survey the relevant research, while highlighting the limitations of the state of the art. Advances on these challenges would permit GCNNs to be applied to wider range of domains, in situations where graph models have previously been limited owing to the obstructions to applying a model owing to the domains’ natures

Fingerabdruckswachstumvorhersage, Bildvorverarbeitung und Multi-level Judgment Aggregation

Im ersten Teil dieser Arbeit wird Fingerwachstum untersucht und eine Methode zur Vorhersage von Wachstum wird vorgestellt. Die Effektivität dieser Methode wird mittels mehrerer Tests validiert. Vorverarbeitung von Fingerabdrucksbildern wird im zweiten Teil behandelt und neue Methoden zur Schätzung des Orientierungsfelds und der Ridge-Frequenz sowie zur Bildverbesserung werden vorgestellt: Die Line Sensor Methode zur Orientierungsfeldschätzung, gebogene Regionen zur Ridge-Frequenz-Schätzung und gebogene Gabor Filter zur Bildverbesserung. Multi-level Jugdment Aggregation wird eingeführt als Design Prinzip zur Kombination mehrerer Methoden auf mehreren Verarbeitungsstufen. Schließlich wird Score Neubewertung vorgestellt, um Informationen aus der Vorverarbeitung mit in die Score Bildung einzubeziehen. Anhand eines Anwendungsbeispiels wird die Wirksamkeit dieses Ansatzes auf den verfügbaren FVC-Datenbanken gezeigt.Finger growth is studied in the first part of the thesis and a method for growth prediction is presented. The effectiveness of the method is validated in several tests. Fingerprint image preprocessing is discussed in the second part and novel methods for orientation field estimation, ridge frequency estimation and image enhancement are proposed: the line sensor method for orientation estimation provides more robustness to noise than state of the art methods. Curved regions are proposed for improving the ridge frequency estimation and curved Gabor filters for image enhancement. The notion of multi-level judgment aggregation is introduced as a design principle for combining different methods at all levels of fingerprint image processing. Lastly, score revaluation is proposed for incorporating information obtained during preprocessing into the score, and thus amending the quality of the similarity measure at the final stage. A sample application combines all proposed methods of the second part and demonstrates the validity of the approach by achieving massive verification performance improvements in comparison to state of the art software on all available databases of the fingerprint verification competitions (FVC)

Sparse variational regularization for visual motion estimation

The computation of visual motion is a key component in numerous computer vision tasks such as object detection, visual object tracking and activity recognition. Despite exten- sive research effort, efficient handling of motion discontinuities, occlusions and illumina- tion changes still remains elusive in visual motion estimation. The work presented in this thesis utilizes variational methods to handle the aforementioned problems because these methods allow the integration of various mathematical concepts into a single en- ergy minimization framework. This thesis applies the concepts from signal sparsity to the variational regularization for visual motion estimation. The regularization is designed in such a way that it handles motion discontinuities and can detect object occlusions

Imagerie de diffusion en temps-réel (correction du bruit et inférence de la connectivité cérébrale)

La plupart des constructeurs de systèmes d'imagerie par résonance magnétique (IRM) proposent un large choix d'applications de post-traitement sur les données IRM reconstruites a posteriori, mais très peu de ces applications peuvent être exécutées en temps réel pendant l'examen. Mises à part certaines solutions dédiées à l'IRM fonctionnelle permettant des expériences relativement simples ainsi que d'autres solutions pour l'IRM interventionnelle produisant des scans anatomiques pendant un acte de chirurgie, aucun outil n'a été développé pour l'IRM pondérée en diffusion (IRMd). Cependant, comme les examens d'IRMd sont extrêmement sensibles à des perturbations du système hardware ou à des perturbations provoquées par le sujet et qui induisent des données corrompues, il peut être intéressant d'investiguer la possibilité de reconstruire les données d'IRMd directement lors de l'examen. Cette thèse est dédiée à ce projet innovant. La contribution majeure de cette thèse a consisté en des solutions de débruitage des données d'IRMd en temps réel. En effet, le signal pondéré en diffusion peut être corrompu par un niveau élevé de bruit qui n'est plus gaussien, mais ricien ou chi non centré. Après avoir réalisé un état de l'art détaillé de la littérature sur le bruit en IRM, nous avons étendu l'estimateur linéaire qui minimise l'erreur quadratique moyenne (LMMSE) et nous l'avons adapté à notre cadre de temps réel réalisé avec un filtre de Kalman. Nous avons comparé les performances de cette solution à celles d'un filtrage gaussien standard, difficile à implémenter car il nécessite une modification de la chaîne de reconstruction pour y être inséré immédiatement après la démodulation du signal acquis dans l'espace de Fourier. Nous avons aussi développé un filtre de Kalman parallèle qui permet d'appréhender toute distribution de bruit et nous avons montré que ses performances étaient comparables à celles de notre méthode précédente utilisant un filtre de Kalman non parallèle. Enfin, nous avons investigué la faisabilité de réaliser une tractographie en temps-réel pour déterminer la connectivité structurelle en direct, pendant l'examen. Nous espérons que ce panel de développements méthodologiques permettra d'améliorer et d'accélérer le diagnostic en cas d'urgence pour vérifier l'état des faisceaux de fibres de la substance blanche.Most magnetic resonance imaging (MRI) system manufacturers propose a huge set of software applications to post-process the reconstructed MRI data a posteriori, but few of them can run in real-time during the ongoing scan. To our knowledge, apart from solutions dedicated to functional MRI allowing relatively simple experiments or for interventional MRI to perform anatomical scans during surgery, no tool has been developed in the field of diffusion-weighted MRI (dMRI). However, because dMRI scans are extremely sensitive to lots of hardware or subject-based perturbations inducing corrupted data, it can be interesting to investigate the possibility of processing dMRI data directly during the ongoing scan and this thesis is dedicated to this challenging topic. The major contribution of this thesis aimed at providing solutions to denoise dMRI data in real-time. Indeed, the diffusion-weighted signal may be corrupted by a significant level of noise which is not Gaussian anymore, but Rician or noncentral chi. After making a detailed review of the literature, we extended the linear minimum mean square error (LMMSE) estimator and adapted it to our real-time framework with a Kalman filter. We compared its efficiency to the standard Gaussian filtering, difficult to implement, as it requires a modification of the reconstruction pipeline to insert the filter immediately after the demodulation of the acquired signal in the Fourier space. We also developed a parallel Kalman filter to deal with any noise distribution and we showed that its efficiency was quite comparable to the non parallel Kalman filter approach. Last, we addressed the feasibility of performing tractography in real-time in order to infer the structural connectivity online. We hope that this set of methodological developments will help improving and accelerating a diagnosis in case of emergency to check the integrity of white matter fiber bundles.PARIS11-SCD-Bib. électronique (914719901) / SudocSudocFranceF

Optical tracking of nerve activity using intrinsic changes in birefringence

Changes in birefringence (or dynamic birefringence) provide an arguably cleaner method of measuring IOS as compared to scattering methods. Other imaging methods have substantial limitations. Nerves inherently exhibit a static (rest condition) birefringence that is associated with the structural anisotropies of axonal protein filaments, membrane phospholipids and proteins, as well as surrounding tissues, which include Schwann cells and axon sheaths. The dynamic birefringence, or “crossed-polarized signal” (XPS), in neurons arises from activity in axons and occurs with a rapid momentary change, typically a decrease, in the birefringence when action potentials (APs) propagate along them. We improved the signal-to-noise to make detecting this signal an easier task, and present the XPS as a viable candidate for detecting AP activity in anisotropic nervous tissue. Our data collectively serves as a strong indication that there is a capacitive-charging-like effect directly inducing a gradual recovery (long tail) of the XPS to baseline, and also causing a smoothing of the XPS trace. A setup was constructed that successfully demonstrated the feasibility of tracking propagating compound APs in a peripheral nerve using the XPS. We made significant progress in the attempt to investigate birefringence of myelination. For the first time, the XPS in a myelinated tissue was detected, and it appears to be bipolar in nature. Further work in investigating the nature of this signal is needed, and is currently underway. Since changes in birefringence in neurons are associated instantaneously with electrophysiological phenomena, they are well-suited for fast imaging of propagating action potentials in neuronal tissue. In summary, imaging based on polarization sensing of changes in birefringence offers promise for an improved noninvasive method of detecting and tracking AP activity in myelinated and unmyelinated nerves and could be designed for pre-clinical and surgical applications

Variational image fusion

The main goal of this work is the fusion of multiple images to a single composite that offers more information than the individual input images. We approach those fusion tasks within a variational framework. First, we present iterative schemes that are well-suited for such variational problems and related tasks. They lead to efficient algorithms that are simple to implement and well-parallelisable. Next, we design a general fusion technique that aims for an image with optimal local contrast. This is the key for a versatile method that performs well in many application areas such as multispectral imaging, decolourisation, and exposure fusion. To handle motion within an exposure set, we present the following two-step approach: First, we introduce the complete rank transform to design an optic flow approach that is robust against severe illumination changes. Second, we eliminate remaining misalignments by means of brightness transfer functions that relate the brightness values between frames. Additional knowledge about the exposure set enables us to propose the first fully coupled method that jointly computes an aligned high dynamic range image and dense displacement fields. Finally, we present a technique that infers depth information from differently focused images. In this context, we additionally introduce a novel second order regulariser that adapts to the image structure in an anisotropic way.Das Hauptziel dieser Arbeit ist die Fusion mehrerer Bilder zu einem Einzelbild, das mehr Informationen bietet als die einzelnen Eingangsbilder. Wir verwirklichen diese Fusionsaufgaben in einem variationellen Rahmen. Zunächst präsentieren wir iterative Schemata, die sich gut für solche variationellen Probleme und verwandte Aufgaben eignen. Danach entwerfen wir eine Fusionstechnik, die ein Bild mit optimalem lokalen Kontrast anstrebt. Dies ist der Schlüssel für eine vielseitige Methode, die gute Ergebnisse für zahlreiche Anwendungsbereiche wie Multispektralaufnahmen, Bildentfärbung oder Belichtungsreihenfusion liefert. Um Bewegungen in einer Belichtungsreihe zu handhaben, präsentieren wir folgenden Zweischrittansatz: Zuerst stellen wir die komplette Rangtransformation vor, um eine optische Flussmethode zu entwerfen, die robust gegenüber starken Beleuchtungsänderungen ist. Dann eliminieren wir verbleibende Registrierungsfehler mit der Helligkeitstransferfunktion, welche die Helligkeitswerte zwischen Bildern in Beziehung setzt. Zusätzliches Wissen über die Belichtungsreihe ermöglicht uns, die erste vollständig gekoppelte Methode vorzustellen, die gemeinsam ein registriertes Hochkontrastbild sowie dichte Bewegungsfelder berechnet. Final präsentieren wir eine Technik, die von unterschiedlich fokussierten Bildern Tiefeninformation ableitet. In diesem Kontext stellen wir zusätzlich einen neuen Regularisierer zweiter Ordnung vor, der sich der Bildstruktur anisotrop anpasst