Deformable Shape Completion with Graph Convolutional Autoencoders

The availability of affordable and portable depth sensors has made scanning objects and people simpler than ever. However, dealing with occlusions and missing parts is still a significant challenge. The problem of reconstructing a (possibly non-rigidly moving) 3D object from a single or multiple partial scans has received increasing attention in recent years. In this work, we propose a novel learning-based method for the completion of partial shapes. Unlike the majority of existing approaches, our method focuses on objects that can undergo non-rigid deformations. The core of our method is a variational autoencoder with graph convolutional operations that learns a latent space for complete realistic shapes. At inference, we optimize to find the representation in this latent space that best fits the generated shape to the known partial input. The completed shape exhibits a realistic appearance on the unknown part. We show promising results towards the completion of synthetic and real scans of human body and face meshes exhibiting different styles of articulation and partiality.Comment: CVPR 201

Proposal Flow: Semantic Correspondences from Object Proposals

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.Comment: arXiv admin note: text overlap with arXiv:1511.0506

Computerized Analysis of Magnetic Resonance Images to Study Cerebral Anatomy in Developing Neonates

The study of cerebral anatomy in developing neonates is of great importance for the understanding of brain development during the early period of life. This dissertation therefore focuses on three challenges in the modelling of cerebral anatomy in neonates during brain development. The methods that have been developed all use Magnetic Resonance Images (MRI) as source data. To facilitate study of vascular development in the neonatal period, a set of image analysis algorithms are developed to automatically extract and model cerebral vessel trees. The whole process consists of cerebral vessel tracking from automatically placed seed points, vessel tree generation, and vasculature registration and matching. These algorithms have been tested on clinical Time-of- Flight (TOF) MR angiographic datasets. To facilitate study of the neonatal cortex a complete cerebral cortex segmentation and reconstruction pipeline has been developed. Segmentation of the neonatal cortex is not effectively done by existing algorithms designed for the adult brain because the contrast between grey and white matter is reversed. This causes pixels containing tissue mixtures to be incorrectly labelled by conventional methods. The neonatal cortical segmentation method that has been developed is based on a novel expectation-maximization (EM) method with explicit correction for mislabelled partial volume voxels. Based on the resulting cortical segmentation, an implicit surface evolution technique is adopted for the reconstruction of the cortex in neonates. The performance of the method is investigated by performing a detailed landmark study. To facilitate study of cortical development, a cortical surface registration algorithm for aligning the cortical surface is developed. The method first inflates extracted cortical surfaces and then performs a non-rigid surface registration using free-form deformations (FFDs) to remove residual alignment. Validation experiments using data labelled by an expert observer demonstrate that the method can capture local changes and follow the growth of specific sulcus

Automatic Reporting of TBI Lesion Location in CT based on Deep Learning and Atlas Mapping

Tese de mestrado integrado, Engenharia Biomédica e Biofísica (Biofísica Médica e Fisiologia de Sistemas), 2021, Universidade de Lisboa, Faculdade de CiênciasThe assessment of Computed Tomography (CT) scans for Traumatic Brain Injury (TBI) management remains a time consuming and challenging task for physicians. Computational methods for quantitative lesion segmentation and localisation may increase consistency in diagnosis and prognosis criteria. Our goal was to develop a registration-based tool to accurately localise several lesion classes (i.e., calculate the volume of lesion per brain region), as an extension of the Brain Lesion Analysis and Segmentation Tool for CT (BLAST-CT). Lesions were located by projecting a Magnetic Resonance Imaging (MRI) labelled atlas from the Montreal Neurological Institute (MNI MRI atlas) to a lesion map in native space. We created a CT template to work as an intermediate step between the two imaging spaces, using 182 non-lesioned CT scans and an unbiased iterative registration approach. We then non-linearly registered the parcellated atlas to the CT template, subsequently registering (affine) the result to native space. From the final atlas alignment, it was possible to calculate the volume of each lesion class per brain region. This pipeline was validated on a multi-centre dataset (n=839 scans), and defined three methods to flag any scans that presented sub-optimal results. The first one was based on the similarity metric of the registration of every scan to the study-specific CT template, the second aimed to identify any scans with regions that were completely collapsed post registration, and the final one identified scans with a significant volume of intra-ventricular haemorrhage outside of the ventricles. Additionally, an assessment of lesion prevalence and of the false negative and false positive rates of the algorithm, per anatomical region, was conducted, along with a bias assessment of the BLAST-CT tool. Our results show that the constructed pipeline is able to successfully localise TBI lesions across the whole brain, although without voxel-wise accuracy. We found the error rates calculated for each brain region to be inversely correlated with the lesion volume within that region. No considerable bias was identified on BLAST-CT, as all the significant correlation coefficients calculated between the Dice scores and clinical variables (i.e., age, Glasgow Coma Scale, Extended Glasgow Outcome Scale and Injury Severity Score) were below 0.2. Our results also suggest that the variation in DSC between male and female patients within a specific age range was caused by the discrepancy in lesion volume presented by the scans included in each sample