GP-SLAM+: real-time 3D lidar SLAM based on improved regionalized Gaussian process map reconstruction

This paper presents a 3D lidar SLAM system based on improved regionalized Gaussian process (GP) map reconstruction to provide both low-drift state estimation and mapping in real-time for robotics applications. We utilize spatial GP regression to model the environment. This tool enables us to recover surfaces including those in sparsely scanned areas and obtain uniform samples with uncertainty. Those properties facilitate robust data association and map updating in our scan-to-map registration scheme, especially when working with sparse range data. Compared with previous GP-SLAM, this work overcomes the prohibitive computational complexity of GP and redesigns the registration strategy to meet the accuracy requirements in 3D scenarios. For large-scale tasks, a two-thread framework is employed to suppress the drift further. Aerial and ground-based experiments demonstrate that our method allows robust odometry and precise mapping in real-time. It also outperforms the state-of-the-art lidar SLAM systems in our tests with light-weight sensors.Comment: Accepted by IROS 202

Statistical Modeling of Craniofacial Shape and Texture

We present a fully-automatic statistical 3D shape modeling approach and apply it to a large dataset of 3D images, the Headspace dataset, thus generating the first public shape-and-texture 3D Morphable Model (3DMM) of the full human head. Our approach is the first to employ a template that adapts to the dataset subject before dense morphing. This is fully automatic and achieved using 2D facial landmarking, projection to 3D shape, and mesh editing. In dense template morphing, we improve on the well-known Coherent Point Drift algorithm, by incorporating iterative data-sampling and alignment. Our evaluations demonstrate that our method has better performance in correspondence accuracy and modeling ability when compared with other competing algorithms. We propose a texture map refinement scheme to build high quality texture maps and texture model. We present several applications that include the first clinical use of craniofacial 3DMMs in the assessment of different types of surgical intervention applied to a craniosynostosis patient group

Detail Enhancing Denoising of Digitized 3D Models from a Mobile Scanning System

The acquisition process of digitizing a large-scale environment produces an enormous amount of raw geometry data. This data is corrupted by system noise, which leads to 3D surfaces that are not smooth and details that are distorted. Any scanning system has noise associate with the scanning hardware, both digital quantization errors and measurement inaccuracies, but a mobile scanning system has additional system noise introduced by the pose estimation of the hardware during data acquisition. The combined system noise generates data that is not handled well by existing noise reduction and smoothing techniques. This research is focused on enhancing the 3D models acquired by mobile scanning systems used to digitize large-scale environments. These digitization systems combine a variety of sensors – including laser range scanners, video cameras, and pose estimation hardware – on a mobile platform for the quick acquisition of 3D models of real world environments. The data acquired by such systems are extremely noisy, often with significant details being on the same order of magnitude as the system noise. By utilizing a unique 3D signal analysis tool, a denoising algorithm was developed that identifies regions of detail and enhances their geometry, while removing the effects of noise on the overall model. The developed algorithm can be useful for a variety of digitized 3D models, not just those involving mobile scanning systems. The challenges faced in this study were the automatic processing needs of the enhancement algorithm, and the need to fill a hole in the area of 3D model analysis in order to reduce the effect of system noise on the 3D models. In this context, our main contributions are the automation and integration of a data enhancement method not well known to the computer vision community, and the development of a novel 3D signal decomposition and analysis tool. The new technologies featured in this document are intuitive extensions of existing methods to new dimensionality and applications. The totality of the research has been applied towards detail enhancing denoising of scanned data from a mobile range scanning system, and results from both synthetic and real models are presented

Aceleración de algoritmos de procesamiento de imágenes para el análisis de partículas individuales con microscopia electrónica

Tesis Doctoral inédita cotutelada por la Masaryk University (República Checa) y la Universidad Autónoma de Madrid, Escuela Politécnica Superior, Departamento de Ingeniería Informática. Fecha de Lectura: 24-10-2022Cryogenic Electron Microscopy (Cryo-EM) is a vital field in current structural biology. Unlike X-ray crystallography and Nuclear Magnetic Resonance, it can be used to analyze membrane proteins and other samples with overlapping spectral peaks. However, one of the significant limitations of Cryo-EM is the computational complexity. Modern electron microscopes can produce terabytes of data per single session, from which hundreds of thousands of particles must be extracted and processed to obtain a near-atomic resolution of the original sample. Many existing software solutions use high-Performance Computing (HPC) techniques to bring these computations to the realm of practical usability. The common approach to acceleration is parallelization of the processing, but in praxis, we face many complications, such as problem decomposition, data distribution, load scheduling, balancing, and synchronization. Utilization of various accelerators further complicates the situation, as heterogeneous hardware brings additional caveats, for example, limited portability, under-utilization due to synchronization, and sub-optimal code performance due to missing specialization. This dissertation, structured as a compendium of articles, aims to improve the algorithms used in Cryo-EM, esp. the SPA (Single Particle Analysis). We focus on the single-node performance optimizations, using the techniques either available or developed in the HPC field, such as heterogeneous computing or autotuning, which potentially needs the formulation of novel algorithms. The secondary goal of the dissertation is to identify the limitations of state-of-the-art HPC techniques. Since the Cryo-EM pipeline consists of multiple distinct steps targetting different types of data, there is no single bottleneck to be solved. As such, the presented articles show a holistic approach to performance optimization. First, we give details on the GPU acceleration of the specific programs. The achieved speedup is due to the higher performance of the GPU, adjustments of the original algorithm to it, and application of the novel algorithms. More specifically, we provide implementation details of programs for movie alignment, 2D classification, and 3D reconstruction that have been sped up by order of magnitude compared to their original multi-CPU implementation or sufficiently the be used on-the-fly. In addition to these three programs, multiple other programs from an actively used, open-source software package XMIPP have been accelerated and improved. Second, we discuss our contribution to HPC in the form of autotuning. Autotuning is the ability of software to adapt to a changing environment, i.e., input or executing hardware. Towards that goal, we present cuFFTAdvisor, a tool that proposes and, through autotuning, finds the best configuration of the cuFFT library for given constraints of input size and plan settings. We also introduce a benchmark set of ten autotunable kernels for important computational problems implemented in OpenCL or CUDA, together with the introduction of complex dynamic autotuning to the KTT tool. Third, we propose an image processing framework Umpalumpa, which combines a task-based runtime system, data-centric architecture, and dynamic autotuning. The proposed framework allows for writing complex workflows which automatically use available HW resources and adjust to different HW and data but at the same time are easy to maintainThe project that gave rise to these results received the support of a fellowship from the “la Caixa” Foundation (ID 100010434). The fellowship code is LCF/BQ/DI18/11660021. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 71367

Motion Segmentation Aided Super Resolution Image Reconstruction

This dissertation addresses Super Resolution (SR) Image Reconstruction focusing on motion segmentation. The main thrust is Information Complexity guided Gaussian Mixture Models (GMMs) for Statistical Background Modeling. In the process of developing our framework we also focus on two other topics; motion trajectories estimation toward global and local scene change detections and image reconstruction to have high resolution (HR) representations of the moving regions. Such a framework is used for dynamic scene understanding and recognition of individuals and threats with the help of the image sequences recorded with either stationary or non-stationary camera systems. We introduce a new technique called Information Complexity guided Statistical Background Modeling. Thus, we successfully employ GMMs, which are optimal with respect to information complexity criteria. Moving objects are segmented out through background subtraction which utilizes the computed background model. This technique produces superior results to competing background modeling strategies. The state-of-the-art SR Image Reconstruction studies combine the information from a set of unremarkably different low resolution (LR) images of static scene to construct an HR representation. The crucial challenge not handled in these studies is accumulating the corresponding information from highly displaced moving objects. In this aspect, a framework of SR Image Reconstruction of the moving objects with such high level of displacements is developed. Our assumption is that LR images are different from each other due to local motion of the objects and the global motion of the scene imposed by non-stationary imaging system. Contrary to traditional SR approaches, we employed several steps. These steps are; the suppression of the global motion, motion segmentation accompanied by background subtraction to extract moving objects, suppression of the local motion of the segmented out regions, and super-resolving accumulated information coming from moving objects rather than the whole scene. This results in a reliable offline SR Image Reconstruction tool which handles several types of dynamic scene changes, compensates the impacts of camera systems, and provides data redundancy through removing the background. The framework proved to be superior to the state-of-the-art algorithms which put no significant effort toward dynamic scene representation of non-stationary camera systems

Skull diversity and evolution in miniaturized amphibians, genus Brachycephalus (Anura: Brachycephalidae)

Miniaturized amphibians of the genus Brachycephalus are phenotypically diverse. The species of Brachycephalus have bufoniform or leptodactyliform baupläne and any of three skeletal states: nonhyperossified, hyperossified without dorsal shield, and hyperossified with dorsal shield. We integrate high-resolution microcomputed tomography, geometric morphometrics, and an estimate of molecular phylogenetic relationships to investigate skull diversity in shape and size-shape space in selected species of Brachycephalus. Skull diversity amongst species of Brachycephalus can be partitioned into shape and size-shape space according to the four conditions of skeletal states-baupläne, namely, nonhyperossified leptodactyliform, nonhyperossified bufoniform, hyperossified bufoniform without dorsal shield, and hyperossified bufoniform with dorsal shield. Skull diversity in shape and size-shape space in nonhyperossified leptodactyliform species of Brachycephalus is markedly larger, when compared to skull diversity in species of the three other conditions of skeletal states-baupläne. Variation in skull shape scales with size across Brachycephalus and, therefore, can be explained by allometry. Skull diversity, baupläne, and skeletal states covary to a large extent with monophyletic lineages of Brachycephalus, as revealed by a mitochondrial DNA species tree. Nonhyperossified bufoniform species and hyperossified bufoniform species with or without dorsal shield are monophyletic lineages, as inferred from a mitochondrial DNA species tree. Nonhyperossified leptodactyliform species of Brachycephalus do not share, however, a most recent common ancestor. The nonhyperossified leptodactyliform species of Brachycephalus, due to their marked skull diversity and lack of monophyly, emerge as evolutionarily complex. Therefore, further sampling of the nonhyperossified leptodactyliform condition of skeletal states-baupläne will be necessary to further understand the evolutionary history of Brachycephalus.Fil: dos Reis, Sérgio F.. Universidade Estadual de Campinas. Instituto de Biología; BrasilFil: Clemente Carvalho, Rute B.G.. Queens University; CanadáFil: dos Santos, Caio M. S. F. F.. Universidade Federal do Rio de Janeiro; BrasilFil: Lopes, Ricardo T.. Universidade Federal do Rio de Janeiro; BrasilFil: Von Zuben, Fernando J.. Universidade Estadual de Campinas; BrasilFil: Laborda, Prianda R.. Universidade Estadual de Campinas. Instituto de Biología; BrasilFil: Perez, Sergio Ivan. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - La Plata; Argentina. Universidad Nacional de La Plata. Facultad de Ciencias Naturales y Museo. Departamento Científico de Antropología; Argentin

Assessment of somatostatin image quantification with SPET and SPET-CT to aid characterisation of disease

Work was undertaken in this thesis to assess the use of somatostatin image quantification with SPET and SPET-CT to aid the characterisation of disease within the body. Two radionuclide somatostatin analogues were used for this assessment, the first was NeoSPECT and the second, OctreoScan. The primary aim of work in this thesis was to assess the role of NeoSPECT imaging in the characterisation of disease within the lungs, that is, to differentiate benign from malignant disease. Two forms of image quantification were used in the NeoSPECT assessment, a tumour to background ratio (T:B) and a value of tumour percentage uptake (% uptake). Values of T:B and % uptake were calculated from SPET images acquired 2 hours post injection. T:B results from the benign group (n = 8) demonstrated a median T:B of 2.21, whilst the malignant group (n = 28) demonstrated a median T:B of 2.01. The differences between the groups were tested statistically via a Mann-Whitney test, which showed there to be no statistical difference between the groups (p=0.90, 95.4% CI of (-0.5598, 0.5498)). To undertake the calculation of % uptake a non-patient acquisition (a standard acquisition) was also required, unfortunately not all of the patient cohort used for the T:B assessment had this additional acquisition. As a result of this numbers were low for the % uptake assessment in patients with a benign histology (n = 2), therefore statistical analysis could not be performed. However, review of the range of values for each histology within the malignant group proved useful as no differences were demonstrated between the ranges of values which could help to differentiate between the histologies. Quantification of dual time point imaging was also assessed to determine if there were any variations in values calculated that could also help differentiate the different histologies. For this assessment patients were images at 2 and 4 hours post injection. Results from the Wilcoxon Signed Rank test of the T:B assessment found there to be no statistically significant difference between values of T:B calculated at 2 and 4 hours that was characteristic of tumour type (p=1.0 and p=0.14). The difference in % uptake between 2 and 4 hours was also assessed via a Wilcoxon Signed Rank test, this test also concluded there to be no significant difference value of % uptake between the two acquisitions of the malignant group (p = 0.73). An attempt was also made to quantify ‘other’ uptake within the mediastinum, however, a lack of anatomical information made correlation with histology impossible and as a result no firm conclusions relating image quantification to histology could be drawn from this work. Work from this thesis concluded no quantitative difference between tissue histology could be demonstrated using NeoSPECT, either from single or dual time point imaging. As a result of the NeoSPECT work a number of factors which limited the accuracy and reproducibility of SPET image quantification were identified. Towards the end of the NeoSPECT work hybrid imaging (SPET-CT) became available within the department at Glasgow Royal Infirmary. It was believed that hybrid imaging could resolve some of the limitations and subsequently improve the accuracy of SPET image quantification. However, NeoSPECT was removed from the market for a short period of time and therefore a similar somatostatin analogue, OctreoScan, was used to investigate if the accuracy of somatostatin image quantification could be improved as a result of SPET-CT and its associated reconstruction algorithms including a CT based attenuation correction. Firstly, a qualitative assessment of image quality using the new hybrid reconstructions techniques was undertaken via an observer study. Images were reconstructed with the existing reconstruction techniques, as used for the NeoSPECT work, and with the new hybrid imaging techniques. Four experienced observers blinded to reconstruction technique were asked to score images in terms of their overall image quality. A Friedman test was performed on the scores for each observer, three of the four observers demonstrated a statistically significant difference in their scores between the existing and new hybrid technique (p = 0.00, p = 0.003, p= 0.00), with the new hybrid technique being assigned the highest scores in terms of image quality. Images were also assessed semi-quantitatively via profile analysis which also demonstrated a clear differentiation between the existing and new hybrid techniques with increased image quality being demonstrated in the hybrid data set. The quantitative accuracy of hybrid imaging was also assessed using phantom data. For 111In the difference of the value of absolute activity calculated and that measured varied by 35% but this improved to 21 % when scatter and CT-attenuation based corrections were applied. For 99mTc a much more notable difference between the existing techniques used in chapter 2 and those available from the use of hybrid imaging was demonstrated, the difference in the value of absolute activity calculated and that measured improved from 67% to 0.04%, respectively. Work in this thesis clearly demonstrated an improvement in image quality and accuracy in SPET quantification as a result of hybrid imaging techniques

Computational methods to create and analyze a digital gene expression atlas of embryo development from microscopy images

Abstract The creation of atlases, or digital models where information from different subjects can be combined, is a field of increasing interest in biomedical imaging. When a single image does not contain enough information to appropriately describe the organism under study, it is then necessary to acquire images of several individuals, each of them containing complementary data with respect to the rest of the components in the cohort. This approach allows creating digital prototypes, ranging from anatomical atlases of human patients and organs, obtained for instance from Magnetic Resonance Imaging, to gene expression cartographies of embryo development, typically achieved from Light Microscopy. Within such context, in this PhD Thesis we propose, develop and validate new dedicated image processing methodologies that, based on image registration techniques, bring information from multiple individuals into alignment within a single digital atlas model. We also elaborate a dedicated software visualization platform to explore the resulting wealth of multi-dimensional data and novel analysis algo-rithms to automatically mine the generated resource in search of bio¬logical insights. In particular, this work focuses on gene expression data from developing zebrafish embryos imaged at the cellular resolution level with Two-Photon Laser Scanning Microscopy. Disposing of quantitative measurements relating multiple gene expressions to cell position and their evolution in time is a fundamental prerequisite to understand embryogenesis multi-scale processes. However, the number of gene expressions that can be simultaneously stained in one acquisition is limited due to optical and labeling constraints. These limitations motivate the implementation of atlasing strategies that can recreate a virtual gene expression multiplex. The developed computational tools have been tested in two different scenarios. The first one is the early zebrafish embryogenesis where the resulting atlas constitutes a link between the phenotype and the genotype at the cellular level. The second one is the late zebrafish brain where the resulting atlas allows studies relating gene expression to brain regionalization and neurogenesis. The proposed computational frameworks have been adapted to the requirements of both scenarios, such as the integration of partial views of the embryo into a whole embryo model with cellular resolution or the registration of anatom¬ical traits with deformable transformation models non-dependent on any specific labeling. The software implementation of the atlas generation tool (Match-IT) and the visualization platform (Atlas-IT) together with the gene expression atlas resources developed in this Thesis are to be made freely available to the scientific community. Lastly, a novel proof-of-concept experiment integrates for the first time 3D gene expression atlas resources with cell lineages extracted from live embryos, opening up the door to correlate genetic and cellular spatio-temporal dynamics. La creación de atlas, o modelos digitales, donde la información de distintos sujetos puede ser combinada, es un campo de creciente interés en imagen biomédica. Cuando una sola imagen no contiene suficientes datos como para describir apropiadamente el organismo objeto de estudio, se hace necesario adquirir imágenes de varios individuos, cada una de las cuales contiene información complementaria respecto al resto de componentes del grupo. De este modo, es posible crear prototipos digitales, que pueden ir desde atlas anatómicos de órganos y pacientes humanos, adquiridos por ejemplo mediante Resonancia Magnética, hasta cartografías de la expresión genética del desarrollo de embrionario, típicamente adquiridas mediante Microscopía Optica. Dentro de este contexto, en esta Tesis Doctoral se introducen, desarrollan y validan nuevos métodos de procesado de imagen que, basándose en técnicas de registro de imagen, son capaces de alinear imágenes y datos provenientes de múltiples individuos en un solo atlas digital. Además, se ha elaborado una plataforma de visualization específicamente diseñada para explorar la gran cantidad de datos, caracterizados por su multi-dimensionalidad, que resulta de estos métodos. Asimismo, se han propuesto novedosos algoritmos de análisis y minería de datos que permiten inspeccionar automáticamente los atlas generados en busca de conclusiones biológicas significativas. En particular, este trabajo se centra en datos de expresión genética del desarrollo embrionario del pez cebra, adquiridos mediante Microscopía dos fotones con resolución celular. Disponer de medidas cuantitativas que relacionen estas expresiones genéticas con las posiciones celulares y su evolución en el tiempo es un prerrequisito fundamental para comprender los procesos multi-escala característicos de la morfogénesis. Sin embargo, el número de expresiones genéticos que pueden ser simultáneamente etiquetados en una sola adquisición es reducido debido a limitaciones tanto ópticas como del etiquetado. Estas limitaciones requieren la implementación de estrategias de creación de atlas que puedan recrear un multiplexado virtual de expresiones genéticas. Las herramientas computacionales desarrolladas han sido validadas en dos escenarios distintos. El primer escenario es el desarrollo embrionario temprano del pez cebra, donde el atlas resultante permite constituir un vínculo, a nivel celular, entre el fenotipo y el genotipo de este organismo modelo. El segundo escenario corresponde a estadios tardíos del desarrollo del cerebro del pez cebra, donde el atlas resultante permite relacionar expresiones genéticas con la regionalización del cerebro y la formación de neuronas. La plataforma computacional desarrollada ha sido adaptada a los requisitos y retos planteados en ambos escenarios, como la integración, a resolución celular, de vistas parciales dentro de un modelo consistente en un embrión completo, o el alineamiento entre estructuras de referencia anatómica equivalentes, logrado mediante el uso de modelos de transformación deformables que no requieren ningún marcador específico. Está previsto poner a disposición de la comunidad científica tanto la herramienta de generación de atlas (Match-IT), como su plataforma de visualización (Atlas-IT), así como las bases de datos de expresión genética creadas a partir de estas herramientas. Por último, dentro de la presente Tesis Doctoral, se ha incluido una prueba conceptual innovadora que permite integrar los mencionados atlas de expresión genética tridimensionales dentro del linaje celular extraído de una adquisición in vivo de un embrión. Esta prueba conceptual abre la puerta a la posibilidad de correlar, por primera vez, las dinámicas espacio-temporales de genes y células