Point Cloud Registration for LiDAR and Photogrammetric Data: a Critical Synthesis and Performance Analysis on Classic and Deep Learning Algorithms

Recent advances in computer vision and deep learning have shown promising performance in estimating rigid/similarity transformation between unregistered point clouds of complex objects and scenes. However, their performances are mostly evaluated using a limited number of datasets from a single sensor (e.g. Kinect or RealSense cameras), lacking a comprehensive overview of their applicability in photogrammetric 3D mapping scenarios. In this work, we provide a comprehensive review of the state-of-the-art (SOTA) point cloud registration methods, where we analyze and evaluate these methods using a diverse set of point cloud data from indoor to satellite sources. The quantitative analysis allows for exploring the strengths, applicability, challenges, and future trends of these methods. In contrast to existing analysis works that introduce point cloud registration as a holistic process, our experimental analysis is based on its inherent two-step process to better comprehend these approaches including feature/keypoint-based initial coarse registration and dense fine registration through cloud-to-cloud (C2C) optimization. More than ten methods, including classic hand-crafted, deep-learning-based feature correspondence, and robust C2C methods were tested. We observed that the success rate of most of the algorithms are fewer than 40% over the datasets we tested and there are still are large margin of improvement upon existing algorithms concerning 3D sparse corresopondence search, and the ability to register point clouds with complex geometry and occlusions. With the evaluated statistics on three datasets, we conclude the best-performing methods for each step and provide our recommendations, and outlook future efforts.Comment: 7 figure

Improving Dose-Response Correlations for Locally Advanced NSCLC Patients Treated with IMRT or PSPT

The standard of care for locally advanced non-small cell lung cancer (NSCLC) is concurrent chemo-radiotherapy. Despite recent advancements in radiation delivery methods, the median survival time of NSCLC patients remains below 28 months. Higher tumor dose has been found to increase survival but also a higher rate of radiation pneumonitis (RP) that affects breathing capability. In fear of such toxicity, less-aggressive treatment plans are often clinically preferred, leading to metastasis and recurrence. Therefore, accurate RP prediction is crucial to ensure tumor coverage to improve treatment outcome. Current models have associated RP with increased dose but with limited accuracy as they lack spatial correlation between accurate dose representation and quantitative RP representation. These models represent lung tissue damage with radiation dose distribution planned pre-treatment, which assumes a fixed patient geometry and inevitably renders imprecise dose delivery due to intra-fractional breathing motion and inter-fractional anatomy response. Additionally, current models employ whole-lung dose metrics as the contributing factor to RP as a qualitative, binary outcome but these global dose metrics discard microscopic, voxel-(3D pixel)-level information and prevent spatial correlations with quantitative RP representation. To tackle these limitations, we developed advanced deformable image registration (DIR) techniques that registered corresponding anatomical voxels between images for tracking and accumulating dose throughout treatment. DIR also enabled voxel-level dose-response correlation when CT image density change (IDC) was used to quantify RP. We hypothesized that more accurate estimates of biologically effective dose distributions actually delivered, achieved through (a) dose accumulation using deformable registration of weekly 4DCT images acquired over the course or radiotherapy and (b) the incorporation of variable relative biological effectiveness (RBE), would lead to statistically and clinically significant improvement in the correlation of RP with biologically effective dose distributions. Our work resulted in a robust intra-4DCT and inter-4DCT DIR workflow, with the accuracy meeting AAPM TG-132 recommendations for clinical implementation of DIR. The automated DIR workflow allowed us to develop a fully automated 4DCT-based dose accumulation pipeline in RayStation (RaySearch Laboratories, Stockholm, Sweden). With a sample of 67 IMRT patients, our results showed that the accumulated dose was statistically different than the planned dose across the entire cohort with an average MLD increase of ~1 Gy and clinically different for individual patients where 16% resulted in difference in the score of the normal tissue complication probability (NTCP) using an established, clinically used model, which could qualify the patients for treatment planning re-evaluation. Lastly, we associated dose difference with accuracy difference by establishing and comparing voxel-level dose-IDC correlations and concluded that the accumulated dose better described the localized damage, thereby a closer representation of the delivered dose. Using the same dose-response correlation strategy, we plotted the dose-IDC relationships for both photon patients (N = 51) and proton patients (N = 67), we measured the variable proton RBE values to be 3.07–1.27 from 9–52 Gy proton voxels. With the measured RBE values, we fitted an established variable proton RBE model with pseudo-R2 of 0.98. Therefore, our results led to statistically and clinically significant improvement in the correlation of RP with accumulated and biologically effective dose distributions and demonstrated the potential of incorporating the effect of anatomical change and biological damage in RP prediction models

A Virtual Testbed for Fish-Tank Virtual Reality: Improving Calibration with a Virtual-in-Virtual Display

With the development of novel calibration techniques for multimedia projectors and curved projection surfaces, volumetric 3D displays are becoming easier and more affordable to build. The basic requirements include a display shape that defines the volume (e.g. a sphere, cylinder, or cuboid) and a tracking system to provide each user's location for the perspective corrected rendering. When coupled with modern graphics cards, these displays are capable of high resolution, low latency, high frame rate, and even stereoscopic rendering; however, like many previous studies have shown, every component must be precisely calibrated for a compelling 3D effect. While human perceptual requirements have been extensively studied for head-tracked displays, most studies featured seated users in front of a flat display. It remains unclear if results from these flat display studies are applicable to newer, walk-around displays with enclosed or curved shapes. To investigate these issues, we developed a virtual testbed for volumetric head-tracked displays that can measure calibration accuracy of the entire system in real-time. We used this testbed to investigate visual distortions of prototype curved displays, improve existing calibration techniques, study the importance of stereo to performance and perception, and validate perceptual calibration with novice users. Our experiments show that stereo is important for task performance, but requires more accurate calibration, and that novice users can make effective use of perceptual calibration tools. We also propose a novel, real-time calibration method that can be used to fine-tune an existing calibration using perceptual feedback. The findings from this work can be used to build better head-tracked volumetric displays with an unprecedented amount of 3D realism and intuitive calibration tools for novice users

Fusion of magnetic resonance and ultrasound images for endometriosis detection

Endometriosis is a gynecologic disorder that typically affects women in their reproductive age and is associated with chronic pelvic pain and infertility. In the context of pre-operative diagnosis and guided surgery, endometriosis is a typical example of pathology that requires the use of both magnetic resonance (MR) and ultrasound (US) modalities. These modalities are used side by sidebecause they contain complementary information. However, MRI and US images have different spatial resolutions, fields of view and contrasts and are corrupted by different kinds of noise, which results in important challenges related to their analysis by radiologists. The fusion of MR and US images is a way of facilitating the task of medical experts and improve the pre-operative diagnosis and the surgery mapping. The object of this PhD thesis is to propose a new automatic fusion method for MRI and US images. First, we assume that the MR and US images to be fused are aligned, i.e., there is no geometric distortion between these images. We propose a fusion method for MR and US images, which aims at combining the advantages of each modality, i.e., good contrast and signal to noise ratio for the MR image and good spatial resolution for the US image. The proposed algorithm is based on an inverse problem, performing a super-resolution of the MR image and a denoising of the US image. A polynomial function is introduced to modelthe relationships between the gray levels of the MR and US images. However, the proposed fusion method is very sensitive to registration errors. Thus, in a second step, we introduce a joint fusion and registration method for MR and US images. Registration is a complicated task in practical applications. The proposed MR/US image fusion performs jointly super-resolution of the MR image and despeckling of the US image, and is able to automatically account for registration errors. A polynomial function is used to link ultrasound and MR images in the fusion process while an appropriate similarity measure is introduced to handle the registration problem. The proposed registration is based on a non-rigid transformation containing a local elastic B-spline model and a global affine transformation. The fusion and registration operations are performed alternatively simplifying the underlying optimization problem. The interest of the joint fusion and registration is analyzed using synthetic and experimental phantom images

Topics in genomic image processing

The image processing methodologies that have been actively studied and developed now play a very significant role in the flourishing biotechnology research. This work studies, develops and implements several image processing techniques for M-FISH and cDNA microarray images. In particular, we focus on three important areas: M-FISH image compression, microarray image processing and expression-based classification. Two schemes, embedded M-FISH image coding (EMIC) and Microarray BASICA: Background Adjustment, Segmentation, Image Compression and Analysis, have been introduced for M-FISH image compression and microarray image processing, respectively. In the expression-based classification area, we investigate the relationship between optimal number of features and sample size, either analytically or through simulation, for various classifiers

hybridGEOTABS project : MPC for controlling the power of the ground by integration

GEOTABS is an acronym for a GEOthermal heat pump combined with a Thermally Activated Building System (TABS). GEOTABS combines the use of geothermal energy, which is an almost limitless and ubiquitous energy source, with radiant heating and cooling systems, which can provide very comfortable conditioning of the indoor space. GEOTABShybrid refers to the integration of GEOTABS with secondary heating and cooling systems and other renewable and residual energy sources (R2ES), offering a huge potential to meet heating and cooling needs in office buildings, elderly care homes, schools and multi-family buildings throughout Europe in a sustainable way. Through the use of Model Predictive Control (MPC), a new control-integrated building design procedure and a readily applicable commercial system solution in GEOTABShybrid, the overall efficiency of heating and cooling will be significantly improved in comparison to current best practice GEOTABS systems and its competitiveness will be strengthened. The present paper is the first of a series that first introduces the hybridGEOTABS project and then specifically focuses on the control-related aspects of the hybridGEOTABS solution, the MPC, providing some interesting insights of its potential development