Neural Deformable Models for 3D Bi-Ventricular Heart Shape Reconstruction and Modeling from 2D Sparse Cardiac Magnetic Resonance Imaging

We propose a novel neural deformable model (NDM) targeting at the reconstruction and modeling of 3D bi-ventricular shape of the heart from 2D sparse cardiac magnetic resonance (CMR) imaging data. We model the bi-ventricular shape using blended deformable superquadrics, which are parameterized by a set of geometric parameter functions and are capable of deforming globally and locally. While global geometric parameter functions and deformations capture gross shape features from visual data, local deformations, parameterized as neural diffeomorphic point flows, can be learned to recover the detailed heart shape.Different from iterative optimization methods used in conventional deformable model formulations, NDMs can be trained to learn such geometric parameter functions, global and local deformations from a shape distribution manifold. Our NDM can learn to densify a sparse cardiac point cloud with arbitrary scales and generate high-quality triangular meshes automatically. It also enables the implicit learning of dense correspondences among different heart shape instances for accurate cardiac shape registration. Furthermore, the parameters of NDM are intuitive, and can be used by a physician without sophisticated post-processing. Experimental results on a large CMR dataset demonstrate the improved performance of NDM over conventional methods.Comment: Accepted by ICCV 202

Meshless deformable models for LV motion analysis

We propose a novel meshless deformable model for in vivo cardiac left ventricle (LV) 3D motion estimation. As a relatively new technology, tagged MRI (tMRI) provides a direct and noninvasive way to reveal local deformation of the myocardium, which creates a large amount of heart motion data which requiring quantitative analysis. In our study, we sample the heart motion sparsely at intersections of three sets of orthogonal tagging planes and then use a new meshless deformable model to recover the dense 3D motion of the myocardium temporally during the cardiac cycle. We compute external forces at tag intersections based on tracked local motion and redistribute the force to meshless particles throughout the myocardium. Internal constraint forces at particles are derived from local strain energy using a Moving Least Squares (MLS) method. The dense 3D motion field is then computed and updated using the Lagrange equation. The new model avoids the singularity problem of mesh-based models and is capable of tracking large deformation with high efficiency and accuracy. In particular, the model performs well even when the control points (tag intersections) are relatively sparse. We tested the performance of the meshless model on a numerical phantom, as well as in vivo heart data of healthy subjects and patients. The experimental results show that the meshless deformable model can fully recover the myocardium motion in 3D. 1

From Fully-Supervised Single-Task to Semi-Supervised Multi-Task Deep Learning Architectures for Segmentation in Medical Imaging Applications

Medical imaging is routinely performed in clinics worldwide for the diagnosis and treatment of numerous medical conditions in children and adults. With the advent of these medical imaging modalities, radiologists can visualize both the structure of the body as well as the tissues within the body. However, analyzing these high-dimensional (2D/3D/4D) images demands a significant amount of time and effort from radiologists. Hence, there is an ever-growing need for medical image computing tools to extract relevant information from the image data to help radiologists perform efficiently. Image analysis based on machine learning has pivotal potential to improve the entire medical imaging pipeline, providing support for clinical decision-making and computer-aided diagnosis. To be effective in addressing challenging image analysis tasks such as classification, detection, registration, and segmentation, specifically for medical imaging applications, deep learning approaches have shown significant improvement in performance. While deep learning has shown its potential in a variety of medical image analysis problems including segmentation, motion estimation, etc., generalizability is still an unsolved problem and many of these successes are achieved at the cost of a large pool of datasets. For most practical applications, getting access to a copious dataset can be very difficult, often impossible. Annotation is tedious and time-consuming. This cost is further amplified when annotation must be done by a clinical expert in medical imaging applications. Additionally, the applications of deep learning in the real-world clinical setting are still limited due to the lack of reliability caused by the limited prediction capabilities of some deep learning models. Moreover, while using a CNN in an automated image analysis pipeline, it’s critical to understand which segmentation results are problematic and require further manual examination. To this extent, the estimation of uncertainty calibration in a semi-supervised setting for medical image segmentation is still rarely reported. This thesis focuses on developing and evaluating optimized machine learning models for a variety of medical imaging applications, ranging from fully-supervised, single-task learning to semi-supervised, multi-task learning that makes efficient use of annotated training data. The contributions of this dissertation are as follows: (1) developing a fully-supervised, single-task transfer learning for the surgical instrument segmentation from laparoscopic images; and (2) utilizing supervised, single-task, transfer learning for segmenting and digitally removing the surgical instruments from endoscopic/laparoscopic videos to allow the visualization of the anatomy being obscured by the tool. The tool removal algorithms use a tool segmentation mask and either instrument-free reference frames or previous instrument-containing frames to fill in (inpaint) the instrument segmentation mask; (3) developing fully-supervised, single-task learning via efficient weight pruning and learned group convolution for accurate left ventricle (LV), right ventricle (RV) blood pool and myocardium localization and segmentation from 4D cine cardiac MR images; (4) demonstrating the use of our fully-supervised memory-efficient model to generate dynamic patient-specific right ventricle (RV) models from cine cardiac MRI dataset via an unsupervised learning-based deformable registration field; and (5) integrating a Monte Carlo dropout into our fully-supervised memory-efficient model with inherent uncertainty estimation, with the overall goal to estimate the uncertainty associated with the obtained segmentation and error, as a means to flag regions that feature less than optimal segmentation results; (6) developing semi-supervised, single-task learning via self-training (through meta pseudo-labeling) in concert with a Teacher network that instructs the Student network by generating pseudo-labels given unlabeled input data; (7) proposing largely-unsupervised, multi-task learning to demonstrate the power of a simple combination of a disentanglement block, variational autoencoder (VAE), generative adversarial network (GAN), and a conditioning layer-based reconstructor for performing two of the foremost critical tasks in medical imaging — segmentation of cardiac structures and reconstruction of the cine cardiac MR images; (8) demonstrating the use of 3D semi-supervised, multi-task learning for jointly learning multiple tasks in a single backbone module – uncertainty estimation, geometric shape generation, and cardiac anatomical structure segmentation of the left atrial cavity from 3D Gadolinium-enhanced magnetic resonance (GE-MR) images. This dissertation summarizes the impact of the contributions of our work in terms of demonstrating the adaptation and use of deep learning architectures featuring different levels of supervision to build a variety of image segmentation tools and techniques that can be used across a wide spectrum of medical image computing applications centered on facilitating and promoting the wide-spread computer-integrated diagnosis and therapy data science

Model-Based Shape and Motion Analysis: Left Ventricle of a Heart

The accurate and clinically useful estimation of the shape, motion, and deformation of the left ventricle of a heart (LV) is an important yet open research problem. Recently, computer vision techniques for reconstructing the 3-D shape and motion of the LV have been developed. The main drawback of these techniques, however, is that their models are formulated in terms of either too many local parameters that require non-trivial processing to be useful for close to real time diagnosis, or too few parameters to offer an adequate approximation to the LV motion. To address the problem, we present a new class of volumetric primitives for a compact and accurate LV shape representation in which model parameters are functions. Lagrangian dynamics are employed to convert geometric models into dynamic models that can deform according to the forces manifested in the data points. It is thus possible to make a precise estimation of the deformation of the LV shape endocardial, epicardial and anywhere in between with a small number of intuitive parameter functions. We believe that the proposed technique has a wide range of potential applications. In this thesis, we demonstrate the possibility by applying it to the 3-D LV shape and motion characterization from magnetic tagging data (MRI-SPAMM). We show that the results of our experiments with normal and abnormal heart data enable us to quantitatively verify the physicians\u27 qualitative conception of the left ventricular wall motion

The effects of explicit versus parameterized convection on the MJO in a large-domain high-resolution tropical case study. Part II: Processes leading to differences in MJO development

High-resolution simulations over a large tropical domain (∼20◦S–20◦N and 42◦E–180◦E) using both explicit and parameterized convection are analyzed and compared during a 10-day case study of an active Madden-Julian Oscillation (MJO) event. In Part II, the moisture budgets and moist entropy budgets are analyzed. Vertical subgrid diabatic heating profiles and vertical velocity profiles are also compared; these are related to the horizontal and vertical advective components of the moist entropy budget which contribute to gross moist stability, GMS, and normalized GMS (NGMS). The 4-km model with explicit convection and good MJO performance has a vertical heating structure that increases with height in the lower troposphere in regions of strong convection (like observations), whereas the 12-km model with parameterized convection and a poor MJO does not show this relationship. The 4-km explicit convection model also has a more top-heavy heating profile for the troposphere as a whole near and to the west of the active MJO-related convection, unlike the 12-km parameterized convection model. The dependence of entropy advection components on moisture convergence is fairly weak in all models, and differences between models are not always related to MJO performance, making comparisons to previous work somewhat inconclusive. However, models with relatively good MJO strength and propagation have a slightly larger increase of the vertical advective component with increasing moisture convergence, and their NGMS vertical terms have more variability in time and longitude, with total NGMS that is comparatively larger to the west and smaller to the east

Force-based atomistic/continuum blending for multilattices

We formulate the blended force-based quasicontinuum method for multilattices and develop rigorous error estimates in terms of the approximation parameters: choice of atomistic region, blending region, and continuum finite element mesh. Balancing the approximation parameters yields a convergent atomistic/continuum multiscale method for multilattices with point defects, including a rigorous convergence rate in terms of the computational cost. The analysis is illustrated with numerical results for a Stone–Wales defect in graphene

Imaging the Molecular Gas in A z=3.9 Quasar Host Galaxy at 0\farcs3 Resolution: A Central, Sub-Kilparsex Scale Star Formation Reservoir in APM 08279+5255

We have mapped the molecular gas content in the host galaxy of the strongly lensed high-redshift quasar APM 08279+5255 (z = 3.911) with the Very Large Array at 0\farcs3 resolution. The CO(J = 1➝0) emission is clearly resolved in our maps. The CO(J = 1➝0) line luminosity derived from these maps is in good agreement with a previous single-dish measurement. In contrast to previous interferometer-based studies, we find that the full molecular gas reservoir is situated in two compact peaks separated by ≲0\farcs4. Our observations reveal, for the first time, that the emission from cold molecular gas is virtually co-spatial with the optical/near-infrared continuum emission of the central active galactic nucleus (AGN) in this source. This striking similarity in morphology indicates that the molecular gas is situated in a compact region close to the AGN. Based on the high-resolution CO maps, we present a revised model for the gravitational lensing in this system, which indicates that the molecular gas emission is magnified by only a factor of 4 (in contrast to previously suggested factors of 100). This model suggests that the CO is situated in a circumnuclear disk of ~550 pc radius that is possibly seen at an inclination of ≲25°, i.e., relatively close to face-on. From the CO luminosity, we derive a molecular gas mass of Mgas = 1.3x10^11 M☉ for this galaxy. From the CO structure and linewidth, we derive a dynamical mass of M dyn sin^2 i = 4.0x10^10 M☉. Based on a revised mass estimate for the central black hole of Mbh = 2.3x10^10 M☉ and the results of our molecular line study, we find that the mass of the stellar bulge of APM 08279+5255 falls short of the local M BH-σbulge relationship of nearby galaxies by more than an order of magnitude, lending support to recent suggestions that this relation may evolve with cosmic time and/or change toward the high-mass end

Self-supervised learning for few-shot medical image segmentation

Fully-supervised deep learning segmentation models are inflexible when encountering new unseen semantic classes and their fine-tuning often requires significant amounts of annotated data. Few-shot semantic segmentation (FSS) aims to solve this inflexibility by learning to segment an arbitrary unseen semantically meaningful class by referring to only a few labeled examples, without involving fine-tuning. State-of-the-art FSS methods are typically designed for segmenting natural images and rely on abundant annotated data of training classes to learn image representations that generalize well to unseen testing classes. However, such a training mechanism is impractical in annotation-scarce medical imaging scenarios. To address this challenge, in this work, we propose a novel self-supervised FSS framework for medical images, named SSL-ALPNet, in order to bypass the requirement for annotations during training. The proposed method exploits superpixel-based pseudo-labels to provide supervision signals. In addition, we propose a simple yet effective adaptive local prototype pooling module which is plugged into the prototype networks to further boost segmentation accuracy. We demonstrate the general applicability of the proposed approach using three different tasks: organ segmentation of abdominal CT and MRI images respectively, and cardiac segmentation of MRI images. The proposed method yields higher Dice scores than conventional FSS methods which require manual annotations for training in our experiments