NON-RIGID BODY MECHANICAL PROPERTY RECOVERY FROM IMAGES AND VIDEOS

Material property has great importance in surgical simulation and virtual reality. The mechanical properties of the human soft tissue are critical to characterize the tissue deformation of each patient. Studies have shown that the tissue stiffness described by the tissue properties may indicate abnormal pathological process. The (recovered) elasticity parameters can assist surgeons to perform better pre-op surgical planning and enable medical robots to carry out personalized surgical procedures. Traditional elasticity parameters estimation methods rely largely on known external forces measured by special devices and strain field estimated by landmarks on the deformable bodies. Or they are limited to mechanical property estimation for quasi-static deformation. For virtual reality applications such as virtual try-on, garment material capturing is of equal significance as the geometry reconstruction. In this thesis, I present novel approaches for automatically estimating the material properties of soft bodies from images or from a video capturing the motion of the deformable body. I use a coupled simulation-optimization-identification framework to deform one soft body at its original, non-deformed state to match the deformed geometry of the same object in its deformed state. The optimal set of material parameters is thereby determined by minimizing the error metric function. This method can simultaneously recover the elasticity parameters of multiple regions of soft bodies using Finite Element Method-based simulation (of either linear or nonlinear materials undergoing large deformation) and particle-swarm optimization methods. I demonstrate the effectiveness of this approach on real-time interaction with virtual organs in patient-specific surgical simulation, using parameters acquired from low-resolution medical images. With the recovered elasticity parameters and the age of the prostate cancer patients as features, I build a cancer grading and staging classifier. The classifier achieves up to 91% for predicting cancer T-Stage and 88% for predicting Gleason score. To recover the mechanical properties of soft bodies from a video, I propose a method which couples statistical graphical model with FEM simulation. Using this method, I can recover the material properties of a soft ball from a high-speed camera video that captures the motion of the ball. Furthermore, I extend the material recovery framework to fabric material identification. I propose a novel method for garment material extraction from a single-view image and a learning based cloth material recovery method from a video recording the motion of the cloth. Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, I propose a method that can compute a 3D model of a human body and its outfit from a single photograph with little human interaction. My proposed learning-based cloth material type recovery method exploits simulated data-set and deep neural network. I demonstrate the effectiveness of my algorithms by re-purposing the reconstructed garments for virtual try-on, garment transfer, and cloth animation on digital characters. With the recovered mechanical properties, one can construct a virtual world with soft objects exhibiting real-world behaviors.Doctor of Philosoph

Inverse-Consistent Determination of Young\u27s Modulus of Human Lung

Human lung undergoes respiration-induced deformation due to sequential inhalation and exhalation. Accurate determination of lung deformation is crucial for tumor localization and targeted radiotherapy in patients with lung cancer. Numerical modeling of human lung dynamics based on underlying physics and physiology enables simulation and virtual visualization of lung deformation. Dynamical modeling is numerically complicated by the lack of information on lung elastic behavior, structural heterogeneity as well as boundary constrains. This study integrates physics-based modeling and image-based data acquisition to develop the patient-specific biomechanical model and consequently establish the first consistent Young\u27s modulus (YM) of human lung. This dissertation has four major components: (i) develop biomechanical model for computation of the flow and deformation characteristics that can utilize subject-specific, spatially-dependent lung material property; (ii) develop a fusion algorithm to integrate deformation results from a deformable image registration (DIR) and physics-based modeling using the theory of Tikhonov regularization; (iii) utilize fusion algorithm to establish unique and consistent patient specific Young\u27s modulus and; (iv) validate biomechanical model utilizing established patient-specific elastic property with imaging data. The simulation is performed on three dimensional lung geometry reconstructed from four-dimensional computed tomography (4DCT) dataset of human subjects. The heterogeneous Young\u27s modulus is estimated from a linear elastic deformation model with the same lung geometry and 4D lung DIR. The biomechanical model adequately predicts the spatio-temporal lung deformation, consistent with data obtained from imaging. The accuracy of the numerical solution is enhanced through fusion with the imaging data beyond the classical comparison of the two sets of data. Finally, the fused displacement results are used to establish unique and consistent patient-specific elastic property of the lung

Developing advanced mathematical models for detecting abnormalities in 2D/3D medical structures.

Detecting abnormalities in two-dimensional (2D) and three-dimensional (3D) medical structures is among the most interesting and challenging research areas in the medical imaging field. Obtaining the desired accurate automated quantification of abnormalities in medical structures is still very challenging. This is due to a large and constantly growing number of different objects of interest and associated abnormalities, large variations of their appearances and shapes in images, different medical imaging modalities, and associated changes of signal homogeneity and noise for each object. The main objective of this dissertation is to address these problems and to provide proper mathematical models and techniques that are capable of analyzing low and high resolution medical data and providing an accurate, automated analysis of the abnormalities in medical structures in terms of their area/volume, shape, and associated abnormal functionality. This dissertation presents different preliminary mathematical models and techniques that are applied in three case studies: (i) detecting abnormal tissue in the left ventricle (LV) wall of the heart from delayed contrast-enhanced cardiac magnetic resonance images (MRI), (ii) detecting local cardiac diseases based on estimating the functional strain metric from cardiac cine MRI, and (iii) identifying the abnormalities in the corpus callosum (CC) brain structure—the largest fiber bundle that connects the two hemispheres in the brain—for subjects that suffer from developmental brain disorders. For detecting the abnormal tissue in the heart, a graph-cut mathematical optimization model with a cost function that accounts for the object’s visual appearance and shape is used to segment the the inner cavity. The model is further integrated with a geometric model (i.e., a fast marching level set model) to segment the outer border of the myocardial wall (the LV). Then the abnormal tissue in the myocardium wall (also called dead tissue, pathological tissue, or infarct area) is identified based on a joint Markov-Gibbs random field (MGRF) model of the image and its region (segmentation) map that accounts for the pixel intensities and the spatial interactions between the pixels. Experiments with real in-vivo data and comparative results with ground truth (identified by a radiologist) and other approaches showed that the proposed framework can accurately detect the pathological tissue and can provide useful metrics for radiologists and clinicians. To estimate the strain from cardiac cine MRI, a novel method based on tracking the LV wall geometry is proposed. To achieve this goal, a partial differential equation (PDE) method is applied to track the LV wall points by solving the Laplace equation between the LV contours of each two successive image frames over the cardiac cycle. The main advantage of the proposed tracking method over traditional texture-based methods is its ability to track the movement and rotation of the LV wall based on tracking the geometric features of the inner, mid-, and outer walls of the LV. This overcomes noise sources that come from scanner and heart motion. To identify the abnormalities in the CC from brain MRI, the CCs are aligned using a rigid registration model and are segmented using a shape-appearance model. Then, they are mapped to a simple unified space for analysis. This work introduces a novel cylindrical mapping model, which is conformal (i.e., one to one transformation and bijective), that enables accurate 3D shape analysis of the CC in the cylindrical domain. The framework can detect abnormalities in all divisions of the CC (i.e., splenium, rostrum, genu and body). In addition, it offers a whole 3D analysis of the CC abnormalities instead of only area-based analysis as done by previous groups. The initial classification results based on the centerline length and CC thickness suggest that the proposed CC shape analysis is a promising supplement to the current techniques for diagnosing dyslexia. The proposed techniques in this dissertation have been successfully tested on complex synthetic and MR images and can be used to advantage in many of today’s clinical applications of computer-assisted medical diagnostics and intervention

Patient-specific, mechanistic models of tumor growth incorporating artificial intelligence and big data

Despite the remarkable advances in cancer diagnosis, treatment, and management that have occurred over the past decade, malignant tumors remain a major public health problem. Further progress in combating cancer may be enabled by personalizing the delivery of therapies according to the predicted response for each individual patient. The design of personalized therapies requires patient-specific information integrated into an appropriate mathematical model of tumor response. A fundamental barrier to realizing this paradigm is the current lack of a rigorous, yet practical, mathematical theory of tumor initiation, development, invasion, and response to therapy. In this review, we begin by providing an overview of different approaches to modeling tumor growth and treatment, including mechanistic as well as data-driven models based on ``big data" and artificial intelligence. Next, we present illustrative examples of mathematical models manifesting their utility and discussing the limitations of stand-alone mechanistic and data-driven models. We further discuss the potential of mechanistic models for not only predicting, but also optimizing response to therapy on a patient-specific basis. We then discuss current efforts and future possibilities to integrate mechanistic and data-driven models. We conclude by proposing five fundamental challenges that must be addressed to fully realize personalized care for cancer patients driven by computational models

Algorithmic Analysis Techniques for Molecular Imaging

This study addresses image processing techniques for two medical imaging modalities: Positron Emission Tomography (PET) and Magnetic Resonance Imaging (MRI), which can be used in studies of human body functions and anatomy in a non-invasive manner. In PET, the so-called Partial Volume Effect (PVE) is caused by low spatial resolution of the modality. The efficiency of a set of PVE-correction methods is evaluated in the present study. These methods use information about tissue borders which have been acquired with the MRI technique. As another technique, a novel method is proposed for MRI brain image segmen- tation. A standard way of brain MRI is to use spatial prior information in image segmentation. While this works for adults and healthy neonates, the large variations in premature infants preclude its direct application. The proposed technique can be applied to both healthy and non-healthy premature infant brain MR images. Diffusion Weighted Imaging (DWI) is a MRI-based technique that can be used to create images for measuring physiological properties of cells on the structural level. We optimise the scanning parameters of DWI so that the required acquisition time can be reduced while still maintaining good image quality. In the present work, PVE correction methods, and physiological DWI models are evaluated in terms of repeatabilityof the results. This gives in- formation on the reliability of the measures given by the methods. The evaluations are done using physical phantom objects, correlation measure- ments against expert segmentations, computer simulations with realistic noise modelling, and with repeated measurements conducted on real pa- tients. In PET, the applicability and selection of a suitable partial volume correction method was found to depend on the target application. For MRI, the data-driven segmentation offers an alternative when using spatial prior is not feasible. For DWI, the distribution of b-values turns out to be a central factor affecting the time-quality ratio of the DWI acquisition. An optimal b-value distribution was determined. This helps to shorten the imaging time without hampering the diagnostic accuracy.Siirretty Doriast

Multimodal Image Fusion and Its Applications.

Image fusion integrates different modality images to provide comprehensive information of the image content, increasing interpretation capabilities and producing more reliable results. There are several advantages of combining multi-modal images, including improving geometric corrections, complementing data for improved classification, and enhancing features for analysis...etc. This thesis develops the image fusion idea in the context of two domains: material microscopy and biomedical imaging. The proposed methods include image modeling, image indexing, image segmentation, and image registration. The common theme behind all proposed methods is the use of complementary information from multi-modal images to achieve better registration, feature extraction, and detection performances. In material microscopy, we propose an anomaly-driven image fusion framework to perform the task of material microscopy image analysis and anomaly detection. This framework is based on a probabilistic model that enables us to index, process and characterize the data with systematic and well-developed statistical tools. In biomedical imaging, we focus on the multi-modal registration problem for functional MRI (fMRI) brain images which improves the performance of brain activation detection.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120701/1/yuhuic_1.pd

Characterization and modelling of complex motion patterns

Movement analysis is the principle of any interaction with the world and the survival of living beings completely depends on the effciency of such analysis. Visual systems have remarkably developed eficient mechanisms that analyze motion at different levels, allowing to recognize objects in dynamical and cluttered environments. In artificial vision, there exist a wide spectrum of applications for which the study of complex movements is crucial to recover salient information. Yet each domain may be different in terms of scenarios, complexity and relationships, a common denominator is that all of them require a dynamic understanding that captures the relevant information. Overall, current strategies are highly dependent on the appearance characterization and usually they are restricted to controlled scenarios. This thesis proposes a computational framework that is inspired in known motion perception mechanisms and structured as a set of modules. Each module is in due turn composed of a set of computational strategies that provide qualitative and quantitative descriptions of the dynamic associated to a particular movement. Diverse applications were herein considered and an extensive validation was performed for each of them. Each of the proposed strategies has shown to be reliable at capturing the dynamic patterns of different tasks, identifying, recognizing, tracking and even segmenting objects in sequences of video.Resumen. El análisis del movimiento es el principio de cualquier interacción con el mundo y la supervivencia de los seres vivos depende completamente de la eficiencia de este tipo de análisis. Los sistemas visuales notablemente han desarrollado mecanismos eficientes que analizan el movimiento en diferentes niveles, lo cual permite reconocer objetos en entornos dinámicos y saturados. En visión artificial existe un amplio espectro de aplicaciones para las cuales el estudio de los movimientos complejos es crucial para recuperar información saliente. A pesar de que cada dominio puede ser diferente en términos de los escenarios, la complejidad y las relaciones de los objetos en movimiento, un común denominador es que todos ellos requieren una comprensión dinámica para capturar información relevante. En general, las estrategias actuales son altamente dependientes de la caracterización de la apariencia y por lo general están restringidos a escenarios controlados. Esta tesis propone un marco computacional que se inspira en los mecanismos de percepción de movimiento conocidas y esta estructurado como un conjunto de módulos. Cada módulo esta a su vez compuesto por un conjunto de estrategias computacionales que proporcionan descripciones cualitativas y cuantitativas de la dinámica asociada a un movimiento particular. Diversas aplicaciones fueron consideradas en este trabajo y una extensa validación se llevó a cabo para cada uno de ellas. Cada una de las estrategias propuestas ha demostrado ser fiable en la captura de los patrones dinámicos de diferentes tareas identificando, reconociendo, siguiendo e incluso segmentando objetos en secuencias de video.Doctorad