Simulation-Based Joint Estimation of Body Deformation and Elasticity Parameters for Medical Image Analysis

Elasticity parameter estimation is essential for generating accurate and controlled simulation results for computer animation and medical image analysis. However, finding the optimal parameters for a particular simulation often requires iterations of simulation, assessment, and adjustment and can become a tedious process. Elasticity values are especially important in medical image analysis, since cancerous tissues tend to be stiffer. Elastography is a popular type of method for finding stiffness values by reconstructing a dense displacement field from medical images taken during the application of forces or vibrations. These methods, however, are limited by the imaging modality and the force exertion or vibration actuation mechanisms, which can be complicated for deep-seated organs. In this thesis, I present a novel method for reconstructing elasticity parameters without requiring a dense displacement field or a force exertion device. The method makes use of natural deformations within the patient and relies on surface information from segmented images taken on different days. The elasticity value of the target organ and boundary forces acting on surrounding organs are optimized with an iterative optimizer, within which the deformation is always generated by a physically-based simulator. Experimental results on real patient data are presented to show the positive correlation between recovered elasticity values and clinical prostate cancer stages. Furthermore, to resolve the performance issue arising from the high dimensionality of boundary forces, I propose to use a reduced finite element model to improve the convergence of the optimizer. To find the set of bases to represent the dimensions for forces, a statistical training based on real patient data is performed. I demonstrate the trade-off between accuracy and performance by using different numbers of bases in the optimization using synthetic data. A speedup of more than an order of magnitude is observed without sacrificing too much accuracy in recovered elasticity.Doctor of Philosoph

Simulation-Based Joint Estimation of Body Deformation and Elasticity Parameters for Medical Image Analysis

Estimation of tissue stiffness is an important means of noninvasive cancer detection. Existing elasticity reconstruction methods usually depend on a dense displacement field (inferred from ultrasound or MR images) and known external forces. Many imaging modalities, however, cannot provide details within an organ and therefore cannot provide such a displacement field. Furthermore, force exertion and measurement can be difficult for some internal organs, making boundary forces another missing parameter. We propose a general method for estimating elasticity and boundary forces automatically using an iterative optimization framework, given the desired (target) output surface. During the optimization, the input model is deformed by the simulator, and an objective function based on the distance between the deformed surface and the target surface is minimized numerically. The optimization framework does not depend on a particular simulation method and is therefore suitable for different physical models. We show a positive correlation between clinical prostate cancer stage (a clinical measure of severity) and the recovered elasticity of the organ. Since the surface correspondence is established, our method also provides a non-rigid image registration, where the quality of the deformation fields is guaranteed, as they are computed using a physics-based simulation

Finite Element Based Tracking of Deforming Surfaces

We present an approach to robustly track the geometry of an object that deforms over time from a set of input point clouds captured from a single viewpoint. The deformations we consider are caused by applying forces to known locations on the object's surface. Our method combines the use of prior information on the geometry of the object modeled by a smooth template and the use of a linear finite element method to predict the deformation. This allows the accurate reconstruction of both the observed and the unobserved sides of the object. We present tracking results for noisy low-quality point clouds acquired by either a stereo camera or a depth camera, and simulations with point clouds corrupted by different error terms. We show that our method is also applicable to large non-linear deformations.Comment: additional experiment

Realistic tool-tissue interaction models for surgical simulation and planning

Surgical simulators present a safe and potentially effective method for surgical training, and can also be used in pre- and intra-operative surgical planning. Realistic modeling of medical interventions involving tool-tissue interactions has been considered to be a key requirement in the development of high-fidelity simulators and planners. The soft-tissue constitutive laws, organ geometry and boundary conditions imposed by the connective tissues surrounding the organ, and the shape of the surgical tool interacting with the organ are some of the factors that govern the accuracy of medical intervention planning.\ud \ud This thesis is divided into three parts. First, we compare the accuracy of linear and nonlinear constitutive laws for tissue. An important consequence of nonlinear models is the Poynting effect, in which shearing of tissue results in normal force; this effect is not seen in a linear elastic model. The magnitude of the normal force for myocardial tissue is shown to be larger than the human contact force discrimination threshold. Further, in order to investigate and quantify the role of the Poynting effect on material discrimination, we perform a multidimensional scaling study. Second, we consider the effects of organ geometry and boundary constraints in needle path planning. Using medical images and tissue mechanical properties, we develop a model of the prostate and surrounding organs. We show that, for needle procedures such as biopsy or brachytherapy, organ geometry and boundary constraints have more impact on target motion than tissue material parameters. Finally, we investigate the effects surgical tool shape on the accuracy of medical intervention planning. We consider the specific case of robotic needle steering, in which asymmetry of a bevel-tip needle results in the needle naturally bending when it is inserted into soft tissue. We present an analytical and finite element (FE) model for the loads developed at the bevel tip during needle-tissue interaction. The analytical model explains trends observed in the experiments. We incorporated physical parameters (rupture toughness and nonlinear material elasticity) into the FE model that included both contact and cohesive zone models to simulate tissue cleavage. The model shows that the tip forces are sensitive to the rupture toughness. In order to model the mechanics of deflection of the needle, we use an energy-based formulation that incorporates tissue-specific parameters such as rupture toughness, nonlinear material elasticity, and interaction stiffness, and needle geometric and material properties. Simulation results follow similar trends (deflection and radius of curvature) to those observed in macroscopic experimental studies of a robot-driven needle interacting with gels

From medical images to individualized cardiac mechanics: A Physiome approach

Cardiac mechanics is a branch of science that deals with forces, kinematics, and material properties of the heart, which is valuable for clinical applications and physiological studies. Although anatomical and biomechanical experiments are necessary to provide the fundamental knowledge of cardiac mechanics, the invasive nature of the procedures limits their further applicability. In consequence, noninvasive alternatives are required, and cardiac images provide an excellent source of subject-specific and in vivo information. Noninvasive and individualized cardiac mechanical studies can be achieved through coupling general physiological models derived from invasive experiments with subject-specific information extracted from medical images. Nevertheless, as data extracted from images are gross, sparse, or noisy, and do not directly provide the information of interest in general, the couplings between models and measurements are complicated inverse problems with numerous issues need to be carefully considered. The goal of this research is to develop a noninvasive framework for studying individualized cardiac mechanics through systematic coupling between cardiac physiological models and medical images according to their respective merits. More specifically, nonlinear state-space filtering frameworks for recovering individualized cardiac deformation and local material parameters of realistic nonlinear constitutive laws have been proposed. To ensure the physiological meaningfulness, clinical relevance, and computational feasibility of the frameworks, five key issues have to be properly addressed, including the cardiac physiological model, the heart representation in the computational environment, the information extraction from cardiac images, the coupling between models and image information, and also the computational complexity. For the cardiac physiological model, a cardiac physiome model tailored for cardiac image analysis has been proposed to provide a macroscopic physiological foundation for the study. For the heart representation, a meshfree method has been adopted to facilitate implementations and spatial accuracy refinements. For the information extraction from cardiac images, a registration method based on free-form deformation has been adopted for robust motion tracking. For the coupling between models and images, state-space filtering has been applied to systematically couple the models with the measurements. For the computational complexity, a mode superposition approach has been adopted to project the system into an equivalent mathematical space with much fewer dimensions for computationally feasible filtering. Experiments were performed on both synthetic and clinical data to verify the proposed frameworks

Quantifying perception of nonlinear elastic tissue models using multidimensional scaling

Simplified soft tissue models used in surgical simulations cannot perfectly reproduce all material behaviors. In particular, many tissues exhibit the Poynting effect, which results in normal forces during shearing of tissue and is only observed in nonlinear elastic material models. In order to investigate and quantify the role of the Poynting effect on material discrimination, we performed a multidimensional scaling (MDS) study. Participants were presented with several pairs of shear and normal forces generated by a haptic device during interaction with virtual soft objects. Participants were asked to rate the similarity between the forces felt. The selection of the material parameters – and thus the magnitude of the shear\ud and normal forces – was based on a pre-study prior to the MDS experiment. It was observed that for nonlinear elastic tissue models exhibiting the Poynting effect, MDS analysis indicated that both shear and normal forces affect user perception

Middle-Ear Imaging and Estimation of The Linear Elastic Properties of The Human Tympanic Membrane

Finite-element (FE) modeling of the human middle ear could improve diagnostic techniques, such as tympanometry. Accurate representation of the mechanical properties and geometry of the middle ear, especially of the soft tissues, are crucial for FE modeling of the middle ear. The objective of this work is to quantitatively evaluate the efficacy of iodine potassium iodide (IKI) solution as a contrast agent for imaging the middle-ear soft tissues and to estimate the linear elastic properties of the human tympanic membrane (TM). In the imaging study, six human temporal bones were used, which were obtained in right-left pairs, from three cadaveric heads. All bones were fixed using formaldehyde. Only one bone from each pair was stained using IKI solution. Samples were scanned using a micro-computed tomography system. Contrast-to-noise ratios of eight middle-ear soft tissues were calculated for each temporal bone. Results from Welch\u27s t-test indicate significant difference between the two soft tissues group, i.e., stained and unstained, at a 95% confidence interval. Results from a paired t-tests for each of the individual soft tissues also indicated significant improvement of contrast in all tissues after staining. The increase in contrast with IKI solution confirms its potential application in sample-specific FE modeling. In the linear elasticity study, experiments were performed on three specimens with a custom-built pressurization unit at a quasi-static pressure of 500 Pa. The shape of each TM before and after pressurization was recorded using a Fourier transform profilometer. The samples were also imaged using micro-CT to create sample-specific FE models. For each sample, the Young’s modulus was then estimated by numerically optimizing its value in the FE model so simulated pressurized shapes matched experimental data. Also, the effects of incorporating two forms of spatial non-uniformity in the distribution of Young’s modulus were studied, including partitioning the TM into 4 quadrants and 4 concentric rings. The estimated Young’s modulus values were 2.2 MPa, 2.4 MPa and 2.0 MPa, which are similar to recent literature values using an alternative method. An improved fit between simulated and experimental data were obtained when spatial non-uniformity was incorporated

NON-RIGID BODY MECHANICAL PROPERTY RECOVERY FROM IMAGES AND VIDEOS

Material property has great importance in surgical simulation and virtual reality. The mechanical properties of the human soft tissue are critical to characterize the tissue deformation of each patient. Studies have shown that the tissue stiffness described by the tissue properties may indicate abnormal pathological process. The (recovered) elasticity parameters can assist surgeons to perform better pre-op surgical planning and enable medical robots to carry out personalized surgical procedures. Traditional elasticity parameters estimation methods rely largely on known external forces measured by special devices and strain field estimated by landmarks on the deformable bodies. Or they are limited to mechanical property estimation for quasi-static deformation. For virtual reality applications such as virtual try-on, garment material capturing is of equal significance as the geometry reconstruction. In this thesis, I present novel approaches for automatically estimating the material properties of soft bodies from images or from a video capturing the motion of the deformable body. I use a coupled simulation-optimization-identification framework to deform one soft body at its original, non-deformed state to match the deformed geometry of the same object in its deformed state. The optimal set of material parameters is thereby determined by minimizing the error metric function. This method can simultaneously recover the elasticity parameters of multiple regions of soft bodies using Finite Element Method-based simulation (of either linear or nonlinear materials undergoing large deformation) and particle-swarm optimization methods. I demonstrate the effectiveness of this approach on real-time interaction with virtual organs in patient-specific surgical simulation, using parameters acquired from low-resolution medical images. With the recovered elasticity parameters and the age of the prostate cancer patients as features, I build a cancer grading and staging classifier. The classifier achieves up to 91% for predicting cancer T-Stage and 88% for predicting Gleason score. To recover the mechanical properties of soft bodies from a video, I propose a method which couples statistical graphical model with FEM simulation. Using this method, I can recover the material properties of a soft ball from a high-speed camera video that captures the motion of the ball. Furthermore, I extend the material recovery framework to fabric material identification. I propose a novel method for garment material extraction from a single-view image and a learning based cloth material recovery method from a video recording the motion of the cloth. Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, I propose a method that can compute a 3D model of a human body and its outfit from a single photograph with little human interaction. My proposed learning-based cloth material type recovery method exploits simulated data-set and deep neural network. I demonstrate the effectiveness of my algorithms by re-purposing the reconstructed garments for virtual try-on, garment transfer, and cloth animation on digital characters. With the recovered mechanical properties, one can construct a virtual world with soft objects exhibiting real-world behaviors.Doctor of Philosoph

Real-time measurement corrected prediction of soft tissue response for medical simulations

Medical simulators, such as in palpation and disease diagnosis, require an efficient model of the biological soft tissue deformation. Hence, a computationally fast and accurate algorithm is required to support and enhance user interactions in near real-time simulations. The visual accuracy of such simulators is dependent on the user¿s reaction time. Static visual images that update at a rate of 25 Hz are perceived as real-time moving images. Hence, visualizing software requires fast algorithms to compute the deformation of soft tissue to facilitate a meaningful simulation. Furthermore, soft tissue behaviour should be modelled accurately while compatible with real-time computation. This work proposes a fast solver for the linearized finite element method (FEM) and validates the proposed algorithm with experimental results. The novelty of the method lies in the utilization of real-time force/displacement measurements that are embedded in the solution via the Kalman filter. A novel computational algorithm that utilizes the strength of the FEM in terms of accuracy and employs direct measurements from the manipulated tissue to overcome the slow computational process of the FEM is proposed in the first part of the thesis. As the behaviour of the mechanically loaded tissue can be regarded as linearly responding at each time step, a constant acceleration temporal discretization method, i.e., the Newmark-ß is employed. In real-time applications, the accuracy of the target variable highly depends on the accuracy of the inputs while differentiating noise from the signal is hardly ever possible. To address this problem, a Kalman filter-based method is developed. The proposed algorithm not only filters the noise from the measurements but also adapts the filter gain to the estimates of the target variable, i.e., the resulting tissue deformation. For a simulated tension test of a cubic model, the proposed algorithm achieves the update frequency of 63.3 Hz. This rate is a significant improvement in computational speed compared to the 5.8 Hz update rate by the classic FEM. Besides, this novel combination of the KF and the FEM makes it possible to expand the displacement estimates in the spatial domain when the measurements are only partially available at certain points. The performance of the above method is validated experimentally through a comparison with indentation tests on artificial human tissue-like material and with the FEM result under identical simulation conditions. The test is repeated on several samples, and the displacement variation from the FEM outcome is considered as the model error. Simulation results show that the proposed method achieves the deformation update frequency of 145.7 Hz compared to the 2.7 Hz from the reference FEM. The proposed method shows the same predictive ability, only 0.47% difference from FEM on average. Experimental validation of the proposed KF-FEM confirms that by consideration of both the measurement noise and the model error, the proposed method is capable of achieving high-frequency response without sacrificing the accuracy. Further to this, the experiments confirmed the linearized model response is reliable within the applied displacement range and therefore proving that KF can be employed. The developed KF-FEM was modified in the next study to address the problem resulting from inaccurate external loads measurements by the force sensors. In the modified version, both the external force, i.e., driving variable, and the displacement, i.e., driven variable, are taken as system states. It is considered that the uncertainty of the model input influences the accuracy of the system estimates. The modified model is calibrated to differentiate the system noise from the input noise. Numerical simulations were conducted on a liver shape geometrical model, and the simulation results demonstrate that more than 90% of the measurement noise is removed. The computational speed is also increased, delivering up to 89 Hz update rate. While the uncertainty of the external load is replicated in the displacements in an FEM solution, the developed algorithm can differentiate the measurement noise, including the displacement and external forces, from the system error, i.e., the FE model error. In the last study, the proposed model was developed to reflect the nonlinear behaviour of the manipulated tissue. The Central Difference time discretization method was used to model large deformations. A novel feature is that the Equation of motion is formulated within the element level rather than in the global spatial domain. This approach helped to improve the computational speed. Indentation with strains of slightly over 10% was simulated to assess the performance of the proposed model. The developed algorithm achieved the 33.85 Hz update frequency on a standard-issue PC and confirmed its suitability for real-time applications. Also, the proposed model achieved estimates with a maximum 5.75% mean absolute error (MAE) concerning the measurements while the classic FEM showed 6.20% MAE under identical simulation condition. Results confirm that deformation estimates for noisy boundary loads of the FEM can be improved with the help of direct measurements and yet be realistic in terms of real-time visual update. This study proposed a novel computational algorithm that achieved update frequencies of higher than 25 Hz to be perceived as real-time in human eyes. The developed KF-FEM model has also shown the potential of improving the FEM accuracy with the help of direct measurements. The proposed algorithm used partially available measurements and expanded its estimates in the spatial domain. The method was experimentally validated, and the model input uncertainty, as well as the nonlinear behaviour of the soft tissue, were assessed and verified

Identification of the Elastic Modulus of an Organ Model Using Reactive Force and Ultrasound Image

制度:新 ; 報告番号:甲3418号 ; 学位の種類:博士(工学) ; 授与年月日:2011/7/28 ; 早大学位記番号:新574