A hybrid patient-specific biomechanical model based image registration method for the motion estimation of lungs

This paper presents a new hybrid biomechanical model-based non-rigid image registration method for lung motion estimation. In the proposed method, a patient-specific biomechanical modelling process captures major physically realistic deformations with explicit physical modelling of sliding motion, whilst a subsequent non-rigid image registration process compensates for small residuals. The proposed algorithm was evaluated with 10 4D CT datasets of lung cancer patients. The target registration error (TRE), defined as the Euclidean distance of landmark pairs, was significantly lower with the proposed method (TRE = 1.37 mm) than with biomechanical modelling (TRE = 3.81 mm) and intensity-based image registration without specific considerations for sliding motion (TRE = 4.57 mm). The proposed method achieved a comparable accuracy as several recently developed intensity-based registration algorithms with sliding handling on the same datasets. A detailed comparison on the distributions of TREs with three non-rigid intensity-based algorithms showed that the proposed method performed especially well on estimating the displacement field of lung surface regions (mean TRE = 1.33 mm, maximum TRE = 5.3 mm). The effects of biomechanical model parameters (such as Poisson’s ratio, friction and tissue heterogeneity) on displacement estimation were investigated. The potential of the algorithm in optimising biomechanical models of lungs through analysing the pattern of displacement compensation from the image registration process has also been demonstrated

A 3D discrete model of the diaphragm and human trunk

In this paper, a 3D discrete model is presented to model the movements of the trunk during breathing. In this model, objects are represented by physical particles on their contours. A simple notion of force generated by a linear actuator allows the model to create forces on each particle by way of a geometrical attractor. Tissue elasticity and contractility are modeled by local shape memory and muscular fibers attractors. A specific dynamic MRI study was used to build a simple trunk model comprised of by three compartments: lungs, diaphragm and abdomen. This model was registered on the real geometry. Simulation results were compared qualitatively as well as quantitatively to the experimental data, in terms of volume and geometry. A good correlation was obtained between the model and the real data. Thanks to this model, pathology such as hemidiaphragm paralysis can also be simulated.Comment: published in: "Lung Modelling", France (2006

Simulation-Based Joint Estimation of Body Deformation and Elasticity Parameters for Medical Image Analysis

Elasticity parameter estimation is essential for generating accurate and controlled simulation results for computer animation and medical image analysis. However, finding the optimal parameters for a particular simulation often requires iterations of simulation, assessment, and adjustment and can become a tedious process. Elasticity values are especially important in medical image analysis, since cancerous tissues tend to be stiffer. Elastography is a popular type of method for finding stiffness values by reconstructing a dense displacement field from medical images taken during the application of forces or vibrations. These methods, however, are limited by the imaging modality and the force exertion or vibration actuation mechanisms, which can be complicated for deep-seated organs. In this thesis, I present a novel method for reconstructing elasticity parameters without requiring a dense displacement field or a force exertion device. The method makes use of natural deformations within the patient and relies on surface information from segmented images taken on different days. The elasticity value of the target organ and boundary forces acting on surrounding organs are optimized with an iterative optimizer, within which the deformation is always generated by a physically-based simulator. Experimental results on real patient data are presented to show the positive correlation between recovered elasticity values and clinical prostate cancer stages. Furthermore, to resolve the performance issue arising from the high dimensionality of boundary forces, I propose to use a reduced finite element model to improve the convergence of the optimizer. To find the set of bases to represent the dimensions for forces, a statistical training based on real patient data is performed. I demonstrate the trade-off between accuracy and performance by using different numbers of bases in the optimization using synthetic data. A speedup of more than an order of magnitude is observed without sacrificing too much accuracy in recovered elasticity.Doctor of Philosoph

NON-RIGID BODY MECHANICAL PROPERTY RECOVERY FROM IMAGES AND VIDEOS

Material property has great importance in surgical simulation and virtual reality. The mechanical properties of the human soft tissue are critical to characterize the tissue deformation of each patient. Studies have shown that the tissue stiffness described by the tissue properties may indicate abnormal pathological process. The (recovered) elasticity parameters can assist surgeons to perform better pre-op surgical planning and enable medical robots to carry out personalized surgical procedures. Traditional elasticity parameters estimation methods rely largely on known external forces measured by special devices and strain field estimated by landmarks on the deformable bodies. Or they are limited to mechanical property estimation for quasi-static deformation. For virtual reality applications such as virtual try-on, garment material capturing is of equal significance as the geometry reconstruction. In this thesis, I present novel approaches for automatically estimating the material properties of soft bodies from images or from a video capturing the motion of the deformable body. I use a coupled simulation-optimization-identification framework to deform one soft body at its original, non-deformed state to match the deformed geometry of the same object in its deformed state. The optimal set of material parameters is thereby determined by minimizing the error metric function. This method can simultaneously recover the elasticity parameters of multiple regions of soft bodies using Finite Element Method-based simulation (of either linear or nonlinear materials undergoing large deformation) and particle-swarm optimization methods. I demonstrate the effectiveness of this approach on real-time interaction with virtual organs in patient-specific surgical simulation, using parameters acquired from low-resolution medical images. With the recovered elasticity parameters and the age of the prostate cancer patients as features, I build a cancer grading and staging classifier. The classifier achieves up to 91% for predicting cancer T-Stage and 88% for predicting Gleason score. To recover the mechanical properties of soft bodies from a video, I propose a method which couples statistical graphical model with FEM simulation. Using this method, I can recover the material properties of a soft ball from a high-speed camera video that captures the motion of the ball. Furthermore, I extend the material recovery framework to fabric material identification. I propose a novel method for garment material extraction from a single-view image and a learning based cloth material recovery method from a video recording the motion of the cloth. Most recent garment capturing techniques rely on acquiring multiple views of clothing, which may not always be readily available, especially in the case of pre-existing photographs from the web. As an alternative, I propose a method that can compute a 3D model of a human body and its outfit from a single photograph with little human interaction. My proposed learning-based cloth material type recovery method exploits simulated data-set and deep neural network. I demonstrate the effectiveness of my algorithms by re-purposing the reconstructed garments for virtual try-on, garment transfer, and cloth animation on digital characters. With the recovered mechanical properties, one can construct a virtual world with soft objects exhibiting real-world behaviors.Doctor of Philosoph

Heterogeneous volumetric data mapping and its medical applications

With the advance of data acquisition techniques, massive solid geometries are being collected routinely in scientific tasks, these complex and unstructured data need to be effectively correlated for various processing and analysis. Volumetric mapping solves bijective low-distortion correspondence between/among 3D geometric data, and can serve as an important preprocessing step in many tasks in compute-aided design and analysis, industrial manufacturing, medical image analysis, to name a few. This dissertation studied two important volumetric mapping problems: the mapping of heterogeneous volumes (with nonuniform inner structures/layers) and the mapping of sequential dynamic volumes. To effectively handle heterogeneous volumes, first, we studied the feature-aligned harmonic volumetric mapping. Compared to previous harmonic mapping, it supports the point, curve, and iso-surface alignment, which are important low-dimensional structures in heterogeneous volumetric data. Second, we proposed a biharmonic model for volumetric mapping. Unlike the conventional harmonic volumetric mapping that only supports positional continuity on the boundary, this new model allows us to have higher order continuity $C^1$ along the boundary surface. This suggests a potential model to solve the volumetric mapping of complex and big geometries through divide-and-conquer. We also studied the medical applications of our volumetric mapping in lung tumor respiratory motion modeling. We were building an effective digital platform for lung tumor radiotherapy based on effective volumetric CT/MRI image matching and analysis. We developed and integrated in this platform a set of geometric/image processing techniques including advanced image segmentation, finite element meshing, volumetric registration and interpolation. The lung organ/tumor and surrounding tissues are treated as a heterogeneous region and a dynamic 4D registration framework is developed for lung tumor motion modeling and tracking. Compared to the previous 3D pairwise registration, our new 4D parameterization model leads to a significantly improved registration accuracy. The constructed deforming model can hence approximate the deformation of the tissues and tumor

Patient-specific simulation environment for surgical planning and preoperative rehearsal

Surgical simulation is common practice in the fields of surgical education and training. Numerous surgical simulators are available from commercial and academic organisations for the generic modelling of surgical tasks. However, a simulation platform is still yet to be found that fulfils the key requirements expected for patient-specific surgical simulation of soft tissue, with an effective translation into clinical practice. Patient-specific modelling is possible, but to date has been time-consuming, and consequently costly, because data preparation can be technically demanding. This motivated the research developed herein, which addresses the main challenges of biomechanical modelling for patient-specific surgical simulation. A novel implementation of soft tissue deformation and estimation of the patient-specific intraoperative environment is achieved using a position-based dynamics approach. This modelling approach overcomes the limitations derived from traditional physically-based approaches, by providing a simulation for patient-specific models with visual and physical accuracy, stability and real-time interaction. As a geometrically- based method, a calibration of the simulation parameters is performed and the simulation framework is successfully validated through experimental studies. The capabilities of the simulation platform are demonstrated by the integration of different surgical planning applications that are found relevant in the context of kidney cancer surgery. The simulation of pneumoperitoneum facilitates trocar placement planning and intraoperative surgical navigation. The implementation of deformable ultrasound simulation can assist surgeons in improving their scanning technique and definition of an optimal procedural strategy. Furthermore, the simulation framework has the potential to support the development and assessment of hypotheses that cannot be tested in vivo. Specifically, the evaluation of feedback modalities, as a response to user-model interaction, demonstrates improved performance and justifies the need to integrate a feedback framework in the robot-assisted surgical setting.Open Acces

파티클 시뮬레이션을 이용한 물리 기반 비강체 정합 기술

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2013. 8. 신영길.Recent advances in computing hardware have enabled the application of physically based simulation techniques to various research fields for improved accuracy. In this paper, we present a novel physically based non-rigid registration method using smoothed particle hydrodynamics (SPH) for hepatic metastasis volume-preserving registration between follow-up liver CT images. Our method models the liver and hepatic metastasis as a set of particles carrying their own physical properties. Based on the fact that the hepatic metastasis is stiffer than other normal cells in the liver parenchyma, the candidate regions of hepatic metastasis are modeled with particles of higher stiffness compared to the liver parenchyma. Particles placed in the liver and candidate regions of hepatic metastasis in the source image are transformed along a gradient vector flow (GVF)-based force field calculated in the target image. In this transformation, the particles are physically interacted and deformed by a novel deformable particle method which is proposed to preserve the hepatic metastasis to the best. In experimental results using 10 clinical datasets, our method matches the liver effectively between follow-up CT images as well as preserves the volume of hepatic metastasis almost completely, enabling the accurate assessment of the volume change of the hepatic metastasis. These results demonstrated a potential of the proposed method that it can deliver a substantial aid in measuring the size change of index lesion (i.e., hepatic metastasis) after the chemotheraphy of metastasis patients in radiation oncology.최근 컴퓨팅 하드웨어의 발달은 정확도 향상을 위해 물리 기반의 시뮬레이션 기술을 다양한 연구 분야에 적용할 수 있게 하였다. 본 논문에서는 입자를 이용하여 시뮬레이션하는 방법 중 하나인 입자 보간 방식의 유체역학(smoothed particle hydrodynamics) 기술을 이용하여, 후속 컴퓨터 단층촬영 영상(computed tomography) 사이에 간전이(hepatic metastasis) 체적을 보전하는 물리 기반의 비정형체 정합 기술을 제안한다. 제안 방법은 간과 간전이를 물리적 속성을 동반하는 일련의 입자로 표현하며, 간전이가 정상 간에 비해 강한 탄성을 보인다는 사실에 기반하여 간전이로 짐작되는 부위를 상대적으로 강한 탄성을 갖는 입자로 표현하였다. 초기에 간과 간전이 후보 영역을 나타내는 입자들은 입력 영상의 해당 영역에 위치되며, 정합하고자 하는 대상 영상으로 부터 경사도 벡터 흐름(gradient vector flow) 방법으로 계산된 힘의 장을 따라 이동된다. 이 때, 각 입자는 간전이의 체적을 최대한 보존하기 위해 제안된 변형 가능 입자 방식에 따라 서로 물리적으로 상호작용하며 변형된다. 10명의 환자 데이터를 이용한 실험 결과에 따르면, 후속 컴퓨터 단층촬영(CT) 영상 간의 정합 과정에서 간의 모양을 효과적으로 일치시킬 뿐만 아니라 간전이의 체적을 거의 완벽하게 보존하여 간전이의 체적 변화를 정확하게 진단할 수 있게 하였다. 이 결과는 간전이 환자가 화학 요법을 시행 한 후 암의 진행 상태를 판단하기 위해 간전이의 크기 변화를 측정하는데 도움을 줄 수 있는 방법임을 시사한다.I. Introduction 1.1 Motivation 1 1.2 Dissertation Goals 3 1.3 Main Contribution 4 1.4 Organization of the Dissertation 5 II. Background 2.1 Medical Image Registration 6 2.1.1 Transformation Models 8 2.1.2 Similarity Metrics 18 2.1.3 Optimization 23 2.1.4 Physically Based Non-Rigid Registration 25 2.2 Smoothed Particle Hydrodynamics 29 2.2.1 Formulation of SPH 30 2.2.2 Kernels 33 2.2.3 Applications 35 III. Volume-Preserving Deformation of Particles 3.1 SPH for Deformable Objects 40 3.2 Volume-Preserving Deformable Particle 44 IV. Non-Rigid Registration with the Deformable Particles 4.1 Automatic Detection of Liver and Candidate Regions of Metastasis 50 4.2 Placement of Initial Particles in Source Image 53 4.3 Generation of GVF-based Force Field in Target Image 55 4.4 Non-Rigid Registration with Particles 58 4.5 Computation of Deformation Field 60 V. Implementation 5.1 Workflow 62 5.2 Neighbor Search 65 5.3 Time Integrator and Time Step 67 5.4 Terminating Condition 69 VI. Results 6.1 Phantom Study 71 6.2 General Observations based on Visual Assessment 73 6.3 Evaluation of Registration Performance 74 6.4 Evaluation of Metastasis Detection Accuracy 77 6.5 Evaluation of Volume Preservation 79 6.6 Parameter Study 80 VII. Conclusion 86 Bibliography 89Docto