NiftySim: A GPU-based nonlinear finite element package for simulation of soft tissue biomechanics

Purpose NiftySim, an open-source finite element toolkit, has been designed to allow incorporation of high-performance soft tissue simulation capabilities into biomedical applications. The toolkit provides the option of execution on fast graphics processing unit (GPU) hardware, numerous constitutive models and solid-element options, membrane and shell elements, and contact modelling facilities, in a simple to use library. Methods The toolkit is founded on the total Lagrangian explicit dynamics (TLEDs) algorithm, which has been shown to be efficient and accurate for simulation of soft tissues. The base code is written in C ++++ , and GPU execution is achieved using the nVidia CUDA framework. In most cases, interaction with the underlying solvers can be achieved through a single Simulator class, which may be embedded directly in third-party applications such as, surgical guidance systems. Advanced capabilities such as contact modelling and nonlinear constitutive models are also provided, as are more experimental technologies like reduced order modelling. A consistent description of the underlying solution algorithm, its implementation with a focus on GPU execution, and examples of the toolkit’s usage in biomedical applications are provided. Results Efficient mapping of the TLED algorithm to parallel hardware results in very high computational performance, far exceeding that available in commercial packages. Conclusion The NiftySim toolkit provides high-performance soft tissue simulation capabilities using GPU technology for biomechanical simulation research applications in medical image computing, surgical simulation, and surgical guidance applications

Comparing Regularized Kelvinlet Functions and the Finite Element Method for Registration of Medical Images to Sparse Organ Data

Image-guided surgery collocates patient-specific data with the physical environment to facilitate surgical decision making in real-time. Unfortunately, these guidance systems commonly become compromised by intraoperative soft-tissue deformations. Nonrigid image-to-physical registration methods have been proposed to compensate for these deformations, but intraoperative clinical utility requires compatibility of these techniques with data sparsity and temporal constraints in the operating room. While linear elastic finite element models are effective in sparse data scenarios, the computation time for finite element simulation remains a limitation to widespread deployment. This paper proposes a registration algorithm that uses regularized Kelvinlets, which are analytical solutions to linear elasticity in an infinite domain, to overcome these barriers. This algorithm is demonstrated and compared to finite element-based registration on two datasets: a phantom dataset representing liver deformations and an in vivo dataset representing breast deformations. The regularized Kelvinlets algorithm resulted in a significant reduction in computation time compared to the finite element method. Accuracy as evaluated by target registration error was comparable between both methods. Average target registration errors were 4.6 +/- 1.0 and 3.2 +/- 0.8 mm on the liver dataset and 5.4 +/- 1.4 and 6.4 +/- 1.5 mm on the breast dataset for the regularized Kelvinlets and finite element method models, respectively. This work demonstrates the generalizability of using a regularized Kelvinlets registration algorithm on multiple soft tissue elastic organs. This method may improve and accelerate registration for image-guided surgery applications, and it shows the potential of using regularized Kelvinlets solutions on medical imaging data.Comment: 17 pages, 9 figure

Detection and modelling of contacts in explicit finite-element simulation of soft tissue biomechanics

Realistic modelling of soft-tissue biomechanics and mechanical interactions between tissues is an important part of surgical simulation, and may become a valuable asset in surgical image-guidance. Unfortunately, it is also computationally very demanding. Explicit matrix-free FEM solvers have been shown to be a good choice for fast tissue simulation, however little work has been done on contact algorithms for such FEM solvers. This work introduces such an algorithm that is capable of handling the scenarios typically encountered in image-guidance. The responses are computed with an evolution of the Lagrange-multiplier method first used by Taylor and Flanagan in PRONTO 3D with spatio-temporal smoothing heuristics for improved stability with coarser meshes and larger time steps. For contact search, a bounding-volume hierarchy (BVH) capable of identifying self collisions, and which is optimised for the small time steps by reducing the number of bounding-volume refittings between iterations through identification of geometry areas with mostly rigid motion and negligible deformation, is introduced. Further optimisation is achieved by integrating the self-collision criterion in the BVH creation and updating algorithms. The effectiveness of the algorithm is demonstrated on a number of artificial test cases and meshes derived from medical image data

Complexity Reduction in Image-Based Breast Cancer Care

The diversity of malignancies of the breast requires personalized diagnostic and therapeutic decision making in a complex situation. This thesis contributes in three clinical areas: (1) For clinical diagnostic image evaluation, computer-aided detection and diagnosis of mass and non-mass lesions in breast MRI is developed. 4D texture features characterize mass lesions. For non-mass lesions, a combined detection/characterisation method utilizes the bilateral symmetry of the breast s contrast agent uptake. (2) To improve clinical workflows, a breast MRI reading paradigm is proposed, exemplified by a breast MRI reading workstation prototype. Instead of mouse and keyboard, it is operated using multi-touch gestures. The concept is extended to mammography screening, introducing efficient navigation aids. (3) Contributions to finite element modeling of breast tissue deformations tackle two clinical problems: surgery planning and the prediction of the breast deformation in a MRI biopsy device

Deep learning reconstruction of digital breast tomosynthesis images for accurate breast density and patient-specific radiation dose estimation

The two-dimensional nature of mammography makes estimation of the overall breast density challenging, and estimation of the true patient-specific radiation dose impossible. Digital breast tomosynthesis (DBT), a pseudo-3D technique, is now commonly used in breast cancer screening and diagnostics. Still, the severely limited 3rd dimension information in DBT has not been used, until now, to estimate the true breast density or the patient-specific dose. This study proposes a reconstruction algorithm for DBT based on deep learning specifically optimized for these tasks. The algorithm, which we name DBToR, is based on unrolling a proximal-dual optimization method. The proximal operators are replaced with convolutional neural networks and prior knowledge is included in the model. This extends previous work on a deep learning-based reconstruction model by providing both the primal and the dual blocks with breast thickness information, which is available in DBT. Training and testing of the model were performed using virtual patient phantoms from two different sources. Reconstruction performance, and accuracy in estimation of breast density and radiation dose, were estimated, showing high accuracy (density <+/-3%; dose <+/-20%) without bias, significantly improving on the current state-of-the-art. This work also lays the groundwork for developing a deep learning-based reconstruction algorithm for the task of image interpretation by radiologists.Comment: Accepted in Medical Image Analysi

FEA-based simulation of breast deformation in real-time using artificial neural network

Treatment of breast cancer involves two stages: diagnosis and treatment. It is difficult to correlate the imaging results at the two stages because as the patient’s posture changes during treatment, the images captured during diagnosis do not represent the tumor location during the treatment. In the absence of real-time imaging during treatment, the visualization of tumor location is challenging for surgeons. There are many challenges for breast deformation simulation. For example, material properties are very important to simulate the deformation accurately. The simulation speed will decide whether the technology is applicable for clinical use. But because of the limit of hardware, achieving real time simulation is difficult. This thesis focuses on investigating visualization of breast deformation for different patient’s positions. We utilized magnetic resonance imaging (MRI) of a patient collected during diagnosis for this study. This data was preprocessed to form a 3D reconstructed model that was used to run a finite element analysis (FEA) simulation. FEA simulates the deformation of breast tissues for different constraints, such as glandular ratio and gravity angle. However, FEA simulation of such deformation can take a few minutes to as much as 40 minutes to complete using a 8 cores computer. To obtain real-time visualization, we constructed a neural network (NN) model that takes breast gravity angle and glandular / fat ratio (breast material) as input to estimate breast deformation for different patient’s positions offline. This NN is used to predict the deformation of the breast and provide visualization in real-time (5 ms prediction time). To further validate our result, we carried out MRI of a breast phantom in several angles (to mimic various patient postures). We also implemented an iterative technique to estimate material properties. This data was used to simulate breast deformations at different posture angles. A similar approach was implemented to build an NN model. Our results show that NN has the ability to map the gravity direction to the breast shape and tumor location accurately, while, keeping run time to a minimum

Technologies for Biomechanically-Informed Image Guidance of Laparoscopic Liver Surgery

Laparoscopic surgery for liver resection has a number medical advantages over open surgery, but also comes with inherent technical challenges. The surgeon only has a very limited field of view through the imaging modalities routinely employed intra-operatively, laparoscopic video and ultrasound, and the pneumoperitoneum required to create the operating space and gaining access to the organ can significantly deform and displace the liver from its pre-operative configuration. This can make relating what is visible intra-operatively to the pre-operative plan and inferring the location of sub-surface anatomy a very challenging task. Image guidance systems can help overcome these challenges by updating the pre-operative plan to the situation in theatre and visualising it in relation to the position of surgical instruments. In this thesis, I present a series of contributions to a biomechanically-informed image-guidance system made during my PhD. The most recent one is work on a pipeline for the estimation of the post-insufflation configuration of the liver by means of an algorithm that uses a database of segmented training images of patient abdomens where the post-insufflation configuration of the liver is known. The pipeline comprises an algorithm for inter and intra-subject registration of liver meshes by means of non-rigid spectral point-correspondence finding. My other contributions are more fundamental and less application specific, and are all contained and made available to the public in the NiftySim open-source finite element modelling package. Two of my contributions to NiftySim are of particular interest with regards to image guidance of laparoscopic liver surgery: 1) a novel general purpose contact modelling algorithm that can be used to simulate contact interactions between, e.g., the liver and surrounding anatomy; 2) membrane and shell elements that can be used to, e.g., simulate the Glisson capsule that has been shown to significantly influence the organ’s measured stiffness

Modified mass-spring system for physically based deformation modeling

Mass-spring systems are considered the simplest and most intuitive of all deformable models. They are computationally efficient, and can handle large deformations with ease. But they suffer several intrinsic limitations. In this book a modified mass-spring system for physically based deformation modeling that addresses the limitations and solves them elegantly is presented. Several implementations in modeling breast mechanics, heart mechanics and for elastic images registration are presented

Proceedings of the International Workshop on Medical Ultrasound Tomography: 1.- 3. Nov. 2017, Speyer, Germany

Ultrasound Tomography is an emerging technology for medical imaging that is quickly approaching its clinical utility. Research groups around the globe are engaged in research spanning from theory to practical applications. The International Workshop on Medical Ultrasound Tomography (1.-3. November 2017, Speyer, Germany) brought together scientists to exchange their knowledge and discuss new ideas and results in order to boost the research in Ultrasound Tomography

Modified mass-spring system for physically based deformation modeling

Mass-spring systems are considered the simplest and most intuitive of all deformable models. They are computationally efficient, and can handle large deformations with ease. But they suffer several intrinsic limitations. In this book a modified mass-spring system for physically based deformation modeling that addresses the limitations and solves them elegantly is presented. Several implementations in modeling breast mechanics, heart mechanics and for elastic images registration are presented