Survey of Finite Element Method-Based Real-Time Simulations

The finite element method (FEM) has deservedly gained the reputation of the most powerful, highly efficient, and versatile numerical method in the field of structural analysis. Though typical application of FE programs implies the so-called “off-line” computations, the rapid pace of hardware development over the past couple of decades was the major impetus for numerous researchers to consider the possibility of real-time simulation based on FE models. Limitations of available hardware components in various phases of developments demanded remarkable innovativeness in the quest for suitable solutions to the challenge. Different approaches have been proposed depending on the demands of the specific field of application. Though it is still a relatively young field of work in global terms, an immense amount of work has already been done calling for a representative survey. This paper aims to provide such a survey, which of course cannot be exhaustive

Predicting the Cancer Tumor Position in Liver Using Finite Element Analysis (FEA) and Artificial Intelligence (AI)

The computational power and advantages of the Finite Element Method (FEM) are noticeable. When dealing with high nonlinearity of the materials and geometrical complexity, FEM is a powerful solution, depending on the correct definition of the problem. The availability of this method has benefited many engineering areas. In the field of biomechanics and, more specifically, in Computer-Assisted Surgery, FEM is even more appreciated. This approach, however, comes at a high computational cost. Thus, a significant delay in the response impedes its implementation for real-time applications in clinical practices, even by using parallelization or utilizing Graphics Processing Unit (GPU). This is where an alternative approach is needed to accelerate FEM-based simulations to provide the desired outputs and minimizing the time lag, preventing using FEM during intra-operative applications. A novel technique that may help to overcome the obstacles mentioned above and improve the response time is the field of Machine Learning (ML). In particular, the Artificial Neural Network (ANN), as a subset of ML, has demonstrated high potentials in computer vision and pattern recognition, whose implementation can be extended to replace a FEM model once it has been trained with sufficient inputs. In this work, a FEM-ML framework is established to drastically increase the response time for predicting tumor and internal structures’ locations in the human liver for surgical applications by using ANN. This technique takes advantage of the FEM results to train a model capable of capturing large deformations of liver tissue during the surgical intervention while reporting back the nodal locations of the components with high accuracy and efficiency. For doing so, a biomechanical model of the liver, accounting for the effect of the stiffness of blood vessels, is developed, and multiple simulations with random nodal loads on the surface of the liver are conducted in the commercial software Abaqus to produce the input required for the ANN. The ANN then predicts the nodes’ coordinates resulting from the applied forces that can be used to reconstruct the deformed model of the organ

Real-time biomechanical modeling of the liver using Machine Learning models trained on Finite Element Method simulations

[EN] The development of accurate real-time models of the biomechanical behavior of different organs and tissues still poses a challenge in the field of biomechanical engineering. In the case of the liver, specifically, such a model would constitute a great leap forward in the implementation of complex applications such as surgical simulators, computed-assisted surgery or guided tumor irradiation. In this work, a relatively novel approach for developing such a model is presented. It consists in the use of a machine learning algorithm, which provides real-time inference, trained on tens of thousands of simulations of the biomechanical behavior of the liver carried out by the finite element method on more than 100 different liver geometries. Considering a target accuracy threshold of 3 mm for the Euclidean Error, four different scenarios were modeled and assessed: a single liver with an arbitrary force applied (99.96% of samples within the accepted error range), a single liver with two simultaneous forces applied (99.84% samples in range), a single liver with different material properties and an arbitrary force applied (98.46% samples in range), and a much more general model capable of modeling the behavior of any liver with an arbitrary force applied (99.01% samples in range for the median liver). The results show that the Machine Learning models perform extremely well on all the scenarios, managing to keep the Mean Euclidean Error under 1 mm in all cases. Furthermore, the proposed model achieves working frequencies above 100Hz on modest hardware (with frequencies above 1000Hz being easily achievable on more powerful GPUs) thus fulfilling the real-time requirements. These results constitute a remarkable improvement in this field and may involve a prompt implementation in clinical practice.This work has been funded by the Spanish Ministry of Economy and Competitiveness (MINECO) through research projects TIN2014-52033-R, also supported by European FEDER funds.Pellicer-Valero, OJ.; Rupérez Moreno, MJ.; Martinez-Sanchis, S.; Martín-Guerrero, JD. (2020). Real-time biomechanical modeling of the liver using Machine Learning models trained on Finite Element Method simulations. Expert Systems with Applications. 143:1-12. https://doi.org/10.1016/j.eswa.2019.113083S112143Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., Kudlur, M., Levenberg, J., Monga, R., Moore, S., Murray, D. G., Steiner, B., Tucker, P., Vasudevan, V., Warden, P., Wicke, M., Yu, Y., & Zheng, X. (2016). TensorFlow: A system for large-scale machine learning. arXiv:1605.08695.Brunon, A., Bruyère-Garnier, K., & Coret, M. (2010). Mechanical characterization of liver capsule through uniaxial quasi-static tensile tests until failure. Journal of Biomechanics, 43(11), 2221-2227. doi:10.1016/j.jbiomech.2010.03.038Chinesta, F., Leygue, A., Bordeu, F., Aguado, J. V., Cueto, E., Gonzalez, D., … Huerta, A. (2013). PGD-Based Computational Vademecum for Efficient Design, Optimization and Control. Archives of Computational Methods in Engineering, 20(1), 31-59. doi:10.1007/s11831-013-9080-xClifford, M. A., Banovac, F., Levy, E., & Cleary, K. (2002). Assessment of Hepatic Motion Secondary to Respiration for Computer Assisted Interventions. Computer Aided Surgery, 7(5), 291-299. doi:10.3109/10929080209146038Cotin, S., Delingette, H., & Ayache, N. (2000). A hybrid elastic model for real-time cutting, deformations, and force feedback for surgery training and simulation. The Visual Computer, 16(8), 437-452. doi:10.1007/pl00007215Duysak, A., Zhang, J. J., & Ilankovan, V. (2003). Efficient modelling and simulation of soft tissue deformation using mass-spring systems. International Congress Series, 1256, 337-342. doi:10.1016/s0531-5131(03)00423-0Fung, Y. C., & Skalak, R. (1981). Biomechanics: Mechanical Properties of Living Tissues. Journal of Biomechanical Engineering, 103(4), 231-298. doi:10.1115/1.3138285González, D., Aguado, J. V., Cueto, E., Abisset-Chavanne, E., & Chinesta, F. (2016). kPCA-Based Parametric Solutions Within the PGD Framework. Archives of Computational Methods in Engineering, 25(1), 69-86. doi:10.1007/s11831-016-9173-4González, D., Cueto, E., & Chinesta, F. (2015). Computational Patient Avatars for Surgery Planning. Annals of Biomedical Engineering, 44(1), 35-45. doi:10.1007/s10439-015-1362-zJahya, A., Herink, M., & Misra, S. (2013). A framework for predicting three-dimensional prostate deformation in real time. The International Journal of Medical Robotics and Computer Assisted Surgery, 9(4), e52-e60. doi:10.1002/rcs.1493Lister, K., Gao, Z., & Desai, J. P. (2010). Development of In Vivo Constitutive Models for Liver: Application to Surgical Simulation. Annals of Biomedical Engineering, 39(3), 1060-1073. doi:10.1007/s10439-010-0227-8Lorente, D., Martínez-Martínez, F., Rupérez, M. J., Lago, M. A., Martínez-Sober, M., Escandell-Montero, P., … Martín-Guerrero, J. D. (2017). A framework for modelling the biomechanical behaviour of the human liver during breathing in real time using machine learning. Expert Systems with Applications, 71, 342-357. doi:10.1016/j.eswa.2016.11.037Maas, S. A., Ellis, B. J., Ateshian, G. A., & Weiss, J. A. (2012). FEBio: Finite Elements for Biomechanics. Journal of Biomechanical Engineering, 134(1). doi:10.1115/1.4005694Myronenko, A., & Xubo Song. (2010). Point Set Registration: Coherent Point Drift. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(12), 2262-2275. doi:10.1109/tpami.2010.46Niroomandi, S., Alfaro, I., Cueto, E., & Chinesta, F. (2012). Accounting for large deformations in real-time simulations of soft tissues based on reduced-order models. Computer Methods and Programs in Biomedicine, 105(1), 1-12. doi:10.1016/j.cmpb.2010.06.012Plantefève, R., Peterlik, I., Haouchine, N., & Cotin, S. (2015). Patient-Specific Biomechanical Modeling for Guidance During Minimally-Invasive Hepatic Surgery. Annals of Biomedical Engineering, 44(1), 139-153. doi:10.1007/s10439-015-1419-zLarge elastic deformations of isotropic materials. I. Fundamental concepts. (1948). Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 240(822), 459-490. doi:10.1098/rsta.1948.0002Large elastic deformations of isotropic materials IV. further developments of the general theory. (1948). Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 241(835), 379-397. doi:10.1098/rsta.1948.0024Ruder, S. (2016). An overview of gradient descent optimization algorithms. (pp. 1–14). arXiv: 1609.04747.Untaroiu, C. D., & Lu, Y.-C. (2013). Material characterization of liver parenchyma using specimen-specific finite element models. Journal of the Mechanical Behavior of Biomedical Materials, 26, 11-22. doi:10.1016/j.jmbbm.2013.05.013Valanis, K. C., & Landel, R. F. (1967). The Strain‐Energy Function of a Hyperelastic Material in Terms of the Extension Ratios. Journal of Applied Physics, 38(7), 2997-3002. doi:10.1063/1.171003

Efficient techniques for soft tissue modeling and simulation

Performing realistic deformation simulations in real time is a challenging problem in computer graphics. Among numerous proposed methods including Finite Element Modeling and ChainMail, we have implemented a mass spring system because of its acceptable accuracy and speed. Mass spring systems have, however, some drawbacks such as, the determination of simulation coefficients with their iterative nature. Given the correct parameters, mass spring systems can accurately simulate tissue deformations but choosing parameters that capture nonlinear deformation behavior is extremely difficult. Since most of the applications require a large number of elements i. e. points and springs in the modeling process it is extremely difficult to reach realtime performance with an iterative method. We have developed a new parameter identification method based on neural networks. The structure of the mass spring system is modified and neural networks are integrated into this structure. The input space consists of changes in spring lengths and velocities while a "teacher" signal is chosen as the total spring force, which is expressed in terms of positional changes and applied external forces. Neural networks are trained to learn nonlinear tissue characteristics represented by spring stiffness and damping in the mass spring algorithm. The learning algorithm is further enhanced by an adaptive learning rate, developed particularly for mass spring systems. In order to avoid the iterative approach in deformation simulations we have developed a new deformation algorithm. This algorithm defines the relationships between points and springs and specifies a set of rules on spring movements and deformations. These rules result in a deformation surface, which is called the search space. The deformation algorithm then finds the deformed points and springs in the search space with the help of the defined rules. The algorithm also sets rules on each element i. e. triangle or tetrahedron so that they do not pass through each other. The new algorithm is considerably faster than the original mass spring systems algorithm and provides an opportunity for various deformation applications. We have used mass spring systems and the developed method in the simulation of craniofacial surgery. For this purpose, a patient-specific head model was generated from MRI medical data by applying medical image processing tools such as, filtering, the segmentation and polygonal representation of such model is obtained using a surface generation algorithm. Prism volume elements are generated between the skin and bone surfaces so that different tissue layers are included to the head model. Both methods produce plausible results verified by surgeons

A framework for modelling the biomechanical behaviour of the human liver during breathing in real time using machine learning

Progress in biomechanical modelling of human soft tissue is the basis for the development of new clinical applications capable of improving the diagnosis and treatment of some diseases (e.g. cancer), as well as the surgical planning and guidance of some interventions. The finite element method (FEM) is one of the most popular techniques used to predict the deformation of the human soft tissue due to its high accuracy. However, FEM has an associated high computational cost, which makes it difficult its integration in real-time computer-aided surgery systems. An alternative for simulating the mechanical behaviour of human organs in real time comes from the use of machine learning (ML) techniques, which are much faster than FEM. This paper assesses the feasibility of ML methods for modelling the biomechanical behaviour of the human liver during the breathing process, which is crucial for guiding surgeons during interventions where it is critical to track this deformation (e.g. some specific kind of biopsies) or for the accurate application of radiotherapy dose to liver tumours. For this purpose, different ML regression models were investigated, including three tree-based methods (decision trees, random forests and extremely randomised trees) and other two simpler regression techniques (dummy model and linear regression). In order to build and validate the ML models, a labelled data set was constructed from modelling the deformation of eight ex-vivo human livers using FEM. The best prediction performance was obtained using extremely randomised trees, with a mean error of 0.07 mm and all the samples with an error under 1 mm. The achieved results lay the foundation for the future development of some real-time software capable of simulating the human liver deformation during the breathing process during clinical interventions.This work has been funded by the Spanish Ministry of Economy and Competitiveness (MINECO) through research projects TIN2014-52033-R and DPI2013-40859-R, both also supported by European FEDER funds. The authors acknowledge the kind collaboration of the personnel from the hospital involved in the research.Lorente, D.; Martínez-Martínez, F.; Rupérez Moreno, MJ.; Lago, MA.; Martínez-Sober, M.; Escandell-Montero, P.; Martínez-Martínez, JM.... (2017). A framework for modelling the biomechanical behaviour of the human liver during breathing in real time using machine learning. Expert Systems with Applications. 71:342-357. doi:10.1016/j.eswa.2016.11.037S3423577

Real-time Knowledge-based Fuzzy Logic Model for Soft Tissue Deformation

In this research, the improved mass spring model is presented to simulate the human liver deformation. The underlying MSM is redesigned where fuzzy knowledge-based approaches are implemented to determine the stiffness values. Results show that fuzzy approaches are in very good agreement to the benchmark model. The novelty of this research is that for liver deformation in particular, no specific contributions in the literature exist reporting on real-time knowledge-based fuzzy MSM for liver deformation

Nonlinear effects in finite elements analysis of colorectal surgical clamping

Minimal Invasive Surgery (MIS) is a procedure that has increased its applications in past few years in different types of surgeries. As number of application fields are increasing day by day, new issues have been arising. In particular, instruments must be inserted through a trocar to access the abdominal cavity without capability of direct manipulation of tissues, so a loss of sensitivity occurs. Generally speaking, the student of medicine or junior surgeons need a lot of practice hours before starting any surgical procedure, since they have to difficulty in acquiring specific skills (hand–eye coordination among others) for this type of surgery. Here is what the surgical simulator present a promising training method using an approach based on Finite Element Method (FEM). The use of continuum mechanics, especially Finite Element Analysis (FEA) has gained an extensive application in medical field in order to simulate soft tissues. In particular, colorectal simulations can be used to understand the interaction between colon and the surrounding tissues and also between colon and instruments. Although several works have been introduced considering small displacements, FEA applied to colorectal surgical procedures with large displacements is a topic that asks for more investigations. This work aims to investigate how FEA can describe non-linear effects induced by material properties and different approximating geometries, focusing as test-case application colorectal surgery. More in detail, it shows a comparison between simulations that are performed using both linear and hyperelastic models. These different mechanical behaviours are applied on different geometrical models (planar, cylindrical, 3D-SS and a real model from digital acquisitions 3D-S) with the aim of evaluating the effects of geometric non-linearity. Final aim of the research is to provide a preliminary contribution to the simulation of the interaction between surgical instrument and colon tissues with multi-purpose FEA in order to help the preliminary set-up of different bioengineering tasks like force-contact evaluation or approximated modelling for virtual reality (surgical simulations). In particular, the contribution of this work is focused on the sensitivity analysis of the nonlinearities by FEA in the tissue-tool interaction through an explicit FEA solver. By doing in this way, we aim to demonstrate that the set-up of FEA computational surgical tools may be simplified in order to provide assistance to non-expert FEA engineers or medicians in more precise way of using FEA tools