Multi-Material Mesh Representation of Anatomical Structures for Deep Brain Stimulation Planning

The Dual Contouring algorithm (DC) is a grid-based process used to generate surface meshes from volumetric data. However, DC is unable to guarantee 2-manifold and watertight meshes due to the fact that it produces only one vertex for each grid cube. We present a modified Dual Contouring algorithm that is capable of overcoming this limitation. The proposed method decomposes an ambiguous grid cube into a set of tetrahedral cells and uses novel polygon generation rules that produce 2-manifold and watertight surface meshes with good-quality triangles. These meshes, being watertight and 2-manifold, are geometrically correct, and therefore can be used to initialize tetrahedral meshes. The 2-manifold DC method has been extended into the multi-material domain. Due to its multi-material nature, multi-material surface meshes will contain non-manifold elements along material interfaces or shared boundaries. The proposed multi-material DC algorithm can (1) generate multi-material surface meshes where each material sub-mesh is a 2-manifold and watertight mesh, (2) preserve the non-manifold elements along the material interfaces, and (3) ensure that the material interface or shared boundary between materials is consistent. The proposed method is used to generate multi-material surface meshes of deep brain anatomical structures from a digital atlas of the basal ganglia and thalamus. Although deep brain anatomical structures can be labeled as functionally separate, they are in fact continuous tracts of soft tissue in close proximity to each other. The multi-material meshes generated by the proposed DC algorithm can accurately represent the closely-packed deep brain structures as a single mesh consisting of multiple material sub-meshes. Each sub-mesh represents a distinct functional structure of the brain. Printed and/or digital atlases are important tools for medical research and surgical intervention. While these atlases can provide guidance in identifying anatomical structures, they do not take into account the wide variations in the shape and size of anatomical structures that occur from patient to patient. Accurate, patient-specific representations are especially important for surgical interventions like deep brain stimulation, where even small inaccuracies can result in dangerous complications. The last part of this research effort extends the discrete deformable 2-simplex mesh into the multi-material domain where geometry-based internal forces and image-based external forces are used in the deformation process. This multi-material deformable framework is used to segment anatomical structures of the deep brain region from Magnetic Resonance (MR) data

Preserving Topology and Elasticity for Embedded Deformable Models

International audienceIn this paper we introduce a new approach for the embedding of linear elastic deformable models. Our technique results in significant improvements in the efficient physically based simulation of highly detailed objects. First, our embedding takes into account topological details, that is, disconnected parts that fall into the same coarse element are simulated independently. Second, we account for the varying material properties by computing stiffness and interpolation functions for coarse elements which accurately approximate the behaviour of the embedded material. Finally, we also take into account empty space in the coarse embeddings, which provides a better simulation of the boundary. The result is a straightforward approach to simulating complex deformable models with the ease and speed associated with a coarse regular embedding, and with a quality of detail that would only be possible at much finer resolution

Appearance Preserving Rendering of Out-of-Core Polygon and NURBS Models

In Computer Aided Design (CAD) trimmed NURBS surfaces are widely used due to their flexibility. For rendering and simulation however, piecewise linear representations of these objects are required. A relatively new field in CAD is the analysis of long-term strain tests. After such a test the object is scanned with a 3d laser scanner for further processing on a PC. In all these areas of CAD the number of primitives as well as their complexity has grown constantly in the recent years. This growth is exceeding the increase of processor speed and memory size by far and posing the need for fast out-of-core algorithms. This thesis describes a processing pipeline from the input data in the form of triangular or trimmed NURBS models until the interactive rendering of these models at high visual quality. After discussing the motivation for this work and introducing basic concepts on complex polygon and NURBS models, the second part of this thesis starts with a review of existing simplification and tessellation algorithms. Additionally, an improved stitching algorithm to generate a consistent model after tessellation of a trimmed NURBS model is presented. Since surfaces need to be modified interactively during the design phase, a novel trimmed NURBS rendering algorithm is presented. This algorithm removes the bottleneck of generating and transmitting a new tessellation to the graphics card after each modification of a surface by evaluating and trimming the surface on the GPU. To achieve high visual quality, the appearance of a surface can be preserved using texture mapping. Therefore, a texture mapping algorithm for trimmed NURBS surfaces is presented. To reduce the memory requirements for the textures, the algorithm is modified to generate compressed normal maps to preserve the shading of the original surface. Since texturing is only possible, when a parametric mapping of the surface - requiring additional memory - is available, a new simplification and tessellation error measure is introduced that preserves the appearance of the original surface by controlling the deviation of normal vectors. The preservation of normals and possibly other surface attributes allows interactive visualization for quality control applications (e.g. isophotes and reflection lines). In the last part out-of-core techniques for processing and rendering of gigabyte-sized polygonal and trimmed NURBS models are presented. Then the modifications necessary to support streaming of simplified geometry from a central server are discussed and finally and LOD selection algorithm to support interactive rendering of hard and soft shadows is described

Segmentation and Deformable Modelling Techniques for a Virtual Reality Surgical Simulator in Hepatic Oncology

Liver surgical resection is one of the most frequently used curative therapies. However, resectability is problematic. There is a need for a computer-assisted surgical planning and simulation system which can accurately and efficiently simulate the liver, vessels and tumours in actual patients. The present project describes the development of these core segmentation and deformable modelling techniques. For precise detection of irregularly shaped areas with indistinct boundaries, the segmentation incorporated active contours - gradient vector flow (GVF) snakes and level sets. To improve efficiency, a chessboard distance transform was used to replace part of the GVF effort. To automatically initialize the liver volume detection process, a rotating template was introduced to locate the starting slice. For shape maintenance during the segmentation process, a simplified object shape learning step was introduced to avoid occasional significant errors. Skeletonization with fuzzy connectedness was used for vessel segmentation. To achieve real-time interactivity, the deformation regime of this system was based on a single-organ mass-spring system (MSS), which introduced an on-the-fly local mesh refinement to raise the deformation accuracy and the mesh control quality. This method was now extended to a multiple soft-tissue constraint system, by supplementing it with an adaptive constraint mesh generation. A mesh quality measure was tailored based on a wide comparison of classic measures. Adjustable feature and parameter settings were thus provided, to make tissues of interest distinct from adjacent structures, keeping the mesh suitable for on-line topological transformation and deformation. More than 20 actual patient CT and 2 magnetic resonance imaging (MRI) liver datasets were tested to evaluate the performance of the segmentation method. Instrument manipulations of probing, grasping, and simple cutting were successfully simulated on deformable constraint liver tissue models. This project was implemented in conjunction with the Division of Surgery, Hammersmith Hospital, London; the preliminary reality effect was judged satisfactory by the consultant hepatic surgeon

Efficient Liquid Animation: New Discretizations for Spatially Adaptive Liquid Viscosity and Reduced-Model Two-Phase Bubbles and Inviscid Liquids

The work presented in this thesis focuses on improving the computational efficiency when simulating viscous liquids and air bubbles immersed in liquids by designing new discretizations to focus computational effort in regions that meaningfully contribute to creating realistic motion. For example, when simulating air bubbles rising through a liquid, the entire bubble volume is traditionally simulated despite the bubble’s interior being visually unimportant. We propose our constraint bubbles model to avoid simulating the interior of the bubble volume by reformulating the usual incompressibility constraint throughout a bubble volume as a constraint over only the bubble’s surface. Our constraint method achieves qualitatively similar results compared to a two-phase simulation ground-truth for bubbles with low densities (e.g., air bubbles in water). For bubbles with higher densities, we propose our novel affine regions to model the bubble’s entire velocity field with a single affine vector field. We demonstrate that affine regions can correctly achieve hydrostatic equilibrium for bubble densities that match the surrounding liquid and correctly sink for higher densities. Finally, we introduce a tiled approach to subdivide large-scale affine regions into smaller subregions. Using this strategy, we are able to accelerate single-phase free surface flow simulations, offering a novel approach to adaptively enforce incompressibility in free surface liquids without complex data structures. While pressure forces are often the bottleneck for inviscid fluid simulations, viscosity can impose orders of magnitude greater computational costs. We observed that viscous liquids require high simulation resolution at the surface to capture detailed viscous buckling and rotational motion but, because viscosity dampens relative motion, do not require the same resolution in the liquid’s interior. We therefore propose a novel adaptive method to solve free surface viscosity equations by discretizing the variational finite difference approach of Batty and Bridson (2008) on an octree grid. Our key insight is that the variational method guarantees a symmetric positive definite linear system by construction, allowing the use of fast numerical solvers like the Conjugate Gradients method. By coarsening simulation grid cells inside the liquid volume, we rapidly reduce the degrees-of-freedom in the viscosity linear system up to a factor of 7.7x and achieve performance improvements for the linear solve between 3.8x and 9.4x compared to a regular grid equivalent. The results of our adaptive method closely match an equivalent regular grid for common scenarios such as: rotation and bending, buckling and folding, and solid-liquid interactions

Non-Uniform Rational B-Splines and Rational Bezier Triangles for Isogeometric Analysis of Structural Applications

Isogeometric Analysis (IGA) is a major advancement in computational analysis that bridges the gap between a computer-aided design (CAD) model, which is typically constructed using Non-Uniform Rational B-splines (NURBS), and a computational model that traditionally uses Lagrange polynomials to represent the geometry and solution variables. In IGA, the same shape functions that are used in CAD are employed for analysis. The direct manipulation of CAD data eliminates approximation errors that emanate from the process of converting the geometry from CAD to Finite Element Analysis (FEA). As a result, IGA allows the exact geometry to be represented at the coarsest level and maintained throughout the analysis process. While IGA was initially introduced to streamline the design and analysis process, this dissertation shows that IGA can also provide improved computational results for complex and highly nonlinear problems in structural mechanics. This dissertation addresses various problems in structural mechanics in the context of IGA, with the use of NURBS and rational Bézier triangles for the description of the parametric and physical spaces. The approaches considered here show that a number of important properties (e.g., high-order smoothness, geometric exactness, reduced number of degrees of freedom, and increased flexibility in discretization) can be achieved, leading to improved numerical solutions. Specifically, using B-splines and a layer-based discretization, a distributed plasticity isogeometric frame model is formulated to capture the spread of plasticity in large-deformation frames. The modeling approach includes an adaptive analysis where the structure of interest is initially modeled with coarse mesh and knots are inserted based on the yielding information at the quadrature points. It is demonstrated that improvement on efficiency and convergence rates is attained. With NURBS, an isogeometric rotation-free multi-layered plate formulation is developed based on a layerwise deformation theory. The derivation assumes a separate displacement field expansion within each layer, and considers transverse displacement component as C0-continuous at dissimilar material interfaces, which is enforced via knot repetition. The separate integration of the in-plane and through-thickness directions allows to capture the complete 3D stresses in a 2D setting. The proposed method is used to predict the behavior of advanced materials such as laminated composites, and the results show advantages in efficiency and accuracy. To increase the flexibility in discretizing complex geometries, rational Bézier triangles for domain triangulation is studied. They are further coupled with a Delaunay-based feature-preserving discretization algorithm for static bending and free vibration analysis of Kirchhoff plates. Lagrange multipliers are employed to explicitly impose high-order continuity constraints and the augmented system is solved iteratively without increasing the matrix size. The resulting discretization is geometrically exact, admits small geometric features, and constitutes C1-continuity. The feature-preserving rational Bézier triangles are further applied to smeared damage modeling of quasi-brittle materials. Due to the ability of Lagrange multipliers to raise global continuity to any desired order, the implicit fourth- and sixth-order gradient damage models are analyzed. The inclusion of higher-order terms in the nonlocal Taylor expansion improves solution accuracy. A local refinement algorithm that resolves marked regions with high resolution while keeping the resulting mesh conforming and well-conditioned is also utilized to improve efficiency. The outcome is a unified modeling framework where the feature-preserving discretization is able to capture the damage initiation and early-stage propagation, and the local refinement technique can then be applied to adaptively refine the mesh in the direction of damage propagation.PHDCivil EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147668/1/ningliu_1.pd

Essential techniques for improving visual realism of laparoscopic surgery simulation.

With the prevalence of laparoscopic surgery, the request for reliable training and assessment is becoming increasingly important. The traditional way of training is both time consuming and cost intensive, and may cause ethical or moral issues. With the development of computer technologies, virtual reality has entered the world of consumer electronics as a new way to enhance tactile and visual sensory experiences. Virtual reality based surgical skill training gradually becomes an effective supplementary to the traditional laparoscopic skill training in many surgical theatres. To provide high fidelity virtual surgery training experiences, the presentation of the virtual world should have the same level of realism as what surgeons see and feel during real operations. However, the weak computing power limits the potential level of details on the graphics presentation and physical behaviour of virtual objects, which will further influence the fidelity of tactile interaction. Achieving visual realism (realistic graphics presentation and accurate physical behaviour) and good user experience using limited computing resources is the main challenge for laparoscopic surgery simulation. The topic of visual realism in laparoscopic surgery simulation has not been well researched. This topic mainly relates to the area of 3D anatomy modeling, soft body simulation and rendering. Current researches in computer graphics and game communities are not tailored for laparoscopic surgery simulation. The direct use of those techniques in developing surgery simulators will often result in poor quality anatomy model, inaccurate simulation, low fidelity visual effect, poor user experience and inefficient production pipeline, which significantly influence the visual realism of the virtual world. The development of laparoscopic surgery simulator is an interdiscipline of computer graphics, computational physics and haptics. However, current researches barely focus on the study of tailored techniques and efficient production pipeline which often result in the long term research cycle and daunting cost for simulator development. This research is aiming at improving the visual realism of laparoscopic surgery simulation from the perspective of computer graphics. In this research, a set of tailor techniques have been proposed to improve the visual realism for laparoscopic surgery simulation. For anatomy modeling, an automatic and efficient 3D anatomy conversion pipeline is proposed which can convert bad quality 3D anatomy into simulation ready state while preserving the original model’s surface parameterization property. For simulation, a soft tissue simulation pipeline is pro- posed which can provide multi-layer heterogeneous soft tissue modeling and intuitive physically editable simulation based on uniform polynomial based hyperelastic material representation. For interaction, a collision detection and interaction system based on adaptive circumphere structure is proposed which supports robust and efficient sliding con- tact, energized dissection and clip. For rendering, a multi-layer soft tissue rendering pipeline is proposed which decomposed the multi-layer structure of soft tissue into corresponding material asset required by state-of-art rendering techniques. Based on this research, a system framework for building a laparoscopic surgery simulator is also proposed to test the feasibility of those tailored techniques

Hierarchical processing, editing and rendering of acquired geometry

La représentation des surfaces du monde réel dans la mémoire d’une machine peut désormais être obtenue automatiquement via divers périphériques de capture tels que les scanners 3D. Ces nouvelles sources de données, précises et rapides, amplifient de plusieurs ordres de grandeur la résolution des surfaces 3D, apportant un niveau de précision élevé pour les applications nécessitant des modèles numériques de surfaces telles que la conception assistée par ordinateur, la simulation physique, la réalité virtuelle, l’imagerie médicale, l’architecture, l’étude archéologique, les effets spéciaux, l’animation ou bien encore les jeux video. Malheureusement, la richesse de la géométrie produite par ces méthodes induit une grande, voire gigantesque masse de données à traiter, nécessitant de nouvelles structures de données et de nouveaux algorithmes capables de passer à l’échelle d’objets pouvant atteindre le milliard d’échantillons. Dans cette thèse, je propose des solutions performantes en temps et en espace aux problèmes de la modélisation, du traitement géométrique, de l’édition intéractive et de la visualisation de ces surfaces 3D complexes. La méthodologie adoptée pendant l’élaboration transverse de ces nouveaux algorithmes est articulée autour de 4 éléments clés : une approche hiérarchique systématique, une réduction locale de la dimension des problèmes, un principe d’échantillonage-reconstruction et une indépendance à l’énumération explicite des relations topologiques aussi appelée approche basée-points. En pratique, ce manuscrit propose un certain nombre de contributions, parmi lesquelles : une nouvelle structure hiérarchique hybride de partitionnement, l’Arbre Volume-Surface (VS-Tree) ainsi que de nouveaux algorithmes de simplification et de reconstruction ; un système d’édition intéractive de grands objets ; un noyau temps-réel de synthèse géométrique par raffinement et une structure multi-résolution offrant un rendu efficace de grands objets. Ces structures, algorithmes et systèmes forment une chaîne capable de traiter les objets en provenance du pipeline d’acquisition, qu’ils soient représentés par des nuages de points ou des maillages, possiblement non 2-variétés. Les solutions obtenues ont été appliquées avec succès aux données issues des divers domaines d’application précités.Digital representations of real-world surfaces can now be obtained automatically using various acquisition devices such as 3D scanners and stereo camera systems. These new fast and accurate data sources increase 3D surface resolution by several orders of magnitude, borrowing higher precision to applications which require digital surfaces. All major computer graphics applications can take benefit of this automatic modeling process, including: computer-aided design, physical simulation, virtual reality, medical imaging, architecture, archaeological study, special effects, computer animation and video games. Unfortunately, the richness of the geometry produced by these media comes at the price of a large, possibility gigantic, amount of data which requires new efficient data structures and algorithms offering scalability for processing such objects. This thesis proposes time and space efficient solutions for modeling, editing and rendering such complex surfaces, solving these problems with new algorithms sharing 4 fundamental elements: a systematic hierarchical approach, a local dimension reduction, a sampling-reconstruction paradigm and a pointbased basis. Basically, this manuscript proposes several contributions, including: a new hierarchical space subdivision structure, the Volume-Surface Tree, for geometry processing such as simplification and reconstruction; a streaming system featuring new algorithms for interactive editing of large objects, an appearancepreserving multiresolution structure for efficient rendering of large point-based surfaces, and a generic kernel for real-time geometry synthesis by refinement. These elements form a pipeline able to process acquired geometry, either represented by point clouds or non-manifold meshes. Effective results have been successfully obtained with data coming from the various applications mentioned

Representation and coding of 3D video data

Livrable D4.1 du projet ANR PERSEECe rapport a été réalisé dans le cadre du projet ANR PERSEE (n° ANR-09-BLAN-0170). Exactement il correspond au livrable D4.1 du projet