Surface reconstruction for planning and navigation of liver resections

AbstractComputer-assisted systems for planning and navigation of liver resection procedures rely on the use of patient-specific 3D geometric models obtained from computed tomography. In this work, we propose the application of Poisson surface reconstruction (PSR) to obtain 3D models of the liver surface with applications to planning and navigation of liver surgery. In order to apply PSR, the introduction of an efficient transformation of the segmentation data, based on computation of gradient fields, is proposed. One of the advantages of PSR is that it requires only one control parameter, allowing the process to be fully automatic once the optimal value is estimated. Validation of our results is performed via comparison with 3D models obtained by state-of-art Marching Cubes incorporating Laplacian smoothing and decimation (MCSD). Our results show that PSR provides smooth liver models with better accuracy/complexity trade-off than those obtained by MCSD. After estimating the optimal parameter, automatic reconstruction of liver surfaces using PSR is achieved keeping similar processing time as MCSD. Models from this automatic approach show an average reduction of 79.59% of the polygons compared to the MCSD models presenting similar smoothness properties. Concerning visual quality, on one hand, and despite this reduction in polygons, clinicians perceive the quality of automatic PSR models to be the same as complex MCSD models. On the other hand, clinicians perceive a significant improvement on visual quality for automatic PSR models compared to optimal (obtained in terms of accuracy/complexity) MCSD models. The median reconstruction error using automatic PSR was as low as 1.03±0.23mm, which makes the method suitable for clinical applications. Automatic PSR is currently employed at Oslo University Hospital to obtain patient-specific liver models in selected patients undergoing laparoscopic liver resection

CAD-Estate: Large-scale CAD Model Annotation in RGB Videos

We propose a method for annotating videos of complex multi-object scenes with a globally-consistent 3D representation of the objects. We annotate each object with a CAD model from a database, and place it in the 3D coordinate frame of the scene with a 9-DoF pose transformation. Our method is semi-automatic and works on commonly-available RGB videos, without requiring a depth sensor. Many steps are performed automatically, and the tasks performed by humans are simple, well-specified, and require only limited reasoning in 3D. This makes them feasible for crowd-sourcing and has allowed us to construct a large-scale dataset by annotating real-estate videos from YouTube. Our dataset CAD-Estate offers 101k instances of 12k unique CAD models placed in the 3D representations of 20k videos. In comparison to Scan2CAD, the largest existing dataset with CAD model annotations on real scenes, CAD-Estate has 7x more instances and 4x more unique CAD models. We showcase the benefits of pre-training a Mask2CAD model on CAD-Estate for the task of automatic 3D object reconstruction and pose estimation, demonstrating that it leads to performance improvements on the popular Scan2CAD benchmark. The dataset is available at https://github.com/google-research/cad-estate.Comment: Project page: https://github.com/google-research/cad-estat

A parametric-assisted method for 3D generation of as-built BIM models for the built heritage

The paper outlines a parametric-assisted method for the 3D reconstruction and creation of BIM models for the built heritage. The research implements the emerging paradigms of open sourcing, cloud computing and interoperability, employing low-cost technologies (digital photogrammetry) and open source software (Grasshopper for Rhinoceros) which can ease the accessibility to a potential reuse of heritage, typically requiring high specialists and expensive equipment. The research examines the abandoned Albergo Diurno “Venezia” in Milan, heritage with a unique architectural value – a blend of Liberty and Art Deco styles. The process of 3D reconstruction of the ceiling is described. Custom algorithms have been developed to automatically rebuild the complex and irregular geometry from mesh, towards the creation of a NURBS-based 3D model. It is shown how the proposed methodology can streamline the process of data elaboration by reducing arbitrary operations and improve accuracy to preserve geometric irregularities. The associative model allows the automatic improvement in the model definition when more precise input data is feeding the algorithm, offering the opportunity to relate the precision of BIM models in accordance with the needed level of detail (LOD)

Angular and linear measurements of adult flexible flatfoot via weight-bearing CT scans and 3D bone reconstruction tools

open10noThis study was partially funded by the Italian Ministry of Economy and Finance, program “5 per mille”.Acquired adult flatfoot is a frequent deformity which implies multiple, complex and combined 3D modifications of the foot skeletal structure. The difficult thorough evaluation of the degree of severity pre-op and the corresponding assessment post-op can now be overcome by cone-beam (CBCT) technology, which can provide access to the 3D skeletal structure in weight-bearing. This study aims to report flatfoot deformities originally in 3D and in weight-bearing, with measurements taken using two different bone segmentation techniques. 21 such patients, with indication for surgical corrections, underwent CBCT (Carestream, US) while standing on one leg. From these scans, 3D models of each bone of the foot were reconstructed by using two different state-of-the-art segmentation tools: a semi-automatic (Mimics Innovation Suite, Materialise, Belgium), and an automatic (Bonelogic Ortho Foot and Ankle, Disior, Finland). From both reconstructed models, Principal Component Analysis was used to define anatomical reference frames, and original foot and ankle angles and other parameters were calculated mostly based on the longitudinal axis of the bones, in anatomical plane projections and in 3D. Both bone model reconstructions revealed a considerable valgus of the calcareous, plantarflexion and internal rotation of the talus, and typical Meary’s angles in the lateral and transverse plane projections. The mean difference from these angles between semi-automatic and automatic segmentations was larger than 3.5 degrees for only 3 of the 32 measurements, and a large number of these differences were not statistically significant. CBCT and the present techniques for bone shape reconstruction finally provide a novel and valuable 3D assessment of complex foot deformities in weight-bearing, eliminating previous limitations associated to unloaded feet and bidimensional measures. Corresponding measurements on the bone models from the two segmentation tools compared well. Other more representative measurements can be defined in the future using CBCT and these techniques.openOrtolani M.; Leardini A.; Pavani C.; Scicolone S.; Girolami M.; Bevoni R.; Lullini G.; Durante S.; Berti L.; Belvedere C.Ortolani M.; Leardini A.; Pavani C.; Scicolone S.; Girolami M.; Bevoni R.; Lullini G.; Durante S.; Berti L.; Belvedere C

Automated recovery of 3D models of plant shoots from multiple colour images

Increased adoption of the systems approach to biological research has focussed attention on the use of quantitative models of biological objects. This includes a need for realistic 3D representations of plant shoots for quantification and modelling. Previous limitations in single or multi-view stereo algorithms have led to a reliance on volumetric methods or expensive hardware to record plant structure. We present a fully automatic approach to image-based 3D plant reconstruction that can be achieved using a single low-cost camera. The reconstructed plants are represented as a series of small planar sections that together model the more complex architecture of the leaf surfaces. The boundary of each leaf patch is refined using the level set method, optimising the model based on image information, curvature constraints and the position of neighbouring surfaces. The reconstruction process makes few assumptions about the nature of the plant material being reconstructed, and as such is applicable to a wide variety of plant species and topologies, and can be extended to canopy-scale imaging. We demonstrate the effectiveness of our approach on datasets of wheat and rice plants, as well as a novel virtual dataset that allows us to compute quantitative measures of reconstruction accuracy. The output is a 3D mesh structure that is suitable for modelling applications, in a format that can be imported in the majority of 3D graphics and software packages

Shape: A 3D Modeling Tool for Astrophysics

We present a flexible interactive 3D morpho-kinematical modeling application for astrophysics. Compared to other systems, our application reduces the restrictions on the physical assumptions, data type and amount that is required for a reconstruction of an object's morphology. It is one of the first publicly available tools to apply interactive graphics to astrophysical modeling. The tool allows astrophysicists to provide a-priori knowledge about the object by interactively defining 3D structural elements. By direct comparison of model prediction with observational data, model parameters can then be automatically optimized to fit the observation. The tool has already been successfully used in a number of astrophysical research projects.Comment: 13 pages, 11 figures, accepted for publication in the "IEEE Transactions on Visualization and Computer Graphics

Methodology for automatic recovering of 3D partitions from unstitched faces of non-manifold CAD models

Data exchanges between different software are currently used in industry to speed up the preparation of digital prototypes for Finite Element Analysis (FEA). Unfortunately, due to data loss, the yield of the transfer of manifold models rarely reaches 1. In the case of non-manifold models, the transfer results are even less satisfactory. This is particularly true for partitioned 3D models: during the data transfer based on the well-known exchange formats, all 3D partitions are generally lost. Partitions are mainly used for preparing mesh models required for advanced FEA: mapped meshing, material separation, definition of specific boundary conditions, etc. This paper sets up a methodology to automatically recover 3D partitions from exported non-manifold CAD models in order to increase the yield of the data exchange. Our fully automatic approach is based on three steps. First, starting from a set of potentially disconnected faces, the CAD model is stitched. Then, the shells used to create the 3D partitions are recovered using an iterative propagation strategy which starts from the so-called manifold vertices. Finally, using the identified closed shells, the 3D partitions can be reconstructed. The proposed methodology has been validated on academic as well as industrial examples.This work has been carried out under a research contract between the Research and Development Direction of the EDF Group and the Arts et Métiers ParisTech Aix-en-Provence