Sketching-out virtual humans: A smart interface for human modelling and animation

In this paper, we present a fast and intuitive interface for sketching out 3D virtual humans and animation. The user draws stick figure key frames first and chooses one for “fleshing-out” with freehand body contours. The system automatically constructs a plausible 3D skin surface from the rendered figure, and maps it onto the posed stick figures to produce the 3D character animation. A “creative model-based method” is developed, which performs a human perception process to generate 3D human bodies of various body sizes, shapes and fat distributions. In this approach, an anatomical 3D generic model has been created with three distinct layers: skeleton, fat tissue, and skin. It can be transformed sequentially through rigid morphing, fatness morphing, and surface fitting to match the original 2D sketch. An auto-beautification function is also offered to regularise the 3D asymmetrical bodies from users’ imperfect figure sketches. Our current system delivers character animation in various forms, including articulated figure animation, 3D mesh model animation, 2D contour figure animation, and even 2D NPR animation with personalised drawing styles. The system has been formally tested by various users on Tablet PC. After minimal training, even a beginner can create vivid virtual humans and animate them within minutes

3D printed ventricular septal defect patch: a primer for the 2015 Radiological Society of North America (RSNA) hands-on course in 3D printing.

Hand-held three dimensional models of the human anatomy and pathology, tailored-made protheses, and custom-designed implants can be derived from imaging modalities, most commonly Computed Tomography (CT). However, standard DICOM format images cannot be 3D printed; instead, additional image post-processing is required to transform the anatomy of interest into Standard Tessellation Language (STL) format is needed. This conversion, and the subsequent 3D printing of the STL file, requires a series of steps. Initial post-processing involves the segmentation-demarcation of the desired for 3D printing parts and creating of an initial STL file. Then, Computer Aided Design (CAD) software is used, particularly for wrapping, smoothing and trimming. Devices and implants that can also be 3D printed, can be designed using this software environment. The purpose of this article is to provide a tutorial on 3D Printing with the test case of complex congenital heart disease (CHD). While the infant was born with double outlet right ventricle (DORV), this hands-on guide to be featured at the 2015 annual meeting of the Radiological Society of North America Hands-on Course in 3D Printing focused on the additional finding of a ventricular septal defect (VSD). The process of segmenting the heart chambers and the great vessels will be followed by optimization of the model using CAD software. A virtual patch that accurately matches the patient's VSD will be designed and both models will be prepared for 3D printing

MeshPipe: a Python-based tool for easy automation and demonstration of geometry processing pipelines

The popularization of inexpensive 3D scanning, 3D printing, 3D publishing and AR/VR display technologies have renewed the interest in open-source tools providing the geometry processing algorithms required to clean, repair, enrich, optimize and modify point-based and polygonal-based models. Nowadays, there is a large variety of such open-source tools whose user community includes 3D experts but also 3D enthusiasts and professionals from other disciplines. In this paper we present a Python-based tool that addresses two major caveats of current solutions: the lack of easy-to-use methods for the creation of custom geometry processing pipelines (automation), and the lack of a suitable visual interface for quickly testing, comparing and sharing different pipelines, supporting rapid iterations and providing dynamic feedback to the user (demonstration). From the user's point of view, the tool is a 3D viewer with an integrated Python console from which internal or external Python code can be executed. We provide an easy-to-use but powerful API for element selection and geometry processing. Key algorithms are provided by a high-level C library exposed to the viewer via Python-C bindings. Unlike competing open-source alternatives, our tool has a minimal learning curve and typical pipelines can be written in a few lines of Python code.Peer ReviewedPostprint (published version

The Topology ToolKit

This system paper presents the Topology ToolKit (TTK), a software platform designed for topological data analysis in scientific visualization. TTK provides a unified, generic, efficient, and robust implementation of key algorithms for the topological analysis of scalar data, including: critical points, integral lines, persistence diagrams, persistence curves, merge trees, contour trees, Morse-Smale complexes, fiber surfaces, continuous scatterplots, Jacobi sets, Reeb spaces, and more. TTK is easily accessible to end users due to a tight integration with ParaView. It is also easily accessible to developers through a variety of bindings (Python, VTK/C++) for fast prototyping or through direct, dependence-free, C++, to ease integration into pre-existing complex systems. While developing TTK, we faced several algorithmic and software engineering challenges, which we document in this paper. In particular, we present an algorithm for the construction of a discrete gradient that complies to the critical points extracted in the piecewise-linear setting. This algorithm guarantees a combinatorial consistency across the topological abstractions supported by TTK, and importantly, a unified implementation of topological data simplification for multi-scale exploration and analysis. We also present a cached triangulation data structure, that supports time efficient and generic traversals, which self-adjusts its memory usage on demand for input simplicial meshes and which implicitly emulates a triangulation for regular grids with no memory overhead. Finally, we describe an original software architecture, which guarantees memory efficient and direct accesses to TTK features, while still allowing for researchers powerful and easy bindings and extensions. TTK is open source (BSD license) and its code, online documentation and video tutorials are available on TTK's website

Anatomical shape reconstruction and manufacturing: solving topological changes of lumen vessel trough geometric approach

Over the last years there has been an increasing growth of interest in Rapid Prototyping (RP) techniques applied to various fields of medicine. RP makes it possible, in vascular surgery, to produce accurate anatomic replicas of patient vessels. These replicas can help the customization of surgical invasive interventions such as in situ stent-graft insertion in carotid region. The main goal of this work is to obtain high quality in lumen reconstruction and manufacturing replicas by RP technique. This goal is achieved through the complete control of each phase of the generating process. We present a semi-automatic method for carotid lumen reconstruction based on Boundary Representation (BRep). All parameters influencing the quality of the shape reconstruction are presented and discussed: shape acquisition, shape reconstruction and shape manufacturing. The shape acquisition starts by extracting the points belonging to the boundary of the lumen vessel, from Computer Tomography (CT) images. These points, parameterised in a vector, are the input data of the shape reconstruction algorithm based on B-Spline interpolation. The B-Spline type for representing curves and surfaces were chosen because of their properties of continuity and local control. In the shape reconstruction stage we had to face problems due to the topological change on the vessel structure. For vessel regions where there are not changes of topology, we use the closed B-Spline curves (belonging to adjacent acquisition planes) as generating curves to build a B-Spline surface. For vessel regions with at least a change of topology (ex. bifurcation region) our algorithm split automatically the involved curves to obtain three rectangular B-Spline patches. Such patches are joined together to obtain the bifurcation vessel lumen. The set of lumen surfaces is then inserted in a Boundary Representation in order to get a valid solid. To analyse the quality of the reconstructed shapes, the final object is compared with the acquisition image. This solid is correctly tessellated in triangles to produce the data format used by the RP devices (STL)

Point Cloud Framework for Rendering 3D Models Using Google Tango

This project seeks to demonstrate the feasibility of point cloud meshing for capturing and modeling three dimensional objects on consumer smart phones and tablets. Traditional methods of capturing objects require hundreds of images, are very slow and consume a large amount of cellular data for the average consumer. Software developers need a starting point for capturing and meshing point clouds to create 3D models as hardware manufacturers provide the tools to capture point cloud data. The project uses Googles Tango computer vision library for Android to capture point clouds on devices with depth-sensing hardware. The point clouds are combined and meshed as models for use in 3D rendering projects. We expect our results to be embraced by the Android market because capturing point clouds is fast and does not carry a large data footprint

A semi-automatic computer-aided method for surgical template design

This paper presents a generalized integrated framework of semi-automatic surgical template design. Several algorithms were implemented including the mesh segmentation, offset surface generation, collision detection, ruled surface generation, etc., and a special software named TemDesigner was developed. With a simple user interface, a customized template can be semi- automatically designed according to the preoperative plan. Firstly, mesh segmentation with signed scalar of vertex is utilized to partition the inner surface from the input surface mesh based on the indicated point loop. Then, the offset surface of the inner surface is obtained through contouring the distance field of the inner surface, and segmented to generate the outer surface. Ruled surface is employed to connect inner and outer surfaces. Finally, drilling tubes are generated according to the preoperative plan through collision detection and merging. It has been applied to the template design for various kinds of surgeries, including oral implantology, cervical pedicle screw insertion, iliosacral screw insertion and osteotomy, demonstrating the efficiency, functionality and generality of our method.Comment: 18 pages, 16 figures, 2 tables, 36 reference

Innovative techniques to devise 3D-printed anatomical brain phantoms for morpho-functional medical imaging

Introduction. The Ph.D. thesis addresses the development of innovative techniques to create 3D-printed anatomical brain phantoms, which can be used for quantitative technical assessments on morpho-functional imaging devices, providing simulation accuracy not obtainable with currently available phantoms. 3D printing (3DP) technology is paving the way for advanced anatomical modelling in biomedical applications. Despite the potential already expressed by 3DP in this field, it is still little used for the realization of anthropomorphic phantoms of human organs with complex internal structures. Making an anthropomorphic phantom is very different from making a simple anatomical model and 3DP is still far from being plug-and-print. Hence, the need to develop ad-hoc techniques providing innovative solutions for the realization of anatomical phantoms with unique characteristics, and greater ease-of-use. Aim. The thesis explores the entire workflow (brain MRI images segmentation, 3D modelling and materialization) developed to prototype a new complex anthropomorphic brain phantom, which can simulate three brain compartments simultaneously: grey matter (GM), white matter (WM) and striatum (caudate nucleus and putamen, known to show a high uptake in nuclear medicine studies). The three separate chambers of the phantom will be filled with tissue-appropriate solutions characterized by different concentrations of radioisotope for PET/SPECT, para-/ferro-magnetic metals for MRI, and iodine for CT imaging. Methods. First, to design a 3D model of the brain phantom, it is necessary to segment MRI images and to extract an error-less STL (Standard Tessellation Language) description. Then, it is possible to materialize the prototype and test its functionality. - Image segmentation. Segmentation is one of the most critical steps in modelling. To this end, after demonstrating the proof-of-concept, a multi-parametric segmentation approach based on brain relaxometry was proposed. It includes a pre-processing step to estimate relaxation parameter maps (R1 = longitudinal relaxation rate, R2 = transverse relaxation rate, PD = proton density) from the signal intensities provided by MRI sequences of routine clinical protocols (3D-GrE T1-weighted, FLAIR and fast-T2-weighted sequences with ≤ 3 mm slice thickness). In the past, maps of R1, R2, and PD were obtained from Conventional Spin Echo (CSE) sequences, which are no longer suitable for clinical practice due to long acquisition times. Rehabilitating the multi-parametric segmentation based on relaxometry, the estimation of pseudo-relaxation maps allowed developing an innovative method for the simultaneous automatic segmentation of most of the brain structures (GM, WM, cerebrospinal fluid, thalamus, caudate nucleus, putamen, pallidus, nigra, red nucleus and dentate). This method allows the segmentation of higher resolution brain images for future brain phantom enhancements. - STL extraction. After segmentation, the 3D model of phantom is described in STL format, which represents the shapes through the approximation in manifold mesh (i.e., collection of triangles, which is continuous, without holes and with a positive – not zero – volume). For this purpose, we developed an automatic procedure to extract a single voxelized surface, tracing the anatomical interface between the phantom's compartments directly on the segmented images. Two tubes were designed for each compartment (one for filling and the other to facilitate the escape of air). The procedure automatically checks the continuity of the surface, ensuring that the 3D model could be exported in STL format, without errors, using a common image-to-STL conversion software. Threaded junctions were added to the phantom (for the hermetic closure) using a mesh processing software. The phantom's 3D model resulted correct and ready for 3DP. Prototyping. Finally, the most suitable 3DP technology is identified for the materialization. We investigated the material extrusion technology, named Fused Deposition Modeling (FDM), and the material jetting technology, named PolyJet. FDM resulted the best candidate for our purposes. It allowed materializing the phantom's hollow compartments in a single print, without having to print them in several parts to be reassembled later. FDM soluble internal support structures were completely removable after the materialization, unlike PolyJet supports. A critical aspect, which required a considerable effort to optimize the printing parameters, was the submillimetre thickness of the phantom walls, necessary to avoid distorting the imaging simulation. However, 3D printer manufacturers recommend maintaining a uniform wall thickness of at least 1 mm. The optimization of printing path made it possible to obtain strong, but not completely waterproof walls, approximately 0.5 mm thick. A sophisticated technique, based on the use of a polyvinyl-acetate solution, was developed to waterproof the internal and external phantom walls (necessary requirement for filling). A filling system was also designed to minimize the residual air bubbles, which could result in unwanted hypo-intensity (dark) areas in phantom-based imaging simulation. Discussions and conclusions. The phantom prototype was scanned trough CT and PET/CT to evaluate the realism of the brain simulation. None of the state-of-the-art brain phantoms allow such anatomical rendering of three brain compartments. Some represent only GM and WM, others only the striatum. Moreover, they typically have a poor anatomical yield, showing a reduced depth of the sulci and a not very faithful reproduction of the cerebral convolutions. The ability to simulate the three brain compartments simultaneously with greater accuracy, as well as the possibility of carrying out multimodality studies (PET/CT, PET/MRI), which represent the frontier of diagnostic imaging, give this device cutting-edge prospective characteristics. The effort to further customize 3DP technology for these applications is expected to increase significantly in the coming years