Analysis and Manipulation of Repetitive Structures of Varying Shape

Self-similarity and repetitions are ubiquitous in man-made and natural objects. Such structural regularities often relate to form, function, aesthetics, and design considerations. Discovering structural redundancies along with their dominant variations from 3D geometry not only allows us to better understand the underlying objects, but is also beneficial for several geometry processing tasks including compact representation, shape completion, and intuitive shape manipulation. To identify these repetitions, we present a novel detection algorithm based on analyzing a graph of surface features. We combine general feature detection schemes with a RANSAC-based randomized subgraph searching algorithm in order to reliably detect recurring patterns of locally unique structures. A subsequent segmentation step based on a simultaneous region growing is applied to verify that the actual data supports the patterns detected in the feature graphs. We introduce our graph based detection algorithm on the example of rigid repetitive structure detection. Then we extend the approach to allow more general deformations between the detected parts. We introduce subspace symmetries whereby we characterize similarity by requiring the set of repeating structures to form a low dimensional shape space. We discover these structures based on detecting linearly correlated correspondences among graphs of invariant features. The found symmetries along with the modeled variations are useful for a variety of applications including non-local and non-rigid denoising. Employing subspace symmetries for shape editing, we introduce a morphable part model for smart shape manipulation. The input geometry is converted to an assembly of deformable parts with appropriate boundary conditions. Our method uses self-similarities from a single model or corresponding parts of shape collections as training input and allows the user also to reassemble the identified parts in new configurations, thus exploiting both the discrete and continuous learned variations while ensuring appropriate boundary conditions across part boundaries. We obtain an interactive yet intuitive shape deformation framework producing realistic deformations on classes of objects that are difficult to edit using repetition-unaware deformation techniques

Sub-pixel Registration In Computational Imaging And Applications To Enhancement Of Maxillofacial Ct Data

In computational imaging, data acquired by sampling the same scene or object at different times or from different orientations result in images in different coordinate systems. Registration is a crucial step in order to be able to compare, integrate and fuse the data obtained from different measurements. Tomography is the method of imaging a single plane or slice of an object. A Computed Tomography (CT) scan, also known as a CAT scan (Computed Axial Tomography scan), is a Helical Tomography, which traditionally produces a 2D image of the structures in a thin section of the body. It uses X-ray, which is ionizing radiation. Although the actual dose is typically low, repeated scans should be limited. In dentistry, implant dentistry in specific, there is a need for 3D visualization of internal anatomy. The internal visualization is mainly based on CT scanning technologies. The most important technological advancement which dramatically enhanced the clinician\u27s ability to diagnose, treat, and plan dental implants has been the CT scan. Advanced 3D modeling and visualization techniques permit highly refined and accurate assessment of the CT scan data. However, in addition to imperfections of the instrument and the imaging process, it is not uncommon to encounter other unwanted artifacts in the form of bright regions, flares and erroneous pixels due to dental bridges, metal braces, etc. Currently, removing and cleaning up the data from acquisition backscattering imperfections and unwanted artifacts is performed manually, which is as good as the experience level of the technician. On the other hand the process is error prone, since the editing process needs to be performed image by image. We address some of these issues by proposing novel registration methods and using stonecast models of patient\u27s dental imprint as reference ground truth data. Stone-cast models were originally used by dentists to make complete or partial dentures. The CT scan of such stone-cast models can be used to automatically guide the cleaning of patients\u27 CT scans from defects or unwanted artifacts, and also as an automatic segmentation system for the outliers of the CT scan data without use of stone-cast models. Segmented data is subsequently used to clean the data from artifacts using a new proposed 3D inpainting approach

A Geometric Approach for Deciphering Protein Structure from Cryo-EM Volumes

Electron Cryo-Microscopy or cryo-EM is an area that has received much attention in the recent past. Compared to the traditional methods of X-Ray Crystallography and NMR Spectroscopy, cryo-EM can be used to image much larger complexes, in many different conformations, and under a wide range of biochemical conditions. This is because it does not require the complex to be crystallisable. However, cryo-EM reconstructions are limited to intermediate resolutions, with the state-of-the-art being 3.6A, where secondary structure elements can be visually identified but not individual amino acid residues. This lack of atomic level resolution creates new computational challenges for protein structure identification. In this dissertation, we present a suite of geometric algorithms to address several aspects of protein modeling using cryo-EM density maps. Specifically, we develop novel methods to capture the shape of density volumes as geometric skeletons. We then use these skeletons to find secondary structure elements: SSEs) of a given protein, to identify the correspondence between these SSEs and those predicted from the primary sequence, and to register high-resolution protein structures onto the density volume. In addition, we designed and developed Gorgon, an interactive molecular modeling system, that integrates the above methods with other interactive routines to generate reliable and accurate protein backbone models

Retopology: a comprehensive study of current automation solutions from an artist’s workflow perspective

Dissertação de mestrado em Engenharia InformáticaTopology (the density, organization and flow of a 3D mesh’s connectivity) constrains the suitability of a 3D model for any given purpose, be it surface showcasing through renders, use in real-time engines, posing or animation. While some of these use cases might not have very strict topology requirements, others may demand optimized polygon counts for performance reasons, or even specific geometry distribution in order to take deformation directions into account. Many processes for creating 3D models such as sculpting try to make the user unaware of the inner workings of geometry, by providing flexible levels of surface detailing through dynamic geometry allocation. The resulting models have a dense, unorganized topology that is inefficient and unfit for most use cases, with the additional drawback of being hard to work with manually. Retopology is the process of providing a new topology to a model such as these, while maintaining the shape of its surface. It’s a technical and time-consuming process that clashes with the rest of the artist’s workflow, which is mainly composed of creative processes. While there’s abundant research in this area focusing on polygon distribution quality based on surface shape, artists are still left with no options but to resort to manual work when it comes to deformation-optimized topology. This document exposes this disconnect, along with a proposed framework that attempts to provide a more complete retopology solution for 3D artists. This framework combines traditional mesh extraction algorithms with adapting manually-made meshes in a pipeline that tries to understand the input on a higher level, in order to solve deficiencies that are present in current retopology tools. Our results are very positive, presenting an improvement over state of the art solutions, which could possibly steer discussion and research in this area to be more in line with the needs of 3D artists.A topologia (a densidade, organização e direções tomadas pela conectividade de uma mesh 3D) limita a adequação de um modelo 3D para um leque variado de usos, entre os quais, visualização da superfície através de renders, uso em motores real-time, poses ou animações. Embora muitos destes usos não possuam requerimentos de topologia muito rigorosos, outros podem exigir número de polígonos mais baixos por questões de performance, ou até distribuição de geometria específica para acomodar direções de deformação corretamente. Muitos processos de criação de modelos 3D, como escultura, permitem que o utilizador não esteja ciente do que se passa em termos de funcionamento da geometria por debaixo da utilização. Isto é conseguido oferecendo níveis de detalhe flexíveis, alocando geometria de forma dinâmica. Os modelos resultantes têm uma topologia densa e desorganizada, que é ineficiente e pouco apropriada para a maior parte dos casos de uso, com a desvantagem adicional de ser difícil de trabalhar com a mesma manualmente. A retopologia é o processo de gerar uma nova topologia para um modelo, ao mesmo tempo que se mantém a forma da superfície. É um processo técnico e demorado, que entra em conflito com o resto do fluxo de trabalho do artista, que é composto maioritariamente por processos artísticos. Apesar de haver investigação abundante nesta área focada na qualidade da distribuição de polígonos baseada na forma da superfície, os artistas continuam a ter de recorrer ao trabalho manual quando se trata de topologia otimizada para deformações. Este documento expõe esta divergência, propondo, em conjunto, uma framework que tenta oferecer uma solução mais completa para os artistas 3D. Esta framework combina algoritmos de extração de meshes tradicionais com adaptação de meshes feitas manualmente, numa pipeline que tenta compreender o input a um nível superior, resolvendo as deficiências presentes nas ferramentas de retopologia atuais. Os nossos resultados são bastante positivos, apresentando melhorias em relação a soluções de estado da arte, facto que poderá mudar o rumo da discussão e investigação neste campo, para melhor se adequar às necessidades dos artistas 3D

Proceedings of the Fourth International Workshop on Mathematical Foundations of Computational Anatomy - Geometrical and Statistical Methods for Biological Shape Variability Modeling (MFCA 2013), Nagoya, Japan

International audienceComputational anatomy is an emerging discipline at the interface of geometry, statistics and image analysis which aims at modeling and analyzing the biological shape of tissues and organs. The goal is to estimate representative organ anatomies across diseases, populations, species or ages, to model the organ development across time (growth or aging), to establish their variability, and to correlate this variability information with other functional, genetic or structural information. The Mathematical Foundations of Computational Anatomy (MFCA) workshop aims at fostering the interactions between the mathematical community around shapes and the MICCAI community in view of computational anatomy applications. It targets more particularly researchers investigating the combination of statistical and geometrical aspects in the modeling of the variability of biological shapes. The workshop is a forum for the exchange of the theoretical ideas and aims at being a source of inspiration for new methodological developments in computational anatomy. A special emphasis is put on theoretical developments, applications and results being welcomed as illustrations. Following the first edition of this workshop in 2006, second edition in New-York in 2008, the third edition in Toronto in 2011, the forth edition was held in Nagoya Japan on September 22 2013

Augmented Reality and Artificial Intelligence in Image-Guided and Robot-Assisted Interventions

In minimally invasive orthopedic procedures, the surgeon places wires, screws, and surgical implants through the muscles and bony structures under image guidance. These interventions require alignment of the pre- and intra-operative patient data, the intra-operative scanner, surgical instruments, and the patient. Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. State of the art approaches often support the surgeon by using external navigation systems or ill-conditioned image-based registration methods that both have certain drawbacks. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. Consequently, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. This dissertation investigates the applications of AR, artificial intelligence, and robotics in interventional medicine. Our solutions were applied in a broad spectrum of problems for various tasks, namely improving imaging and acquisition, image computing and analytics for registration and image understanding, and enhancing the interventional visualization. The benefits of these approaches were also discovered in robot-assisted interventions. We revealed how exemplary workflows are redefined via AR by taking full advantage of head-mounted displays when entirely co-registered with the imaging systems and the environment at all times. The proposed AR landscape is enabled by co-localizing the users and the imaging devices via the operating room environment and exploiting all involved frustums to move spatial information between different bodies. The system's awareness of the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. We also leveraged the principles governing image formation and combined it with deep learning and RGBD sensing to fuse images and reconstruct interventional data. We hope that our holistic approaches towards improving the interface of surgery and enhancing the usability of interventional imaging, not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications

Symmetry Detection in Geometric Models

PUNTIS (LO1506), SGS-2019-016Symmetry occurs very commonly in real world objects as well as in artificially created geometric models. The knowledge about symmetry of a given object can be very useful in many applications in computer graphics and geometry processing, such as compression, object alignment, symmetric editing or completion of partial objects. In order to use the symmetry of any object in any given application, it first needs to be found. In this work, we provide some background about symmetry in general and about different types of symmetry, mainly in 3D objects. Then we focus on the task of automatic symmetry detection in 3D objects and we also describe the link between symmetry detection and the problem of registration. Most importantly, we present our own contribution in these fields. First, we show a new method of evaluating consensus in RANSAC surface registration together with a thorough analysis of various distance metrics for rigid transformations that can be used in this new approach. Afterwards, we provide an analysis of different representations of the space of planes in context of symmetry plane detection. At last, we propose a new, robust, fast and flexible method for symmetry plane detection based on a novel differentiable symmetry measure

Magnetism in curved geometries

Curvature impacts physical properties across multiple length scales, ranging from the macroscopic scale, where the shape and size vary drastically with the curvature, to the nanoscale at interfaces and inhomogeneities in materials with structural, chemical, electronic, and magnetic short-range order. In quantum materials, where correlations, entanglement, and topology dominate, the curvature opens the path to novel characteristics and phenomena that have recently emerged and could have a dramatic impact on future fundamental and applied studies of materials. Particularly, magnetic systems hosting non-collinear and topological states and 3D magnetic nanostructures strongly benefit from treating curvature as a new design parameter to explore prospective applications in the magnetic field and stress sensing, microrobotics, and information processing and storage. This Perspective gives an overview of recent progress in synthesis, theory, and characterization studies and discusses future directions, challenges, and application potential of the harnessing curvature for 3D nanomagnetism