Interactively Cutting and Constraining Vertices in Meshes Using Augmented Matrices

We present a finite-element solution method that is well suited for interactive simulations of cutting meshes in the regime of linear elastic models. Our approach features fast updates to the solution of the stiffness system of equations to account for real-time changes in mesh connectivity and boundary conditions. Updates are accomplished by augmenting the stiffness matrix to keep it consistent with changes to the underlying model, without refactoring the matrix at each step of cutting. The initial stiffness matrix and its Cholesky factors are used to implicitly form and solve a Schur complement system using an iterative solver. As changes accumulate over many simulation timesteps, the augmented solution method slows down due to the size of the augmented matrix. However, by periodically refactoring the stiffness matrix in a concurrent background process, fresh Cholesky factors that incorporate recent model changes can replace the initial factors. This controls the size of the augmented matrices and provides a way to maintain a fast solution rate as the number of changes to a model grows. We exploit sparsity in the stiffness matrix, the right-hand-side vectors and the solution vectors to compute the solutions fast, and show that the time complexity of the update steps is bounded linearly by the size of the Cholesky factor of the initial matrix. Our complexity analysis and experimental results demonstrate that this approach scales well with problem size. Results for cutting and deformation of 3D linear elastic models are reported for meshes representing the brain, eye, and model problems with element counts up to 167,000; these show the potential of this method for real-time interactivity. An application to limbal incisions for surgical correction of astigmatism, for which linear elastic models and small deformations are sufficient, is included

A Revisit of Shape Editing Techniques: from the Geometric to the Neural Viewpoint

3D shape editing is widely used in a range of applications such as movie production, computer games and computer aided design. It is also a popular research topic in computer graphics and computer vision. In past decades, researchers have developed a series of editing methods to make the editing process faster, more robust, and more reliable. Traditionally, the deformed shape is determined by the optimal transformation and weights for an energy term. With increasing availability of 3D shapes on the Internet, data-driven methods were proposed to improve the editing results. More recently as the deep neural networks became popular, many deep learning based editing methods have been developed in this field, which is naturally data-driven. We mainly survey recent research works from the geometric viewpoint to those emerging neural deformation techniques and categorize them into organic shape editing methods and man-made model editing methods. Both traditional methods and recent neural network based methods are reviewed

Surface Deformation Potentials on Meshes for Computer Graphics and Visualization

Shape deformation models have been used in computer graphics primarily to describe the dynamics of physical deformations like cloth draping, collisions of elastic bodies, fracture, or animation of hair. Less frequent is their application to problems not directly related to a physical process. In this thesis we apply deformations to three problems in computer graphics that do not correspond to physical deformations. To this end, we generalize the physical model by modifying the energy potential. Originally, the energy potential amounts to the physical work needed to deform a body from its rest state into a given configuration and relates material strain to internal restoring forces that act to restore the original shape. For each of the three problems considered, this potential is adapted to reflect an application specific notion of shape. Under the influence of further constraints, our generalized deformation results in shapes that balance preservation of certain shape properties and application specific objectives similar to physical equilibrium states. The applications discussed in this thesis are surface parameterization, interactive shape editing and automatic design of panorama maps. For surface parameterization, we interpret parameterizations over a planar domain as deformations from a flat initial configuration onto a given surface. In this setting, we review existing parameterization methods by analyzing properties of their potential functions and derive potentials accounting for distortion of geometric properties. Interactive shape editing allows an untrained user to modify complex surfaces, be simply grabbing and moving parts of interest. A deformation model interactively extrapolates the transformation from those parts to the rest of the surface. This thesis proposes a differential shape representation for triangle meshes leading to a potential that can be optimized interactively with a simple, tailored algorithm. Although the potential is not physically accurate, it results in intuitive deformation behavior and can be parameterized to account for different material properties. Panorama maps are blends between landscape illustrations and geographic maps that are traditionally painted by an artist to convey geographic surveyknowledge on public places like ski resorts or national parks. While panorama maps are not drawn to scale, the shown landscape remains recognizable and the observer can easily recover details necessary for self location and orientation. At the same time, important features as trails or ski slopes appear not occluded and well visible. This thesis proposes the first automatic panorama generation method. Its basis is again a surface deformation, that establishes the necessary compromise between shape preservation and feature visibility.Potentiale zur Flächendeformation auf Dreiecksnetzen für Anwendungen in der Computergrafik und Visualisierung Deformationsmodelle werden in der Computergrafik bislang hauptsächlich eingesetzt, um die Dynamik physikalischer Deformationsprozesse zu modellieren. Gängige Beispiele sind Bekleidungssimulationen, Kollisionen elastischer Körper oder Animation von Haaren und Frisuren. Deutlich seltener ist ihre Anwendung auf Probleme, die nicht direkt physikalischen Prozessen entsprechen. In der vorliegenden Arbeit werden Deformationsmodelle auf drei Probleme der Computergrafik angewandt, die nicht unmittelbar einem physikalischen Deformationsprozess entsprechen. Zu diesem Zweck wird das physikalische Modell durch eine passende Änderung der potentiellen Energie verallgemeinert. Die potentielle Energie entspricht normalerweise der physikalischen Arbeit, die aufgewendet werden muss, um einen Körper aus dem Ruhezustand in eine bestimmte Konfiguration zu verformen. Darüber hinaus setzt sie die aktuelle Verformung in Beziehung zu internen Spannungskräften, die wirken um die ursprüngliche Form wiederherzustellen. In dieser Arbeit passen wir für jedes der drei betrachteten Problemfelder die potentielle Energie jeweils so an, dass sie eine anwendungsspezifische Definition von Form widerspiegelt. Unter dem Einfluss weiterer Randbedingungen führt die so verallgemeinerte Deformation zu einer Fläche, die eine Balance zwischen der Erhaltung gewisser Formeigenschaften und Zielvorgaben der Anwendung findet. Diese Balance entspricht dem Equilibrium einer physikalischen Deformation. Die drei in dieser Arbeit diskutierten Anwendungen sind Oberflächenparameterisierung, interaktives Bearbeiten von Flächen und das vollautomatische Erzeugen von Panoramakarten im Stile von Heinrich Berann. Zur Oberflächenparameterisierung interpretieren wir Parameterisierungen über einem flachen Parametergebiet als Deformationen, die ein ursprünglich ebenes Flächenstück in eine gegebene Oberfläche verformen. Innerhalb dieses Szenarios vergleichen wir dann existierende Methoden zur planaren Parameterisierung, indem wir die resultierenden potentiellen Energien analysieren, und leiten weitere Potentiale her, die die Störung geometrischer Eigenschaften wie Fläche und Winkel erfassen. Verfahren zur interaktiven Flächenbearbeitung ermöglichen schnelle und intuitive Änderungen an einer komplexen Oberfläche. Dazu wählt der Benutzer Teile der Fläche und bewegt diese durch den Raum. Ein Deformationsmodell extrapoliert interaktiv die Transformation der gewählten Teile auf die restliche Fläche. Diese Arbeit stellt eine neue differentielle Flächenrepräsentation für diskrete Flächen vor, die zu einem einfach und interaktiv zu optimierendem Potential führt. Obwohl das vorgeschlagene Potential nicht physikalisch korrekt ist, sind die resultierenden Deformationen intuitiv. Mittels eines Parameters lassen sich außerdem bestimmte Materialeigenschaften einstellen. Panoramakarten im Stile von Heinrich Berann sind eine Verschmelzung von Landschaftsillustration und geographischer Karte. Traditionell werden sie so von Hand gezeichnet, dass bestimmt Merkmale wie beispielsweise Skipisten oder Wanderwege in einem Gebiet unverdeckt und gut sichtbar bleiben, was große Kunstfertigkeit verlangt. Obwohl diese Art der Darstellung nicht maßstabsgetreu ist, sind Abweichungen auf den ersten Blick meistens nicht zu erkennen. Dadurch kann der Betrachter markante Details schnell wiederfinden und sich so innerhalb des Gebietes orientieren. Diese Arbeit stellt das erste, vollautomatische Verfahren zur Erzeugung von Panoramakarten vor. Grundlage ist wiederum eine verallgemeinerte Oberflächendeformation, die sowohl auf Formerhaltung als auch auf die Sichtbarkeit vorgegebener geographischer Merkmale abzielt

Mobile Wound Assessment and 3D Modeling from a Single Image

The prevalence of camera-enabled mobile phones have made mobile wound assessment a viable treatment option for millions of previously difficult to reach patients. We have designed a complete mobile wound assessment platform to ameliorate the many challenges related to chronic wound care. Chronic wounds and infections are the most severe, costly and fatal types of wounds, placing them at the center of mobile wound assessment. Wound physicians assess thousands of single-view wound images from all over the world, and it may be difficult to determine the location of the wound on the body, for example, if the wound is taken at close range. In our solution, end-users capture an image of the wound by taking a picture with their mobile camera. The wound image is segmented and classified using modern convolution neural networks, and is stored securely in the cloud for remote tracking. We use an interactive semi-automated approach to allow users to specify the location of the wound on the body. To accomplish this we have created, to the best our knowledge, the first 3D human surface anatomy labeling system, based off the current NYU and Anatomy Mapper labeling systems. To interactively view wounds in 3D, we have presented an efficient projective texture mapping algorithm for texturing wounds onto a 3D human anatomy model. In so doing, we have demonstrated an approach to 3D wound reconstruction that works even for a single wound image

Virtual Reality Methods for Research in the Geosciences

In the presented work, I evaluate if and how Virtual Reality (VR) technologies can be used to support researchers working in the geosciences by providing immersive, collaborative visualization systems as well as virtual tools for data analysis. Technical challenges encountered in the development of theses systems are identified and solutions for these are provided. To enable geologists to explore large digital terrain models (DTMs) in an immersive, explorative fashion within a VR environment, a suitable terrain rendering algorithm is required. For realistic perception of planetary curvature at large viewer altitudes, spherical rendering of the surface is necessary. Furthermore, rendering must sustain interactive frame rates of about 30 frames per second to avoid sensory confusion of the user. At the same time, the data structures used for visualization should also be suitable for efficiently computing spatial properties such as height profiles or volumes in order to implement virtual analysis tools. To address these requirements, I have developed a novel terrain rendering algorithm based on tiled quadtree hierarchies using the HEALPix parametrization of a sphere. For evaluation purposes, the system is applied to a 500 GiB dataset representing the surface of Mars. Considering the current development of inexpensive remote surveillance equipment such as quadcopters, it seems inevitable that these devices will play a major role in future disaster management applications. Virtual reality installations in disaster management headquarters which provide an immersive visualization of near-live, three-dimensional situational data could then be a valuable asset for rapid, collaborative decision making. Most terrain visualization algorithms, however, require a computationally expensive pre-processing step to construct a terrain database. To address this problem, I present an on-the-fly pre-processing system for cartographic data. The system consists of a frontend for rendering and interaction as well as a distributed processing backend executing on a small cluster which produces tiled data in the format required by the frontend on demand. The backend employs a CUDA based algorithm on graphics cards to perform efficient conversion from cartographic standard projections to the HEALPix-based grid used by the frontend. Measurement of spatial properties is an important step in quantifying geological phenomena. When performing these tasks in a VR environment, a suitable input device and abstraction for the interaction (a “virtual tool”) must be provided. This tool should enable the user to precisely select the location of the measurement even under a perspective projection. Furthermore, the measurement process should be accurate to the resolution of the data available and should not have a large impact on the frame rate in order to not violate interactivity requirements. I have implemented virtual tools based on the HEALPix data structure for measurement of height profiles as well as volumes. For interaction, a ray-based picking metaphor was employed, using a virtual selection ray extending from the user’s hand holding a VR interaction device. To provide maximum accuracy, the algorithms access the quad-tree terrain database at the highest available resolution level while at the same time maintaining interactivity in rendering. Geological faults are cracks in the earth’s crust along which a differential movement of rock volumes can be observed. Quantifying the direction and magnitude of such translations is an essential requirement in understanding earth’s geological history. For this purpose, geologists traditionally use maps in top-down projection which are cut (e.g. using image editing software) along the suspected fault trace. The two resulting pieces of the map are then translated in parallel against each other until surface features which have been cut by the fault motion come back into alignment. The amount of translation applied is then used as a hypothesis for the magnitude of the fault action. In the scope of this work it is shown, however, that performing this study in a top-down perspective can lead to the acceptance of faulty reconstructions, since the three-dimensional structure of topography is not considered. To address this problem, I present a novel terrain deformation algorithm which allows the user to trace a fault line directly within a 3D terrain visualization system and interactively deform the terrain model while inspecting the resulting reconstruction from arbitrary perspectives. I demonstrate that the application of 3D visualization allows for a more informed interpretation of fault reconstruction hypotheses. The algorithm is implemented on graphics cards and performs real-time geometric deformation of the terrain model, guaranteeing interactivity with respect to all parameters. Paleoceanography is the study of the prehistoric evolution of the ocean. One of the key data sources used in this research are coring experiments which provide point samples of layered sediment depositions at the ocean floor. The samples obtained in these experiments document the time-varying sediment concentrations within the ocean water at the point of measurement. The task of recovering the ocean flow patterns based on these deposition records is a challenging inverse numerical problem, however. To support domain scientists working on this problem, I have developed a VR visualization tool to aid in the verification of model parameters by providing simultaneous visualization of experimental data from coring as well as the resulting predicted flow field obtained from numerical simulation. Earth is visualized as a globe in the VR environment with coring data being presented using a billboard rendering technique while the time-variant flow field is indicated using Line-Integral-Convolution (LIC). To study individual sediment transport pathways and their correlation with the depositional record, interactive particle injection and real-time advection is supported

CAPS--Computer-aided plastic surgery

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Architecture, 1992.Includes bibliographical references (leaves 166-173).by Steven Donald Pieper.Ph.D

Authoring virtual crowds: a survey

Recent advancements in crowd simulation unravel a wide range of functionalities for virtual agents, delivering highly-realistic,natural virtual crowds. Such systems are of particular importance to a variety of applications in fields such as: entertainment(e.g., movies, computer games); architectural and urban planning; and simulations for sports and training. However, providingtheir capabilities to untrained users necessitates the development of authoring frameworks. Authoring virtual crowds is acomplex and multi-level task, varying from assuming control and assisting users to realise their creative intents, to deliveringintuitive and easy to use interfaces, facilitating such control. In this paper, we present a categorisation of the authorable crowdsimulation components, ranging from high-level behaviours and path-planning to local movements, as well as animation andvisualisation. We provide a review of the most relevant methods in each area, emphasising the amount and nature of influencethat the users have over the final result. Moreover, we discuss the currently available authoring tools (e.g., graphical userinterfaces, drag-and-drop), identifying the trends of early and recent work. Finally, we suggest promising directions for futureresearch that mainly stem from the rise of learning-based methods, and the need for a unified authoring framework.This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 860768 (CLIPE project). This project has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 739578 and the Government of the Republic of Cyprus through the Deputy Ministry of Research, Innovation and Digital PolicyPeer ReviewedPostprint (author's final draft

Appearance Modelling and Reconstruction for Navigation in Minimally Invasive Surgery

Minimally invasive surgery is playing an increasingly important role for patient care. Whilst its direct patient benefit in terms of reduced trauma, improved recovery and shortened hospitalisation has been well established, there is a sustained need for improved training of the existing procedures and the development of new smart instruments to tackle the issue of visualisation, ergonomic control, haptic and tactile feedback. For endoscopic intervention, the small field of view in the presence of a complex anatomy can easily introduce disorientation to the operator as the tortuous access pathway is not always easy to predict and control with standard endoscopes. Effective training through simulation devices, based on either virtual reality or mixed-reality simulators, can help to improve the spatial awareness, consistency and safety of these procedures. This thesis examines the use of endoscopic videos for both simulation and navigation purposes. More specifically, it addresses the challenging problem of how to build high-fidelity subject-specific simulation environments for improved training and skills assessment. Issues related to mesh parameterisation and texture blending are investigated. With the maturity of computer vision in terms of both 3D shape reconstruction and localisation and mapping, vision-based techniques have enjoyed significant interest in recent years for surgical navigation. The thesis also tackles the problem of how to use vision-based techniques for providing a detailed 3D map and dynamically expanded field of view to improve spatial awareness and avoid operator disorientation. The key advantage of this approach is that it does not require additional hardware, and thus introduces minimal interference to the existing surgical workflow. The derived 3D map can be effectively integrated with pre-operative data, allowing both global and local 3D navigation by taking into account tissue structural and appearance changes. Both simulation and laboratory-based experiments are conducted throughout this research to assess the practical value of the method proposed