Feature preserving noise removal for binary voxel volumes using 3D surface skeletons

Skeletons are well-known descriptors that capture the geometry and topology of 2D and 3D shapes. We leverage these properties by using surface skeletons to remove noise from 3D shapes. For this, we extend an existing method that removes noise, but keeps important (salient) corners for 2D shapes. Our method detects and removes large-scale, complex, and dense multiscale noise patterns that contaminate virtually the entire surface of a given 3D shape, while recovering its main (salient) edges and corners. Our method can treat any (voxelized) 3D shapes and surface-noise types, is computationally scalable, and has one easy-to-set parameter. We demonstrate the added-value of our approach by comparing our results with several known 3D shape denoising methods

Capture, Learning, and Synthesis of 3D Speaking Styles

Audio-driven 3D facial animation has been widely explored, but achieving realistic, human-like performance is still unsolved. This is due to the lack of available 3D datasets, models, and standard evaluation metrics. To address this, we introduce a unique 4D face dataset with about 29 minutes of 4D scans captured at 60 fps and synchronized audio from 12 speakers. We then train a neural network on our dataset that factors identity from facial motion. The learned model, VOCA (Voice Operated Character Animation) takes any speech signal as input - even speech in languages other than English - and realistically animates a wide range of adult faces. Conditioning on subject labels during training allows the model to learn a variety of realistic speaking styles. VOCA also provides animator controls to alter speaking style, identity-dependent facial shape, and pose (i.e. head, jaw, and eyeball rotations) during animation. To our knowledge, VOCA is the only realistic 3D facial animation model that is readily applicable to unseen subjects without retargeting. This makes VOCA suitable for tasks like in-game video, virtual reality avatars, or any scenario in which the speaker, speech, or language is not known in advance. We make the dataset and model available for research purposes at http://voca.is.tue.mpg.de.Comment: To appear in CVPR 201

Adaptive Methods for Point Cloud and Mesh Processing

Point clouds and 3D meshes are widely used in numerous applications ranging from games to virtual reality to autonomous vehicles. This dissertation proposes several approaches for noise removal and calibration of noisy point cloud data and 3D mesh sharpening methods. Order statistic filters have been proven to be very successful in image processing and other domains as well. Different variations of order statistics filters originally proposed for image processing are extended to point cloud filtering in this dissertation. A brand-new adaptive vector median is proposed in this dissertation for removing noise and outliers from noisy point cloud data. The major contributions of this research lie in four aspects: 1) Four order statistic algorithms are extended, and one adaptive filtering method is proposed for the noisy point cloud with improved results such as preserving significant features. These methods are applied to standard models as well as synthetic models, and real scenes, 2) A hardware acceleration of the proposed method using Microsoft parallel pattern library for filtering point clouds is implemented using multicore processors, 3) A new method for aerial LIDAR data filtering is proposed. The objective is to develop a method to enable automatic extraction of ground points from aerial LIDAR data with minimal human intervention, and 4) A novel method for mesh color sharpening using the discrete Laplace-Beltrami operator is proposed. Median and order statistics-based filters are widely used in signal processing and image processing because they can easily remove outlier noise and preserve important features. This dissertation demonstrates a wide range of results with median filter, vector median filter, fuzzy vector median filter, adaptive mean, adaptive median, and adaptive vector median filter on point cloud data. The experiments show that large-scale noise is removed while preserving important features of the point cloud with reasonable computation time. Quantitative criteria (e.g., complexity, Hausdorff distance, and the root mean squared error (RMSE)), as well as qualitative criteria (e.g., the perceived visual quality of the processed point cloud), are employed to assess the performance of the filters in various cases corrupted by different noisy models. The adaptive vector median is further optimized for denoising or ground filtering aerial LIDAR data point cloud. The adaptive vector median is also accelerated on multi-core CPUs using Microsoft Parallel Patterns Library. In addition, this dissertation presents a new method for mesh color sharpening using the discrete Laplace-Beltrami operator, which is an approximation of second order derivatives on irregular 3D meshes. The one-ring neighborhood is utilized to compute the Laplace-Beltrami operator. The color for each vertex is updated by adding the Laplace-Beltrami operator of the vertex color weighted by a factor to its original value. Different discretizations of the Laplace-Beltrami operator have been proposed for geometrical processing of 3D meshes. This work utilizes several discretizations of the Laplace-Beltrami operator for sharpening 3D mesh colors and compares their performance. Experimental results demonstrated the effectiveness of the proposed algorithms

Analysis of the Path Tracing rendering method on CPU and GPU

Fa molts anys que es realitza recerca al camp de la renderització realista. Tenim diversos mètodes capaços d'aconseguir imatges amb un gran grau de realisme, com pot ser Ray Tracing, Path Tracing, Photon Mapping o Metropolis Light Transport. A excepció del Ray Tracing, la resta de mètodes tracten de resoldre l'Equació de Renderitzat mitjançant aproximacions (és impossible calcular-la íntegrament, ja que necessitaríem temps i potència de càlcul infinita). Gràcies a poder aproximar aquesta equació es poden aconseguir efectes de forma natural, sense necessitat d'un postprocessat, com motion blur, depth of field, càustiques, etc. El mètode que implementarem i estudiarem és el Path Tracing. Realitzarem diverses versions d'aquest mètode amb les quals podrem explorar quina arquitectura (GPU o CPU) ens ofereix una major avantatge pel que fa al rendiment per al nostre algoritme. Per això comptarem amb diverses màquines, una amb un hardware d'última generació, una amb un hardware més econòmic i una última màquina pensada per un entorn professional. És molt usual que les aplicacions que implementen aquest mètode s'utilitzen d'estructures de dades que permeten millorar de forma molt notable el rendiment d'aquesta. Per aquesta raó, implementarem una Bounding Volume Hierarchy, un estructura de tipus arbre, per a la representació de l'escena i així augmentar el rendiment. Estudiarem com recorre-la de dues formes diferents, una recursiva molt més natural en aquest tipus d'estructures i un altre iterativa, per veure com afecta al rendiment de l'aplicació en la GPU. És ben sabut que les funcions recursives no són gens òptimes a la GPU. Per últim, implementarem un seguit de filtres d'eliminació de soroll. El mètode de Path Tracing produeix imatges molt sorolloses si s'utilitzen poques mostres per píxel, per això l'ús de filtres d'eliminació de soroll és molt comú. Això ens permetrà trobar un equilibri entre el nombre de mostres per píxel i la necessitat d'un postfiltratge de la imatge resultant.The field of realistic rendering has been investigated for many years. We have different methods capable of creating images with a high degree of realism, such as Ray Tracing, Path Tracing, Photon Mapping or Metropolis Light Transport. Except for Ray Tracing, the rest of the cited methods try to solve the Rendering Equation by approximations (it is impossible to calculate it completely because we would need time and infinite computing power). Thanks to being able to approximate this equation, effects can be achieved naturally, without needing any post-processing, such as motion blur, depth of field, caustics, etc. The method that we will implement and study is Path Tracing. We will make several versions of this method with which we will explore which architecture (GPU or CPU) gives us a greater advantage in terms of performance for our algorithm. For this, we will have different machines, one with the last generation hardware, one with cheaper hardware and the last machine with hardware thought for a professional environment. It is very usual that the applications that implement this method are assisted by accelerating structures that allow improving in a very notable way the performance of this one. For this same reason, we will implement a Bounding Volume Hierarchy, a tree-type structure, to represent our scene and thus increase performance. We will study how to go through it in two different ways, one recursive much more natural in this type of structure and another iterative, to see how it affects the performance of the application on the GPU. It's well known that the GPU is not optimal for recursive functions. Finally, we'll implement a set of denoising filters. Path tracing produces very noisy images when using a few samples per pixel, so the use of denoising filters is very common. This will also help us find a balance between the number of samples per pixel and the need for post-filtering of the output image

Cardiac Motion Analysis Based on Optical Flow on Real-Time Three-Dimensional Ultrasound Data

With relatively high frame rates and the ability to acquire volume data sets with a stationary transducer, 3D ultrasound systems, based on matrix phased array transducers, provide valuable three-dimensional information, from which quantitative measures of cardiac function can be extracted. Such analyses require segmentation and visual tracking of the left ventricular endocardial border. Due to the large size of the volumetric data sets, manual tracing of the endocardial border is tedious and impractical for clinical applications. Therefore the development of automatic methods for tracking three-dimensional endocardial motion is essential. In this study, we evaluate a four-dimensional optical flow motion tracking algorithm to determine its capability to follow the endocardial border in three dimensional ultrasound data through time. The four-dimensional optical flow method was implemented using three-dimensional correlation. We tested the algorithm on an experimental open-chest dog data set and a clinical data set acquired with a Philips' iE33 three-dimensional ultrasound machine. Initialized with left ventricular endocardial data points obtained from manual tracing at end-diastole, the algorithm automatically tracked these points frame by frame through the whole cardiac cycle. Finite element surfaces were fitted through the data points obtained by both optical flow tracking and manual tracing by an experienced observer for quantitative comparison of the results. Parameterization of the finite element surfaces was performed and maps displaying relative differences between the manual and semi-automatic methods were compared. The results showed good consistency with less than 10% difference between manual tracing and optical flow estimation on 73% of the entire surface. In addition, the optical flow motion tracking algorithm greatly reduced processing time (about 94% reduction compared to human involvement per cardiac cycle) for analyzing cardiac function in three-dimensional ultrasound data sets. A displacement field was computed from the optical flow output, and a framework for computation of dynamic cardiac information is introduced. The method was applied to a clinical data set from a heart transplant patient and dynamic measurements agreed with known physiology as well as experimental results

A hierarchical reduced-order model applied to nuclear reactors

Modelling the neutron transport of a nuclear reactor is a very computationally demanding task that requires a large number of degrees of freedom to accurately capture all of the physics. For a complete reactor picture, other physics must be incorporated, through coupling, further exacerbating the computational demand. Computational modelling has many benefits: optimisation, real-time analysis, and safety analysis are some of the more important ones. However, nuclear modelling has yet to capitalise on these, and existing approaches are too computationally demanding. Machine Learning has seen incredible growth over the last decade, but it has yet to be utilised within the nuclear modelling community to the same extent. The frameworks available represent incredibly efficient and optimised code, having been written to run on GPUs and AI computers. Presented here is a physics-driven neural network that solves neutron transport, first for the diffusion approximation and then extended to the whole transport problem. One method that can potentially reduce the computational complexity is Reduced-Order Modelling (ROM), which is a way to define a low-dimensional space in which a high-dimensional system can be approximated. These established methods can be used with machine learning methods, potentially reducing computational costs further than either method individually. A method to utilise autoencoders with a projection-based framework is also presented here. The structure of a reactor can be broken down, forming a hierarchy which starts with the reactor core, which is populated by fuel assemblies, which are then populated by fuel rods. This hierarchy means that materials are repeated within a solution, and many existing methods do not capitalise on this and instead resolve the entire global domain. This research presents two ways to utilise this structure with ROM. The first involves combining it with domain decomposition, producing ROMs for the sub-structures. The second presents a hierarchical interpolating method, reducing the number of sub-domains within the solution that need to be resolved.Open Acces

Catalog Matching with Astrometric Correction and its Application to the Hubble Legacy Archive

Object cross-identification in multiple observations is often complicated by the uncertainties in their astrometric calibration. Due to the lack of standard reference objects, an image with a small field of view can have significantly larger errors in its absolute positioning than the relative precision of the detected sources within. We present a new general solution for the relative astrometry that quickly refines the World Coordinate System of overlapping fields. The efficiency is obtained through the use of infinitesimal 3-D rotations on the celestial sphere, which do not involve trigonometric functions. They also enable an analytic solution to an important step in making the astrometric corrections. In cases with many overlapping images, the correct identification of detections that match together across different images is difficult to determine. We describe a new greedy Bayesian approach for selecting the best object matches across a large number of overlapping images. The methods are developed and demonstrated on the Hubble Legacy Archive, one of the most challenging data sets today. We describe a novel catalog compiled from many Hubble Space Telescope observations, where the detections are combined into a searchable collection of matches that link the individual detections. The matches provide descriptions of astronomical objects involving multiple wavelengths and epochs. High relative positional accuracy of objects is achieved across the Hubble images, often sub-pixel precision in the order of just a few milli-arcseconds. The result is a reliable set of high-quality associations that are publicly available online.Comment: 9 pages, 9 figures, accepted for publication in the Astrophysical Journa