PC-MSDM: A quality metric for 3D point clouds

International audienceIn this paper, we present PC-MSDM, an objective metric for visual quality assessment of 3D point clouds. This full-reference metric is based on local curvature statistics and can be viewed as an extension for point clouds of the MSDM metric suited for 3D meshes. We evaluate its performance on an open subjective dataset of point clouds compressed by octree pruning; results show that the proposed metric outperforms its counterparts in terms of correlation with mean opinion scores

Lightweight integration of 3D features to improve 2D image segmentation

Scene understanding has made tremendous progress over the past few years, as data acquisition systems are now providing an increasing amount of data of various modalities (point cloud, depth, RGB...). However, this improvement comes at a large cost on computation resources and data annotation requirements. To analyze geometric information and images jointly, many approaches rely on both a 2D loss and 3D loss, requiring not only 2D per pixel-labels but also 3D per-point labels. However, obtaining a 3D groundtruth is challenging, time-consuming and error-prone. In this paper, we show that image segmentation can benefit from 3D geometric information without requiring a 3D groundtruth, by training the geometric feature extraction and the 2D segmentation network jointly, in an end-to-end fashion, using only the 2D segmentation loss. Our method starts by extracting a map of 3D features directly from a provided point cloud by using a lightweight 3D neural network. The 3D feature map, merged with the RGB image, is then used as an input to a classical image segmentation network. Our method can be applied to many 2D segmentation networks, improving significantly their performance with only a marginal network weight increase and light input dataset requirements, since no 3D groundtruth is required

An Analysis and Implementation of a Parallel Ball Pivoting Algorithm

The problem of surface reconstruction from a set of 3D points given by their coordinates andoriented normals is a difficult problem, which has been tackled with many different approaches.In 1999, Bernardini and colleagues introduced a very elegant and efficient reconstruction methodthat uses a ball pivoting around triangle edges and adds new triangles if the ball is incidentto three points and contains no other points. This paper details an implementation and parallelization of this algorithm

Géométrie inverse : du nuage de points brut à la surface 3D : théorie et algorithmes

Many laser devices acquire directly 3D objects and reconstruct their surface. Nevertheless, the final reconstructed surface is usually smoothed out as a result of the scanner internal de-noising process and the offsets between different scans. This thesis, working on results from high precision scans, adopts the somewhat extreme conservative position, not to loose or alter any raw sample throughout the whole processing pipeline, and to attempt to visualize them. Indeed, it is the only way to discover all surface imperfections (holes, offsets). Furthermore, since high precision data can capture the slightest surface variation, any smoothing and any sub-sampling can incur in the loss of textural detail.The thesis attempts to prove that one can triangulate the raw point cloud with almost no sample loss. It solves the exact visualization problem on large data sets of up to 35 million points made of 300 different scan sweeps and more. Two major problems are addressed. The first one is the orientation of the complete raw point set, an the building of a high precision mesh. The second one is the correction of the tiny scan misalignments which can cause strong high frequency aliasing and hamper completely a direct visualization.The second development of the thesis is a general low-high frequency decomposition algorithm for any point cloud. Thus classic image analysis tools, the level set tree and the MSER representations, are extended to meshes, yielding an intrinsic mesh segmentation method.The underlying mathematical development focuses on an analysis of a half dozen discrete differential operators acting on raw point clouds which have been proposed in the literature. By considering the asymptotic behavior of these operators on a smooth surface, a classification by their underlying curvature operators is obtained.This analysis leads to the development of a discrete operator consistent with the mean curvature motion (the intrinsic heat equation) defining a remarkably simple and robust numerical scale space. By this scale space all of the above mentioned problems (point set orientation, raw point set triangulation, scan merging, segmentation), usually addressed by separated techniques, are solved in a unified framework.De nombreux scanners laser permettent d'obtenir la surface 3D a partir d'un objet. Néanmoins, la surface reconstruite est souvent lisse, ce qui est du au débruitage interne du scanner et aux décalages entre les scans. Cette these utilise des scans haute precision et choisit de ne pas perdre ni alterer les echantillons initiaux au cours du traitement afin de les visualiser. C'est en effet la seule façon de decouvrir les imperfections (trous, decalages de scans). De plus, comme les donnees haute precision capturent meme le plus leger detail, tout debruitage ou sous-echantillonnage peut amener a perdre ces details.La these s'attache a prouver que l'on peut trianguler le nuage de point initial en ne perdant presque aucun echantillon. Le probleme de la visualisation exacte sur des donnees de plus de 35 millions de points et de 300 scans differents est ainsi resolu. Deux problemes majeurs sont traites: le premier est l'orientation du nuage de point brut complet et la creation d'un maillage. Le second est la correction des petits decalages entre les scans qui peuvent creer un tres fort aliasing et compromettre la visualisation de la surface. Le second developpement de la these est une decomposition des nuages de points en hautes/basses frequences. Ainsi, des methodes classiques pour l'analyse d'image, l'arbre des ensembles de niveau et la representation MSER, sont etendues aux maillages, ce qui donne une methode intrinseque de segmentation de maillages. Une analyse mathematiques d'operateurs differentiels discrets, proposes dans la litterature et operant sur des nuages de points est realisee. En considerant les developpements asymptotiques de ces operateurs sur une surface reguliere, ces operateurs peuvent etre classifies. Cette analyse amene au developpement d'un operateur discret consistant avec Ie mouvement par courbure moyenne (l'equation de la chaleur intrinseque) definissant ainsi un espace-echelle numerique simple et remarquablement robuste. Cet espace-echelle permet de resoudre de maniere unifiee tous les problemes mentionnes auparavant (orientation et triangulation du nuage de points, fusion de scans, segmentation de maillages) qui sont ordinairement traites avec des techniques distinctes

Surface derivatives computation using Fourier Transform

National audienceWe present a method for computing high order derivatives on a smooth surface S at a point p by analyzing the vibrations of the surface along circles in the tangent plane, centered at p. By computing the Discrete Fourier Transform of the deviation of S from the tangent plane restricted to those circles, a linear relation between the Fourier coefficients and the derivatives can be expressed. Thus, given a smooth scalar field defined on the surface, all its derivatives at p can be computed simultaneously. The originality of this method is that no direct derivation process is applied to the data. Instead, integration is performed through the Discrete Fourier Transform, and the result is expressed as a one dimensional polynomial. We derive two applications of our framework namely normal correction and curvature estimation which we demonstrate on synthetic and real data

A survey of Optimal Transport for Computer Graphics and Computer Vision

International audienceOptimal transport is a long-standing theory that has been studied in depth from both theoretical and numerical point of views. Starting from the 50s this theory has also found a lot of applications in operational research. Over the last 30 years it has spread to computer vision and computer graphics and is now becoming hard to ignore. Still, its mathematical complexity can make it difficult to comprehend, and as such, computer vision and computer graphics researchers may find it hard to follow recent developments in their field related to optimal transport. This survey first briefly introduces the theory of optimal transport in layman's terms as well as most common numerical techniques to solve it. More importantly, it presents applications of these numerical techniques to solve various computer graphics and vision related problems. This involves applications ranging from image processing, geometry processing, rendering, fluid simulation, to computational optics, and many more. It is aimed at computer graphics researchers desiring to follow optimal transport research in their field as well as optimal transport researchers willing to find applications for their numerical algorithms

Neural skeleton: Implicit neural representation away from the surface

International audienceImplicit Neural Representations are powerful tools for representing 3D shapes. They encode an implicit field in the parameters of a Neural Network, leveraging the power of auto-differentiation for optimizing the implicit function and avoiding the need for a manually crafted function. So far, Implicit Neural Representations have been mainly designed to extract or render object surfaces and methods primarily focus on improving the implicit function near the surface. In this paper we argue that implicit fields are useful for other shape analysis tasks, in particular skeleton (medial axis) extraction. Indeed, a medial axis is defined through distances to the surface, which can be provided by an implicit neural representation, making it robust to noise and missing data. However this requires the implicit field to be reliable away from the surface, something most representations are not optimized for. To achieve this, inspired by variational image denoising techniques, we propose to add a Total Variation term, to regularize the implicit field. We further design a skeleton sampling method working directly on the GPU, and link the extracted points using a coverage formulation. We show that our resulting neural skeleton is more robust to sample defects such as noise or missing data compared to other medial axis extraction methods

The Bilateral Filter for Point Clouds

International audiencePoint sets obtained by 3D scanners are often corrupted with noise, that can have several causes, such as a tangential acquisition direction, changing environmental lights or a reflective object material. It is thus crucial to design efficient tools to remove noise from the acquired data without removing important information such as sharp edges or shape details. To do so, Fleish-man et al. introduced a bilateral filter for meshes adapted from the bilateral filter for gray level images. This anisotropic filter denoises a point with respect to its neighbors by considering not only the distance from the neighbors to the point but also the distance along a normal direction. This simple fact allows for a much better preservation of sharp edges. In this paper, we analyze a parallel implementation of the bilateral filter adapted for point clouds. Source Code The ANSI C++ source code permitting to reproduce results from the on-line demo is available on the web page of the article 1