Journal Staff

We investigate the mesh colors method for per-face parameterization for texture-mapping of geometry, implemented in the game engine Frostbite 3, for the purpose of evaluating the method compared to traditional texture-mapping in a real-time application. Traditional UV-mapping often causes discontinuities which commonly results in visible seams in the end results. If any change is done to the vertex positions or the topology a remapping of the UV-map has to be done. Mesh colors aims to avoid these problems by skipping the transformation to 2D space as in UV-mapping, and associating color samples directly with the geometry of a mesh. The implementation was done in Frostbite 3 in C++ and HLSL shader code, and rendered with the DirectX graphics API. The results show that mesh colors is a viable alternative in a real-time renderer. Though not as fast as regular UV-mapped textures due to lack of hardware accelerated filtering operations, mesh colors is a realistic alternative for special cases where regular texture-mapping would be cumbersome to work with or produce sub-par results. Possible areas of future research are efficient data structures suitable to handle data insertion dynamically, compression of mesh colors data, and mesh colors in the context of mesh LOD generation

Plane-Based Optimization of Geometry and Texture for RGB-D Reconstruction of Indoor Scenes

We present a novel approach to reconstruct RGB-D indoor scene with plane primitives. Our approach takes as input a RGB-D sequence and a dense coarse mesh reconstructed by some 3D reconstruction method on the sequence, and generate a lightweight, low-polygonal mesh with clear face textures and sharp features without losing geometry details from the original scene. To achieve this, we firstly partition the input mesh with plane primitives, simplify it into a lightweight mesh next, then optimize plane parameters, camera poses and texture colors to maximize the photometric consistency across frames, and finally optimize mesh geometry to maximize consistency between geometry and planes. Compared to existing planar reconstruction methods which only cover large planar regions in the scene, our method builds the entire scene by adaptive planes without losing geometry details and preserves sharp features in the final mesh. We demonstrate the effectiveness of our approach by applying it onto several RGB-D scans and comparing it to other state-of-the-art reconstruction methods.Comment: in International Conference on 3D Vision 2018; Models and Code: see https://github.com/chaowang15/plane-opt-rgbd. arXiv admin note: text overlap with arXiv:1905.0885

Scribble-based gradient mesh recoloring

Previous gradient mesh recoloring methods usually have dependencies on an additional reference image and the rasterized gradient mesh. To circumvent such dependencies, we propose a user scribble-based recoloring method, in which users are allowed to annotate gradient meshes with a few color scribbles. Our approach builds an auxiliary mesh from gradient meshes, namely control net, by taking both colors and local color gradients at mesh points into account. We then develop an extended chrominance blending method to propagate the user specified colors over the control net. The recolored gradient mesh is finally reconstructed from the recolored control net. Experiments validate the effectiveness of our approach on multiple gradient meshes. Compared with various alternative solutions, our method has no color bleedings nor sampling artifacts, and can achieve fast performance

Approximation Algorithms for Multicoloring Planar Graphs and Powers of Square and Triangular Meshes

International audienceA multicoloring of a weighted graph G is an assignment of sets of colors to the vertices of G so that two adjacent vertices receive two disjoint sets of colors. A multicoloring problem on G is to find a multicoloring of G. In particular, we are interested in a minimum multicoloring that uses the least total number of colors. The main focus of this work is to obtain upper bounds on the weighted chromatic number of some classes of graphs in terms of the weighted clique number. We first propose an 11/6-approximation algorithm for multicoloring any weighted planar graph. We then study the multicoloring problem on powers of square and triangular meshes. Among other results, we show that the infinite triangular mesh is an induced subgraph of the fourth power of the infinite square mesh and we present 2-approximation algorithms for multicoloring a power square mesh and the second power of a triangular mesh, 3-approximation algorithms for multicoloring powers of semi-toroidal meshes and of triangular meshes and 4-approximation algorithm for multicoloring the power of a toroidal mesh. We also give similar algorithms for the Cartesian product of powers of paths and of cycles

NeRFlame: FLAME-based conditioning of NeRF for 3D face rendering

Traditional 3D face models are based on mesh representations with texture. One of the most important models is FLAME (Faces Learned with an Articulated Model and Expressions), which produces meshes of human faces that are fully controllable. Unfortunately, such models have problems with capturing geometric and appearance details. In contrast to mesh representation, the neural radiance field (NeRF) produces extremely sharp renders. But implicit methods are hard to animate and do not generalize well to unseen expressions. It is not trivial to effectively control NeRF models to obtain face manipulation. The present paper proposes a novel approach, named NeRFlame, which combines the strengths of both NeRF and FLAME methods. Our method enables high-quality rendering capabilities of NeRF while also offering complete control over the visual appearance, similar to FLAME. Unlike conventional NeRF-based architectures that utilize neural networks to model RGB colors and volume density, NeRFlame employs FLAME mesh as an explicit density volume. As a result, color values are non-zero only in the proximity of the FLAME mesh. This FLAME backbone is then integrated into the NeRF architecture to predict RGB colors, allowing NeRFlame to explicitly model volume density and implicitly model RGB colors

Static/Dynamic Filtering for Mesh Geometry

The joint bilateral filter, which enables feature-preserving signal smoothing according to the structural information from a guidance, has been applied for various tasks in geometry processing. Existing methods either rely on a static guidance that may be inconsistent with the input and lead to unsatisfactory results, or a dynamic guidance that is automatically updated but sensitive to noises and outliers. Inspired by recent advances in image filtering, we propose a new geometry filtering technique called static/dynamic filter, which utilizes both static and dynamic guidances to achieve state-of-the-art results. The proposed filter is based on a nonlinear optimization that enforces smoothness of the signal while preserving variations that correspond to features of certain scales. We develop an efficient iterative solver for the problem, which unifies existing filters that are based on static or dynamic guidances. The filter can be applied to mesh face normals followed by vertex position update, to achieve scale-aware and feature-preserving filtering of mesh geometry. It also works well for other types of signals defined on mesh surfaces, such as texture colors. Extensive experimental results demonstrate the effectiveness of the proposed filter for various geometry processing applications such as mesh denoising, geometry feature enhancement, and texture color filtering

The min-max edge q-coloring problem

In this paper we introduce and study a new problem named \emph{min-max edge $q$-coloring} which is motivated by applications in wireless mesh networks. The input of the problem consists of an undirected graph and an integer $q$. The goal is to color the edges of the graph with as many colors as possible such that: (a) any vertex is incident to at most $q$ different colors, and (b) the maximum size of a color group (i.e. set of edges identically colored) is minimized. We show the following results: 1. Min-max edge $q$-coloring is NP-hard, for any $q \ge 2$. 2. A polynomial time exact algorithm for min-max edge $q$-coloring on trees. 3. Exact formulas of the optimal solution for cliques and almost tight bounds for bicliques and hypergraphs. 4. A non-trivial lower bound of the optimal solution with respect to the average degree of the graph. 5. An approximation algorithm for planar graphs.Comment: 16 pages, 5 figure

ITEM: Inter-Texture Error Measurement for 3D Meshes

We introduce a simple and innovative method to compare any two texture maps, regardless of their sizes, aspect ratios, or even masks, as long as they are both meant to be mapped onto the same 3D mesh. Our system is based on a zero-distortion 3D mesh unwrapping technique which compares two new adapted texture atlases with the same mask but different texel colors, and whose every texel covers the same area in 3D. Once these adapted atlases are created, we measure their difference with ITEM-RMSE, a slightly modified version of the standard RMSE defined for images. ITEM-RMSE is more meaningful and reliable than RMSE because it only takes into account the texels inside the mask, since they are the only ones that will actually be used during rendering. Our method is not only very useful to compare the space efficiency of different texture atlas generation algorithms, but also to quantify texture loss in compression schemes for multi-resolution textured 3D meshes

SurfelMeshing: Online Surfel-Based Mesh Reconstruction

We address the problem of mesh reconstruction from live RGB-D video, assuming a calibrated camera and poses provided externally (e.g., by a SLAM system). In contrast to most existing approaches, we do not fuse depth measurements in a volume but in a dense surfel cloud. We asynchronously (re)triangulate the smoothed surfels to reconstruct a surface mesh. This novel approach enables to maintain a dense surface representation of the scene during SLAM which can quickly adapt to loop closures. This is possible by deforming the surfel cloud and asynchronously remeshing the surface where necessary. The surfel-based representation also naturally supports strongly varying scan resolution. In particular, it reconstructs colors at the input camera's resolution. Moreover, in contrast to many volumetric approaches, ours can reconstruct thin objects since objects do not need to enclose a volume. We demonstrate our approach in a number of experiments, showing that it produces reconstructions that are competitive with the state-of-the-art, and we discuss its advantages and limitations. The algorithm (excluding loop closure functionality) is available as open source at https://github.com/puzzlepaint/surfelmeshing .Comment: Version accepted to IEEE Transactions on Pattern Analysis and Machine Intelligenc