Mesh Denoising with Facet Graph Convolutions

code available at the following address:https://gitlab.inria.fr/marmando/deep-mesh-denoizingInternational audienceWe examine the problem of mesh denoising, which consists of removing noise from corrupted 3D meshes while preserving existing geometric features. Most mesh denoising methods require a lot of mesh-specific parameter fine-tuning, to account for specific features and noise types. In recent years, data-driven methods have demonstrated their robustness and effectiveness with respect to noise and feature properties on a wide variety of geometry and image problems. Most existing mesh denoising methods still use hand-crafted features, and locally denoise facets rather than examine the mesh globally. In this work, we propose the use of a fully end-to-end learning strategy based on graph convolutions, where meaningful features are learned directly by our network. It operates on a graph of facets, directly on the existing topology of the mesh, without resampling, and follows a multi-scale design to extract geometric features at different resolution levels. Similar to most recent pipelines, given a noisy mesh, we first denoise face normals with our novel approach, then update vertex positions accordingly. Our method performs significantly better than the current state-of-the-art learning-based methods. Additionally, we show that it can be trained on noisy data, without explicit correspondence between noisy and ground-truth facets. We also propose a multi-scale denoising strategy, better suited to correct noise with a low spatial frequency

Cross-view and Cross-pose Completion for 3D Human Understanding

Human perception and understanding is a major domain of computer vision which, like many other vision subdomains recently, stands to gain from the use of large models pre-trained on large datasets. We hypothesize that the most common pre-training strategy of relying on general purpose, object-centric image datasets such as ImageNet, is limited by an important domain shift. On the other hand, collecting domain specific ground truth such as 2D or 3D labels does not scale well. Therefore, we propose a pre-training approach based on self-supervised learning that works on human-centric data using only images. Our method uses pairs of images of humans: the first is partially masked and the model is trained to reconstruct the masked parts given the visible ones and a second image. It relies on both stereoscopic (cross-view) pairs, and temporal (cross-pose) pairs taken from videos, in order to learn priors about 3D as well as human motion. We pre-train a model for body-centric tasks and one for hand-centric tasks. With a generic transformer architecture, these models outperform existing self-supervised pre-training methods on a wide set of human-centric downstream tasks, and obtain state-of-the-art performance for instance when fine-tuning for model-based and model-free human mesh recovery

Automated Design of Salient Object Detection Algorithms with Brain Programming

Despite recent improvements in computer vision, artificial visual systems' design is still daunting since an explanation of visual computing algorithms remains elusive. Salient object detection is one problem that is still open due to the difficulty of understanding the brain's inner workings. Progress on this research area follows the traditional path of hand-made designs using neuroscience knowledge. In recent years two different approaches based on genetic programming appear to enhance their technique. One follows the idea of combining previous hand-made methods through genetic programming and fuzzy logic. The other approach consists of improving the inner computational structures of basic hand-made models through artificial evolution. This research work proposes expanding the artificial dorsal stream using a recent proposal to solve salient object detection problems. This approach uses the benefits of the two main aspects of this research area: fixation prediction and detection of salient objects. We decided to apply the fusion of visual saliency and image segmentation algorithms as a template. The proposed methodology discovers several critical structures in the template through artificial evolution. We present results on a benchmark designed by experts with outstanding results in comparison with the state-of-the-art.Comment: 35 pages, 5 figure

Refractive index sensor based on slot waveguide cavity

The experimental study of a gold slot waveguide cavity is presented. The resonance of this cavity working in the telecom wavelength range is highly dependent on the refractive index of the medium located in or around the slots array, because of the high confinement of the electromagnetic field in the structure. We will demonstrate the application of this structure to local refractive index sensors at the nanoscale. The measured sensitivity of this device is S = 730 nm/RIU (refractive index unit). The structure has been optimized by adding another array of slots cascaded with the first one. The consequence is an improvement in the time efficiency of the experiments. A discussion about the effect of the volume of liquids used and the filling percentage of the slots by the liquids is also presented as parameters affecting the measurements and the sensitivity of the sensor

Theoretical and experimental study of a 30nm metallic slot array

We present the simulation, fabrication, and characterization of a metallic slot grating on a silicon waveguide (30nm slot width for a period of 500nm). According to the simulation, the experimental measured spectrum of the structure exhibits a dip in the near-infrared region. Moreover, the finite-difference time-domain simulation shows that this device can be used as a sensor, due to its high sensitivity to the refractive index variation of the medium above and inside the cavity (η ¼ Δλ=Δn ¼ 750nm=refractive index unit)

Planar Integrated Sensors on Waveguides for sensing applications

We investigate the use of a metallic nano-cavity on dielectric waveguides for sensing applications. Nowadays the fabrication of metallic structures featured by challenging dimensions is feasible thanks to the most recent technologies such as E-Beam Lithography (EBL) and lift-off [1]. We propose here to study the simulation, fabrication and characterization of such structures at the telecom wavelength (λ=1.55 μm)

In-plane illuminated metallic annular aperture array for sensing application

We propose here an application to sensing of annular aperture arrays (AAA). We theoretically investigate the optical properties of the reflective AAA device when illuminated in-plane. The cavity presents almost perfect absorption due to the waveguide mode resonance with strong field localization in the aperture. Additionally, the reflective cavity is modeled to be available for on-chip sensing with a theoretically expected sensitivity of 764 nm∕RIU (refractive index unit)

SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction

Recent hand-object interaction datasets show limited real object variability and rely on fitting the MANO parametric model to obtain groundtruth hand shapes. To go beyond these limitations and spur further research, we introduce the SHOWMe dataset which consists of 96 videos, annotated with real and detailed hand-object 3D textured meshes. Following recent work, we consider a rigid hand-object scenario, in which the pose of the hand with respect to the object remains constant during the whole video sequence. This assumption allows us to register sub-millimetre-precise groundtruth 3D scans to the image sequences in SHOWMe. Although simpler, this hypothesis makes sense in terms of applications where the required accuracy and level of detail is important eg., object hand-over in human-robot collaboration, object scanning, or manipulation and contact point analysis. Importantly, the rigidity of the hand-object systems allows to tackle video-based 3D reconstruction of unknown hand-held objects using a 2-stage pipeline consisting of a rigid registration step followed by a multi-view reconstruction (MVR) part. We carefully evaluate a set of non-trivial baselines for these two stages and show that it is possible to achieve promising object-agnostic 3D hand-object reconstructions employing an SfM toolbox or a hand pose estimator to recover the rigid transforms and off-the-shelf MVR algorithms. However, these methods remain sensitive to the initial camera pose estimates which might be imprecise due to lack of textures on the objects or heavy occlusions of the hands, leaving room for improvements in the reconstruction. Code and dataset are available at https://europe.naverlabs.com/research/showmeComment: Paper and Appendix, Accepted in ACVR workshop at ICCV conferenc

Plasmonic Slot Waveguides for Localized Biomolecular Sensing Applications

A plasmonic slot waveguide excited by evanescent wave coupling of a silicon strip waveguide is studied to be used as a label-free biosensor. The plasmonic slot waveguide enables strong electric-field enhancement in a small volume inducing higher interaction between the light and the analyte

4DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacements

This work presents 4DHumanOutfit, a new dataset of densely sampled spatio-temporal 4D human motion data of different actors, outfits and motions. The dataset is designed to contain different actors wearing different outfits while performing different motions in each outfit. In this way, the dataset can be seen as a cube of data containing 4D motion sequences along 3 axes with identity, outfit and motion. This rich dataset has numerous potential applications for the processing and creation of digital humans, e.g. augmented reality, avatar creation and virtual try on. 4DHumanOutfit is released for research purposes at https://kinovis.inria.fr/4dhumanoutfit/. In addition to image data and 4D reconstructions, the dataset includes reference solutions for each axis. We present independent baselines along each axis that demonstrate the value of these reference solutions for evaluation tasks