Learning SO(3) Equivariant Representations with Spherical CNNs

We address the problem of 3D rotation equivariance in convolutional neural networks. 3D rotations have been a challenging nuisance in 3D classification tasks requiring higher capacity and extended data augmentation in order to tackle it. We model 3D data with multi-valued spherical functions and we propose a novel spherical convolutional network that implements exact convolutions on the sphere by realizing them in the spherical harmonic domain. Resulting filters have local symmetry and are localized by enforcing smooth spectra. We apply a novel pooling on the spectral domain and our operations are independent of the underlying spherical resolution throughout the network. We show that networks with much lower capacity and without requiring data augmentation can exhibit performance comparable to the state of the art in standard retrieval and classification benchmarks.Comment: Camera-ready. Accepted to ECCV'18 as oral presentatio

Multi Voxel Descriptor for 3D Texture Retrieval

In this paper, we present a new feature descriptors  which exploit voxels for 3D textured retrieval system when models vary either by geometric shape or texture or both. First, we perform pose normalisation to modify arbitrary 3D models  in order to have same orientation. We then map the structure of 3D models into voxels. This purposes to make all the 3D models have the same dimensions. Through this voxels, we can capture information from a number of ways.  First, we build biner voxel histogram and color voxel histogram.  Second, we compute distance from centre voxel into other voxels and generate histogram. Then we also compute fourier transform in spectral space.  For capturing texture feature, we apply voxel tetra pattern. Finally, we merge all features by linear combination. For experiment, we use standard evaluation measures such as Nearest Neighbor (NN), First Tier (FT), Second Tier (ST), Average Dynamic Recall (ADR). Dataset in SHREC 2014  and its evaluation program is used to verify the proposed method. Experiment result show that the proposed method  is more accurate when compared with some methods of state-of-the-art

Local Color Voxel and Spatial Pattern for 3D Textured Recognition

3D textured retrieval including shape, color dan pattern is still a challenging research. Some approaches are proposed, but voxel-based approach has not much been made yet, where by using this approach, it still keeps both geometry and texture information. It also maps all 3D models into the same dimension. Based on this fact, a novel voxel pattern based is proposed by considering local pattern on a voxel called local color voxel pattern (LCVP). Voxels textured is observed by considering voxel to its neighbors. LCVP is computed around each voxel to its neighbors. LCVP value will indicate uniq pattern on each 3D models. LCVP also quantizes color on each voxel to generate a specific pattern. Shift and reflection circular also will be done. In an additional way, inspired by promising recent results from image processing, this paper also implement spatial pattern which utilizing Weber, Oriented Gradient to extract global spatial descriptor. Finally, a combination of local spectra and spatial and established global features approach called multi Fourier descriptor are proposed. For optimal retrieval, the rank combination is performed between local and global approaches. Experiments were performed by using dataset SHREC'13 and SHREC'14 and showed that the proposed method could outperform some performances to state-of-the-art

Learning Equivariant Representations

State-of-the-art deep learning systems often require large amounts of data and computation. For this reason, leveraging known or unknown structure of the data is paramount. Convolutional neural networks (CNNs) are successful examples of this principle, their defining characteristic being the shift-equivariance. By sliding a filter over the input, when the input shifts, the response shifts by the same amount, exploiting the structure of natural images where semantic content is independent of absolute pixel positions. This property is essential to the success of CNNs in audio, image and video recognition tasks. In this thesis, we extend equivariance to other kinds of transformations, such as rotation and scaling. We propose equivariant models for different transformations defined by groups of symmetries. The main contributions are (i) polar transformer networks, achieving equivariance to the group of similarities on the plane, (ii) equivariant multi-view networks, achieving equivariance to the group of symmetries of the icosahedron, (iii) spherical CNNs, achieving equivariance to the continuous 3D rotation group, (iv) cross-domain image embeddings, achieving equivariance to 3D rotations for 2D inputs, and (v) spin-weighted spherical CNNs, generalizing the spherical CNNs and achieving equivariance to 3D rotations for spherical vector fields. Applications include image classification, 3D shape classification and retrieval, panoramic image classification and segmentation, shape alignment and pose estimation. What these models have in common is that they leverage symmetries in the data to reduce sample and model complexity and improve generalization performance. The advantages are more significant on (but not limited to) challenging tasks where data is limited or input perturbations such as arbitrary rotations are present

Retrieval and classification methods for textured 3D models: a comparative study

International audienceThis paper presents a comparative study of six methods for the retrieval and classification of tex-tured 3D models, which have been selected as representative of the state of the art. To better analyse and control how methods deal with specific classes of geometric and texture deformations, we built a collection of 572 synthetic textured mesh models, in which each class includes multiple texture and geometric modifications of a small set of null models. Results show a challenging, yet lively, scenario and also reveal interesting insights in how to deal with texture information according to different approaches, possibly working in the CIELab as well as in modifications of the RGB colour space