Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres

Many computer vision challenges require continuous outputs, but tend to be solved by discrete classification. The reason is classification's natural containment within a probability $n$-simplex, as defined by the popular softmax activation function. Regular regression lacks such a closed geometry, leading to unstable training and convergence to suboptimal local minima. Starting from this insight we revisit regression in convolutional neural networks. We observe many continuous output problems in computer vision are naturally contained in closed geometrical manifolds, like the Euler angles in viewpoint estimation or the normals in surface normal estimation. A natural framework for posing such continuous output problems are $n$-spheres, which are naturally closed geometric manifolds defined in the $\mathbb{R}^{(n+1)}$ space. By introducing a spherical exponential mapping on $n$-spheres at the regression output, we obtain well-behaved gradients, leading to stable training. We show how our spherical regression can be utilized for several computer vision challenges, specifically viewpoint estimation, surface normal estimation and 3D rotation estimation. For all these problems our experiments demonstrate the benefit of spherical regression. All paper resources are available at https://github.com/leoshine/Spherical_Regression.Comment: CVPR 2019 camera read

Experimentally measuring rolling and sliding in three-dimensional dense granular packings

We experimentally measure a three-dimensional (3D) granular system's reversibility under cyclic compression. We image the grains using a refractive-index-matched fluid, then analyze the images using the artificial intelligence of variational autoencoders. These techniques allow us to track all the grains' translations and 3D rotations with accuracy sufficient to infer sliding and rolling displacements. Our observations reveal unique roles played by 3D rotational motions in granular flows. We find that rotations and contact-point motion dominate the dynamics in the bulk, far from the perturbation's source. Furthermore, we determine that 3D rotations are irreversible under cyclic compression. Consequently, contact-point sliding, which is dissipative, accumulates throughout the cycle. Using numerical simulations whose accuracy our experiment supports, we discover that much of the dissipation occurs in the bulk, where grains rotate more than they translate. Our observations suggest that the analysis of 3D rotations is needed for understanding granular materials' unique and powerful ability to absorb and dissipate energy

Two Turns Must Take Turns: Primary School Students' Cognition about 3D Rotation in a Virtual Reality Learning Environment

This paper reports on five primary school students’ explorations of 3D rotation in a virtual reality learning environment (VRLE) named VRMath. When asked to investigate if you would face the same direction when you turn right 45 degrees first then roll up 45 degrees, or when you roll up 45 degrees first then turn right 45 degrees, the students found that the different order of the two turns ended up with different directions in the VRLE. This was contrary to the students’ prior predictions based on using pen, paper and body movements. The findings of this study showed the difficulty young children have in perceiving and understanding the non-commutative nature of 3D rotation and the power of the computational VRLE in giving students experiences that they rarely have in real life with 3D manipulations and 3D mental movements

Learning SO(3) Equivariant Representations with Spherical CNNs

We address the problem of 3D rotation equivariance in convolutional neural networks. 3D rotations have been a challenging nuisance in 3D classification tasks requiring higher capacity and extended data augmentation in order to tackle it. We model 3D data with multi-valued spherical functions and we propose a novel spherical convolutional network that implements exact convolutions on the sphere by realizing them in the spherical harmonic domain. Resulting filters have local symmetry and are localized by enforcing smooth spectra. We apply a novel pooling on the spectral domain and our operations are independent of the underlying spherical resolution throughout the network. We show that networks with much lower capacity and without requiring data augmentation can exhibit performance comparable to the state of the art in standard retrieval and classification benchmarks.Comment: Camera-ready. Accepted to ECCV'18 as oral presentatio

The closure constraint for the hyperbolic tetrahedron as a Bianchi identity

The closure constraint is a central piece of the mathematics of loop quantum gravity. It encodes the gauge invariance of the spin network states of quantum geometry and provides them with a geometrical interpretation: each decorated vertex of a spin network is dual to a quantized polyhedron in $\mathbb{R}^{3}$. For instance, a 4-valent vertex is interpreted as a tetrahedron determined by the four normal vectors of its faces. We develop a framework where the closure constraint is re-interpreted as a Bianchi identity, with the normals defined as holonomies around the polyhedron faces of a connection (constructed from the spinning geometry interpretation of twisted geometries). This allows us to define closure constraints for hyperbolic tetrahedra (living in the 3-hyperboloid of unit future-oriented spacelike vectors in $\mathbb{R}^{3,1}$) in terms of normals living all in $SU(2)$ or in $SB(2,\mathbb{C})$. The latter fits perfectly with the classical phase space developed for $q$-deformed loop quantum gravity supposed to account for a non-vanishing cosmological constant $\Lambda>0$. This is the first step towards interpreting $q$-deformed twisted geometries as actual discrete hyperbolic triangulations.Comment: 31 page