4 research outputs found
Multi-view Consensus CNN for 3D Facial Landmark Placement
The rapid increase in the availability of accurate 3D scanning devices has
moved facial recognition and analysis into the 3D domain. 3D facial landmarks
are often used as a simple measure of anatomy and it is crucial to have
accurate algorithms for automatic landmark placement. The current
state-of-the-art approaches have yet to gain from the dramatic increase in
performance reported in human pose tracking and 2D facial landmark placement
due to the use of deep convolutional neural networks (CNN). Development of deep
learning approaches for 3D meshes has given rise to the new subfield called
geometric deep learning, where one topic is the adaptation of meshes for the
use of deep CNNs. In this work, we demonstrate how methods derived from
geometric deep learning, namely multi-view CNNs, can be combined with recent
advances in human pose tracking. The method finds 2D landmark estimates and
propagates this information to 3D space, where a consensus method determines
the accurate 3D face landmark position. We utilise the method on a standard 3D
face dataset and show that it outperforms current methods by a large margin.
Further, we demonstrate how models trained on 3D range scans can be used to
accurately place anatomical landmarks in magnetic resonance images.Comment: This is a pre-print of an article published in proceedings of the
asian conference on computer vision 2018 (LNCS 11361). The final
authenticated version is available online at:
https://doi.org/10.1007/978-3-030-20887-5_4
Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
Motivated by the vast success of deep convolutional networks, there is a
great interest in generalizing convolutions to non-Euclidean manifolds. A major
complication in comparison to flat spaces is that it is unclear in which
alignment a convolution kernel should be applied on a manifold. The underlying
reason for this ambiguity is that general manifolds do not come with a
canonical choice of reference frames (gauge). Kernels and features therefore
have to be expressed relative to arbitrary coordinates. We argue that the
particular choice of coordinatization should not affect a network's inference
-- it should be coordinate independent. A simultaneous demand for coordinate
independence and weight sharing is shown to result in a requirement on the
network to be equivariant under local gauge transformations (changes of local
reference frames). The ambiguity of reference frames depends thereby on the
G-structure of the manifold, such that the necessary level of gauge
equivariance is prescribed by the corresponding structure group G. Coordinate
independent convolutions are proven to be equivariant w.r.t. those isometries
that are symmetries of the G-structure. The resulting theory is formulated in a
coordinate free fashion in terms of fiber bundles. To exemplify the design of
coordinate independent convolutions, we implement a convolutional network on
the M\"obius strip. The generality of our differential geometric formulation of
convolutional networks is demonstrated by an extensive literature review which
explains a large number of Euclidean CNNs, spherical CNNs and CNNs on general
surfaces as specific instances of coordinate independent convolutions.Comment: The implementation of orientation independent M\"obius convolutions
is publicly available at https://github.com/mauriceweiler/MobiusCNN