516 research outputs found
Learning Equivariant Representations
State-of-the-art deep learning systems often require large amounts of data
and computation. For this reason, leveraging known or unknown structure of the
data is paramount. Convolutional neural networks (CNNs) are successful examples
of this principle, their defining characteristic being the shift-equivariance.
By sliding a filter over the input, when the input shifts, the response shifts
by the same amount, exploiting the structure of natural images where semantic
content is independent of absolute pixel positions. This property is essential
to the success of CNNs in audio, image and video recognition tasks. In this
thesis, we extend equivariance to other kinds of transformations, such as
rotation and scaling. We propose equivariant models for different
transformations defined by groups of symmetries. The main contributions are (i)
polar transformer networks, achieving equivariance to the group of similarities
on the plane, (ii) equivariant multi-view networks, achieving equivariance to
the group of symmetries of the icosahedron, (iii) spherical CNNs, achieving
equivariance to the continuous 3D rotation group, (iv) cross-domain image
embeddings, achieving equivariance to 3D rotations for 2D inputs, and (v)
spin-weighted spherical CNNs, generalizing the spherical CNNs and achieving
equivariance to 3D rotations for spherical vector fields. Applications include
image classification, 3D shape classification and retrieval, panoramic image
classification and segmentation, shape alignment and pose estimation. What
these models have in common is that they leverage symmetries in the data to
reduce sample and model complexity and improve generalization performance. The
advantages are more significant on (but not limited to) challenging tasks where
data is limited or input perturbations such as arbitrary rotations are present
A Unified Single-stage Learning Model for Estimating Fiber Orientation Distribution Functions on Heterogeneous Multi-shell Diffusion-weighted MRI
Diffusion-weighted (DW) MRI measures the direction and scale of the local
diffusion process in every voxel through its spectrum in q-space, typically
acquired in one or more shells. Recent developments in micro-structure imaging
and multi-tissue decomposition have sparked renewed attention to the radial
b-value dependence of the signal. Applications in tissue classification and
micro-architecture estimation, therefore, require a signal representation that
extends over the radial as well as angular domain. Multiple approaches have
been proposed that can model the non-linear relationship between the DW-MRI
signal and biological microstructure. In the past few years, many deep
learning-based methods have been developed towards faster inference speed and
higher inter-scan consistency compared with traditional model-based methods
(e.g., multi-shell multi-tissue constrained spherical deconvolution). However,
a multi-stage learning strategy is typically required since the learning
process relied on various middle representations, such as simple harmonic
oscillator reconstruction (SHORE) representation. In this work, we present a
unified dynamic network with a single-stage spherical convolutional neural
network, which allows efficient fiber orientation distribution function (fODF)
estimation through heterogeneous multi-shell diffusion MRI sequences. We study
the Human Connectome Project (HCP) young adults with test-retest scans. From
the experimental results, the proposed single-stage method outperforms prior
multi-stage approaches in repeated fODF estimation with shell dropoff and
single-shell DW-MRI sequences
Scalable and equivariant spherical CNNs by discrete-continuous (DISCO) convolutions
No existing spherical convolutional neural network (CNN) framework is both
computationally scalable and rotationally equivariant. Continuous approaches
capture rotational equivariance but are often prohibitively computationally demanding. Discrete approaches offer more favorable computational performance
but at the cost of equivariance. We develop a hybrid discrete-continuous (DISCO)
group convolution that is simultaneously equivariant and computationally scalable
to high-resolution. While our framework can be applied to any compact group, we
specialize to the sphere. Our DISCO spherical convolutions exhibit SO(3) rotational equivariance, where SO(n) is the special orthogonal group representing
rotations in n-dimensions. When restricting rotations of the convolution to the
quotient space SO(3)/SO(2) for further computational enhancements, we recover
a form of asymptotic SO(3) rotational equivariance. Through a sparse tensor implementation we achieve linear scaling in number of pixels on the sphere for both
computational cost and memory usage. For 4k spherical images we realize a saving of 109
in computational cost and 104
in memory usage when compared to the
most efficient alternative equivariant spherical convolution. We apply the DISCO
spherical CNN framework to a number of benchmark dense-prediction problems
on the sphere, such as semantic segmentation and depth estimation, on all of which
we achieve the state-of-the-art performance
- …