2 research outputs found
Hierarchical Spatial Transformer Network
Computer vision researchers have been expecting that neural networks have
spatial transformation ability to eliminate the interference caused by
geometric distortion for a long time. Emergence of spatial transformer network
makes dream come true. Spatial transformer network and its variants can handle
global displacement well, but lack the ability to deal with local spatial
variance. Hence how to achieve a better manner of deformation in the neural
network has become a pressing matter of the moment. To address this issue, we
analyze the advantages and disadvantages of approximation theory and optical
flow theory, then we combine them to propose a novel way to achieve image
deformation and implement it with a hierarchical convolutional neural network.
This new approach solves for a linear deformation along with an optical flow
field to model image deformation. In the experiments of cluttered MNIST
handwritten digits classification and image plane alignment, our method
outperforms baseline methods by a large margin
Linearized Multi-Sampling for Differentiable Image Transformation
We propose a novel image sampling method for differentiable image
transformation in deep neural networks. The sampling schemes currently used in
deep learning, such as Spatial Transformer Networks, rely on bilinear
interpolation, which performs poorly under severe scale changes, and more
importantly, results in poor gradient propagation. This is due to their strict
reliance on direct neighbors. Instead, we propose to generate random auxiliary
samples in the vicinity of each pixel in the sampled image, and create a linear
approximation with their intensity values. We then use this approximation as a
differentiable formula for the transformed image. We demonstrate that our
approach produces more representative gradients with a wider basin of
convergence for image alignment, which leads to considerable performance
improvements when training networks for classification tasks. This is not only
true under large downsampling, but also when there are no scale changes. We
compare our approach with multi-scale sampling and show that we outperform it.
We then demonstrate that our improvements to the sampler are compatible with
other tangential improvements to Spatial Transformer Networks and that it
further improves their performance