23 research outputs found
Inability of spatial transformations of CNN feature maps to support invariant recognition
A large number of deep learning architectures use spatial transformations of
CNN feature maps or filters to better deal with variability in object
appearance caused by natural image transformations. In this paper, we prove
that spatial transformations of CNN feature maps cannot align the feature maps
of a transformed image to match those of its original, for general affine
transformations, unless the extracted features are themselves invariant. Our
proof is based on elementary analysis for both the single- and multi-layer
network case. The results imply that methods based on spatial transformations
of CNN feature maps or filters cannot replace image alignment of the input and
cannot enable invariant recognition for general affine transformations,
specifically not for scaling transformations or shear transformations. For
rotations and reflections, spatially transforming feature maps or filters can
enable invariance but only for networks with learnt or hardcoded rotation- or
reflection-invariant featuresComment: 22 pages, 3 figure
SFNet: Learning Object-aware Semantic Correspondence
We address the problem of semantic correspondence, that is, establishing a
dense flow field between images depicting different instances of the same
object or scene category. We propose to use images annotated with binary
foreground masks and subjected to synthetic geometric deformations to train a
convolutional neural network (CNN) for this task. Using these masks as part of
the supervisory signal offers a good compromise between semantic flow methods,
where the amount of training data is limited by the cost of manually selecting
point correspondences, and semantic alignment ones, where the regression of a
single global geometric transformation between images may be sensitive to
image-specific details such as background clutter. We propose a new CNN
architecture, dubbed SFNet, which implements this idea. It leverages a new and
differentiable version of the argmax function for end-to-end training, with a
loss that combines mask and flow consistency with smoothness terms.
Experimental results demonstrate the effectiveness of our approach, which
significantly outperforms the state of the art on standard benchmarks.Comment: cvpr 2019 oral pape