2 research outputs found
Geometry-aware Deep Transform
Many recent efforts have been devoted to designing sophisticated deep
learning structures, obtaining revolutionary results on benchmark datasets. The
success of these deep learning methods mostly relies on an enormous volume of
labeled training samples to learn a huge number of parameters in a network;
therefore, understanding the generalization ability of a learned deep network
cannot be overlooked, especially when restricted to a small training set, which
is the case for many applications. In this paper, we propose a novel deep
learning objective formulation that unifies both the classification and metric
learning criteria. We then introduce a geometry-aware deep transform to enable
a non-linear discriminative and robust feature transform, which shows
competitive performance on small training sets for both synthetic and
real-world data. We further support the proposed framework with a formal
-robustness analysis.Comment: to appear in ICCV2015, updated with minor revisio
Deep Neural Networks with Random Gaussian Weights: A Universal Classification Strategy?
Three important properties of a classification machinery are: (i) the system
preserves the core information of the input data; (ii) the training examples
convey information about unseen data; and (iii) the system is able to treat
differently points from different classes. In this work we show that these
fundamental properties are satisfied by the architecture of deep neural
networks. We formally prove that these networks with random Gaussian weights
perform a distance-preserving embedding of the data, with a special treatment
for in-class and out-of-class data. Similar points at the input of the network
are likely to have a similar output. The theoretical analysis of deep networks
here presented exploits tools used in the compressed sensing and dictionary
learning literature, thereby making a formal connection between these important
topics. The derived results allow drawing conclusions on the metric learning
properties of the network and their relation to its structure, as well as
providing bounds on the required size of the training set such that the
training examples would represent faithfully the unseen data. The results are
validated with state-of-the-art trained networks.Comment: 14 pages, 13 figure