6 research outputs found
OSLNet:Deep Small-Sample Classification with an Orthogonal Softmax Layer
A deep neural network of multiple nonlinear layers forms a large function
space, which can easily lead to overfitting when it encounters small-sample
data. To mitigate overfitting in small-sample classification, learning more
discriminative features from small-sample data is becoming a new trend. To this
end, this paper aims to find a subspace of neural networks that can facilitate
a large decision margin. Specifically, we propose the Orthogonal Softmax Layer
(OSL), which makes the weight vectors in the classification layer remain
orthogonal during both the training and test processes. The Rademacher
complexity of a network using the OSL is only , where is the
number of classes, of that of a network using the fully connected
classification layer, leading to a tighter generalization error bound.
Experimental results demonstrate that the proposed OSL has better performance
than the methods used for comparison on four small-sample benchmark datasets,
as well as its applicability to large-sample datasets. Codes are available at:
https://github.com/dongliangchang/OSLNet.Comment: TIP 2020. Code available at https://github.com/dongliangchang/OSLNe
Domain Adaptation on Graphs by Learning Aligned Graph Bases
A common assumption in semi-supervised learning with graph models is that the
class label function varies smoothly on the data graph, resulting in the rather
strict prior that the label function has low-frequency content. Meanwhile, in
many classification problems, the label function may vary abruptly in certain
graph regions, resulting in high-frequency components. Although the
semi-supervised estimation of class labels is an ill-posed problem in general,
in several applications it is possible to find a source graph on which the
label function has similar frequency content to that on the target graph where
the actual classification problem is defined. In this paper, we propose a
method for domain adaptation on graphs motivated by these observations. Our
algorithm is based on learning the spectrum of the label function in a source
graph with many labeled nodes, and transferring the information of the spectrum
to the target graph with fewer labeled nodes. While the frequency content of
the class label function can be identified through the graph Fourier transform,
it is not easy to transfer the Fourier coefficients directly between the two
graphs, since no one-to-one match exists between the Fourier basis vectors of
independently constructed graphs in the domain adaptation setting. We solve
this problem by learning a transformation between the Fourier bases of the two
graphs that flexibly ``aligns'' them. The unknown class label function on the
target graph is then reconstructed such that its spectrum matches that on the
source graph while also ensuring the consistency with the available labels. The
proposed method is tested in the classification of image, online product
review, and social network data sets. Comparative experiments suggest that the
proposed algorithm performs better than recent domain adaptation methods in the
literature in most settings
An embarrassingly simple approach to visual domain adaptation
We show that it is possible to achieve high-quality domain adaptation without explicit adaptation. The nature of the classification problem means that when samples from the same class in different domains are sufficiently close, and samples from differing classes are separated by large enough margins, there is a high probability that each will be classified correctly. Inspired by this, we propose an embarrassingly simple yet effective approach to domain adaptation-only the class mean is used to learn class-specific linear projections. Learning these projections is naturally cast into a linear-discriminant-analysis-like framework, which gives an efficient, closed form solution. Furthermore, to enable to application of this approach to unsupervised learning, an iterative validation strategy is developed to infer target labels. Extensive experiments on cross-domain visual recognition demonstrate that, even with the simplest formulation, our approach outperforms existing non-deep adaptation methods and exhibits classification performance comparable with that of modern deep adaptation methods. An analysis of potential issues effecting the practical application of the method is also described, including robustness, convergence, and the impact of small sample sizes.Hao Lu, Chunhua Shen, Zhiguo Cao , Yang Xiao , and Anton van den Henge