21 research outputs found
Recycling Randomness with Structure for Sublinear time Kernel Expansions
We propose a scheme for recycling Gaussian random vectors into structured
matrices to approximate various kernel functions in sublinear time via random
embeddings. Our framework includes the Fastfood construction as a special case,
but also extends to Circulant, Toeplitz and Hankel matrices, and the broader
family of structured matrices that are characterized by the concept of
low-displacement rank. We introduce notions of coherence and graph-theoretic
structural constants that control the approximation quality, and prove
unbiasedness and low-variance properties of random feature maps that arise
within our framework. For the case of low-displacement matrices, we show how
the degree of structure and randomness can be controlled to reduce statistical
variance at the cost of increased computation and storage requirements.
Empirical results strongly support our theory and justify the use of a broader
family of structured matrices for scaling up kernel methods using random
features
Deep Structured Features for Semantic Segmentation
We propose a highly structured neural network architecture for semantic
segmentation with an extremely small model size, suitable for low-power
embedded and mobile platforms. Specifically, our architecture combines i) a
Haar wavelet-based tree-like convolutional neural network (CNN), ii) a random
layer realizing a radial basis function kernel approximation, and iii) a linear
classifier. While stages i) and ii) are completely pre-specified, only the
linear classifier is learned from data. We apply the proposed architecture to
outdoor scene and aerial image semantic segmentation and show that the accuracy
of our architecture is competitive with conventional pixel classification CNNs.
Furthermore, we demonstrate that the proposed architecture is data efficient in
the sense of matching the accuracy of pixel classification CNNs when trained on
a much smaller data set.Comment: EUSIPCO 2017, 5 pages, 2 figure
Matrix Infinitely Divisible Series: Tail Inequalities and Applications in Optimization
In this paper, we study tail inequalities of the largest eigenvalue of a
matrix infinitely divisible (i.d.) series, which is a finite sum of fixed
matrices weighted by i.d. random variables. We obtain several types of tail
inequalities, including Bennett-type and Bernstein-type inequalities. This
allows us to further bound the expectation of the spectral norm of a matrix
i.d. series. Moreover, by developing a new lower-bound function for
that appears in the Bennett-type inequality, we derive
a tighter tail inequality of the largest eigenvalue of the matrix i.d. series
than the Bernstein-type inequality when the matrix dimension is high. The
resulting lower-bound function is of independent interest and can improve any
Bennett-type concentration inequality that involves the function . The
class of i.d. probability distributions is large and includes Gaussian and
Poisson distributions, among many others. Therefore, our results encompass the
existing work \cite{tropp2012user} on matrix Gaussian series as a special case.
Lastly, we show that the tail inequalities of a matrix i.d. series have
applications in several optimization problems including the chance constrained
optimization problem and the quadratic optimization problem with orthogonality
constraints.Comment: Comments Welcome