26 research outputs found
Compressively Sensed Image Recognition
Compressive Sensing (CS) theory asserts that sparse signal reconstruction is
possible from a small number of linear measurements. Although CS enables
low-cost linear sampling, it requires non-linear and costly reconstruction.
Recent literature works show that compressive image classification is possible
in CS domain without reconstruction of the signal. In this work, we introduce a
DCT base method that extracts binary discriminative features directly from CS
measurements. These CS measurements can be obtained by using (i) a random or a
pseudo-random measurement matrix, or (ii) a measurement matrix whose elements
are learned from the training data to optimize the given classification task.
We further introduce feature fusion by concatenating Bag of Words (BoW)
representation of our binary features with one of the two state-of-the-art
CNN-based feature vectors. We show that our fused feature outperforms the
state-of-the-art in both cases.Comment: 6 pages, submitted/accepted, EUVIP 201
Random Projections For Large-Scale Regression
Fitting linear regression models can be computationally very expensive in
large-scale data analysis tasks if the sample size and the number of variables
are very large. Random projections are extensively used as a dimension
reduction tool in machine learning and statistics. We discuss the applications
of random projections in linear regression problems, developed to decrease
computational costs, and give an overview of the theoretical guarantees of the
generalization error. It can be shown that the combination of random
projections with least squares regression leads to similar recovery as ridge
regression and principal component regression. We also discuss possible
improvements when averaging over multiple random projections, an approach that
lends itself easily to parallel implementation.Comment: 13 pages, 3 Figure
Private Incremental Regression
Data is continuously generated by modern data sources, and a recent challenge
in machine learning has been to develop techniques that perform well in an
incremental (streaming) setting. In this paper, we investigate the problem of
private machine learning, where as common in practice, the data is not given at
once, but rather arrives incrementally over time.
We introduce the problems of private incremental ERM and private incremental
regression where the general goal is to always maintain a good empirical risk
minimizer for the history observed under differential privacy. Our first
contribution is a generic transformation of private batch ERM mechanisms into
private incremental ERM mechanisms, based on a simple idea of invoking the
private batch ERM procedure at some regular time intervals. We take this
construction as a baseline for comparison. We then provide two mechanisms for
the private incremental regression problem. Our first mechanism is based on
privately constructing a noisy incremental gradient function, which is then
used in a modified projected gradient procedure at every timestep. This
mechanism has an excess empirical risk of , where is the
dimensionality of the data. While from the results of [Bassily et al. 2014]
this bound is tight in the worst-case, we show that certain geometric
properties of the input and constraint set can be used to derive significantly
better results for certain interesting regression problems.Comment: To appear in PODS 201