860 research outputs found
Private Incremental Regression
Data is continuously generated by modern data sources, and a recent challenge
in machine learning has been to develop techniques that perform well in an
incremental (streaming) setting. In this paper, we investigate the problem of
private machine learning, where as common in practice, the data is not given at
once, but rather arrives incrementally over time.
We introduce the problems of private incremental ERM and private incremental
regression where the general goal is to always maintain a good empirical risk
minimizer for the history observed under differential privacy. Our first
contribution is a generic transformation of private batch ERM mechanisms into
private incremental ERM mechanisms, based on a simple idea of invoking the
private batch ERM procedure at some regular time intervals. We take this
construction as a baseline for comparison. We then provide two mechanisms for
the private incremental regression problem. Our first mechanism is based on
privately constructing a noisy incremental gradient function, which is then
used in a modified projected gradient procedure at every timestep. This
mechanism has an excess empirical risk of , where is the
dimensionality of the data. While from the results of [Bassily et al. 2014]
this bound is tight in the worst-case, we show that certain geometric
properties of the input and constraint set can be used to derive significantly
better results for certain interesting regression problems.Comment: To appear in PODS 201
On the Power of Manifold Samples in Exploring Configuration Spaces and the Dimensionality of Narrow Passages
We extend our study of Motion Planning via Manifold Samples (MMS), a general
algorithmic framework that combines geometric methods for the exact and
complete analysis of low-dimensional configuration spaces with sampling-based
approaches that are appropriate for higher dimensions. The framework explores
the configuration space by taking samples that are entire low-dimensional
manifolds of the configuration space capturing its connectivity much better
than isolated point samples. The contributions of this paper are as follows:
(i) We present a recursive application of MMS in a six-dimensional
configuration space, enabling the coordination of two polygonal robots
translating and rotating amidst polygonal obstacles. In the adduced experiments
for the more demanding test cases MMS clearly outperforms PRM, with over
20-fold speedup in a coordination-tight setting. (ii) A probabilistic
completeness proof for the most prevalent case, namely MMS with samples that
are affine subspaces. (iii) A closer examination of the test cases reveals that
MMS has, in comparison to standard sampling-based algorithms, a significant
advantage in scenarios containing high-dimensional narrow passages. This
provokes a novel characterization of narrow passages which attempts to capture
their dimensionality, an attribute that had been (to a large extent) unattended
in previous definitions.Comment: 20 page
Recent advances in directional statistics
Mainstream statistical methodology is generally applicable to data observed
in Euclidean space. There are, however, numerous contexts of considerable
scientific interest in which the natural supports for the data under
consideration are Riemannian manifolds like the unit circle, torus, sphere and
their extensions. Typically, such data can be represented using one or more
directions, and directional statistics is the branch of statistics that deals
with their analysis. In this paper we provide a review of the many recent
developments in the field since the publication of Mardia and Jupp (1999),
still the most comprehensive text on directional statistics. Many of those
developments have been stimulated by interesting applications in fields as
diverse as astronomy, medicine, genetics, neurology, aeronautics, acoustics,
image analysis, text mining, environmetrics, and machine learning. We begin by
considering developments for the exploratory analysis of directional data
before progressing to distributional models, general approaches to inference,
hypothesis testing, regression, nonparametric curve estimation, methods for
dimension reduction, classification and clustering, and the modelling of time
series, spatial and spatio-temporal data. An overview of currently available
software for analysing directional data is also provided, and potential future
developments discussed.Comment: 61 page
The achievable performance of convex demixing
Demixing is the problem of identifying multiple structured signals from a
superimposed, undersampled, and noisy observation. This work analyzes a general
framework, based on convex optimization, for solving demixing problems. When
the constituent signals follow a generic incoherence model, this analysis leads
to precise recovery guarantees. These results admit an attractive
interpretation: each signal possesses an intrinsic degrees-of-freedom
parameter, and demixing can succeed if and only if the dimension of the
observation exceeds the total degrees of freedom present in the observation
- …