14 research outputs found
Efficient Statistics, in High Dimensions, from Truncated Samples
We provide an efficient algorithm for the classical problem, going back to
Galton, Pearson, and Fisher, of estimating, with arbitrary accuracy the
parameters of a multivariate normal distribution from truncated samples.
Truncated samples from a -variate normal means a samples is only revealed if it falls
in some subset ; otherwise the samples are hidden and
their count in proportion to the revealed samples is also hidden. We show that
the mean and covariance matrix can be
estimated with arbitrary accuracy in polynomial-time, as long as we have oracle
access to , and has non-trivial measure under the unknown -variate
normal distribution. Additionally we show that without oracle access to ,
any non-trivial estimation is impossible.Comment: to appear at 59th Annual IEEE Symposium on Foundations of Computer
Science (FOCS), 201
Learning from Censored and Dependent Data: The case of Linear Dynamics
Observations from dynamical systems often exhibit irregularities, such as
censoring, where values are recorded only if they fall within a certain range.
Censoring is ubiquitous in practice, due to saturating sensors,
limit-of-detection effects, and image-frame effects. In light of recent
developments on learning linear dynamical systems (LDSs), and on censored
statistics with independent data, we revisit the decades-old problem of
learning an LDS, from censored observations (Lee and Maddala (1985); Zeger and
Brookmeyer (1986)). Here, the learner observes the state
if and only if belongs to some set . We
develop the first computationally and statistically efficient algorithm for
learning the system, assuming only oracle access to the sets . Our
algorithm, Stochastic Online Newton with Switching Gradients, is a novel
second-order method that builds on the Online Newton Step (ONS) of Hazan et al.
(2007). Our Switching-Gradient scheme does not always use (stochastic)
gradients of the function we want to optimize, which we call "censor-aware"
function. Instead, in each iteration, it performs a simple test to decide
whether to use the censor-aware, or another "censor-oblivious" function, for
getting a stochastic gradient.
In our analysis, we consider a "generic" Online Newton method, which uses
arbitrary vectors instead of gradients, and we prove an error-bound for it.
This can be used to appropriately design these vectors, leading to our
Switching-Gradient scheme. This framework significantly deviates from the
recent long line of works on censored statistics (e.g., Daskalakis et al.
(2018); Kontonis et al. (2019); Daskalakis et al. (2019)), which apply
Stochastic Gradient Descent (SGD), and their analysis reduces to establishing
conditions for off-the-shelf SGD-bounds