12,115 research outputs found
Importance sampling the union of rare events with an application to power systems analysis
We consider importance sampling to estimate the probability of a union
of rare events defined by a random variable . The
sampler we study has been used in spatial statistics, genomics and
combinatorics going back at least to Karp and Luby (1983). It works by sampling
one event at random, then sampling conditionally on that event
happening and it constructs an unbiased estimate of by multiplying an
inverse moment of the number of occuring events by the union bound. We prove
some variance bounds for this sampler. For a sample size of , it has a
variance no larger than where is the union
bound. It also has a coefficient of variation no larger than
regardless of the overlap pattern among the
events. Our motivating problem comes from power system reliability, where the
phase differences between connected nodes have a joint Gaussian distribution
and the rare events arise from unacceptably large phase differences. In the
grid reliability problems even some events defined by constraints in
dimensions, with probability below , are estimated with a
coefficient of variation of about with only sample
values
Multiscale inference for a multivariate density with applications to X-ray astronomy
In this paper we propose methods for inference of the geometric features of a
multivariate density. Our approach uses multiscale tests for the monotonicity
of the density at arbitrary points in arbitrary directions. In particular, a
significance test for a mode at a specific point is constructed. Moreover, we
develop multiscale methods for identifying regions of monotonicity and a
general procedure for detecting the modes of a multivariate density. It is is
shown that the latter method localizes the modes with an effectively optimal
rate. The theoretical results are illustrated by means of a simulation study
and a data example. The new method is applied to and motivated by the
determination and verification of the position of high-energy sources from
X-ray observations by the Swift satellite which is important for a
multiwavelength analysis of objects such as Active Galactic Nuclei.Comment: Keywords and Phrases: multiple tests, modes, multivariate density,
X-ray astronomy AMS Subject Classification: 62G07, 62G10, 62G2
Dynamic Tensor Clustering
Dynamic tensor data are becoming prevalent in numerous applications. Existing
tensor clustering methods either fail to account for the dynamic nature of the
data, or are inapplicable to a general-order tensor. Also there is often a gap
between statistical guarantee and computational efficiency for existing tensor
clustering solutions. In this article, we aim to bridge this gap by proposing a
new dynamic tensor clustering method, which takes into account both sparsity
and fusion structures, and enjoys strong statistical guarantees as well as high
computational efficiency. Our proposal is based upon a new structured tensor
factorization that encourages both sparsity and smoothness in parameters along
the specified tensor modes. Computationally, we develop a highly efficient
optimization algorithm that benefits from substantial dimension reduction. In
theory, we first establish a non-asymptotic error bound for the estimator from
the structured tensor factorization. Built upon this error bound, we then
derive the rate of convergence of the estimated cluster centers, and show that
the estimated clusters recover the true cluster structures with a high
probability. Moreover, our proposed method can be naturally extended to
co-clustering of multiple modes of the tensor data. The efficacy of our
approach is illustrated via simulations and a brain dynamic functional
connectivity analysis from an Autism spectrum disorder study.Comment: Accepted at Journal of the American Statistical Associatio
On clustering procedures and nonparametric mixture estimation
This paper deals with nonparametric estimation of conditional den-sities in
mixture models in the case when additional covariates are available. The
proposed approach consists of performing a prelim-inary clustering algorithm on
the additional covariates to guess the mixture component of each observation.
Conditional densities of the mixture model are then estimated using kernel
density estimates ap-plied separately to each cluster. We investigate the
expected L 1 -error of the resulting estimates and derive optimal rates of
convergence over classical nonparametric density classes provided the
clustering method is accurate. Performances of clustering algorithms are
measured by the maximal misclassification error. We obtain upper bounds of this
quantity for a single linkage hierarchical clustering algorithm. Lastly,
applications of the proposed method to mixture models involving elec-tricity
distribution data and simulated data are presented
- …