11,713 research outputs found
Testing Over-Identifying Restrictions without Consistent Estimation of the Asymptotic Covariance Matrix
We extend the KVB approach of Kiefer, Vogelsang, and Bunzel (2000, Econometrica) and Kiefer and Vogelsang (2002b, Econometric Theory) to construct a class of robust tests for over-identifying restrictions in the context of GMM. The proposed test does not require consistent estimation of the asymptotic covariance matrix but relies on kernel-based normalizing matrices to eliminate the nuisance parameters in the limit. Moreover, the proposed test is valid for any consistent GMM estimator, in contrast with the conventional test that requires the optimal GMM estimator, and hence is easy to implement. Our simulations show that the proposed test is properly sized and may even be more powerful than the conventional test computed with an inappropriate user-chosen parameter.generalized method of moments, kernel function, KVB approach, overidentifying restrictions, robust test
Communications-Inspired Projection Design with Application to Compressive Sensing
We consider the recovery of an underlying signal x \in C^m based on
projection measurements of the form y=Mx+w, where y \in C^l and w is
measurement noise; we are interested in the case l < m. It is assumed that the
signal model p(x) is known, and w CN(w;0,S_w), for known S_W. The objective is
to design a projection matrix M \in C^(l x m) to maximize key
information-theoretic quantities with operational significance, including the
mutual information between the signal and the projections I(x;y) or the Renyi
entropy of the projections h_a(y) (Shannon entropy is a special case). By
capitalizing on explicit characterizations of the gradients of the information
measures with respect to the projections matrix, where we also partially extend
the well-known results of Palomar and Verdu from the mutual information to the
Renyi entropy domain, we unveil the key operations carried out by the optimal
projections designs: mode exposure and mode alignment. Experiments are
considered for the case of compressive sensing (CS) applied to imagery. In this
context, we provide a demonstration of the performance improvement possible
through the application of the novel projection designs in relation to
conventional ones, as well as justification for a fast online projections
design method with which state-of-the-art adaptive CS signal recovery is
achieved.Comment: 25 pages, 7 figures, parts of material published in IEEE ICASSP 2012,
submitted to SIIM
Time Series Cluster Kernel for Learning Similarities between Multivariate Time Series with Missing Data
Similarity-based approaches represent a promising direction for time series
analysis. However, many such methods rely on parameter tuning, and some have
shortcomings if the time series are multivariate (MTS), due to dependencies
between attributes, or the time series contain missing data. In this paper, we
address these challenges within the powerful context of kernel methods by
proposing the robust \emph{time series cluster kernel} (TCK). The approach
taken leverages the missing data handling properties of Gaussian mixture models
(GMM) augmented with informative prior distributions. An ensemble learning
approach is exploited to ensure robustness to parameters by combining the
clustering results of many GMM to form the final kernel.
We evaluate the TCK on synthetic and real data and compare to other
state-of-the-art techniques. The experimental results demonstrate that the TCK
is robust to parameter choices, provides competitive results for MTS without
missing data and outstanding results for missing data.Comment: 23 pages, 6 figure
Automatic positive semidefinate HAC covariance matrix and GMM estimation
This paper proposes a new class of heteroskedastic and autocorrelation consistent (HAC) covariance matrix estimators. The standard HAC estimation method reweights estimators of the autocovariances. Here we initially smooth the data observations themselves using kernel functionâbased weights. The resultant HAC covariance matrix estimator is the normalized outer product of the smoothed random vectors and is therefore automatically positive semidefinite. A corresponding efficient GMM criterion may also be defined as a quadratic form in the smoothed moment indicators whose normalized minimand provides a test statistic for the overidentifying moment conditions
Sketching for Large-Scale Learning of Mixture Models
Learning parameters from voluminous data can be prohibitive in terms of
memory and computational requirements. We propose a "compressive learning"
framework where we estimate model parameters from a sketch of the training
data. This sketch is a collection of generalized moments of the underlying
probability distribution of the data. It can be computed in a single pass on
the training set, and is easily computable on streams or distributed datasets.
The proposed framework shares similarities with compressive sensing, which aims
at drastically reducing the dimension of high-dimensional signals while
preserving the ability to reconstruct them. To perform the estimation task, we
derive an iterative algorithm analogous to sparse reconstruction algorithms in
the context of linear inverse problems. We exemplify our framework with the
compressive estimation of a Gaussian Mixture Model (GMM), providing heuristics
on the choice of the sketching procedure and theoretical guarantees of
reconstruction. We experimentally show on synthetic data that the proposed
algorithm yields results comparable to the classical Expectation-Maximization
(EM) technique while requiring significantly less memory and fewer computations
when the number of database elements is large. We further demonstrate the
potential of the approach on real large-scale data (over 10 8 training samples)
for the task of model-based speaker verification. Finally, we draw some
connections between the proposed framework and approximate Hilbert space
embedding of probability distributions using random features. We show that the
proposed sketching operator can be seen as an innovative method to design
translation-invariant kernels adapted to the analysis of GMMs. We also use this
theoretical framework to derive information preservation guarantees, in the
spirit of infinite-dimensional compressive sensing
Probability density estimation with tunable kernels using orthogonal forward regression
A generalized or tunable-kernel model is proposed for probability density function estimation based on an orthogonal forward regression procedure. Each stage of the density estimation process determines a tunable kernel, namely, its center vector and diagonal covariance matrix, by minimizing a leave-one-out test criterion. The kernel mixing weights of the constructed sparse density estimate are finally updated using the multiplicative nonnegative quadratic programming algorithm to ensure the nonnegative and unity constraints, and this weight-updating process additionally has the desired ability to further reduce the model size. The proposed tunable-kernel model has advantages, in terms of model generalization capability and model sparsity, over the standard fixed-kernel model that restricts kernel centers to the training data points and employs a single common kernel variance for every kernel. On the other hand, it does not optimize all the model parameters together and thus avoids the problems of high-dimensional ill-conditioned nonlinear optimization associated with the conventional finite mixture model. Several examples are included to demonstrate the ability of the proposed novel tunable-kernel model to effectively construct a very compact density estimate accurately
Uncovering Causality from Multivariate Hawkes Integrated Cumulants
We design a new nonparametric method that allows one to estimate the matrix
of integrated kernels of a multivariate Hawkes process. This matrix not only
encodes the mutual influences of each nodes of the process, but also
disentangles the causality relationships between them. Our approach is the
first that leads to an estimation of this matrix without any parametric
modeling and estimation of the kernels themselves. A consequence is that it can
give an estimation of causality relationships between nodes (or users), based
on their activity timestamps (on a social network for instance), without
knowing or estimating the shape of the activities lifetime. For that purpose,
we introduce a moment matching method that fits the third-order integrated
cumulants of the process. We show on numerical experiments that our approach is
indeed very robust to the shape of the kernels, and gives appealing results on
the MemeTracker database
- âŠ