244 research outputs found
Parametric estimation and tests through divergences and duality technique
We introduce estimation and test procedures through divergence optimization
for discrete or continuous parametric models. This approach is based on a new
dual representation for divergences. We treat point estimation and tests for
simple and composite hypotheses, extending maximum likelihood technique. An
other view at the maximum likelihood approach, for estimation and test, is
given. We prove existence and consistency of the proposed estimates. The limit
laws of the estimates and test statistics (including the generalized likelihood
ratio one) are given both under the null and the alternative hypotheses, and
approximation of the power functions is deduced. A new procedure of
construction of confidence regions, when the parameter may be a boundary value
of the parameter space, is proposed. Also, a solution to the irregularity
problem of the generalized likelihood ratio test pertaining to the number of
components in a mixture is given, and a new test is proposed, based on -divergence on signed finite measures and duality technique
Functional kernel estimators of conditional extreme quantiles
We address the estimation of "extreme" conditional quantiles i.e. when their
order converges to one as the sample size increases. Conditions on the rate of
convergence of their order to one are provided to obtain asymptotically
Gaussian distributed kernel estimators. A Weissman-type estimator and kernel
estimators of the conditional tail-index are derived, permitting to estimate
extreme conditional quantiles of arbitrary order.Comment: arXiv admin note: text overlap with arXiv:1107.226
Self-consistent method for density estimation
The estimation of a density profile from experimental data points is a
challenging problem, usually tackled by plotting a histogram. Prior assumptions
on the nature of the density, from its smoothness to the specification of its
form, allow the design of more accurate estimation procedures, such as Maximum
Likelihood. Our aim is to construct a procedure that makes no explicit
assumptions, but still providing an accurate estimate of the density. We
introduce the self-consistent estimate: the power spectrum of a candidate
density is given, and an estimation procedure is constructed on the assumption,
to be released \emph{a posteriori}, that the candidate is correct. The
self-consistent estimate is defined as a prior candidate density that precisely
reproduces itself. Our main result is to derive the exact expression of the
self-consistent estimate for any given dataset, and to study its properties.
Applications of the method require neither priors on the form of the density
nor the subjective choice of parameters. A cutoff frequency, akin to a bin size
or a kernel bandwidth, emerges naturally from the derivation. We apply the
self-consistent estimate to artificial data generated from various
distributions and show that it reaches the theoretical limit for the scaling of
the square error with the dataset size.Comment: 21 pages, 5 figure
Solving Support Vector Machines in Reproducing Kernel Banach Spaces with Positive Definite Functions
In this paper we solve support vector machines in reproducing kernel Banach
spaces with reproducing kernels defined on nonsymmetric domains instead of the
traditional methods in reproducing kernel Hilbert spaces. Using the
orthogonality of semi-inner-products, we can obtain the explicit
representations of the dual (normalized-duality-mapping) elements of support
vector machine solutions. In addition, we can introduce the reproduction
property in a generalized native space by Fourier transform techniques such
that it becomes a reproducing kernel Banach space, which can be even embedded
into Sobolev spaces, and its reproducing kernel is set up by the related
positive definite function. The representations of the optimal solutions of
support vector machines (regularized empirical risks) in these reproducing
kernel Banach spaces are formulated explicitly in terms of positive definite
functions, and their finite numbers of coefficients can be computed by fixed
point iteration. We also give some typical examples of reproducing kernel
Banach spaces induced by Mat\'ern functions (Sobolev splines) so that their
support vector machine solutions are well computable as the classical
algorithms. Moreover, each of their reproducing bases includes information from
multiple training data points. The concept of reproducing kernel Banach spaces
offers us a new numerical tool for solving support vector machines.Comment: 26 page
Reproducing Kernels of Generalized Sobolev Spaces via a Green Function Approach with Distributional Operators
In this paper we introduce a generalized Sobolev space by defining a
semi-inner product formulated in terms of a vector distributional operator
consisting of finitely or countably many distributional operators
, which are defined on the dual space of the Schwartz space. The types of
operators we consider include not only differential operators, but also more
general distributional operators such as pseudo-differential operators. We
deduce that a certain appropriate full-space Green function with respect to
now becomes a conditionally positive
definite function. In order to support this claim we ensure that the
distributional adjoint operator of is
well-defined in the distributional sense. Under sufficient conditions, the
native space (reproducing-kernel Hilbert space) associated with the Green
function can be isometrically embedded into or even be isometrically
equivalent to a generalized Sobolev space. As an application, we take linear
combinations of translates of the Green function with possibly added polynomial
terms and construct a multivariate minimum-norm interpolant to data
values sampled from an unknown generalized Sobolev function at data sites
located in some set . We provide several examples, such
as Mat\'ern kernels or Gaussian kernels, that illustrate how many
reproducing-kernel Hilbert spaces of well-known reproducing kernels are
isometrically equivalent to a generalized Sobolev space. These examples further
illustrate how we can rescale the Sobolev spaces by the vector distributional
operator . Introducing the notion of scale as part of the
definition of a generalized Sobolev space may help us to choose the "best"
kernel function for kernel-based approximation methods.Comment: Update version of the publish at Num. Math. closed to Qi Ye's Ph.D.
thesis (\url{http://mypages.iit.edu/~qye3/PhdThesis-2012-AMS-QiYe-IIT.pdf}
Reproducing Kernel Banach Spaces with the l1 Norm
Targeting at sparse learning, we construct Banach spaces B of functions on an
input space X with the properties that (1) B possesses an l1 norm in the sense
that it is isometrically isomorphic to the Banach space of integrable functions
on X with respect to the counting measure; (2) point evaluations are continuous
linear functionals on B and are representable through a bilinear form with a
kernel function; (3) regularized learning schemes on B satisfy the linear
representer theorem. Examples of kernel functions admissible for the
construction of such spaces are given.Comment: 28 pages, an extra section was adde
Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many
statistics and machine learning applications ranging from support vector
machines to Gaussian processes and kernel embeddings of distributions.
Operators acting on such spaces are, for instance, required to embed
conditional probability distributions in order to implement the kernel Bayes
rule and build sequential data models. It was recently shown that transfer
operators such as the Perron-Frobenius or Koopman operator can also be
approximated in a similar fashion using covariance and cross-covariance
operators and that eigenfunctions of these operators can be obtained by solving
associated matrix eigenvalue problems. The goal of this paper is to provide a
solid functional analytic foundation for the eigenvalue decomposition of RKHS
operators and to extend the approach to the singular value decomposition. The
results are illustrated with simple guiding examples
- …