7 research outputs found
Characteristic Kernels and Infinitely Divisible Distributions
We connect shift-invariant characteristic kernels to infinitely divisible
distributions on . Characteristic kernels play an important
role in machine learning applications with their kernel means to distinguish
any two probability measures. The contribution of this paper is two-fold.
First, we show, using the L\'evy-Khintchine formula, that any shift-invariant
kernel given by a bounded, continuous and symmetric probability density
function (pdf) of an infinitely divisible distribution on is
characteristic. We also present some closure property of such characteristic
kernels under addition, pointwise product, and convolution. Second, in
developing various kernel mean algorithms, it is fundamental to compute the
following values: (i) kernel mean values , , and
(ii) kernel mean RKHS inner products , for probability measures . If , and
kernel are Gaussians, then computation (i) and (ii) results in Gaussian
pdfs that is tractable. We generalize this Gaussian combination to more general
cases in the class of infinitely divisible distributions. We then introduce a
{\it conjugate} kernel and {\it convolution trick}, so that the above (i) and
(ii) have the same pdf form, expecting tractable computation at least in some
cases. As specific instances, we explore -stable distributions and a
rich class of generalized hyperbolic distributions, where the Laplace, Cauchy
and Student-t distributions are included
Matrix Infinitely Divisible Series: Tail Inequalities and Applications in Optimization
In this paper, we study tail inequalities of the largest eigenvalue of a
matrix infinitely divisible (i.d.) series, which is a finite sum of fixed
matrices weighted by i.d. random variables. We obtain several types of tail
inequalities, including Bennett-type and Bernstein-type inequalities. This
allows us to further bound the expectation of the spectral norm of a matrix
i.d. series. Moreover, by developing a new lower-bound function for
that appears in the Bennett-type inequality, we derive
a tighter tail inequality of the largest eigenvalue of the matrix i.d. series
than the Bernstein-type inequality when the matrix dimension is high. The
resulting lower-bound function is of independent interest and can improve any
Bennett-type concentration inequality that involves the function . The
class of i.d. probability distributions is large and includes Gaussian and
Poisson distributions, among many others. Therefore, our results encompass the
existing work \cite{tropp2012user} on matrix Gaussian series as a special case.
Lastly, we show that the tail inequalities of a matrix i.d. series have
applications in several optimization problems including the chance constrained
optimization problem and the quadratic optimization problem with orthogonality
constraints.Comment: Comments Welcome
Random variate generation for exponential and gamma tilted stable distributions
We develop a new efficient simulation scheme for sampling two families of tilted stable distributions: exponential tilted stable (ETS) and gamma tilted stable (GTS) distributions. Our scheme is based on two-dimensional single rejection. For the ETS family, its complexity is uniformly bounded over all ranges of parameters. This new algorithm outperforms all existing schemes. In particular, it is more efficient than the well-known double rejection scheme, which is the only algorithm with uniformly bounded complexity that we can find in the current literature. Beside the ETS family, our scheme is also flexible to be further extended for generating the GTS family, which cannot easily be done by extending the double rejection scheme. Our algorithms are straightforward to implement, and numerical experiments and tests are conducted to demonstrate the accuracy and efficiency
Machine learning with kernels for portfolio valuation and risk management
We introduce a simulation method for dynamic portfolio valuation and risk
management building on machine learning with kernels. We learn the dynamic
value process of a portfolio from a finite sample of its cumulative cash flow.
The learned value process is given in closed form thanks to a suitable choice
of the kernel. We show asymptotic consistency and derive finite sample error
bounds under conditions that are suitable for finance applications. Numerical
experiments show good results in large dimensions for a moderate training
sample size
Model-based kernel sum rule: kernel Bayesian inference with probabilistic model
Kernel Bayesian inference is a principled approach to nonparametric inference in probabilistic graphical models, where probabilistic relationships between variables are learned from data in a nonparametric manner. Various algorithms of kernel Bayesian inference have been developed by combining kernelized basic probabilistic operations such as the kernel sum rule and kernel Bayesâ rule. However, the current framework is fully nonparametric, and it does not allow a user to flexibly combine nonparametric and model-based inferences. This is inefficient when there are good probabilistic models (or simulation models) available for some parts of a graphical model; this is in particular true in scientific fields where âmodelsâ are the central topic of study. Our contribution in this paper is to introduce a novel approach, termed the model-based kernel sum rule (Mb-KSR), to combine a probabilistic model and kernel Bayesian inference. By combining the Mb-KSR with the existing kernelized probabilistic rules, one can develop various algorithms for hybrid (i.e., nonparametric and model-based) inferences. As an illustrative example, we consider Bayesian filtering in a state space model, where typically there exists an accurate probabilistic model for the state transition process. We propose a novel filtering method that combines model-based inference for the state transition process and data-driven, nonparametric inference for the observation generating process. We empirically validate our approach with synthetic and real-data experiments, the latter being the problem of vision-based mobile robot localization in robotics, which illustrates the effectiveness of the proposed hybrid approach