Kernel estimation of Greek weights by parameter randomization

A Greek weight associated to a parameterized random variable $Z(\lambda)$ is a random variable $\pi$ such that $\nabla_{\lambda}E[\phi(Z(\lambda))]=E[\phi(Z(\lambda))\pi]$ for any function $\phi$. The importance of the set of Greek weights for the purpose of Monte Carlo simulations has been highlighted in the recent literature. Our main concern in this paper is to devise methods which produce the optimal weight, which is well known to be given by the score, in a general context where the density of $Z(\lambda)$ is not explicitly known. To do this, we randomize the parameter $\lambda$ by introducing an a priori distribution, and we use classical kernel estimation techniques in order to estimate the score function. By an integration by parts argument on the limit of this first kernel estimator, we define an alternative simpler kernel-based estimator which turns out to be closely related to the partial gradient of the kernel-based estimator of $\mathbb{E}[\phi(Z(\lambda))]$. Similarly to the finite differences technique, and unlike the so-called Malliavin method, our estimators are biased, but their implementation does not require any advanced mathematical calculation. We provide an asymptotic analysis of the mean squared error of these estimators, as well as their asymptotic distributions. For a discontinuous payoff function, the kernel estimator outperforms the classical finite differences one in terms of the asymptotic rate of convergence. This result is confirmed by our numerical experiments.Comment: Published in at http://dx.doi.org/10.1214/105051607000000186 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

FFT-Based Fast Computation of Multivariate Kernel Estimators with Unconstrained Bandwidth Matrices

The problem of fast computation of multivariate kernel density estimation (KDE) is still an open research problem. In our view, the existing solutions do not resolve this matter in a satisfactory way. One of the most elegant and efficient approach utilizes the fast Fourier transform. Unfortunately, the existing FFT-based solution suffers from a serious limitation, as it can accurately operate only with the constrained (i.e., diagonal) multivariate bandwidth matrices. In this paper we describe the problem and give a satisfactory solution. The proposed solution may be successfully used also in other research problems, for example for the fast computation of the optimal bandwidth for KDE.Comment: 10 pages, 1 figure, R source code

JIDT: An information-theoretic toolkit for studying the dynamics of complex systems

Complex systems are increasingly being viewed as distributed information processing systems, particularly in the domains of computational neuroscience, bioinformatics and Artificial Life. This trend has resulted in a strong uptake in the use of (Shannon) information-theoretic measures to analyse the dynamics of complex systems in these fields. We introduce the Java Information Dynamics Toolkit (JIDT): a Google code project which provides a standalone, (GNU GPL v3 licensed) open-source code implementation for empirical estimation of information-theoretic measures from time-series data. While the toolkit provides classic information-theoretic measures (e.g. entropy, mutual information, conditional mutual information), it ultimately focusses on implementing higher-level measures for information dynamics. That is, JIDT focusses on quantifying information storage, transfer and modification, and the dynamics of these operations in space and time. For this purpose, it includes implementations of the transfer entropy and active information storage, their multivariate extensions and local or pointwise variants. JIDT provides implementations for both discrete and continuous-valued data for each measure, including various types of estimator for continuous data (e.g. Gaussian, box-kernel and Kraskov-Stoegbauer-Grassberger) which can be swapped at run-time due to Java's object-oriented polymorphism. Furthermore, while written in Java, the toolkit can be used directly in MATLAB, GNU Octave, Python and other environments. We present the principles behind the code design, and provide several examples to guide users.Comment: 37 pages, 4 figure

Bidirectional PageRank Estimation: From Average-Case to Worst-Case

We present a new algorithm for estimating the Personalized PageRank (PPR) between a source and target node on undirected graphs, with sublinear running-time guarantees over the worst-case choice of source and target nodes. Our work builds on a recent line of work on bidirectional estimators for PPR, which obtained sublinear running-time guarantees but in an average-case sense, for a uniformly random choice of target node. Crucially, we show how the reversibility of random walks on undirected networks can be exploited to convert average-case to worst-case guarantees. While past bidirectional methods combine forward random walks with reverse local pushes, our algorithm combines forward local pushes with reverse random walks. We also discuss how to modify our methods to estimate random-walk probabilities for any length distribution, thereby obtaining fast algorithms for estimating general graph diffusions, including the heat kernel, on undirected networks.Comment: Workshop on Algorithms and Models for the Web-Graph (WAW) 201

High-Dimensional Density Ratio Estimation with Extensions to Approximate Likelihood Computation

The ratio between two probability density functions is an important component of various tasks, including selection bias correction, novelty detection and classification. Recently, several estimators of this ratio have been proposed. Most of these methods fail if the sample space is high-dimensional, and hence require a dimension reduction step, the result of which can be a significant loss of information. Here we propose a simple-to-implement, fully nonparametric density ratio estimator that expands the ratio in terms of the eigenfunctions of a kernel-based operator; these functions reflect the underlying geometry of the data (e.g., submanifold structure), often leading to better estimates without an explicit dimension reduction step. We show how our general framework can be extended to address another important problem, the estimation of a likelihood function in situations where that function cannot be well-approximated by an analytical form. One is often faced with this situation when performing statistical inference with data from the sciences, due the complexity of the data and of the processes that generated those data. We emphasize applications where using existing likelihood-free methods of inference would be challenging due to the high dimensionality of the sample space, but where our spectral series method yields a reasonable estimate of the likelihood function. We provide theoretical guarantees and illustrate the effectiveness of our proposed method with numerical experiments.Comment: With supplementary materia