On the Generalized Ratio of Uniforms as a Combination of Transformed Rejection and Extended Inverse of Density Sampling

Documento depositado en el repositorio arXiv.org. Versión: arXiv:1205.0482v6 [stat.CO]In this work we investigate the relationship among three classical sampling techniques: the inverse of density (Khintchine's theorem), the transformed rejection (TR) and the generalized ratio of uniforms (GRoU). Given a monotonic probability density function (PDF), we show that the transformed area obtained using the generalized ratio of uniforms method can be found equivalently by applying the transformed rejection sampling approach to the inverse function of the target density. Then we provide an extension of the classical inverse of density idea, showing that it is completely equivalent to the GRoU method for monotonic densities. Although we concentrate on monotonic probability density functions (PDFs), we also discuss how the results presented here can be extended to any non-monotonic PDF that can be decomposed into a collection of intervals where it is monotonically increasing or decreasing. In this general case, we show the connections with transformations of certain random variables and the generalized inverse PDF with the GRoU technique. Finally, we also introduce a GRoU technique to handle unbounded target densities

Nonparametric Likelihood Ratio Test for Univariate Shape-constrained Densities

We provide a comprehensive study of a nonparametric likelihood ratio test on whether a random sample follows a distribution in a prespecified class of shape-constrained densities. While the conventional definition of likelihood ratio is not well-defined for general nonparametric problems, we consider a working sub-class of alternative densities that leads to test statistics with desirable properties. Under the null, a scaled and centered version of the test statistic is asymptotic normal and distribution-free, which comes from the fact that the asymptotic dominant term under the null depends only on a function of spacings of transformed outcomes that are uniform distributed. The nonparametric maximum likelihood estimator (NPMLE) under the hypothesis class appears only in an average log-density ratio which often converges to zero at a faster rate than the asymptotic normal term under the null, while diverges in general test so that the test is consistent. The main technicality is to show these results for log-density ratio which requires a case-by-case analysis, including new results for k-monotone densities with unbounded support and completely monotone densities that are of independent interest. A bootstrap method by simulating from the NPMLE is shown to have the same limiting distribution as the test statistic

Exact Box-Cox Analysis

The Box-Cox method has been widely used to improve estimation accuracy in different fields, especially in econometrics and time series. In this thesis, we initially review the Box-Cox transformation [Box and Cox, 1964] and other alternative parametric power transformations. Following, the maximum likelihood method for the Box-Cox transformation is presented by discussing the problems of previous approaches in the literature. This work consists of the exact analysis of Box-Cox transformation taking into account the truncation effect in the transformed domain. We introduce a new family of distributions for the Box-Cox transformation in the original and transformed data scales. A likelihood analysis of the Box-Cox distribution is presented when truncation is considered. It is shown that numerical problems may arise in prediction and simulation when the truncation effect is ignored. A new algorithm has been developed for simulating Box-Cox transformed time series since previous methods are inefficient or unreliable. An application to sunspot data is discussed. Box-Cox analysis is employed for random forest regression prediction using cross-validation instead of MLE to estimate the transformation. An application to Boston housing dataset demonstrates that this technique can substantially improve prediction accuracy

Bayesian inference with optimal maps

We present a new approach to Bayesian inference that entirely avoids Markov chain simulation, by constructing a map that pushes forward the prior measure to the posterior measure. Existence and uniqueness of a suitable measure-preserving map is established by formulating the problem in the context of optimal transport theory. We discuss various means of explicitly parameterizing the map and computing it efficiently through solution of an optimization problem, exploiting gradient information from the forward model when possible. The resulting algorithm overcomes many of the computational bottlenecks associated with Markov chain Monte Carlo. Advantages of a map-based representation of the posterior include analytical expressions for posterior moments and the ability to generate arbitrary numbers of independent posterior samples without additional likelihood evaluations or forward solves. The optimization approach also provides clear convergence criteria for posterior approximation and facilitates model selection through automatic evaluation of the marginal likelihood. We demonstrate the accuracy and efficiency of the approach on nonlinear inverse problems of varying dimension, involving the inference of parameters appearing in ordinary and partial differential equations.United States. Dept. of Energy. Office of Advanced Scientific Computing Research (Grant DE-SC0002517)United States. Dept. of Energy. Office of Advanced Scientific Computing Research (Grant DE-SC0003908

Novel schemes for adaptive rejection sampling

We address the problem of generating random samples from a target probability distribution, with density function ₒ, using accept/reject methods. An "accept/reject sampler" (or, simply, a "rejection sampler") is an algorithm that first draws a random variate from a proposal distribution with density (where ≠ ₒ, in general) and then performs a test to determine whether the variate can be accepted as a sample from the target distribution or not. If we apply the algorithm repeatedly until we accept times, then we obtain a collection of independent and identically distributed (i.i.d.) samples from the distribution with density ₒ. The goal of the present work is to devise and analyze adaptive rejection samplers that can be applied to generate i.i.d. random variates from the broadest possible class of probability distributions. Adaptive rejection sampling algorithms typically construct a sequence of proposal functions ₒ, ₁,... ₁..., such that (a) it is easy to draw i.i.d. samples from them and (b) they converge, in some way, to the density ₒ of the target probability distribution. When surveying the literature, it is simple to identify several such methods but virtually all of them present severe limitations in the class of target densities,ₒ, for which they can be applied. The "standard" adaptive rejection sampler by Gilks and Wild, for instance, only works when ₒ is strictly log-concave. Through Chapters 3, 4 and 5 we introduce a new methodology for adaptive rejection sampling that can be used with a broad family of target probability densities (including, e.g., multimodal functions) and subsumes Gilks and Wild's method as a particular case. We discuss several variations of the main algorithm that enable, e.g., sampling from some particularly "difficult" distributions (for instance, cases where ₒ has log-convex tails and in nite support) or yield "automatic" software implementations using little analytical information about the target density ₒ. Several numerical examples, including comparisons with some of the most relevant techniques in the literature, are also shown in Chapter 6. ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------En este trabajo abordamos el problema de generar muestras aleatorias de una distribución de probabilidad objetivo, con densidad ₒ , empleando métodos de aceptación/rechazo. En un algoritmo de aceptación/rechazo, primero se genera una realización aleatoria de una distribución tentativa con densidad (donde ≠ ₒ , en general) y a continuación se realiza un test para determinar si la muestra se puede aceptar como proveniente de la distribución objetivo o no. Si se aplica el algoritmo repetidamente hasta aceptar veces, obtenemos una colección de muestras independientes y idénticamente distribuidas (i.i.d.) de la distribución con densidad ₒ. El objetivo del trabajo es proponer y analizar nuevos m étodos de aceptación/rechazo adaptativos que pueden ser aplicados para generar muestras i.i.d. de la clase más amplia posible de distribuciones de probabilidad. Los algoritmos de aceptación/rechazo adaptativos suelen construir una secuencia de funciones tentativas ₒ, ₁,... ₁..., tales que (a) sea fácil generar muestras i.i.d. a partir de ellas y (b) converjan, de manera adecuada, hacia la densidad ₒ de la distribución objetivo. Al revisar la literatura, es sencillo identificar varios métodos de este tipo pero todos ellos presentan limitaciones importantes en cuanto a las clases de densidades objetivo a las que se pueden aplicar. El método original de Gilks y Wild, por ejemplo, sólo funciona si ₒ es estrictamente log-cóncava. En los Capí tulos 3, 4 y 5 presentamos una nueva metodología para muestreo adaptativo por aceptación/rechazo que se puede utilizar con una amplia clase de densidades de probabilidad objetivo (incluyendo, por ejemplo, funciones multimodales) y comprende al método de Gilks y Wild como un caso particular. Se discuten diversas variaciones del algoritmo principal que facilitan, por ejemplo, el muestreo de algunas distribuciones particularmente "difíciles" (e.g., casos en los que ₒ tiene colas log-convexas y con soporte infinito) o una implementación software prácticamente "automática", en el sentido de que necesitamos poca información analítica acerca de la función ₒ. En el Capítulo 6 mostramos varios ejemplos numéricos, incluyendo comparaciones con algunas de las técnicas máas relevantes que se pueden encontrar en la literatura

A new class of nonparametric tests for second-order stochastic dominance based on the Lorenz P-P plot

Given samples from two non-negative random variables, we propose a family of tests for the null hypothesis that one random variable stochastically dominates the other at the second order. Test statistics are obtained as functionals of the difference between the identity and the Lorenz P-P plot, defined as the composition between the inverse unscaled Lorenz curve of one distribution and the unscaled Lorenz curve of the other. We determine upper bounds for such test statistics under the null hypothesis and derive their limit distribution, to be approximated via bootstrap procedures. We then establish the asymptotic validity of the tests under relatively mild conditions and investigate finite sample properties through simulations. The results show that our testing approach can be a valid alternative to classic methods based on the difference of the integrals of the cumulative distribution functions, which require bounded support and struggle to detect departures from the null in some cases