Bi-log-concave distribution functions

Nonparametric statistics for distribution functions F or densities f=F' under qualitative shape constraints provides an interesting alternative to classical parametric or entirely nonparametric approaches. We contribute to this area by considering a new shape constraint: F is said to be bi-log-concave, if both log(F) and log(1 - F) are concave. Many commonly considered distributions are compatible with this constraint. For instance, any c.d.f. F with log-concave density f = F' is bi-log-concave. But in contrast to the latter constraint, bi-log-concavity allows for multimodal densities. We provide various characterizations. It is shown that combining any nonparametric confidence band for F with the new shape-constraint leads to substantial improvements, particularly in the tails. To pinpoint this, we show that these confidence bands imply non-trivial confidence bounds for arbitrary moments and the moment generating function of F

logcondens: Computations Related to Univariate Log-Concave Density Estimation

Maximum likelihood estimation of a log-concave density has attracted considerable attention over the last few years. Several algorithms have been proposed to estimate such a density. Two of those algorithms, an iterative convex minorant and an active set algorithm, are implemented in the R package logcondens. While these algorithms are discussed elsewhere, we describe in this paper the use of the logcondens package and discuss functions and datasets related to log-concave density estimation contained in the package. In particular, we provide functions to (1) compute the maximum likelihood estimate (MLE) as well as a smoothed log-concave density estimator derived from the MLE, (2) evaluate the estimated density, distribution and quantile functions at arbitrary points, (3) compute the characterizing functions of the MLE, (4) sample from the estimated distribution, and finally (5) perform a two-sample permutation test using a modified Kolmogorov-Smirnov test statistic. In addition, logcondens makes two datasets available that have been used to illustrate log-concave density estimation.

Nonparametric confidence bands in deconvolution density estimation

Uniform confidence bands for densities f via nonparametric kernel estimates were first constructed by Bickel and Rosenblatt [Ann. Statist. 1, 1071.1095]. In this paper this is extended to confidence bands in the deconvolution problem g = f for an ordinary smooth error density . Under certain regularity conditions, we obtain asymptotic uniform confidence bands based on the asymptotic distribution of the maximal deviation (LÉ-distance) between a deconvolution kernel estimator . f and f. Further consistency of the simple nonparametric bootstrap is proved. For our theoretical developments the bias is simply corrected by choosing an undersmoothing bandwidth. For practical purposes we propose a new data-driven bandwidth selector based on heuristic arguments, which aims --

Convergence analysis of generalized iteratively reweighted least squares algorithms on convex function spaces

The computation of robust regression estimates often relies on minimization of a convex functional on a convex set. In this paper we discuss a general technique for a large class of convex functionals to compute the minimizers iteratively which is closely related to majorization-minimization algorithms. Our approach is based on a quadratic approximation of the functional to be minimized and includes the iteratively reweighted least squares algorithm as a special case. We prove convergence on convex function spaces for general coercive and convex functionals F and derive geometric convergence in certain unconstrained settings. The algorithm is applied to TV penalized quantile regression and is compared with a step size corrected Newton-Raphson algorithm. It is found that typically in the first steps the iteratively reweighted least squares algorithm performs significantly better, whereas the Newton type method outpaces the former only after many iterations. Finally, in the setting of bivariate regression with unimodality constraints we illustrate how this algorithm allows to utilize highly efficient algorithms for special quadratic programs in more complex settings. --regression analysis,monotone regression,quantile regression,shape constraints,L1 regression,nonparametric regression,total variation semi-norm,reweighted least squares,Fermat's problem,convex approximation,quadratic approximation,pool adjacent violators algorithm

Modulation Estimators and Confidence Sets

An unknown signal plus white noise is observed at n discretetime points. Within a large convex class of linear estimators of the signal, we choose the one which minimizes estimated quadratic risk. By construction,the resulting estimator is nonlinear. This estimation is done after orthogonal transformation of the data to a reasonable coordinate system. The procedure adaptively tapers the coefficients of the transformed data. If the class of candidate estimators satisfies a uniform entropy condition, then our estimator is asymptotically minimax in Pinsker's sense over certain ellipsoids in the parameter space and dominates the James-Stein estimatorasymptotically. We describe computational algorithms for the modulation estimator and construct confidence sets for the unknown signal.These confidence sets are centered at the estimator, have correctasymptotic coverage probability, and have relatively small risk asset-valued estimators of the signal

Optimal Nonparametric Testing of Qualitative Hypotheses

Suppose one observes a process Y on the unit interval, where dY = ƒ + n-1/2dW with an unknown function parameter ƒ, given scale parameter n ≥ 1 ("sample size") and standard Brownian motion W. We propose two classes of tests of qualitative nonparametric hypotheses about ƒ such as monotonicity or convexity. These tests are asymptotically optimal and adaptive with respect to two different criteria. As a by-product we obtain an extension of L'evy extasciiacutes modulus of continuity of Brownian motion. It is of independent interest because of its potential applications to simultaneous confidence intervals in nonparametric curve estimation

Least squares and shrinkage estimation under bimonotonicity constraints

In this paper we describe active set type algorithms for minimization of a smooth function under general order constraints, an important case being functions on the set of bimonotone r×s matrices. These algorithms can be used, for instance, to estimate a bimonotone regression function via least squares or (a smooth approximation of) least absolute deviations. Another application is shrinkage estimation in image denoising or, more generally, regression problems with two ordinal factors after representing the data in a suitable basis which is indexed by pairs (i,j)∈{1, ,r}×{1, ,s}. Various numerical examples illustrate our method

Some new inequalities for beta distributions

This note provides some new tail inequalities and exponential inequalities of Hoeffding and Bernstein type for beta distributions

Covariate Selection Based on a Model-free Approach to Linear Regression with Exact Probabilities

In this paper we propose a completely new approach to the problem of covariate selection in linear regression. It is intuitive, very simple, fast and powerful, non-frequentist and non-Bayesian. It does not overfit, there is no shrinkage of the least squares coefficients, and it is model-free. A covariate or a set of covariates is included only if they are better in the sense of least squares than the same number of Gaussian covariates consisting of i.i.d. $N(0,1)$ random variables. The degree to which they are better is measured by the P-value which is the probability that the Gaussian covariates are better. This probability is given in terms of the Beta distribution, it is exact and it holds for the data at hand whatever this may be. The idea extends to a stepwise procedure, the main contribution of the paper, where the best of the remaining covariates is only accepted if it is better than the best of the same number of random Gaussian covariates. Again this probability is given in terms of the Beta distribution, it is exact and it holds for the data at hand whatever this may be. We use a version with default parameters which works for a large collection of known data sets with up to a few hundred thousand covariates. The computing time for the largest data sets was about four seconds, and it outperforms all other selection procedures of which we are aware. The paper gives the results of simulations, applications to real data sets and theorems on the asymptotic behaviour under the standard linear model. An R-package {\it gausscov} is available. \Comment: 40 pages, 5 figure