Estimation of a regression function by maxima of minima of linear functions

The estimation of a multivariate regression function from independent and identically distributed random variables is considered. First we propose and analyse estimates which are defined by minimisation of the empirical L_{2} risk over a class of functions consisting of maxima of minima of linear functions. It is shown that the estimates are strongly universally consistent. Moreover results concerning the rate of convergence of the estimates with data-dependent parameter choice using "splitting the sample\u27; are derived in the case of an unbounded response variable. In particular it is shown that, for smooth regression functions satisfying the assumptions of single index models, the estimate is able to achieve (up to some logarithmic factor) the corresponding optimal one—dimesional rate of convergence. In this context it is remarkable that this newly proposed estimate can be computed in applications (see the appendix). Furthermore an L_{2} boosting algorithm for estimation of a regression function is presented. This method repeatedly fits a function from a fixed function space to the residuals of the data and the number of iteration steps is chosen data—dependently by "splitting the sample\u27;. A general result concerning the rate of convergence of the algorithm is derived in the case of an unbounded response variable. Finally this method is used to fit a sum of maxima of minima of linear functions to a given set of data. The derived rate of convergence of the corresponding estimate does not depend on the dimension of the observation variable.Die vorliegende Arbeit beschäftigt sich mit der Schätzung multivariater Regressionsfunktionen anhand von unabhängig und identisch verteilten Zufallsvariablen. Zunächst wird ein neues Schätzverfahren vorgestellt, welches auf der Minimierung des empirischen L_{2}—Risikos bezüglich einer Funktionenklasse, die aus Maxima von Minima von linearen Funtionen besteht, basiert. Für dieses Schätzverfahren wird zunächst die starke universelle Konsistenz nachgewiesen. Weiterhin werden sowohl für diesen Schätzer als auch für das entsprechende Schätzverfahren mit datenabhängiger Parameterwahl (mittels "Splitting the Sample") die entsprechenden Konvergenzraten hergeleitet. Diese Konvergenzraten gelten insbesondere auch dann, wenn die abhängige Variable unbeschränkt ist. Insbesondere wird gezeigt, dass unter den Voraussetzungen des "Single Index Models" die (bis auf einen logarithmischen Faktor) zugehörige optimale eindimensionale Konvergenzrate erreicht wird. Weiterhin wird in dieser Arbeit ein L_{2}—Boosting—Algorithmus zur Schätzung multivariater Regressionsfunktionen vorgestellt. Bei diesem Verfahren werden schrittweise Funktionen eines festgewählten Funktionenraumes an die Residuen der Daten angepasst. Auch hierbei erfolgt die Wahl der Anzahl der Iterationsschritte wieder datenabhängig. Es wird für diesen L_{2}—Boosting—Algorithmus zunächst ein allgemeines Resultat bezüglich der Konvergenzrate hergeleitet, welches auch in dem Fall einer unbeschränkten abhängigen Variablen gilt. Abschließend wird dieses Verfahren verwendet, um einen Schätzer zu konstruieren, der als Summe von Maxima von Minima von linearen Funktionen dargestellt werden kann. Die für diesen Schätzer hergeleitete Konvergenzrate hängt nicht mehr von der Dimension der unabhängigen Variablen ab

Estimation of a regression function by maxima of minima of linear functions

In this paper, estimation of a regression function from independent and identically distributed random variables is considered. Estimates are defined by minimization of the empirical L2 risk over a class of functions, which are defined as maxima of minima of linear functions. Results concerning the rate of convergence of the estimates are derived. In particular, it is shown that for smooth regression functions satisfying the assumption of single index models, the estimate is able to achieve (up to some logarithmic factor) the corresponding optimal one-dimensional rate of convergence. Hence, under these assumptions, the estimate is able to circumvent the so-called curse of dimensionality. The small sample behavior of the estimates is illustrated by applying them to simulated data. © 2009 IEEE

Modelling beyond Regression Functions: an Application of Multimodal Regression to Speed-Flow Data

An enormous amount of publications deals with smoothing in the sense of nonparametric regression. However, nearly all of the literature treats the case where predictors and response are related in the form of a function y=m(x)+noise. In many situations this simple functional model does not capture adequately the essential relation between predictor and response. We show by means of speed-flow diagrams, that a more general setting may be required, allowing for multifunctions instead of only functions. It turns out that in this case the conditional modes are more appropriate for the estimation of the underlying relation than the commonly used mean or the median. Estimation is achieved using a conditional mean-shift procedure, which is adapted to the present situation

A linear regression based cost function for WSN localization

Localization with Wireless Sensor Networks (WSN) creates new opportunities for location-based consumer communication applications. There is a great need for cost functions of maximum likelihood localization algorithms that are not only accurate but also lack local minima. In this paper we present Linear Regression based Cost Function for Localization (LiReCoFuL), a new cost function based on regression tools that fulfills these requirements. With empirical test results on a real-life test bed, we show that our cost function outperforms the accuracy of a minimum mean square error cost function. Furthermore we show that LiReCoFuL is as accurate as relative location estimation error cost functions and has very few local extremes

Statistical Inference using the Morse-Smale Complex

The Morse-Smale complex of a function $f$ decomposes the sample space into cells where $f$ is increasing or decreasing. When applied to nonparametric density estimation and regression, it provides a way to represent, visualize, and compare multivariate functions. In this paper, we present some statistical results on estimating Morse-Smale complexes. This allows us to derive new results for two existing methods: mode clustering and Morse-Smale regression. We also develop two new methods based on the Morse-Smale complex: a visualization technique for multivariate functions and a two-sample, multivariate hypothesis test.Comment: 45 pages, 13 figures. Accepted to Electronic Journal of Statistic

Landmark-Based Registration of Curves via the Continuous Wavelet Transform

This paper is concerned with the problem of the alignment of multiple sets of curves. We analyze two real examples arising from the biomedical area for which we need to test whether there are any statistically significant differences between two subsets of subjects. To synchronize a set of curves, we propose a new nonparametric landmark-based registration method based on the alignment of the structural intensity of the zero-crossings of a wavelet transform. The structural intensity is a multiscale technique recently proposed by Bigot (2003, 2005) which highlights the main features of a signal observed with noise. We conduct a simulation study to compare our landmark-based registration approach with some existing methods for curve alignment. For the two real examples, we compare the registered curves with FANOVA techniques, and a detailed analysis of the warping functions is provided

Locally Adaptive Function Estimation for Binary Regression Models

In this paper we present a nonparametric Bayesian approach for fitting unsmooth or highly oscillating functions in regression models with binary responses. The approach extends previous work by Lang et al. (2002) for Gaussian responses. Nonlinear functions are modelled by first or second order random walk priors with locally varying variances or smoothing parameters. Estimation is fully Bayesian and uses latent utility representations of binary regression models for efficient block sampling from the full conditionals of nonlinear functions

2-D Prony-Huang Transform: A New Tool for 2-D Spectral Analysis

This work proposes an extension of the 1-D Hilbert Huang transform for the analysis of images. The proposed method consists in (i) adaptively decomposing an image into oscillating parts called intrinsic mode functions (IMFs) using a mode decomposition procedure, and (ii) providing a local spectral analysis of the obtained IMFs in order to get the local amplitudes, frequencies, and orientations. For the decomposition step, we propose two robust 2-D mode decompositions based on non-smooth convex optimization: a "Genuine 2-D" approach, that constrains the local extrema of the IMFs, and a "Pseudo 2-D" approach, which constrains separately the extrema of lines, columns, and diagonals. The spectral analysis step is based on Prony annihilation property that is applied on small square patches of the IMFs. The resulting 2-D Prony-Huang transform is validated on simulated and real data.Comment: 24 pages, 7 figure