Highly efficient hierarchical online nonlinear regression using second order methods

We introduce highly efficient online nonlinear regression algorithms that are suitable for real life applications. We process the data in a truly online manner such that no storage is needed, i.e., the data is discarded after being used. For nonlinear modeling we use a hierarchical piecewise linear approach based on the notion of decision trees where the space of the regressor vectors is adaptively partitioned based on the performance. As the first time in the literature, we learn both the piecewise linear partitioning of the regressor space as well as the linear models in each region using highly effective second order methods, i.e., Newton–Raphson Methods. Hence, we avoid the well known over fitting issues by using piecewise linear models, however, since both the region boundaries as well as the linear models in each region are trained using the second order methods, we achieve substantial performance compared to the state of the art. We demonstrate our gains over the well known benchmark data sets and provide performance results in an individual sequence manner guaranteed to hold without any statistical assumptions. Hence, the introduced algorithms address computational complexity issues widely encountered in real life applications while providing superior guaranteed performance in a strong deterministic sense. © 2017 Elsevier B.V

A comparative evaluation of nonlinear dynamics methods for time series prediction

A key problem in time series prediction using autoregressive models is to fix the model order, namely the number of past samples required to model the time series adequately. The estimation of the model order using cross-validation may be a long process. In this paper, we investigate alternative methods to cross-validation, based on nonlinear dynamics methods, namely Grassberger-Procaccia, K,gl, Levina-Bickel and False Nearest Neighbors algorithms. The experiments have been performed in two different ways. In the first case, the model order has been used to carry out the prediction, performed by a SVM for regression on three real data time series showing that nonlinear dynamics methods have performances very close to the cross-validation ones. In the second case, we have tested the accuracy of nonlinear dynamics methods in predicting the known model order of synthetic time series. In this case, most of the methods have yielded a correct estimate and when the estimate was not correct, the value was very close to the real one

Blood pressure estimation using pulse transit time models

Abstract. Blood pressure (BP) is an important indicator of human health. Common methods for measuring BP continuously are either invasive, intermittent or they require using a cumbersome cuff. Pulse Transmit Time (PTT) -based measurement can be an alternative for such methods, as it ensures continue and non-invasive monitoring. However, since the method is indirect, it requires careful modelling of PTT-BP relation. In this thesis, three approaches of BP estimation from PTT are tested: linear regression, nonlinear Moens and Korteweg model and nonlinear model developed by Gesche. In the experiments, cardiovascular pulses for PTT were sensed using two fiber optics based accelerometers developed at the University of Oulu. To evaluate feasibility of presented models, the results were compared with reference BP values, measured using methods accepted for the commercial use. There were two groups of data. One was compared with BP measured using invasive catheter. Second group was compared with BP measured using volume clamp method. Obtained results suggest, that the presented calculation methods in present state still require further development in order to provide accurate BP values, however, they can be potentially used for observation of BP changes

DETERMINATION OF THE BEST METHODOLOGY FOR BATHYMETRY MAPPING USING SPOT 6 IMAGERY: A STUDY OF 12 EMPIRICAL ALGORITHMS

For the past four decades, many researchers have published a novel empirical methodology for bathymetry extraction using remote sensing data. However, a comparative analysis of each method has not yet been done. Which is important to determine the best method that gives a good accuracy prediction. This study focuses on empirical bathymetry extraction methodology for multispectral data with three visible band, specifically SPOT 6 Image. Twelve algorithms have been chosen intentionally, namely, 1) Ratio transform (RT); 2) Multiple linear regression (MLR); 3) Multiple nonlinear regression (RF); 4) Second-order polynomial of ratio transform (SPR); 5) Principle component (PC); 6) Multiple linear regression using relaxing uniformity assumption on water and atmosphere (KNW); 7) Semiparametric regression using depth-independent variables (SMP); 8) Semiparametric regression using spatial coordinates (STR); 9) Semiparametric regression using depth-independent variables and spatial coordinates (TNP), 10) bagging fitting ensemble (BAG); 11) least squares boosting fitting ensemble (LSB); and 12) support vector regression (SVR). This study assesses the performance of 12 empirical models for bathymetry calculations in two different areas: Gili Mantra Islands, West Nusa Tenggara and Menjangan Island, Bali. The estimated depth from each method was compared with echosounder data; RF, STR, and TNP results demonstrate higher accuracy ranges from 0.02 to 0.63 m more than other nine methods. The TNP algorithm, producing the most accurate results (Gili Mantra Island RMSE = 1.01 m and R2=0.82, Menjangan Island RMSE = 1.09 m and R2=0.45), proved to be the preferred algorithm for bathymetry mapping

Nonlinear quantile mixed models

In regression applications, the presence of nonlinearity and correlation among observations offer computational challenges not only in traditional settings such as least squares regression, but also (and especially) when the objective function is non-smooth as in the case of quantile regression. In this paper, we develop methods for the modeling and estimation of nonlinear conditional quantile functions when data are clustered within two-level nested designs. This work represents an extension of the linear quantile mixed models of Geraci and Bottai (2014, Statistics and Computing). We develop a novel algorithm which is a blend of a smoothing algorithm for quantile regression and a second order Laplacian approximation for nonlinear mixed models. To assess the proposed methods, we present a simulation study and two applications, one in pharmacokinetics and one related to growth curve modeling in agriculture.Comment: 26 pages, 8 figures, 8 table

High-order regularized regression in Electrical Impedance Tomography

We present a novel approach for the inverse problem in electrical impedance tomography based on regularized quadratic regression. Our contribution introduces a new formulation for the forward model in the form of a nonlinear integral transform, that maps changes in the electrical properties of a domain to their respective variations in boundary data. Using perturbation theory the transform is approximated to yield a high-order misfit unction which is then used to derive a regularized inverse problem. In particular, we consider the nonlinear problem to second-order accuracy, hence our approximation method improves upon the local linearization of the forward mapping. The inverse problem is approached using Newton's iterative algorithm and results from simulated experiments are presented. With a moderate increase in computational complexity, the method yields superior results compared to those of regularized linear regression and can be implemented to address the nonlinear inverse problem