Robust Multivariate Regression

A robust multivariate linear regression estimator can be obtained by replacing the least squares estimator with the robust hbreg estimator. Then the robust multivariate linear regression estimator is asymptotically equivalent to the classical multivariate linear regression estimator since the probability that the robust estimator is equal to the classical estimator goes to one in probability as the sample size n tends to infinity for a large class of iid zero mean error distributions. This paper discusses the robust estimator and some tests using the robust estimator that are asymptotically equivalent to those using the classical estimator

A cautionary note on robust covariance plug-in methods

Many multivariate statistical methods rely heavily on the sample covariance matrix. It is well known though that the sample covariance matrix is highly non-robust. One popular alternative approach for "robustifying" the multivariate method is to simply replace the role of the covariance matrix with some robust scatter matrix. The aim of this paper is to point out that in some situations certain properties of the covariance matrix are needed for the corresponding robust "plug-in" method to be a valid approach, and that not all scatter matrices necessarily possess these important properties. In particular, the following three multivariate methods are discussed in this paper: independent components analysis, observational regression and graphical modeling. For each case, it is shown that using a symmetrized robust scatter matrix in place of the covariance matrix results in a proper robust multivariate method.Comment: 24 pages, 7 figure

Multivariate generalized S-estimators.

In this paper we introduce generalized S-estimators for the multivariate regression model. This class of estimators combines high robustness and high efficiency. They are defined by minimizing the determinant of a robust estimator of the scatter matrix of differences of residuals. In the special case of a multivariate location model, the generalized S-estimator has the important independency property, and can be used for high breakdown estimation in independent component analysis. Robustness properties of the estimators are investigated by deriving their breakdown point and the influence function. We also study the efficiency of the estimators, both a symptotically and a finite samples. To obtain inference for the regression parameters, we discuss the fast and robust boot strap for multivariate generalized S-estimators. The method is illustrated on several real data examples.bootstrap; efficiency; multivariate regression; robustness;

Robust online signal extraction from multivariate time series

We introduce robust regression-based online filters for multivariate time series and discuss their performance in real time signal extraction settings. We focus on methods that can deal with time series exhibiting patterns such as trends, level changes, outliers and a high level of noise as well as periods of a rather steady state. In particular, the data may be measured on a discrete scale which often occurs in practice. Our new filter is based on a robust two-step online procedure. We investigate its relevant properties and its performance by means of simulations and a medical application. --Multivariate time series,signal extraction,robust regression,online methods

Estimators of the multiple correlation coefficient: local robustness and confidence intervals.

Many robust regression estimators are defined by minimizing a measure of spread of the residuals. An accompanying R-2-measure, or multiple correlation coefficient, is then easily obtained. In this paper, local robustness properties of these robust R-2-coefficients axe investigated. It is also shown how confidence intervals for the population multiple correlation coefficient can be constructed in the case of multivariate normality.Cautionary note; High breakdown-point; Influence function; Intervals; Model; Multiple correlation coefficient; R-2-measure; Regression analysis; Residuals; Robustness; Squares regression;

Robust continuum regression.

Several applications of continuum regression to non-contaminated data have shown that a significant improvement in predictive power can be obtained compared to the three standard techniques which it encompasses (Ordinary least Squares, Principal Component Regression and Partial Least Squares). For contaminated data continuum regression may yield aberrant estimates due to its non-robustness with respect to outliers. Also for data originating from a distribution which significantly differs from the normal distribution, continuum regression may yield very inefficient estimates. In the current paper, robust continuum regression (RCR) is proposed. To construct the estimator, an algorithm based on projection pursuit is proposed. The robustness and good efficiency properties of RCR are shown by means of a simulation study. An application to an X-ray fluorescence analysis of hydrometallurgical samples illustrates the method's applicability in practice.Advantages; Applications; Calibration; Continuum regression (CR); Data; Distribution; Efficiency; Estimator; Least-squares; M-estimators; Methods; Model; Optimal; Ordinary least squares; Outliers; Partial least squares; Precision; Prediction; Projection-pursuit; Regression; Research; Robust continuum regression (RCR); Robust multivariate calibration; Robust regression; Robustness; Simulation; Squares; Studies; Variables; Yield;

Bounded influence regression using high breakdown scatter matrices.

In this paper we estimate the parameters of a regression model using S-estimators of multivariate location and scatter. The approach is proven to be Fisher-consistent, and the influence functions are derived. The corresponding asymptotic variances are obtained and it is shown how they can be estimated in practice. A comparison with other recently proposed robust regression estimators is made.fisher-consistency; influence function; robust regression; s-estimators; scatter matrices; multivariate location; robust estimation; s-estimators; linear-regression; rank regression; covariance; squares; diagnostics; stability; efficiency;

Robust multivariate least angle regression

The least angle regression selection (LARS) algorithms that use the classical sample means, variances, and correlations between the original variables are very sensitive to the presence of outliers and other contamination. To remedy this problem, a simple modification of this algorithm is to replace the non-robust estimates with their robust counterparts. Khan, Van Aelst, and Zamar employed the robust correlation for winsorized data based on adjusted winsorization correlation as a robust bivariate correlation approach for plug-in LARS. However, the robust least angle regression selection has some drawbacks in the presence of multivariate outliers. We propose to incorporate the Olive and Hawkins reweighted and fast consistent high breakdown estimator into the robust plug-in LARS method based on correlations. Our proposed method is tested by using a numerical example and a simulation study

Robust and Misspecification Resistant Model Selection in Regression Models with Information Complexity and Genetic Algorithms

In this dissertation, we develop novel computationally effiient model subset selection methods for multiple and multivariate linear regression models which are both robust and misspecification resistant. Our approach is to use a three-way hybrid method which employs the information theoretic measure of complexity (ICOMP) computed on robust M-estimators as model subset selection criteria, integrated with genetic algorithms (GA) as the subset model searching engine. Despite the rich literature on the robust estimation techniques, bridging the theoretical and applied aspects related to robust model subset selection has been somewhat neglected. A few information criteria in the multiple regression literature are robust. However, none of them is model misspecification resistant and none of them could be generalized to the misspecified multivariate regression. In this dissertation, we introduce for the first time both robust and misspecification resistant information complexity (ICOMP) criterion to fill in the gap in the literature. More specifically in multiple linear regression, we introduce robust M-estimators with misspecification resistant ICOMP and use the new information criterion as the fitness fuction in GA to carry out the model subset selection. For multivariate linear regression, we derive the two-stage robust Mahalanobis distance (RMD) estimator and introduce this RMD estimator in the computation of information criteria. The new information criteria are used as the fitness function in the GA to perform the model subset selection. Comparative studies on the simulated data for both multiple and multivariate regression show that the robust and misspecification resistant ICOMP outperforms the other robust information criteria and the non-robust ICOMP computed using OLS (or MLE) when the data contain outliers and error terms in the model deviate from a normal distribution. Compared with the all possible model subset selection, GA combined with the robust and misspecification resistant infromation criteria is proved to be an effective method which can quickly find the a near subset, if not the best, without having to search the whole subset model space