Outliers in dynamic factor models

Dynamic factor models have a wide range of applications in econometrics and applied economics. The basic motivation resides in their capability of reducing a large set of time series to only few indicators (factors). If the number of time series is large compared to the available number of observations then most information may be conveyed to the factors. This way low dimension models may be estimated for explaining and forecasting one or more time series of interest. It is desirable that outlier free time series be available for estimation. In practice, outlying observations are likely to arise at unknown dates due, for instance, to external unusual events or gross data entry errors. Several methods for outlier detection in time series are available. Most methods, however, apply to univariate time series while even methods designed for handling the multivariate framework do not include dynamic factor models explicitly. A method for discovering outliers occurrences in a dynamic factor model is introduced that is based on linear transforms of the observed data. Some strategies to separate outliers that add to the model and outliers within the common component are discussed. Applications to simulated and real data sets are presented to check the effectiveness of the proposed method.Comment: Published in at http://dx.doi.org/10.1214/07-EJS082 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by the Institute of Mathematical Statistics (http://www.imstat.org

Outlier detection in multivariate time series via projection pursuit

This article uses Projection Pursuit methods to develop a procedure for detecting outliers in a multivariate time series. We show that testing for outliers in some projection directions could be more powerful than testing the multivariate series directly. The optimal directions for detecting outliers are found by numerical optimization of the kurtosis coefficient of the projected series. We propose an iterative procedure to detect and handle multiple outliers based on univariate search in these optimal directions. In contrast with the existing methods, the proposed procedure can identify outliers without pre-specifying a vector ARMA model for the data. The good performance of the proposed method is verified in a Monte Carlo study and in a real data analysis

OUTLIER DETECTION IN MULTIVARIATE TIME SERIES VIA PROJECTION PURSUIT

This article uses Projection Pursuit methods to develop a procedure for detecting outliers in a multivariate time series. We show that testing for outliers in some projection directions could be more powerful than testing the multivariate series directly. The optimal directions for detecting outliers are found by numerical optimization of the kurtosis coefficient of the projected series. We propose an iterative procedure to detect and handle multiple outliers based on univariate search in these optimal directions. In contrast with the existing methods, the proposed procedure can identify outliers without pre-specifying a vector ARMA model for the data. The good performance of the proposed method is verified in a Monte Carlo study and in a real data analysis.

Estimation of Commodity Specific Production Costs Using German Farm Accountancy Data

A central problem in estimating per unit costs of production originates from the fact that most farms produce multiple outputs and standard farm-accounting data are only available at the whole-farm level. The seemingly unrelated regression (SUR) approach is used to estimate per unit production costs based on German farm accountancy data. Special emphasis is put on outlier detection prior to the estimation of production costs to increase the robustness of the results. Outlier observations are identified based on the Mahalanobis distance for each observation on the data set. It was observed that less negative cost coefficients are estimated after the exclusion of the outliers. The time series analysis of cost estimation based on SUR regression shows the costs of arable crops after 2004, affected by rising prices of fertilizer, seeds and energy, while the increase of livestock production costs after 2006 is attributed to feed costs.Multi-output, outlier detection, production costs, Seemingly Unrelated Regression, Agricultural Finance,

Identification of unusual events in multi-channel bridge monitoring data

Peer reviewedPostprin

Robust Orthogonal Complement Principal Component Analysis

Recently, the robustification of principal component analysis has attracted lots of attention from statisticians, engineers and computer scientists. In this work we study the type of outliers that are not necessarily apparent in the original observation space but can seriously affect the principal subspace estimation. Based on a mathematical formulation of such transformed outliers, a novel robust orthogonal complement principal component analysis (ROC-PCA) is proposed. The framework combines the popular sparsity-enforcing and low rank regularization techniques to deal with row-wise outliers as well as element-wise outliers. A non-asymptotic oracle inequality guarantees the accuracy and high breakdown performance of ROC-PCA in finite samples. To tackle the computational challenges, an efficient algorithm is developed on the basis of Stiefel manifold optimization and iterative thresholding. Furthermore, a batch variant is proposed to significantly reduce the cost in ultra high dimensions. The paper also points out a pitfall of a common practice of SVD reduction in robust PCA. Experiments show the effectiveness and efficiency of ROC-PCA in both synthetic and real data