Robust online signal extraction from multivariate time series

We introduce robust regression-based online filters for multivariate time series and discuss their performance in real time signal extraction settings. We focus on methods that can deal with time series exhibiting patterns such as trends, level changes, outliers and a high level of noise as well as periods of a rather steady state. In particular, the data may be measured on a discrete scale which often occurs in practice. Our new filter is based on a robust two-step online procedure. We investigate its relevant properties and its performance by means of simulations and a medical application. --Multivariate time series,signal extraction,robust regression,online methods

On rank tests for shift detection in time series

Robustified rank tests, applying a robust scale estimator, are investigated for reliable and fast shift detection in time series. The tests show good power for sufficiently large shifts, low false detection rates for Gaussian noise and high robustness against outliers. Wilcoxon scores in combination with a robust and efficient scale estimator achieve good performance in many situations. --signal extraction,jumps,outliers,test resistance

Jensen's inequality for the Tukey median

--Shape analysis,spherical harmonic descriptors,optimal designs,mean square error,3D-image data,minimax optimal designs,robust designs,dependent data

A note on the choice of the number of slices in sliced inverse regression

Sliced inverse regression (SIR) is a clever technique for reducing the dimension of the predictor in regression problems, thus avoiding the curse of dimensionality. There exist many contributions on various aspects of the performance of SIR. Up to now, few attention has been paid to the problem of choosing the number of slices within the SIR procedure appropriately. The aim of this paper is to show that especially the estimation of the reduced dimension can be strongly in?uenced by the chosen number of slices. --dimension reduction,estimation of dimension

Robust Trend Estimation for AR(1) Disturbances

We discuss the robust estimation of a linear trend if the noise follows an autoregressive process of first order. We find the ordinary repeated median to perform well except for negative correlations. In this case it can be improved by a Prais-Winsten transformation using a robust autocorrelation estimator. -- Wir behandeln die robuste Schätzung eines linearen Trends bei autoregressiven Fehlern erster Ordnung. Die Repeated Median Regression zeigt ein gutes Verhalten bei positiven Korrelationen. Bei negativen Korrelationen ist eine Verbesserung durch eine Prais-Winsten Transformation mittels eines robusten Korrelationsschätzers möglich.Robust Regression,Autocorrelations,Detrending,Cochrane-Orcutt Estimator,Prais-Winsten Estimator

Methods and algorithms for robust filtering

We discuss filtering procedures for robust extraction of a signal from noisy time series. Moving averages and running medians are standard methods for this, but they have shortcomings when large spikes (outliers) respectively trends occur. Modified trimmed means and linear median hybrid filters combine advantages of both approaches, but they do not completely overcome the difficulties. Improvements can be achieved by using robust regression methods, which work even in real time because of increased computational power and faster algorithms. Extending recent work we present filters for robust online signal extraction and discuss their merits for preserving trends, abrupt shifts and extremes and for the removal of spikes. --Signal extraction,drift,edge,outlier,update algorithm

Breakdown and Groups II

The notion of breakdown point was introduced by Hampel (1968, 1971) and has since played an important role in the theory and practice of robust statistics. In Davies and Gather (2004) it was argued that the success of the concept is connected to the existence of a group of transformations on the sample space and the linking of breakdown and equivariance. For example the highest breakdown point of any translation equivariant functional on the real line is 1/2 whereas without equivariance considerations the highest breakdown point is the trivial upper bound of 1. --

Robust Statistics

The first example involves the real data given in Table 1 which are the results of an interlaboratory test. The boxplots are shown in Fig. 1 where the dotted line denotes the mean of the observations and the solid line the median. We note that only the results of the Laboratories 1 and 3 lie below the mean whereas all the remaining laboratories return larger values. In the case of the median, 7 of the readings coincide with the median, 24 readings are smaller and 24 are larger. A glance at Fig. 1 suggests that in the absence of further information the Laboratories 1 and 3 should be treated as outliers. This is the course which we recommend although the issues involved require careful thought. For the moment we note simply that the median is a robust statistic whereas the mean is not. --

Repeated median and hybrid filters

Standard median filters preserve abrupt shifts (edges) and remove impulsive noise (outliers) from a constant signal but they deteriorate in trend periods. FIR median hybrid (FMH) filters are more flexible and also preserve shifts, but they are much more vulnerable to outliers. Application of robust regression methods, in particular of the repeated median, has been suggested for removing subsequent outliers from a signal with trends. A fast algorithm for updating the repeated median in linear time using quadratic space is given in Bernholt and Fried (2003). We construct repeated median hybrid filters to combine the robustness properties of the repeated median with the edge preservation ability of FMH filters. An algorithm for updating the repeated median is presented which needs only linear space. We also investigate analytical properties of these filters and compare their performance via simulations. --Signal extraction,Drifts,Jumps,Outliers,Update algorithm