Feature Extraction for Change-Point Detection using Stationary Subspace Analysis

Detecting changes in high-dimensional time series is difficult because it involves the comparison of probability densities that need to be estimated from finite samples. In this paper, we present the first feature extraction method tailored to change point detection, which is based on an extended version of Stationary Subspace Analysis. We reduce the dimensionality of the data to the most non-stationary directions, which are most informative for detecting state changes in the time series. In extensive simulations on synthetic data we show that the accuracy of three change point detection algorithms is significantly increased by a prior feature extraction step. These findings are confirmed in an application to industrial fault monitoring.Comment: 24 pages, 20 figures, journal preprin

FDR-control in multiscale change-point segmentation.

Fast multiple change-point segmentation methods, which additionally provide faithful statistical statements on the number, locations and sizes of the segments, have recently received great attention. In this paper, we propose a multiscale segmentation method, FDRSeg, which controls the false discovery rate (FDR) in the sense that the number of false jumps is bounded linearly by the number of true jumps. In this way, it adapts the detection power to the number of true jumps. We prove a non-asymptotic upper bound for its FDR in a Gaussian setting, which allows to calibrate the only parameter of FDRSeg properly. Moreover, we show that FDRSeg estimates change-point locations, as well as the signal, in a uniform sense at optimal minimax convergence rates up to a log-factor. The latter is w.r.t. Lp-risk, p≥1, over classes of step functions with bounded jump sizes and either bounded, or even increasing, number of change-points. FDRSeg can be efficiently computed by an accelerated dynamic program; its computational complexity is shown to be linear in the number of observations when there are many change-points. The performance of the proposed method is examined by comparisons with some state of the art methods on both simulated and real datasets. An R-package is available online

Topics in Nonstationary Time Series Analysis

Several interesting applications in areas such as neuroscience, economics, finance and seismology have led to the collection nonstationary time series data wherein the statistical properties of the observed process change across time. The analysis of nonstationary time series data is an important and challenging task with useful applications. In comparison to stationarity, modeling temporal dependence in nonstationary time series is more non-trivial, and numerous methods have been proposed to tackle this problem. Stationarity in time series is more coveted than nonstationarity and many of the existing techniques attempt to transform the problem of nonstationarity to a stationary time series setting. Change point detection is one such method that attempts to find time points wherein the statistical properties of the time series changed. We develop a nonparametric method to detect multiple change points in multivariate piecewise stationary processes when the locations and number of change points are unknown. Based on a test statistic that measures differences in the spectral density matrices through the L₂ norm, we sequentially identify points of local maxima in the test statistic and test for the significance of each of them being change points. In addition, the components responsible for the change in the covariance structure at each detected change point are identified. The asymptotic properties of the test for significant change points under the null and alternative hypothesis are derived. Another related method for handling nonstationarity is the recent technique of stationary subspace analysis (SSA) that aims at finding linear transformations of nonstationary processes that are stationary. We propose an SSA procedure for general multivariate second-order nonstationary processes. It relies on the asymptotic uncorrelatedness of the discrete Fourier transform of a stationary time series to define a measure of departure from stationarity; it is then minimized to find the stationary subspace. The dimension of the subspace is estimated using a sequential testing procedure and its asymptotic properties are discussed. We illustrate the broader applicability and better performance of our method in comparison to existing SSA methods through simulations and discuss an application in neuroeconomics. Here we apply our method to filter out noise in EEG brain signals from an economic choice task experiment. This improves prediction performance and more importantly reduces the number of trials needed from individuals in neuroeconomic experiments thereby aligning with the principle of simple and controlled designs in experimental and behavioral economics

Topics in Nonstationary Time Series Analysis

Several interesting applications in areas such as neuroscience, economics, finance and seismology have led to the collection nonstationary time series data wherein the statistical properties of the observed process change across time. The analysis of nonstationary time series data is an important and challenging task with useful applications. In comparison to stationarity, modeling temporal dependence in nonstationary time series is more non-trivial, and numerous methods have been proposed to tackle this problem. Stationarity in time series is more coveted than nonstationarity and many of the existing techniques attempt to transform the problem of nonstationarity to a stationary time series setting. Change point detection is one such method that attempts to find time points wherein the statistical properties of the time series changed. We develop a nonparametric method to detect multiple change points in multivariate piecewise stationary processes when the locations and number of change points are unknown. Based on a test statistic that measures differences in the spectral density matrices through the L₂ norm, we sequentially identify points of local maxima in the test statistic and test for the significance of each of them being change points. In addition, the components responsible for the change in the covariance structure at each detected change point are identified. The asymptotic properties of the test for significant change points under the null and alternative hypothesis are derived. Another related method for handling nonstationarity is the recent technique of stationary subspace analysis (SSA) that aims at finding linear transformations of nonstationary processes that are stationary. We propose an SSA procedure for general multivariate second-order nonstationary processes. It relies on the asymptotic uncorrelatedness of the discrete Fourier transform of a stationary time series to define a measure of departure from stationarity; it is then minimized to find the stationary subspace. The dimension of the subspace is estimated using a sequential testing procedure and its asymptotic properties are discussed. We illustrate the broader applicability and better performance of our method in comparison to existing SSA methods through simulations and discuss an application in neuroeconomics. Here we apply our method to filter out noise in EEG brain signals from an economic choice task experiment. This improves prediction performance and more importantly reduces the number of trials needed from individuals in neuroeconomic experiments thereby aligning with the principle of simple and controlled designs in experimental and behavioral economics

Multiscale Change-Point Inference

We introduce a new estimator SMUCE (simultaneous multiscale change-point estimator) for the change-point problem in exponential family regression. An unknown step function is estimated by minimizing the number of change-points over the acceptance region of a multiscale test at a level \alpha. The probability of overestimating the true number of change-points K is controlled by the asymptotic null distribution of the multiscale test statistic. Further, we derive exponential bounds for the probability of underestimating K. By balancing these quantities, \alpha will be chosen such that the probability of correctly estimating K is maximized. All results are even non-asymptotic for the normal case. Based on the aforementioned bounds, we construct asymptotically honest confidence sets for the unknown step function and its change-points. At the same time, we obtain exponential bounds for estimating the change-point locations which for example yield the minimax rate O(1/n) up to a log term. Finally, SMUCE asymptotically achieves the optimal detection rate of vanishing signals. We illustrate how dynamic programming techniques can be employed for efficient computation of estimators and confidence regions. The performance of the proposed multiscale approach is illustrated by simulations and in two cutting-edge applications from genetic engineering and photoemission spectroscopy