Improving Monitoring and Diagnosis for Process Control using Independent Component Analysis

Statistical Process Control (SPC) is the general field concerned with monitoring the operation and performance of systems. SPC consists of a collection of techniques for characterizing the operation of a system using a probability distribution consistent with the system\u27s inputs and outputs. Classical SPC monitors a single variable to characterize the operation of a single machine tool or process step using tools such as Shewart charts. The traditional approach works well for simple small to medium size processes. For more complex processes a number of multivariate SPC techniques have been developed in recent decades. These advanced methods suffer from several disadvantages compared to univariate techniques: they tend to be statistically less powerful, and they tend to complicate process diagnosis when a disturbance is detected. This research introduces a general method for simplifying multivariate process monitoring in such a manner as to allow the use of traditional SPC tools while facilitating process diagnosis. Latent variable representations of complex processes are developed which directly relate disturbances with process steps or segments. The method models disturbances in the process rather than the process itself. The basic tool used is Independent Component Analysis (ICA). The methodology is illustrated on the problem of monitoring Electrical Test (E-Test) data from a semiconductor manufacturing process. Development and production data from a working semiconductor plant are used to estimate a factor model that is then used to develop univariate control charts for particular types of process disturbances. Detection and false alarm rates for data with known disturbances are given. The charts correctly detect and classify all the disturbance cases with a very low false alarm rate. A secondary contribution is the introduction of a method for performing an ICA like analysis using possibilistic data instead of probabilistic data. This technique extends the general ICA framework to apply to a broader range of uncertainty types. Further development of this technique could lead to the capability to use extremely sparse data to estimate ICA process models

Time series segmentation procedures to detect, locate and estimate change-points

This thesis deals with the problem of modeling an univariate nonstationary time series by a set of approximately stationary processes. The observed period is segmented into intervals, also called partitions, blocks or segments, in which the time series behaves as approximately stationary. Thus, by segmenting a time series, we aim to obtain the periods of stability and homogeneity in the behavior of the process; identify the moments of change, called change-points; represent the regularities and features of each piece or block; and, use this information in order to determine the pattern in the nonstationary time series. When the time series exhibits multiple change-points, a more intricate and difficult issue is to use an efficient procedure to detect, locate and estimate them. Thus, the main goal of the thesis consists on describing, studying comparatively with simulated data, and applying to real data, a number of segmentation and/or change-points detection procedures, which involve both, different type of statistics indicating when the data is exhibiting a potential break, and, searching algorithms to locate multiple patterns variations. The thesis is structured in five chapters. Chapter 1 introduces the main concepts involved in the segmentation problem in the context of time series. First, a summary of the main statistics to detect a single change-point is presented. Second, we point out the multiple change-points searching algorithms presented in the literature and the linear models for representing time series, both in the parametric and the non-parametric approach. Third, we introduce the locally stationary and piecewise stationary processes. Finally, we show examples of piecewise and locally stationary simulated and real time series where the detection of change-point and segmentation seems to be important. Chapter 2 deals with the problem of detecting, locating and estimating a single or multiple changes in the parameters of a stationary process. We consider changes in the marginal mean, the marginal variance, and both the mean and the variance. This is done for both uncorrelated, or serial correlated processes. The main contributions of this chapter are: a) introducing a modification in the theoretical model proposed by Al Ibrahim et al. (2003) that is useful to look for changes in the mean and the autoregressive coefficients in piecewise autoregressive processes, by using a procedure based on the Bayesian information criterion; we allow also the presence of changes in the variance of the perturbation term; b) comparing this procedure with several procedures available in the literature which are based on cusum methods (Inclán and Tiao (1994), Lee et al. (2003)), minimum description length principle (Davis et al. (2006)), the time varying spectrum (Ombao et al. (2002)) and the likelihood ratio test (Killick et al. (2012)). For that, we compute the empirical size and power properties in several scenarios and; c)apply them to neurology and speech recognition datasets. Chapter 3 studies processes, with constant conditional mean and dynamic behavior in the conditional variance, which are also affected by structural changes. Thus, the goal is to explore, analyse and apply the change-point detection and estimation methods to the situation when the conditional variance of a univariate process is heteroskedastic and exhibits change-points. Procedures based on informational approach, cusum statistics, minimum description length and the spectrum assuming an heteroskedastic time series are presented. We propose a method to detect and locate change-points by using the BIC as an extension of its application in linear models. We analyse comparatively the size and power properties of the procedures presented for single and multiple change-point scenarios and illustrate their performance with the S&P 500 returns. Chapter 4 analyses the problem of detecting and estimating smooth change-points in the data, where the Linear Trend change-point (LTCP) model is considered to represent a smooth change. We propose a procedure based on the Bayesian information criterion to distinguish a smooth from an abrupt change-point. The likelihood function of the LTCP model is obtained, as well as the conditional maximum likelihood estimator of the parameters in the model. The proposed procedure is compared with the outliers analysis techniques (Fox (1972), Chang (1982), Chen and Liu (1993), Kaiser (1999), among others) performing simulation experiments. We also present an iterative procedure to detect multiple smooth and abrupt change-points. This procedure is illustrated with the number of deaths in traffic accidents in Spanish motorways. Finally, Chapter 5 summarizes the main results of the thesis and proposes some extensions for future research

Essays on testing for nonlinearity in time series : issues in nonlinear cointegration, structural breaks and changes in persistence

Besides obvious nonlinear relations like a nonlinear error correction model are nonlinearities in time series closely related to structural breaks and changes in persistence. Since both kinds of changes can induce regime-switching, they qualify well to capture the characteristic of time-variability. On the contrary, linear models are often an insufficient simplification of the real underlying DGP because they fail ro reproduce trends, shocks like finance crises and stylized facts such as long-range dependencies as well as volatility clustering. This is why testing for the presence of these nonlinear properties as the first step of any statistical analysis is very crucial, especially with regard to effective model building. Nonlinear Cointegration In the first chapter, a nonlinear cointegration test is proposed which builds on Kapetanios et al. (2006) who where the first who addressed cointegration in a nonlinear error correction framework under the alternative. The switch between regimes is modeled to follow a second order logistic smooth transition (D-LSTR) function and a null hypothesis of no cointegration is tested against globally stationary D-LSTR cointegration. From the nonlinear error correction regression, t-type and F-type statistics are derived and finite-sample investigations are conducted. The results of the modified nonlinear cointegration test are compared to a comparable linear cointegration test, namely the test proposed by Johansen (1991). The D-LSTR function qualifies well as an overall-generalization of transition functions and it is found that the D-LSTR error correction model has power against both alternatives, D-LSTR as well as 3-regime TAR nonlinearity which is nested for large gamma in the D-LSTR function. Structural breaks The topic of the second paper is to survey the most frequently applied volatility break tests when they are employed to a broad range of different DGPs. Within a simulation study, the break tests are applied to DGPs which can exhibit either single- or double-shifting or the process can experience a smooth increase in the magnitude of the volatility break. The surveyed tests are a CUSUM test in a version proposed by Deng and Perron (2008) and conventional Wald and LM tests. Besides size and power comparisons the break tests are empirically validated and it is found that more breaks are found in equity series than in exchange rate series. One main finding is that huge outliers in the data can impact the long-run variance of the squared return process to be no longer finite which renders non-monotonic power functions. Changes in Persistence Chapter three addresses the specific question whether either structural breaks or nonstationarity in the conditional volatility affect the testing decision of the R test proposed by Leybourne et al. (2007). The additional structural breaks in the conditional volatility process are not specified under the null hypothesis and may therefore lead to a non-pivotal limiting distribution. Hence, heteroskedasticity of an unknown form is encountered and in order to potentially robustify the testing procedure, a wild-bootstrapped version of Leybourne et al. (2007)'s R test is suggested. Within a simulation study, size and power of the originally proposed test and the wild-bootstrap analogue are compared for various constellations of simultaneous breaks in the AR parameter as well as the GARCH parameter. It is found that the Leybourne et al. (2007) test seems heavily impacted by additional structural breaks in the conditional volatility, especially in very finite sample sizes. In an empirical application the two testing procedures are applied and evaluated

The econometrics of structural change: statistical analysis and forecasting in the context of the South African economy

Philosophiae Doctor - PhDOne of the assumptions of conventional regression analysis is I that the parameters are constant over all observations. It has often been suggested that this may not be a valid assumption to make, particularly if the econometric model is to be used for economic forecasting0 Apart from this it is also found that econometric models, in particular, are used to investigate the underlying interrelationships of the system under consideration in order to understand and to explain relevant phenomena in structural analysis. The pre-requisite of such use of econometrics is that the regression parameters of the model is assumed to be constant over time or across different crosssectional units

Empirical analysis of the dynamics of the South African rand (Post-1994)

Thesis (Ph.D. (Economics))--University of the Witwatersrand, Faculty of Commerce, Law and Management, School of Economic & Business Sciences, 2016.The objective of this thesis is to investigate the recent historical dynamics of the four major nominal bilateral spot foreign exchange rates and the fifteen currency-basket nominal effective exchange rate of the South African rand (hereafter referred to as the rand). The thesis has been organised as three separate studies that add to the advancement of the knowledge of the characteristics and behaviour (causal effects) of the rand. The common thread that holds the individual chapters together is the study of the dynamics of the rand. In particular, the study establishes whether the apparent nonstationarity of the exchange rate is a product of unit root test misspecification (a failure to account for structural change), considers the connexions between the timing of the identified structural shifts and important economic and noneconomic events, and analyses rand volatility and the temporal effect of monetary policy surprises on both the spot foreign exchange market returns and volatility of the rand. In order to do this, low- and high-frequency data are employed. With regard to exchange rate modelling, the theoretical economic-exchange rate frameworks are approached both from the traditional macro-based view of exchange rate determination and a micro-based perspective. The various methodologies applied here tackle different aspects of the exchange rate dynamics. To preview the results, we find that adjusting for structural shifts in the unit root tests does not render any of the exchange rates stationary. However, the results show a remarkable fall in the estimates of volatility persistence when structural breaks are integrated into the autoregressive conditional heteroskedasticity (ARCH) framework. The empirical results also shed light on the impact of modelling exchange rates as long memory processes, the extent of asymmetric responses to ‘good news’ and ‘bad news’, the consistencies and contrasts in the five exchange rate series’ volatility dynamics, and the timing and likely triggers of volatility regime switching. Additionally, there are convincing links between the timing of structural changes and important economic (and noneconomic) events, and commonality in the structural breaks detected in the levels and volatility of the rand. We also find statistically and economically significant high-frequency exchange rate returns and volatility responses to domestic interest rate surprises. Furthermore, the rapid response of the rand to monetary policy surprises suggests a relatively high degree of market efficiency (from a mechanical perspective) in processing this information. Keywords: Exchange rate, expectations, long memory, monetary policy surprises, repo rate, structural breaks, volatility; unit root. JEL Code: C22, E52, E58, F31, F41, G14 and G1

Unsupervised methods to discover events from spatio-temporal data

University of Minnesota Ph.D. dissertation. May 2016. Major: Computer Science. Advisor: Vipin Kumar. 1 computer file (PDF); ix, 110 pages.Unsupervised event detection in spatio-temporal data aims to autonomously identify when and/or where events occurred with little or no human supervision. It is an active field of research with notable applications in social, Earth, and medical sciences. While event detection has enjoyed tremendous success in many domains, it is still a challenging problem due to the vastness of data points, presence of noise and missing values, the heterogeneous nature of spatio-temporal signals, and the large variety of event types. Unsupervised event detection is a broad and yet open research area. Instead of exploring every aspect in this area, this dissertation focuses on four novel algorithms that covers two types of important events in spatio-temporal data: change-points and moving regions. The first algorithm in this dissertation is the Persistence-Consistency (PC) framework. It is a general framework that can increase the robustness of change-point detection algorithms to noise and outliers. The major advantage of the PC framework is that it can work with most modeling-based change-point detection algorithms and improve their performance without modifying the selected change-point detection algorithm. We use two real-world applications, forest fire detection using a satellite dataset and activity segmentation from a mobile health dataset, to test the effectiveness of this framework. The second and third algorithms in this dissertation are proposed to detect a novel type of change point, which is named as contextual change points. While most existing change points more or less indicate that the time series is different from what it was before, a contextual change point typically suggests an event that causes the relationship of several time series changes. Each of these two algorithms introduces one type of contextual change point and also presents an algorithm to detect the corresponding type of change point. We demonstrate the unique capabilities of these approaches with two applications: event detection in stock market data and forest fire detection using remote sensing data. The final algorithm in this dissertation is a clustering method that discovers a particular type of moving regions (or dynamic spatio-temporal patterns) in noisy, incomplete, and heterogeneous data. This task faces two major challenges: First, the regions (or clusters) are dynamic and may change in size, shape, and statistical properties over time. Second, numerous spatio-temporal data are incomplete, noisy, heterogeneous, and highly variable (over space and time). Our proposed approach fully utilizes the spatial contiguity and temporal similarity in the spatio-temporal data and, hence, can address the above two challenges. We demonstrate the performance of the proposed method on a real-world application of monitoring in-land water bodies on a global scale

Statistical Analysis and Forecasting of Economic Structural Change

In 1984, the University of Bonn (FRG) and IIASA created a joint research group to analyze the relationship between economic growth and structural change. The research team was to examine the commodity composition as well as the size and direction of commodity and credit flows among countries and regions. Krelle (1988) reports on the results of this "Bonn-IIASA" research project. At the same time, an informal IIASA Working Group was initiated to deal with problems of the statistical analysis of economic data in the context of structural change: What tools do we have to identify nonconstancy of model parameters? What type of models are particularly applicable to nonconstant structure? How is forecasting affected by the presence of nonconstant structure? What problems should be anticipated in applying these tools and models? Some 50 experts, mainly statisticians or econometricians from about 15 countries, came together in Lodz, Poland (May 1985); Berlin, GDR (June 1986); and Sulejov, Poland (September 1986) to present and discuss their findings. This volume contains a selected set of those conference contributions as well as several specially invited chapters. The introductory chapter "What can statistics contribute to the analysis of economic structural change?", discusses not only the role of statistics in the detection and assimilation of structural changes, but also the relevance of respective methods in the evaluation of econometric models. Trends in the development of these methods are indicated, and the contributions to the present volume are put into a broader context of empirical economics to help to bridge the gap between economists and statisticians. The chapters in the first section are concerned with the detection of parameter nonconstancy. The procedures discussed range from classical methods, such as the CUSUM test, to new concepts, particularly those based on nonparametric statistics. Several chapters assess the conditions under which these methods can be applied and their robustness under such conditions. The second section addresses models that are in some sense generalizations of nonconstant-parameter models, so that they can assimilate structural changes. The last section deals with real-life structural change situations

Modeling and Forecasting Stock Return Volatility and the Term Structure of Interest Rates

This dissertation consists of a collection of studies on two topics: stock return volatility and the term structure of interest rates. _Part A_ consists of three studies and contributes to the literature that focuses on the modeling and forecasting of financial market volatility. In this part we first of all discuss how to apply CUSUM tests to identify structural changes in the level of volatility. The main focus of part A is, however, on the use of high-frequency intraday return data to measure the volatility of individual asset eturns as well as the correlations between asset returns. A nonlinear long-memory model for realized volatility is developed which is shown to accurately forecast future volatility. Furthermore, we show that daily covariance matrix estimates based on intraday return data are of economic significance to an investor. We investigate what the optimal intraday sampling frequency is for constructing estimates of the daily covariance matrix and we find that the optimal frequency is substantially lower than the commonly used 5-minute frequency. _Part B_ consists of two studies and investigates the modeling and forecasting of the term structure of interest rates. In the first study we examine the class of Nelson-Siegel models for their in-sample fit and out-of-sample forecasting performance. We show that a four-factor model has a good performance in both areas. In the second study we analyze the forecasting performance of a panel of term structure models. We show that the performance varies substantially across models and subperiods. To mitigate model uncertainty we therefore analyze forecast combination techniques and we find that combined forecasts are consistently accurate over time