Evaluation of Test Statistics for Detection of Outliers and Shifts

Existence of outliers and structural breaks having mutually unknown nature, in time series data, offer challenges to data analysts in model identification, estimation and validation. Detection of these outliers has been an important area of research in time series since long. To analyze the impact of these structural breaks and outliers on model identification, estimation and their inferential analysis, we use two data generating processes: MA(1) and ARMA(1,1). The performance of the test statistics for detecting additive outlier(AO), innovative outlier(IO), level shift(LS) and transient change(TC) is investigated using simulation strategy through power of a test, empirical level of significance, empirical critical values, misspecification frequencies and sampling distribution of estimators for the two models. The empirical critical values are found higher than the theoretical cut-off points, empirical power of the test statistics is not satisfactory for small sample size, large cut-off points and large model coefficient. We have explored confusion between LS, AO, TC and IO at different critical values(c) by varying sample size. We have also collected empirical evidence from time series data for Pakistan using 3-stage iterative procedure to detect multiple outliers and structural breaks. We find that neglecting shocks lead to wrong identification, biased estimation and excess kurtosis. JEL Classification Codes: C15, C18, C63, C32, C87, C51, C52, C82 AMS Classification Codes: 62, 65, 91, DI, 62-08, 62J20, 00A72, 91-08, 91-10, 91-11 62P20, 91B82, 91B84, 62M07, 62M09, 62M10, 62M15, 62M2

Identification of unusual events in multi-channel bridge monitoring data

Peer reviewedPostprin

Identifying change point in production time-series volatility using control charts and stochastic differential equations

The article focuses on volatility change point detection using SPC (Statistical Process Control) methods, specifically time-series control charts and stochastic differential equations (SDEs). Contribution will review recent advances in change point detection for the volatility component of a process satisfying stochastic differential equation (SDE) based on discrete observations, and also by using time-series control charts. Theoretical part will discuss methodology of time-series control charts and SDEs driven by a Brownian motion. Research part will demonstrate the methodologies in a simulation study focusing on analysis of the AR(1) process by means of time-series control charts and SDEs. The aim is to make use of change point detection in time series of production processes and highlight versatility of control charts not only in manufacturing but also in managing financial cash flow stability. © 2014, World Scientific and Engineering Academy and Society. All rights reserved

Identification of unusual events in multi-channel bridge monitoring data

Continuously operating instrumented structural health monitoring (SHM) systems are becoming a practical alternative to replace visual inspection for assessment of condition and soundness of civil infrastructure such as bridges. However, converting large amounts of data from an SHM system into usable information is a great challenge to which special signal processing techniques must be applied. This study is devoted to identification of abrupt, anomalous and potentially onerous events in the time histories of static, hourly sampled strains recorded by a multi-sensor SHM system installed in a major bridge structure and operating continuously for a long time. Such events may result, among other causes, from sudden settlement of foundation, ground movement, excessive traffic load or failure of post-tensioning cables. A method of outlier detection in multivariate data has been applied to the problem of finding and localising sudden events in the strain data. For sharp discrimination of abrupt strain changes from slowly varying ones wavelet transform has been used. The proposed method has been successfully tested using known events recorded during construction of the bridge, and later effectively used for detection of anomalous post-construction events

The Effectiveness of the Huber's Weight on Dispersion and Tuning Constant: A Simulation Study

Dispersion measurement and tuning constants are critical aspects of a model's robustness and efficiency. However, in the presence of outliers, the standard deviation is not a reliable measure of dispersion in Huber's weight. This research aimed to assess the efficacy of the Huber weight function in terms of dispersion measurement and tuning constant. The simulation study was conducted on a hybrid of the autoregressive (AR) model and the generalized autoregressive conditional heteroscedasticity (GARCH) model with 10% and 20% additive outlier contamination. In the simulation analysis, three dispersion measurements were compared: median absolute deviation (MAD), interquartile range (IQR), and IQR/3, with two tuning constant values (1.345 and 1.5). The numerical simulation results showed that during contamination with 10% and 20% additive outliers, the IQR/3 outperformed the MAD and IQR. Our findings also showed that IQR/3 is a potentially more robust dispersion measurement in Huber's weight. The tuning constant of 1.5 revealed a decrease in resistance to outliers and increased efficiency. The proposed IQR/3 model with a constant tuning value (h) of 1.5 outperformed the AR(1)-GARCH(1,2) model while minimising the effect of additive outliers

Impacts of Permanent and Temporary Shocks on Optimal Length of Moving Average to Forecast a Time Series

Moving averages are often used for forecasting and the optimal length of the moving average depends on the size and frequency of structural breaks. A new time series model is proposed to describe permanent shocks related to structural breaks and temporary shocks with probability distributions. In the proposed model, permanent shocks are captured by a Poisson-jump or a Bernoulli-jump process, and temporary shocks are independent and identically normally distributed. This model requires a time series to have negative autocorrelation created by overdifferencing the temporary shocks. The proposed model is adapted to allow for positive autocorrelation by permitting autocorrelation of the jump process. The models are estimated with Oklahoma hard red winter wheat basis, Illinois corn basis and soybean basis, money stock, stock prices, total employment and total unemployment rate macroeconomic series. The parameters of the models are the probability of occurrence of jumps, the variance and the mean of the jump process, a time trend, and the variance of temporary shocks. The parameters are estimated with generalized method of moments estimation. In order to deal with autocorrelation in each series, we add an additional moment condition about autocorrelation to the generalized method of moments estimation. Most shocks are permanent shocks. The findings imply that shorter moving averages are the best for forecasting these series. The developed models are used to estimate the relative impacts of permanent and temporary shocks on the optimal length of moving average to use for forecasts. One year is the optimal length due to the large proportion of permanent shocks occur. The autoregressive integrated moving average (ARIMA) model with outliers is selected as a competing model. The proposed models for both a Poisson-jump model and a Bernoulli-jump model fit actual series better than the competing ARIMA models with outliers.Agricultural Economic

Data cleaning and knowledge discovery in process data

This dissertation presents several methods for overcoming the Big Data challenges, with an emphasis on data cleaning and knowledge discovery in process data. Data cleaning and knowledge discovery is chosen as a main research area here due to its importance from both theoretical and practical points of view. Theoretical background and recent developments of data cleaning methods are reviewed from four aspects: missing data imputation, outlier detection, noise removal and time delay estimation. Moreover, the impact of contaminated data on model performance and corresponding improvement obtained by data cleaning methods are analyzed through both simulated and industrial case studies. The results provide a starting point for further advanced methodology development. It is hard to find a universally applicable method for data cleaning since every data set may have its own distinctive features. Thus, we have to customize available methods so that the quality of the data set is guaranteed. An integrated data cleaning scheme is proposed, which incorporates model building and performance evaluation, to provide guidance in tuning the parameters of data cleaning methods and prevent over-cleaning. A case study based on industrial data has been used to verify the feasibility and effectiveness of the proposed new method, during which a partial least squares (PLS) model was built and three univariate data cleaning procedures is tested. A time series Kalman filter (TSKF) is proposed that successfully handles outlier detection in dynamic systems, where normal process changes often mask the existence of outliers. The TSKF method combines a time series model fitting procedure with a modified Kalman filter to deal with additive outlier (AO) and innovational outlier (IO) detection problems in dynamic process data set. A comparative analysis of TSKF and available methods is performed on simulated and real chemical plant data. Root cause diagnosis of plant-wide oscillations, as a concrete example of data cleaning and knowledge discovery in the process data, is provided. Plant-wide oscillations can negatively influence the overall control performance of the process and the detection results are often affected by noise at different frequency ranges. To address such a problem, an information transfer method combining spectral envelope algorithm with spectral transfer entropy is proposed to detect and diagnose such oscillations within a specific frequency range, mitigating the effects from measurement noise. The feasibility and effectiveness of the proposed method are verified and compared with available methods through both simulated and industrial case studies.Chemical Engineerin

Integrating technology and organization for manufacturing sector performance: evidence from Finland

This dissertation investigates the complex factors shaping the future of manufacturing, focusing on innovation, competitiveness, and employment trends within the European context. Leveraging the extensive 2022 European Manufacturing Survey dataset, it models relationships between critical technological and organizational variables impacting manufacturing resilience using cross-lagged panel path analysis. Against the 2019–2021 economic and environmental backdrop, the research examines manufacturers’ integral survival strategies derived from challenges faced. Factors like business innovation models, organizational concepts, key technologies, and relocation approaches are assessed for performance. The study reveals competitive standards: automation, robotics, additive manufacturing, accessbased business models, maintenance services, and production organization. These discoveries have profound implications for enabling the transition to next-generation sustainable manufacturing through technology integration frameworks. The research marks the need for investments in cross-sectoral research coordination. As climate change intensifies, reimagining manufacturing is critical. While acknowledging limitations like sample size and scope, the dissertation offers a detailed understanding of the manufacturing system’s components and the relationships of success, forward strategies, and human-technology-environment interlinkages. This multidimensional perspective provides insight to catalyze the creation of integrated manufacturing ecosystems worldwide

The relationship between gross domestic product (GDP), inflation, import and export from a statistical point of view

The term relationship in a general statistical concept connotes a wide range of meanings and applications. However, the resultant meaning of the term usually focus on the principle of connectivity, association, causation, inter-relationship, or linkages between variables. In view of this, the thesis reports on the statistical relationships between GDP, Inflation, Export and import. The study utilized 65 countries with data ranging from 1970 to 2011. The research, which is an applied empirical, involves two phases. The first phase dealt with the exploration of nature and pattern of Granger causality concept by using GDP and inflation. In this phase, we first ensured the stationarity and stability of our time series variables are maintained. The stationary and non-stationary instruments utilized include ADF, PP, KPSS, Chow and Quandt tests. After these, we carried out extensive computations using the Granger causality. It should be noted that the concept of Granger causality is concerned with how a variable X can enhance or better the prediction of other variable Y by using the principle of cause and effect. In the second phase of the study, we explored the possible linkages of exports and imports to the Granger causality of GDP and Inflation that were established in Phase 1. To achieve this, we first carried out pairwise Granger causality tests on the four variables (GDP, Inflation, Export and Import) and then considered further computations and testing on the said variables by utilizing the principles of Bayes theorem, assignment problem models, coefficient of variation and other relevant statistical concepts. In fact, the results at this phase are the major contributions to knowledge. The general description of the study embraced the conceptual steps, where we considered relevant literatures on Granger causality and theory of some statistical principles and practices as earlier mentioned above. Next, we have the empirical studies description in which the methodology, results/findings and interpretations on the study were considered. Based on our findings, we conclude that Inflation “Granger causes” GDP most often occurred than the other combinations of Granger causality between Inflation and GDP. Also, it was established that countries with developed economies supported the Granger causality concept better than the developing economies. This result can be attributed to the stability of most of the developed economy variables, while it is unstable with most of the developing economy countries. With countries supporting Granger causality, we have uniformly distributed pattern for the three types in the developed economies whilst skewed toward Inflation “Granger causes” GDP for the developing economies. For other important conclusions, we could establish that less volatility of export over import supports the bidirectional Granger causality whilst higher volatility of exports over import is relationally linked to the unidirectional Granger causality. We inferred also that when there is unidirectional Granger causality between inflation and import (or export), there is also unidirectional causality between GDP and inflation by the Bayes’ Rule; and when there is bidirectional Granger causality between GDP and import only, there is bidirectional causality between GDP and inflation

LSST Science Book, Version 2.0

A survey that can cover the sky in optical bands over wide fields to faint magnitudes with a fast cadence will enable many of the exciting science opportunities of the next decade. The Large Synoptic Survey Telescope (LSST) will have an effective aperture of 6.7 meters and an imaging camera with field of view of 9.6 deg^2, and will be devoted to a ten-year imaging survey over 20,000 deg^2 south of +15 deg. Each pointing will be imaged 2000 times with fifteen second exposures in six broad bands from 0.35 to 1.1 microns, to a total point-source depth of r~27.5. The LSST Science Book describes the basic parameters of the LSST hardware, software, and observing plans. The book discusses educational and outreach opportunities, then goes on to describe a broad range of science that LSST will revolutionize: mapping the inner and outer Solar System, stellar populations in the Milky Way and nearby galaxies, the structure of the Milky Way disk and halo and other objects in the Local Volume, transient and variable objects both at low and high redshift, and the properties of normal and active galaxies at low and high redshift. It then turns to far-field cosmological topics, exploring properties of supernovae to z~1, strong and weak lensing, the large-scale distribution of galaxies and baryon oscillations, and how these different probes may be combined to constrain cosmological models and the physics of dark energy.Comment: 596 pages. Also available at full resolution at http://www.lsst.org/lsst/sciboo