The theory of linear prediction

Linear prediction theory has had a profound impact in the field of digital signal processing. Although the theory dates back to the early 1940s, its influence can still be seen in applications today. The theory is based on very elegant mathematics and leads to many beautiful insights into statistical signal processing. Although prediction is only a part of the more general topics of linear estimation, filtering, and smoothing, this book focuses on linear prediction. This has enabled detailed discussion of a number of issues that are normally not found in texts. For example, the theory of vector linear prediction is explained in considerable detail and so is the theory of line spectral processes. This focus and its small size make the book different from many excellent texts which cover the topic, including a few that are actually dedicated to linear prediction. There are several examples and computer-based demonstrations of the theory. Applications are mentioned wherever appropriate, but the focus is not on the detailed development of these applications. The writing style is meant to be suitable for self-study as well as for classroom use at the senior and first-year graduate levels. The text is self-contained for readers with introductory exposure to signal processing, random processes, and the theory of matrices, and a historical perspective and detailed outline are given in the first chapter

Persistence and Anti-persistence: Theory and Software

Persistent and anti-persistent time series processes show what is called hyperbolic decay. Such series play an important role in the study of many diverse areas such as geophysics and financial economics. They are also of theoretical interest. Fractional Gaussian noise (FGN) and fractionally-differeneced white noise are two widely known examples of time series models with hyperbolic decay. New closed form expressions are obtained for the spectral density functions of these models. Two lesser known time series models exhibiting hyperbolic decay are introduced and their basic properties are derived. A new algorithm for approximate likelihood estimation of the models using frequency domain methods is derived and implemented in R. The issue of mean estimation and multimodality in time series, particularly in the simple case of one short memory component and one hyperbolic component is discussed. Methods for visualizing bimodal surfaces are discussed. The exact prediction variance is derived for any model that admits an autocovariance function and integrated (inverse-differenced) by integer d. A new software package in R, arfima, for exact simulation, estimation, and forecasting of mixed short-memory and hyperbolic decay time series. This package has a wider functionality and increased reliability over other software that is available in R and elsewhere

푸리에 계수의 비모수적 추정을 이용한 베이지안 회귀분석과 그 응용

학위논문 (박사) -- 서울대학교 대학원 : 자연과학대학 통계학과, 2021. 2. 임채영.We illustrate a nonparmetric modeling of Fourier coefficients, known as a spectral density, under a Bayesian framework to forecast a stationary one-dimensional random process and to predict a stationary two-dimensional random field on a regular grid. We switch from the time/space domain to the frequency domain, and introduce a Gaussian process prior to the log-spectral density. First, we propose Bayesian modeling of spectral density for spatial regression on a regular lattice grid. An interpolation technique to convert an estimated spectral density to a covariance matrix is also proposed to avoid matrix inversion for the spatial prediction. Simulation study shows that our approach is robust in that it does not require a parametric form and/or isotropic assumption of a covariance function. Also, our approach gives better prediction results over conventional spatial prediction under most parametric covariance models that we considered. We also compare our approach with other existing spatial prediction approaches using two datasets of Korean ozone concentration. Our approach performs reasonably good in terms of mean absolute error and root mean squared error. Second, we propose Bayesian modeling of spectral density for time series regression with heteroscedastic autocovariance. Heteroskedastic autocovariance is modeled as time varying marginal variance multiplied by stationary autocorrelation. Bayesian Markov-Chain-Monte-Carlo(MCMC) is used to estimate coefficients of the B-spline basis representation of the log marginal variance function as well as a log spectral density at Fourier frequencies so that we can estimate time varying autocovariance function. Simulation results show that the proposed approach successfully detected the temporal pattern of the autocovariance structure. Even though we need to estimate spectral density at all Fourier frequencies during Bayesian procedure, our approach does not lose much efficiency on computation compared to estimating only a few parameters in a parametric model such as $ARMA$ or $GARCH$. We applied the proposed method to forecast foreign exchange rate data and it shows good prediction accuracy in a sense of overall low root mean squared errors.본 박사학위논문에서는 스펙트럴 밀도라 불리우는 일종의 푸리에(Fourier) 계수를 베이지안(Bayesian) 마코프-체인-몬테-카를로(MCMC) 관점에서 비모수적으로 모형화하는 통계적방법론을 제안하는데, 이는 등간격의 격자점에서 정의된, 정상성(stationarity)을 가진 1 차원 또는 2차원 확률과정을 예측하는 역할을 수행한다. 핵심 원리는 시간 또는 공간 영역에서 정의된 자기공분산함수를 푸리에변환을 통해 주파수 영역에서의 스펙트럴 밀도함수로 전환하는 것, 그리고 사후분석(posterior analysis)을 위해서 그 스펙트럼 밀도의 로그변환된 함수에 가우시안(Gaussian)과정 사전분포를 부여하는 것이다. 먼저 공간 자료 예측 문제에 적용할 수 있는 모형을 제안한다. 스펙트럴 밀도함수를 공분산 함수로 변환할 때 본 논문에서 제안하는 보간 기법은 전통적인 공간예측 모형에서 필요로 했던 역행렬 계산을 생략함으로서 계산 부담을 줄여준다. 본 모형은 어떠한 알려진 형태의 함수나 등방성 등의 가정을 필요로하지 않으면서도 기존에 대표적인 공간예측모형들과 비교했을 때 비슷하거나 혹은 더 나은 예측력을 가져다 준다는 것이 시뮬레이션 연구를 통해 입증되었다. 또한 이 모형을 MODIS, AURA와 같은 공신력을 가진 위성자료를 이용하여 한국 지역의 오존농도를 예측하는 문제에 적용했을 때에도 비교적 좋은 예측력을 갖는다는 것이 입증되었다. 다음으로 시계열 자료 예측 문제에 적용할 수 있는 모형을 제안한다. 여기서는 특히 정상성(stationarity) 가정이 일부 완화되어 자기공분산의 한계치(marginal auto-covariance)가 시간에 따라 변하는 이분산성(heteroscedasticity) 확률과정을 생각한다. 이 때 자기공분산은 시간에 따라 변하는 한계분산함수와 정상성을 가진 자기상관함수 사이의 곱으로 표현된다. 자기상관함수의 추정은 기존의 아이디어를 따르고, 한계분산함수의 추정에 있어서는 B-spline 기저함수를 이용한 비모수 추정법을 도입한다. 새로 도입된 과정 역시 하나의 베이지안 마코프-체인-몬테-카를로(MCMC) 안에서 구현된다. 시뮬레이션 연구를 통해 제안한 방법이 등분산성 혹은 이분산성을 지닌 시계열자료의 시간에 따른 패턴을 잘 잡아내는 것이 밝혀졌다. 기존에 잘 알려진 방법인 $ARMA$나 $GARCH$와 같은 모수적 방법론보다 훨씬 많은 수의 모수를 추정해야 함에도 계산 효율은 크게 떨어지지 않는 모습을 보여주고 있다. 본 모형을 대표적인 외국환율 자료 분석에 응용했을 때, 많은 경우 평균제곱오차의 관점에서 전반적으로 예측지점별 오차가 비교적 적게 나오는 것으로 확인되었다.Contents Abstract i 1 Introduction 1 2 Bayesian spatial regression using non-parametric modeling of Fourier coefficients 12 2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . 12 2.1.1 Spectral representation theorem . . . . . . 13 2.1.2 Whittle Likelihood Approximation . . . . . 15 2.1.3 Fast Fourier Transform algorithm . . . . . . 18 2.2 Proposed model . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Periodogram . . . . . . . . . . . . . . . . . 19 2.2.2 Gaussian Mixture Approximation . . . . . . 20 2.2.3 Proposed Gibbs sampler . . . . . . . . . . . 21 2.2.4 Prediction procedures . . . . . . . . . . . . 22 2.3 Proposed model for observations on an incomplete grid . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.4 Simulation Study . . . . . . . . . . . . . . . . . . . 25 2.5 Real Data Analysis . . . . . . . . . . . . . . . . . . 33 iii 3 Bayesian time series regression using non-parametric modeling of Fourier coefficients 42 3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . 42 3.2 Proposed Method . . . . . . . . . . . . . . . . . . . 44 3.3 Simulation Study . . . . . . . . . . . . . . . . . . . 46 3.4 Real Data Analysis . . . . . . . . . . . . . . . . . . 52 4 Concluding remarks 59 A Conditional posterior distributions 71 Bibliography 75 B Proofs of the main results 75 Abstract (in Korean) 79Docto

Modeling Volatility of Financial Time Series Using Arc Length

This thesis explores how arc length can be modeled and used to measure the risk involved with a financial time series. Having arc length as a measure of volatility can help an investor in sorting which stocks are safer/riskier to invest in. A Gamma autoregressive model of order one(GAR(1)) is proposed to model arc length series. Kernel regression based bias correction is studied when model parameters are estimated using method of moment procedure. As an application, a model-based clustering involving thirty different stocks is presented using k-means++ and hierarchical clustering techniques

Nonlinearity, Nonstationarity, and Thick Tails: How They Interact to Generate Persistency in Memory

We consider nonlinear transformations of random walks driven by thick-tailed innovations that may have infinite means or variances. These three nonstandard characteristics: nonlinearity, nonstationarity, and thick tails interact to generate a spectrum of asymptotic autocorrelation patterns consistent with long-memory processes. Such autocorrelations may decay very slowly as the number of lags increases or may not decay at all and remain constant at all lags. Depending upon the type of transformation considered and how the model error is speci- fied, the autocorrelation functions are given by random constants, deterministic functions that decay slowly at hyperbolic rates, or mixtures of the two. Such patterns, along with other sample characteristics of the transformed time series, such as jumps in the sample path, excessive volatility, and leptokurtosis, suggest the possibility that these three ingredients are involved in the data generating processes of many actual economic and financial time series data. In addition to time series characteristics, we explore nonlinear regression asymptotics when the regressor is observable and an alternative regression technique when it is unobservable. To illustrate, we examine two empirical applications: wholesale electricity price spikes driven by capacity shortfalls and exchange rates governed by a target zone.persistency in memory, nonlinear transformations, random walks, thick tails, stable distributions, wholesale electricity prices, target zone exchange rates

The Application of Adaptive Linear and N on-Linear Filters to Fringe Order Identification in White-Light Interferometry Systems

Conventional optical interferometry systems driven by highly coherent light sources have a very short unambiguous operating range, a direct consequence of the flatness of the interference fringes visibility profile at the output of the system. The range can be extended by using a white-light interferometer (WU), which is driven by a low-coherence source and produces a Gaussian visibility profile with a unique maximum in correspondence of the central fringe. Due to system and/or measurement noise, however, the position of the maximum (from which an accurate measurement of the measurand - displacement, temperature, pressure, flow, etc. - can be derived) is not easily detectable, and can lead to large measurement errors. This is especially true in a multiplexing scheme, where the source power is distributed evenly among various sensors, with a corresponding drop in the overall signal-to-noise ratio. The inclusion of a signal processing scheme at the receiver end is thus a necessity. As the fringe pattern at the output of a WLI system is basically a noisy sine wave amplitude modulated by a Gaussian envelope, it can be classified as a non-stationary, narrow-band, linear but non-Gaussian signa\. So far, no attempt has been made to apply digital filtering techniques, as understood in the signal processing community, to the output signal of a WLI system. This thesis constitutes a first step in that direction. Since the only measurable information given by the system is contained in the output signal, the system is modelled as a "black box" driven by the system and measurement noise processes and containing an unknown set of parameters. Standard least squares techniques can then be applied to estimate the parameters of the model, as is usually done in the field of system identification when only noisy output measurements are available. It is shown that identification of the model parameters is equivalent to finding a set of coefficients for an inverse filter which takes the WU signal at its input and delivers the unknown noise process at the output. The non-stationarity of the signal is accounted for by allowing for time variations of the model parameters; this justifies the use of adaptive filters with time-varying coefficients. A new central fringe identification scheme is proposed, based on a modification of the standard least mean square (LMS) adaptive filtering algorithm in combination with amplitude thresholding of the fringe pattern. The new scheme is shown to offer considerable improvement in the identification rate when tested against current schemes over comparable operating ranges, while retaining the computational simplicity and operational speed of the standard LMS. Its performance is also shown to be largely independent of the step-size parameter controlling the rate of convergence and tracking in the standard LMS, which is known to be the main obstacle for a successful application of the algorithm in a practical setting. The non-Gaussianity of the signal is explored and an attempt is made to apply higher-order statistics (HOS) algorithms to central fringe identification. The effectiveness of Gaussianity tests on pilot Gaussian data is seen to depend not only on the number and length of records available but, perhaps more importantly, on the bandwidth of the process. Violation of the stationarity assumption is shown to lead to mis-classification of a seemingly non-Gaussian signal into a Gaussian one, as the visibility profile may alter the distribution of the underlying sinusoid making it appear Gaussian, even when beam diffraction and wavefront aberrations combine to produce a nonGaussian profile. HOS-based adaptive algorithms may still be of some benefit, however, if processing is confined to that region of the fringe pattern where sufficient non-Gaussianity is allowed to develop. Non-linear adaptive filters based on the Volterra theories are finally applied to compensate for possible non-linearities introduced by mismatches in optical components, chromatic aberrations, and analogue-to-digital converters. It is shown that although a Volterra filter is able to reproduce the low-amplitude distortions of the fringe pattern better than a linear filter does, the identification rate does not improve. Reasons are given for such behaviour

On adaptive filter structure and performance

SIGLEAvailable from British Library Document Supply Centre- DSC:D75686/87 / BLDSC - British Library Document Supply CentreGBUnited Kingdo