Widely Linear State Space Filtering of Improper Complex Signals

Complex signals are the backbone of many modern applications, such as power systems, communication systems, biomedical sciences and military technologies. However, standard complex valued signal processing approaches are suited to only a subset of complex signals known as proper, and are inadequate of the generality of complex signals, as they do not fully exploit the available information. This is mainly due to the inherent blindness of the algorithms to the complete second order statistics of the signals, or due to under-modelling of the underlying system. The aim of this thesis is to provide enhanced complex valued, state space based, signal processing solutions for the generality of complex signals and systems. This is achieved based on the recent advances in the so called augmented complex statistics and widely linear modelling, which have brought to light the limitations of conventional statistical complex signal processing approaches. Exploiting these developments, we propose a class of widely linear adaptive state space estimation techniques, which provide a unified framework and enhanced performance for the generality of complex signals, compared with conventional approaches. These include the linear and nonlinear Kalman and particle filters, whereby it is shown that catering for the complete second order information and system models leads to significant performance gains. The proposed techniques are also extended to the case of cooperative distributed estimation, where nodes in a network collaborate locally to estimate signals, under a framework that caters for general complex signals, as well as the cross-correlations between observation noises, unlike earlier solutions. The analysis of the algorithms are supported by numerous case studies, including frequency estimation in three phase power systems, DIFAR sonobuoy underwater target tracking, and real-world wind modeling and prediction.Open Acces

Assessing causality in financial time series

We develop new classes of semiparametric multivariate time series models based on Multi-Output Gaussian Processes and warped Multi-Output Gaussian Processes. These describe relationships between a current vector of observations and the lagged history of each marginal time series. We encode a serial dependence structure through mean and covariance functions and introduce a more complex dependence structure using copulas to couple each warped marginal Gaussian process. Within this class of models our primary goal is to detect causality and to study the interplay between the causal structure and the dependence structure. We do not, however, require true representation of the data generating process, but we model structural hypotheses regarding how causality may have manifested in the observed vector valued processes. With our framework we test the dependence with regards to the structures that are specified, and can use testing for causality under different model assumptions as a way to explore the data and the potentially complex dependence relationships. To perform the testing we consider several families of causality testing and develop compound tests which first require estimation/calibration of the mean and covariance functions parametrising the nonparametric vector valued time series. Our approach allows very general nonlinear dependence and causal relationships which are not often considered in classical parametric time series models, including causality in higher order information and joint extreme dependence features. We provide a generic framework which can be applied to a variety of different problem classes and discuss a number of examples to illustrate the ideas developed. Throughout, we will consider, without loss of generality, two multivariate time series denoted by X_t in R^d, Y_t in R^d' where one may assume, for instance, that these have been generated by observing partial realisations of a generalised diffusion processes: dX_t = mu_X(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dt + Sigma_X(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dW_t dY_t = mu_Y(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dt + Sigma_Y(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dW'_t, where Z_t, which may or may not be included, is some real process that we will call side information, dW_t, dW'_t are two different Brownian motions, possibly with marginal serial correlation and/or instantaneous cross-correlation. All of those processes are only partially observed, and may be sampled at irregular intervals. The form of drift and volatility described by the diffusion equation means that the processes X_t and Y_t can be conditionally dependent on each other, and this dependence can be introduced through both the drift and the volatility. Such generalised diffusion models can induce in the marginal process between X_t and Y_t different types of extremal dependence, depending on the forms of the drift and volatility functions. We propose a smooth stochastic process statistical model to capture the smooth variation of the partially observed time series represented by data X_t, Y_t, Z_t using multiple output Warped Gaussian Process models. In this work we are interested in partial observations of these processes, for which the partially observed time series of X_t and Y_t will have different types of extremal dependence characteristics. We wish to detect the presence or absence of statistical causality where such extremal dependence features may or may not obfuscate the ability to detect causality in nonlinear partially observed time series models. The rationale for developing a semiparametric solution for modelling the partially observed time series is that we may accommodate, through the use of Gaussian Process models, a wide variety of features for the hypotheses about the trends and volatility and importantly their possible causal structures, which can be formally tested in our framework. Furthermore the use of Warped Gaussian Process models allows to incorporate higher order dependence such as extremal tail dependence features. Statistical Causality. The notion of causality that lies at the centre of our research is the concept of statistical causality, based on comparing two predictive models. Quoting Wiener [1956]: "For two simultaneously measured signals, if we can predict the first signal better by using the past information from the second one than by using the information without it, then we call the second signal causal to the first one". The null hypothesis of no causal relationship from time series X_t to Y_t means that including the past of X_t does not improve the prediction of future of Y_t. In a most general form this can be written as equality of conditional distribution of Y, conditioning on either set of explanatory variables (X_t^{-k}, Y_t^{-l}, Z_t^{-m}) denote past of the X_t, Y_t, Z_t time series up to lags k,l,m respectively): H_0: p(Y_t | X_{t-1}^{-k}, Y_{t-1}^{-l}, Z_{t-1}^{-m}) = p(Y_t \mid Y_{t-1}^{-l}, \bZ_{t-1}^{-m}) H_1: p(Y_t | X_{t-1}^{-k}, Y_{t-1}^{-l}, Z_{t-1}^{-m}) p(Y_t \mid Y_{t-1}^{-l}, \bZ_{t-1}^{-m}). The type of casual dependence that is described by statistical causality is a mechanism that occurs at multiple lags over time - which could have been triggered by a sequence of processes, not an individual one. It can help to gain an insight into both cross-sectional and temporal dynamics of the data analysed. Warped Multi-Output Gaussian Processes. A Gaussian process is a Markov process, such that all finite dimensional distributions are Gaussian. While Gaussian processes models can accommodate wide range of properties and are very attractive for their easy implementation and optimisation, but they do not allow higher order dependence such as extremal tail dependence features. One way to generalise Gaussian process models so that higher order dependence can be handled, is to apply a transformation to the joint collection of Gaussian processes for each marginal time series model. We apply mean-variance transformation that results in the transformed variables having multivariate skew-t distributions and being finite dimensional realisations of a general multivariate skew-t process. Motivation for the Model Choice. There are numerous advantages of using Gaussian Processes, beginning with: ease of optimisation and interpretability of hyperparameters, flexibility, richness of covariance functions, allowing for various model structures. Using a likelihood ratio type test with a GP is a very natural choice, as estimating GP model parameters is often done on the basis of maximising likelihood, and therefore this estimation can be incorporated into the compound version of the likelihood ratio test (Generalised Likelihood Ratio Test, GLRT). From Gaussian variables, GPs inherited the property of being fully specified by the mean and the covariance, and so testing for model equivalence inherently means testing for equivalence of the mean and covariance functions. But many popular kernels do not have the ARD property, and using them for a likelihood ratio test settings gives no easy way to account for causal structures in covariance. Consequently, it is using GLRT with an ARD-GP that gives a uniformly most powerful test with an unparalleled flexibility: known asymptotic distribution under the null, explicit evaluation and in a closed form, and usefulness also for misspecified models. The proposed use of copula warping allows introduction of additional dependence, in particular tail dependence, while keeping the likelihood in closed form. Application. We provide a generic framework which can be applied to a wide range of problems, and which can be readily tailored or further extended. The illustrative examples included demonstrate how a range of data properties can be encoded in the model, and how they might affect the detection of causality. We present two real data application: to commodity futures data and inflation and interest rates. We show how the framework can be used in practice, and how it can be combined with, or enhance, more common approaches to analysing financial time series. Our observations are in line with financial interpretations, but they also offer additional insight and pose thought-provoking questions. Structure of the thesis. This thesis presents the research as it evolved: starting from an overview of a range of the causality methods already known, and demonstrating out why they are unsatisfactory. Subsequently, a new approach is presented -- a method based on Gaussian processes, that was developed to solve the drawbacks of the methods presented in the first part. Afterwards, an extension is proposed to widen the range of dependence structures, as well as marginal properties of the data that can be incorporated. Chapter 1 introduces the topic of the thesis, and reviews relevant literature. Chapter 2 discusses philosophical roots of the concept of statistical causality, as well as alternative notions of causality. After illustrating some of the varied ways of conceptual representation of causality, we present four distinct ways of modelling statistical causality. Chapter 3 contains background on the models considered: Gaussian processes, copulas and selected distributions. Chapter 4 describes inference procedures used: assessing hypothesis tests, generalised likelihood ratio test, permutation tests, and likelihood ratio test. The second part, New Perspectives on Causality Representation and Inference, presents the main contribution of our work. It starts with Chapter 5 containing the theoretical background for describing and testing causality with GP models. Chapter 6 extends the model from the previous chapter by introducing mean-variance transformation that results in a warped GP model, which can describe causality in the presence of skewness and tail dependence. Chapter 7 describes how synthetic data has been simulated, details the algorithm for approximating likelihood in the warped GP, and provides information on other relevant algorithms and the software used to implement our method. Chapter 8 presents an extensive experiment section, which aim to show, firstly, the good behaviour of the proposed procedures (model sensitivity and misspecification analysis), secondly, good power of the test for a range of structures, and, thirdly, the interaction of causality and tail dependence. Applications to real-world data are described in Chapter 9, where time series for commodities and currency markets are analysed. Finally, Chapter 10 presents the conclusions and directions for further development, and Appendices provide supplementary material

Distributed adaptive signal processing for frequency estimation

It is widely recognised that future smart grids will heavily rely upon intelligent communication and signal processing as enabling technologies for their operation. Traditional tools for power system analysis, which have been built from a circuit theory perspective, are a good match for balanced system conditions. However, the unprecedented changes that are imposed by smart grid requirements, are pushing the limits of these old paradigms. To this end, we provide new signal processing perspectives to address some fundamental operations in power systems such as frequency estimation, regulation and fault detection. Firstly, motivated by our finding that any excursion from nominal power system conditions results in a degree of non-circularity in the measured variables, we cast the frequency estimation problem into a distributed estimation framework for noncircular complex random variables. Next, we derive the required next generation widely linear, frequency estimators which incorporate the so-called augmented data statistics and cater for the noncircularity and a widely linear nature of system functions. Uniquely, we also show that by virtue of augmented complex statistics, it is possible to treat frequency tracking and fault detection in a unified way. To address the ever shortening time-scales in future frequency regulation tasks, the developed distributed widely linear frequency estimators are equipped with the ability to compensate for the fewer available temporal voltage data by exploiting spatial diversity in wide area measurements. This contribution is further supported by new physically meaningful theoretical results on the statistical behavior of distributed adaptive filters. Our approach avoids the current restrictive assumptions routinely employed to simplify the analysis by making use of the collaborative learning strategies of distributed agents. The efficacy of the proposed distributed frequency estimators over standard strictly linear and stand-alone algorithms is illustrated in case studies over synthetic and real-world three-phase measurements. An overarching theme in this thesis is the elucidation of underlying commonalities between different methodologies employed in classical power engineering and signal processing. By revisiting fundamental power system ideas within the framework of augmented complex statistics, we provide a physically meaningful signal processing perspective of three-phase transforms and reveal their intimate connections with spatial discrete Fourier transform (DFT), optimal dimensionality reduction and frequency demodulation techniques. Moreover, under the widely linear framework, we also show that the two most widely used frequency estimators in the power grid are in fact special cases of frequency demodulation techniques. Finally, revisiting classic estimation problems in power engineering through the lens of non-circular complex estimation has made it possible to develop a new self-stabilising adaptive three-phase transformation which enables algorithms designed for balanced operating conditions to be straightforwardly implemented in a variety of real-world unbalanced operating conditions. This thesis therefore aims to help bridge the gap between signal processing and power communities by providing power system designers with advanced estimation algorithms and modern physically meaningful interpretations of key power engineering paradigms in order to match the dynamic and decentralised nature of the smart grid.Open Acces

Heterogeneous data fusion for brain psychology applications

This thesis aims to apply Empirical Mode Decomposition (EMD), Multiscale Entropy (MSE), and collaborative adaptive filters for the monitoring of different brain consciousness states. Both block based and online approaches are investigated, and a possible extension to the monitoring and identification of Electromyograph (EMG) states is provided. Firstly, EMD is employed as a multiscale time-frequency data driven tool to decompose a signal into a number of band-limited oscillatory components; its data driven nature makes EMD an ideal candidate for the analysis of nonlinear and non-stationary data. This methodology is further extended to process multichannel real world data, by making use of recent theoretical advances in complex and multivariate EMD. It is shown that this can be used to robustly measure higher order features in multichannel recordings to robustly indicate ‘QBD’. In the next stage, analysis is performed in an information theory setting on multiple scales in time, using MSE. This enables an insight into the complexity of real world recordings. The results of the MSE analysis and the corresponding statistical analysis show a clear difference in MSE between the patients in different brain consciousness states. Finally, an online method for the assessment of the underlying signal nature is studied. This method is based on a collaborative adaptive filtering approach, and is shown to be able to approximately quantify the degree of signal nonlinearity, sparsity, and non-circularity relative to the constituent subfilters. To further illustrate the usefulness of the proposed data driven multiscale signal processing methodology, the final case study considers a human-robot interface based on a multichannel EMG analysis. A preliminary analysis shows that the same methodology as that applied to the analysis of brain cognitive states gives robust and accurate results. The analysis, simulations, and the scope of applications presented suggest great potential of the proposed multiscale data processing framework for feature extraction in multichannel data analysis. Directions for future work include further development of real-time feature map approaches and their use across brain-computer and brain-machine interface applications

Kalman-filter-based EEG source localization

This thesis uses the Kalman filter (KF) to solve the electroencephalographic (EEG) inverse problem to image its neuronal sources. Chapter 1 introduces EEG source localization and the KF and discusses how it can solve the inverse problem. Chapter 2 introduces an EEG inverse solution using a spatially whitened KF (SWKF) to reduce the computational burden. Likelihood maximization is used to fit spatially uniform neural model parameters to simulated and clinical EEGs. The SWKF accurately reconstructs source dynamics. Filter performance is analyzed by computing the innovations’ statistical properties and identifying spatial variations in performance that could be improved by use of spatially varying parameters. Chapter 3 investigates the SWKF via one-dimensional (1D) simulations. Motivated by Chapter 2, two model parameters are given Gaussian spatial profiles to better reflect brain dynamics. Constrained optimization ensures estimated parameters have clear biophysical interpretations. Inverse solutions are also computed using the optimal linear KF. Both filters produce accurate state estimates. Spatially varying parameters are correctly identified from datasets with transient dynamics, but estimates for driven datasets are degraded by the unmodeled drive term. Chapter 4 treats the whole-brain EEG inverse problem and applies features of the 1D simulations to the SWKF of Chapter 2. Spatially varying parameters are used to model spatial variation of the alpha rhythm. The simulated EEG here exhibits wave-like patterns and spatially varying dynamics. As in Chapter 3, optimization constrains model parameters to appropriate ranges. State estimation is again reliable for simulated and clinical EEG, although spatially varying parameters do not improve accuracy and parameter estimation is unreliable, with wave velocity underestimated. Contributing factors are identified and approaches to overcome them are discussed. Chapter 5 summarizes the main findings and outlines future work

Kalman-filter-based EEG source localization

This thesis uses the Kalman filter (KF) to solve the electroencephalographic (EEG) inverse problem to image its neuronal sources. Chapter 1 introduces EEG source localization and the KF and discusses how it can solve the inverse problem. Chapter 2 introduces an EEG inverse solution using a spatially whitened KF (SWKF) to reduce the computational burden. Likelihood maximization is used to fit spatially uniform neural model parameters to simulated and clinical EEGs. The SWKF accurately reconstructs source dynamics. Filter performance is analyzed by computing the innovations’ statistical properties and identifying spatial variations in performance that could be improved by use of spatially varying parameters. Chapter 3 investigates the SWKF via one-dimensional (1D) simulations. Motivated by Chapter 2, two model parameters are given Gaussian spatial profiles to better reflect brain dynamics. Constrained optimization ensures estimated parameters have clear biophysical interpretations. Inverse solutions are also computed using the optimal linear KF. Both filters produce accurate state estimates. Spatially varying parameters are correctly identified from datasets with transient dynamics, but estimates for driven datasets are degraded by the unmodeled drive term. Chapter 4 treats the whole-brain EEG inverse problem and applies features of the 1D simulations to the SWKF of Chapter 2. Spatially varying parameters are used to model spatial variation of the alpha rhythm. The simulated EEG here exhibits wave-like patterns and spatially varying dynamics. As in Chapter 3, optimization constrains model parameters to appropriate ranges. State estimation is again reliable for simulated and clinical EEG, although spatially varying parameters do not improve accuracy and parameter estimation is unreliable, with wave velocity underestimated. Contributing factors are identified and approaches to overcome them are discussed. Chapter 5 summarizes the main findings and outlines future work

Time Series Modelling

The analysis and modeling of time series is of the utmost importance in various fields of application. This Special Issue is a collection of articles on a wide range of topics, covering stochastic models for time series as well as methods for their analysis, univariate and multivariate time series, real-valued and discrete-valued time series, applications of time series methods to forecasting and statistical process control, and software implementations of methods and models for time series. The proposed approaches and concepts are thoroughly discussed and illustrated with several real-world data examples