Causality and Network Graph in General Bilinear State-Space Representations

International audienc

Assessing causality in financial time series

We develop new classes of semiparametric multivariate time series models based on Multi-Output Gaussian Processes and warped Multi-Output Gaussian Processes. These describe relationships between a current vector of observations and the lagged history of each marginal time series. We encode a serial dependence structure through mean and covariance functions and introduce a more complex dependence structure using copulas to couple each warped marginal Gaussian process. Within this class of models our primary goal is to detect causality and to study the interplay between the causal structure and the dependence structure. We do not, however, require true representation of the data generating process, but we model structural hypotheses regarding how causality may have manifested in the observed vector valued processes. With our framework we test the dependence with regards to the structures that are specified, and can use testing for causality under different model assumptions as a way to explore the data and the potentially complex dependence relationships. To perform the testing we consider several families of causality testing and develop compound tests which first require estimation/calibration of the mean and covariance functions parametrising the nonparametric vector valued time series. Our approach allows very general nonlinear dependence and causal relationships which are not often considered in classical parametric time series models, including causality in higher order information and joint extreme dependence features. We provide a generic framework which can be applied to a variety of different problem classes and discuss a number of examples to illustrate the ideas developed. Throughout, we will consider, without loss of generality, two multivariate time series denoted by X_t in R^d, Y_t in R^d' where one may assume, for instance, that these have been generated by observing partial realisations of a generalised diffusion processes: dX_t = mu_X(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dt + Sigma_X(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dW_t dY_t = mu_Y(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dt + Sigma_Y(t, X_t^{-k}, Y_t^{-l}, Z_t^{-m}) dW'_t, where Z_t, which may or may not be included, is some real process that we will call side information, dW_t, dW'_t are two different Brownian motions, possibly with marginal serial correlation and/or instantaneous cross-correlation. All of those processes are only partially observed, and may be sampled at irregular intervals. The form of drift and volatility described by the diffusion equation means that the processes X_t and Y_t can be conditionally dependent on each other, and this dependence can be introduced through both the drift and the volatility. Such generalised diffusion models can induce in the marginal process between X_t and Y_t different types of extremal dependence, depending on the forms of the drift and volatility functions. We propose a smooth stochastic process statistical model to capture the smooth variation of the partially observed time series represented by data X_t, Y_t, Z_t using multiple output Warped Gaussian Process models. In this work we are interested in partial observations of these processes, for which the partially observed time series of X_t and Y_t will have different types of extremal dependence characteristics. We wish to detect the presence or absence of statistical causality where such extremal dependence features may or may not obfuscate the ability to detect causality in nonlinear partially observed time series models. The rationale for developing a semiparametric solution for modelling the partially observed time series is that we may accommodate, through the use of Gaussian Process models, a wide variety of features for the hypotheses about the trends and volatility and importantly their possible causal structures, which can be formally tested in our framework. Furthermore the use of Warped Gaussian Process models allows to incorporate higher order dependence such as extremal tail dependence features. Statistical Causality. The notion of causality that lies at the centre of our research is the concept of statistical causality, based on comparing two predictive models. Quoting Wiener [1956]: "For two simultaneously measured signals, if we can predict the first signal better by using the past information from the second one than by using the information without it, then we call the second signal causal to the first one". The null hypothesis of no causal relationship from time series X_t to Y_t means that including the past of X_t does not improve the prediction of future of Y_t. In a most general form this can be written as equality of conditional distribution of Y, conditioning on either set of explanatory variables (X_t^{-k}, Y_t^{-l}, Z_t^{-m}) denote past of the X_t, Y_t, Z_t time series up to lags k,l,m respectively): H_0: p(Y_t | X_{t-1}^{-k}, Y_{t-1}^{-l}, Z_{t-1}^{-m}) = p(Y_t \mid Y_{t-1}^{-l}, \bZ_{t-1}^{-m}) H_1: p(Y_t | X_{t-1}^{-k}, Y_{t-1}^{-l}, Z_{t-1}^{-m}) p(Y_t \mid Y_{t-1}^{-l}, \bZ_{t-1}^{-m}). The type of casual dependence that is described by statistical causality is a mechanism that occurs at multiple lags over time - which could have been triggered by a sequence of processes, not an individual one. It can help to gain an insight into both cross-sectional and temporal dynamics of the data analysed. Warped Multi-Output Gaussian Processes. A Gaussian process is a Markov process, such that all finite dimensional distributions are Gaussian. While Gaussian processes models can accommodate wide range of properties and are very attractive for their easy implementation and optimisation, but they do not allow higher order dependence such as extremal tail dependence features. One way to generalise Gaussian process models so that higher order dependence can be handled, is to apply a transformation to the joint collection of Gaussian processes for each marginal time series model. We apply mean-variance transformation that results in the transformed variables having multivariate skew-t distributions and being finite dimensional realisations of a general multivariate skew-t process. Motivation for the Model Choice. There are numerous advantages of using Gaussian Processes, beginning with: ease of optimisation and interpretability of hyperparameters, flexibility, richness of covariance functions, allowing for various model structures. Using a likelihood ratio type test with a GP is a very natural choice, as estimating GP model parameters is often done on the basis of maximising likelihood, and therefore this estimation can be incorporated into the compound version of the likelihood ratio test (Generalised Likelihood Ratio Test, GLRT). From Gaussian variables, GPs inherited the property of being fully specified by the mean and the covariance, and so testing for model equivalence inherently means testing for equivalence of the mean and covariance functions. But many popular kernels do not have the ARD property, and using them for a likelihood ratio test settings gives no easy way to account for causal structures in covariance. Consequently, it is using GLRT with an ARD-GP that gives a uniformly most powerful test with an unparalleled flexibility: known asymptotic distribution under the null, explicit evaluation and in a closed form, and usefulness also for misspecified models. The proposed use of copula warping allows introduction of additional dependence, in particular tail dependence, while keeping the likelihood in closed form. Application. We provide a generic framework which can be applied to a wide range of problems, and which can be readily tailored or further extended. The illustrative examples included demonstrate how a range of data properties can be encoded in the model, and how they might affect the detection of causality. We present two real data application: to commodity futures data and inflation and interest rates. We show how the framework can be used in practice, and how it can be combined with, or enhance, more common approaches to analysing financial time series. Our observations are in line with financial interpretations, but they also offer additional insight and pose thought-provoking questions. Structure of the thesis. This thesis presents the research as it evolved: starting from an overview of a range of the causality methods already known, and demonstrating out why they are unsatisfactory. Subsequently, a new approach is presented -- a method based on Gaussian processes, that was developed to solve the drawbacks of the methods presented in the first part. Afterwards, an extension is proposed to widen the range of dependence structures, as well as marginal properties of the data that can be incorporated. Chapter 1 introduces the topic of the thesis, and reviews relevant literature. Chapter 2 discusses philosophical roots of the concept of statistical causality, as well as alternative notions of causality. After illustrating some of the varied ways of conceptual representation of causality, we present four distinct ways of modelling statistical causality. Chapter 3 contains background on the models considered: Gaussian processes, copulas and selected distributions. Chapter 4 describes inference procedures used: assessing hypothesis tests, generalised likelihood ratio test, permutation tests, and likelihood ratio test. The second part, New Perspectives on Causality Representation and Inference, presents the main contribution of our work. It starts with Chapter 5 containing the theoretical background for describing and testing causality with GP models. Chapter 6 extends the model from the previous chapter by introducing mean-variance transformation that results in a warped GP model, which can describe causality in the presence of skewness and tail dependence. Chapter 7 describes how synthetic data has been simulated, details the algorithm for approximating likelihood in the warped GP, and provides information on other relevant algorithms and the software used to implement our method. Chapter 8 presents an extensive experiment section, which aim to show, firstly, the good behaviour of the proposed procedures (model sensitivity and misspecification analysis), secondly, good power of the test for a range of structures, and, thirdly, the interaction of causality and tail dependence. Applications to real-world data are described in Chapter 9, where time series for commodities and currency markets are analysed. Finally, Chapter 10 presents the conclusions and directions for further development, and Appendices provide supplementary material

Causality inference between time series data and its applications

Ever since Granger first proposed the idea of quantitatively testing the causal relationship between data streams, the endeavor of accurately inferring the causality in data and using that information to predict the future has not stopped. Artificial Intelligence (AI), by utilizing the massive amounts of data, helps to solve complex problems, whether they include the diagnosis and detection of disease through medical imaging, email spam detection, or self-driving vehicles. Perhaps, this thesis will be trivial in ten years from now. AI has pushed humankind to reach the next technological level in technology. Nowadays, among most machine leaning inquiries, statistical relationships are determined using correlation measures. By feeding data into machine learning algorithms, computers update the algorithm’s parameters iteratively by extracting and mapping features to learning targets until the correlation increases to a significant level to cease the training process. However, with the increasing developments of powerful AI, there is really a shortage of exploring causality in data. It is almost self-evident that ”correlation is not causality." Sometimes, the strong correlation established between variables through machine learning can be absurd and meaningless. Providing insight into causality information through data, which most of the machine learning methods fall short to do, is of paramount importance. The subsequent chapters detail the four endeavors of studying causality in financial markets, earthquakes, animal/human brain signals, the predictivity of data sets. In Chapter 2, we further developed the concept of causality networks into a higher-order causality network. We applied these to financial data and tested their validity and ability to capture the system’s causal relationship. In next Chapter 3, We examined another type of time series-earthquakes. Violent seismic activities decimate people's lives and destroy entire cities and areas. This begs us to understand how earthquakes work and help us make reliably and evacuation-actionable predictions. The causal relationships of seismic activities in different areas are studied and established. Biological data, specifically brain signals, are time-series data and their causal pattern are explored and studied. Different human and mice brain signals are analyzed and clustered in Chapter 4 using their unique causal pattern to understand different brain cell activity. Finally, we realized that the causal pattern in the time series can be used to compress data. A causal compression ratio is invented and used as the data stream’s predictivity index. We describe this in Chapter 5

Sparse Learning for Variable Selection with Structures and Nonlinearities

In this thesis we discuss machine learning methods performing automated variable selection for learning sparse predictive models. There are multiple reasons for promoting sparsity in the predictive models. By relying on a limited set of input variables the models naturally counteract the overfitting problem ubiquitous in learning from finite sets of training points. Sparse models are cheaper to use for predictions, they usually require lower computational resources and by relying on smaller sets of inputs can possibly reduce costs for data collection and storage. Sparse models can also contribute to better understanding of the investigated phenomenons as they are easier to interpret than full models.Comment: PhD thesi

Nonlinear Impulse Response Functions and Local Projections

The goal of this paper is to extend the method of estimating Impluse Response Functions (IRFs) by means of Local Projection (LP) in a nonlinear dynamic framework. We discuss the existence of a nonlinear autoregressive representation for a Markov process, and explain how their Impulse Response Functions are directly linked to the nonlinear Local Projection, as in the case for the linear setting. We then present a nonparametric LP estimator, and compare its asymptotic properties to that of IRFs obtained through direct estimation. We also explore issues of identification for the nonlinear IRF in the multivariate framework, which remarkably differs in comparison to the Gaussian linear case. In particular, we show that identification is conditional on the uniqueness of deconvolution. Then, we consider IRF and LP in augmented Markov models.Comment: 44 pages, 4 figure

Limit Theory under Network Dependence and Nonstationarity

These lecture notes represent supplementary material for a short course on time series econometrics and network econometrics. We give emphasis on limit theory for time series regression models as well as the use of the local-to-unity parametrization when modeling time series nonstationarity. Moreover, we present various non-asymptotic theory results for moderate deviation principles when considering the eigenvalues of covariance matrices as well as asymptotics for unit root moderate deviations in nonstationary autoregressive processes. Although not all applications from the literature are covered we also discuss some open problems in the time series and network econometrics literature.Comment: arXiv admin note: text overlap with arXiv:1705.08413 by other author