Hidden tail chains and recurrence equations for dependence parameters associated with extremes of higher-order Markov chains

We derive some key extremal features for kth order Markov chains, which can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a threshold, as the threshold tends to the upper endpoint of the distribution. Unlike previous studies with k>1 we consider processes where standard limit theory describes each extreme event as a single observation without any information about the transition to and from the body of the distribution. The extremal properties of the Markov chain at lags up to k are determined by the kernel of the chain, through a joint initialisation distribution, with the subsequent values determined by the conditional independence structure through a transition behaviour. We study the extremal properties of each of these elements under weak assumptions for broad classes of extremal dependence structures. For chains with k>1, these transitions involve novel functions of the k previous states, in comparison to just the single value, when k=1. This leads to an increase in the complexity of determining the form of this class of functions, their properties and the method of their derivation in applications. We find that it is possible to find an affine normalization, dependent on the threshold excess, such that non-degenerate limiting behaviour of the process is assured for all lags. These normalization functions have an attractive structure that has parallels to the Yule-Walker equations. Furthermore, the limiting process is always linear in the innovations. We illustrate the results with the study of kth order stationary Markov chains based on widely studied families of copula dependence structures.Comment: 35 page

Testing for equal correlation matrices with application to paired gene expression data

We present a novel method for testing the hypothesis of equality of two correlation matrices using paired high-dimensional datasets. We consider test statistics based on the average of squares, maximum and sum of exceedances of Fisher transform sample correlations and we derive approximate null distributions using asymptotic and non-parametric distributions. Theoretical results on the power of the tests are presented and backed up by a range of simulation experiments. We apply the methodology to a case study of colorectal tumour gene expression data with the aim of discovering biological pathway lists of genes that present significantly different correlation matrices on healthy and tumour samples. We find strong evidence for a large part of the pathway lists correlation matrices to change among the two medical conditions.Comment: 31 pages, 3 figure

Statistical inference for radially-stable generalized Pareto distributions and return level-sets in geometric extremes

We obtain a functional analogue of the quantile function for probability measures admitting a continuous Lebesgue density on $\mathbb{R}^d$, and use it to characterize the class of non-trivial limit distributions of radially recentered and rescaled multivariate exceedances in geometric extremes. A new class of multivariate distributions is identified, termed radially stable generalized Pareto distributions, and is shown to admit certain stability properties that permit extrapolation to extremal sets along any direction in $\mathbb{R}^d$. Based on the limit Poisson point process likelihood of the radially renormalized point process of exceedances, we develop parsimonious statistical models that exploit theoretical links between structural star-bodies and are amenable to Bayesian inference. The star-bodies determine the mean measure of the limit Poisson process through a hierarchical structure. Our framework sharpens statistical inference by suitably including additional information from the angular directions of the geometric exceedances and facilitates efficient computations in dimensions $d=2$ and $d=3$. Additionally, it naturally leads to the notion of the return level-set, which is a canonical quantile set expressed in terms of its average recurrence interval, and a geometric analogue of the uni-dimensional return level. We illustrate our methods with a simulation study showing superior predictive performance of probabilities of rare events, and with two case studies, one associated with river flow extremes, and the other with oceanographic extremes.Comment: 65 pages, 34 figure

Hidden tail chains and recurrence equations for dependence parameters associated with extremes of stationary higher-order Markov chains

We derive some key extremal features for stationary kth-order Markov chains that can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a threshold, as the threshold tends to the upper endpoint of the distribution. Unlike previous studies with k>1, we consider processes where standard limit theory describes each extreme event as a single observation without any information about the transition to and from the body of the distribution. Our work uses different asymptotic theory which results in non-degenerate limit laws for such processes. We study the extremal properties of the initial distribution and the transition probability kernel of the Markov chain under weak assumptions for broad classes of extremal dependence structures that cover both asymptotically dependent and asymptotically independent Markov chains. For chains with k>1, the transition of the chain away from the exceedance involves novel functions of the k previous states, in comparison to just the single value, when k=1. This leads to an increase in the complexity of determining the form of this class of functions, their properties and the method of their derivation in applications. We find that it is possible to derive an affine normalization, dependent on the threshold excess, such that non-degenerate limiting behaviour of the process, in the neighbourhood of the threshold excess, is assured for all lags. We find that these normalization functions have an attractive structure that has parallels to the Yule-Walker equations. Furthermore, the limiting process is always linear in the innovations. We illustrate the results with the study of kth order stationary Markov chains with exponential margins based on widely studied families of copula dependence structures

Conditional independence and conditioned limit laws

Conditioned limit laws constitute an important and well developed framework of extreme value theory that describe a broad range of extremal dependence forms including asymptotic independence. We explore the assumption of conditional independence of $X_1$ and $X_2$ given $X_0$ and study its implication in the limiting distribution of $(X_1,X_2)$ conditionally on $X_0$ being large. We show that under random norming, conditional independence is always preserved in the conditioned limit law but might fail to do so when the normalisation does not include the precise value of the random variable in the conditioning event.Comment: 8 page