Statistical signal processing of nonstationary tensor-valued data

Real-world signals, such as the evolution of three-dimensional vector fields over time, can exhibit highly structured probabilistic interactions across their multiple constitutive dimensions. This calls for analysis tools capable of directly capturing the inherent multi-way couplings present in such data. Yet, current analyses typically employ multivariate matrix models and their associated linear algebras which are agnostic to the global data structure and can only describe local linear pairwise relationships between data entries. To address this issue, this thesis uses the property of linear separability -- a notion intrinsic to multi-dimensional data structures called tensors -- as a linchpin to consider the probabilistic, statistical and spectral separability under one umbrella. This helps to both enhance physical meaning in the analysis and reduce the dimensionality of tensor-valued problems. We first introduce a new identifiable probability distribution which appropriately models the interactions between random tensors, whereby linear relationships are considered between tensor fibres as opposed to between individual entries as in standard matrix analysis. Unlike existing models, the proposed tensor probability distribution formulation is shown to yield a unique maximum likelihood estimator which is demonstrated to be statistically efficient. Both matrices and vectors are lower-order tensors, and this gives us a unique opportunity to consider some matrix signal processing models under the more powerful framework of multilinear tensor algebra. By introducing a model for the joint distribution of multiple random tensors, it is also possible to treat random tensor regression analyses and subspace methods within a unified separability framework. Practical utility of the proposed analysis is demonstrated through case studies over synthetic and real-world tensor-valued data, including the evolution over time of global atmospheric temperatures and international interest rates. Another overarching theme in this thesis is the nonstationarity inherent to real-world signals, which typically consist of both deterministic and stochastic components. This thesis aims to help bridge the gap between formal probabilistic theory of stochastic processes and empirical signal processing methods for deterministic signals by providing a spectral model for a class of nonstationary signals, whereby the deterministic and stochastic time-domain signal properties are designated respectively by the first- and second-order moments of the signal in the frequency domain. By virtue of the assumed probabilistic model, novel tests for nonstationarity detection are devised and demonstrated to be effective in low-SNR environments. The proposed spectral analysis framework, which is intrinsically complex-valued, is facilitated by augmented complex algebra in order to fully capture the joint distribution of the real and imaginary parts of complex random variables, using a compact formulation. Finally, motivated by the need for signal processing algorithms which naturally cater for the nonstationarity inherent to real-world tensors, the above contributions are employed simultaneously to derive a general statistical signal processing framework for nonstationary tensors. This is achieved by introducing a new augmented complex multilinear algebra which allows for a concise description of the multilinear interactions between the real and imaginary parts of complex tensors. These contributions are further supported by new physically meaningful empirical results on the statistical analysis of nonstationary global atmospheric temperatures.Open Acces

High Dimensional Forecasting via Interpretable Vector Autoregression

Vector autoregression (VAR) is a fundamental tool for modeling multivariate time series. However, as the number of component series is increased, the VAR model becomes overparameterized. Several authors have addressed this issue by incorporating regularized approaches, such as the lasso in VAR estimation. Traditional approaches address overparameterization by selecting a low lag order, based on the assumption of short range dependence, assuming that a universal lag order applies to all components. Such an approach constrains the relationship between the components and impedes forecast performance. The lasso-based approaches work much better in high-dimensional situations but do not incorporate the notion of lag order selection. We propose a new class of hierarchical lag structures (HLag) that embed the notion of lag selection into a convex regularizer. The key modeling tool is a group lasso with nested groups which guarantees that the sparsity pattern of lag coefficients honors the VAR's ordered structure. The HLag framework offers three structures, which allow for varying levels of flexibility. A simulation study demonstrates improved performance in forecasting and lag order selection over previous approaches, and a macroeconomic application further highlights forecasting improvements as well as HLag's convenient, interpretable output

Nonlinear Gaussian Filtering : Theory, Algorithms, and Applications

By restricting to Gaussian distributions, the optimal Bayesian filtering problem can be transformed into an algebraically simple form, which allows for computationally efficient algorithms. Three problem settings are discussed in this thesis: (1) filtering with Gaussians only, (2) Gaussian mixture filtering for strong nonlinearities, (3) Gaussian process filtering for purely data-driven scenarios. For each setting, efficient algorithms are derived and applied to real-world problems

Channel Prediction for Mobile MIMO Wireless Communication Systems

Temporal variation and frequency selectivity of wireless channels constitute a major drawback to the attainment of high gains in capacity and reliability offered by multiple antennas at the transmitter and receiver of a mobile communication system. Limited feedback and adaptive transmission schemes such as adaptive modulation and coding, antenna selection, power allocation and scheduling have the potential to provide the platform of attaining the high transmission rate, capacity and QoS requirements in current and future wireless communication systems. Theses schemes require both the transmitter and receiver to have accurate knowledge of Channel State Information (CSI). In Time Division Duplex (TDD) systems, CSI at the transmitter can be obtained using channel reciprocity. In Frequency Division Duplex (FDD) systems, however, CSI is typically estimated at the receiver and fed back to the transmitter via a low-rate feedback link. Due to the inherent time delays in estimation, processing and feedback, the CSI obtained from the receiver may become outdated before its actual usage at the transmitter. This results in significant performance loss, especially in high mobility environments. There is therefore a need to extrapolate the varying channel into the future, far enough to account for the delay and mitigate the performance degradation. The research in this thesis investigates parametric modeling and prediction of mobile MIMO channels for both narrowband and wideband systems. The focus is on schemes that utilize the additional spatial information offered by multiple sampling of the wave-field in multi-antenna systems to aid channel prediction. The research has led to the development of several algorithms which can be used for long range extrapolation of time-varyingchannels. Based on spatial channel modeling approaches, simple and efficient methods for the extrapolation of narrowband MIMO channels are proposed. Various extensions were also developed. These include methods for wideband channels, transmission using polarized antenna arrays, and mobile-to-mobile systems. Performance bounds on the estimation and prediction error are vital when evaluating channel estimation and prediction schemes. For this purpose, analytical expressions for bound on the estimation and prediction of polarized and non-polarized MIMO channels are derived. Using the vector formulation of the Cramer Rao bound for function of parameters, readily interpretable closed-form expressions for the prediction error bounds were found for cases with Uniform Linear Array (ULA) and Uniform Planar Array (UPA). The derived performance bounds are very simple and so provide insight into system design. The performance of the proposed algorithms was evaluated using standardized channel models. The effects of the temporal variation of multipath parameters on prediction is studied and methods for jointly tracking the channel parameters are developed. The algorithms presented can be utilized to enhance the performance of limited feedback and adaptive MIMO transmission schemes

Tensor approximation methods for stochastic problems

Spektrale stochastische Methoden haben sich als effizientes Werkzeug zur Modellierung von Systemen mit Unsicherheiten etabliert. Der Vorteil dieser Methoden ist, dass sie nicht nur Statistiken liefern, sondern auch eine direkte Darstellung der Lösung als sogenanntes Surrogatmodell. Besonders attraktiv für elliptische stochastische partielle Differentialgleichungen (SPDGln) ist das stochastische Galerkin Verfahren, da in diesem wesentliche Eigenschaften des Differentialoperators erhalten bleiben. Ein Nachteil der Methode ist jedoch, dass enorme Mengen an Speicherplatz benötigt werden, da die Lösung in einem Tensorprodukt der räumlichen und stochastischen Ansatzräume liegt. Bisher wurden verschiedene Ansätze erprobt, um diese Anforderung zu verringern. Hierzu zählen Modellreduktionstechniken, Unterraumiterationen, um den Lösungsraum auf einen beherrschbaren Unterraum einzuschränken, oder Methoden, welche die Lösung schrittweise aus Rang-1 Produkten aufzubauen. In der vorliegenden Arbeit werden Bestapproximationen der Lösungen linearer SPDGln als Niedrig-Rang-Darstellungen gesucht. Dies wird dadurch erreicht, dass Tensordarstellungen sowohl für die Eingangsdaten als auch für die Lösung verwendet und während des ganzen iterativen Lösungsprozesses beibehalten werden. Da diese Darstellungen weitere Näherungen während des Lösungsprozesses erfordern, ist es wesentlich die Konvergenz der Lösung genau zu überwachen. Ferner müssen Besonderheiten der Präkonditionierung der diskreten Systeme und der Stagnation der iterativen Verfahren beachtet werden. Mit dem Ziel der praktischen Anwendbarkeit als einem wesentlichen Bestandteil dieser Arbeit wurde großer Wert auf eine detaillierte Beschreibung der Implementierungstechniken gelegt.Spectral stochastic methods have gained wide acceptance as a tool for efficient modelling of uncertain stochastic systems. The advantage of those methods is that they provide not only statistics, but give a direct representation of the measure of the solution as a so-called surrogate model, which can be used for very fast sampling. Especially attractive for elliptic stochastic partial differential equations (SPDEs) is the stochastic Galerkin method, since it preserves essential properties of the differential operator. One drawback of the method is, however, that it requires huge amounts of memory, as the solution is represented in a tensor product space of spatial and stochastic basis functions. Different approaches have been investigated to reduce the memory requirements, for example, model reduction techniques using subspace iterations to reduce the approximation space or methods of approximating the solution from successive rank-1 updates. In the present thesis best approximations to the solutions of linear elliptic SPDEs are constructed in low-rank tensor representations. By using tensor formats for all random quantities, the best subsets for representing the solution are computed “on the fly” during the entire process of solving the SPDE. As those representations require additional approximations during the solution process it is essential to control the convergence of the solution. Furthermore, special issues with preconditioning of the discrete system and stagnation of the iterative methods need adequate treatment. Since one goal of this work was practical usability, special emphasis has been given to implementation techniques and their description in the necessary detail