Online Inference for Mixture Model of Streaming Graph Signals with Non-White Excitation

This paper considers a joint multi-graph inference and clustering problem for simultaneous inference of node centrality and association of graph signals with their graphs. We study a mixture model of filtered low pass graph signals with possibly non-white and low-rank excitation. While the mixture model is motivated from practical scenarios, it presents significant challenges to prior graph learning methods. As a remedy, we consider an inference problem focusing on the node centrality of graphs. We design an expectation-maximization (EM) algorithm with a unique low-rank plus sparse prior derived from low pass signal property. We propose a novel online EM algorithm for inference from streaming data. As an example, we extend the online algorithm to detect if the signals are generated from an abnormal graph. We show that the proposed algorithms converge to a stationary point of the maximum-a-posterior (MAP) problem. Numerical experiments support our analysis

Online Machine Learning for Graph Topology Identification from Multiple Time Series

High dimensional time series data are observed in many complex systems. In networked data, some of the time series are influenced by other time series. Identifying these relations encoded in a graph structure or topology among the time series is of paramount interest in certain applications since the identified structure can provide insights about the underlying system and can assist in inference tasks. In practice, the underlying topology is usually sparse, that is, not all the participating time series in influence each other. The goal of this dissertation pertains to study the problem of sparse topology identification under various settings. Topology identification from time series is a challenging task. The first major challenge in topology identification is that the assumption of static topology does not hold always in practice since most of the practical systems are evolving with time. For instance, in econometrics, social networks, etc., the relations among the time series can change over time. Identifying the topologies of such dynamic networks is a major challenge. The second major challenge is that in most practical scenarios, the data is not available at once - it is coming in a streaming fashion. Hence, batch approaches are either not applicable or they become computationally expensive since a batch algorithm is needed to be run when a new datum becomes available. The third challenge is that the multi-dimensional time series data can contain missing values due faulty sensors, privacy and security reasons, or due to saving energy. We address the aforementioned challenges in this dissertation by proposing online/-batch algorithms to solve the problem of time-varying topology identification. A model based on vector autoregressive (VAR) process is adopted initially. The parameters of the VAR model reveal the topology of the underlying network. First, two online algorithms are proposed for the case of streaming data. Next, using the same VAR model, two online algorithms under the framework of online optimization are presented to track the time-varying topologies. To evaluate the performance of propose online algorithms, we show that both the proposed algorithms incur a sublinear static regret. To characterize the performance theoretically in time-varying scenarios, a bound on the dynamic regret for one of the proposed algorithms (TIRSO) is derived. Next, using a structural equation model (SEM) for topology identification, an online algorithm for tracking time-varying topologies is proposed, and a bound on the dynamic regret is also derived for the proposed algorithm. Moreover, using a non-stationary VAR model, an algorithm for dynamic topology identification and breakpoint detection is also proposed, where the notion of local structural breakpoint is introduced to accommodate the concept of breakpoint where instead of the whole topology, only a few edges vary. Finally, the problem of tracking VAR-based time-varying topologies with missing data is investigated. Online algorithms are proposed where the joint signal and topology estimation is carried out. Dynamic regret analysis is also presented for the proposed algorithm. For all the previously mentioned works, simulation tests about the proposed algorithms are also presented and discussed in this dissertation. The numerical results of the proposed algorithms corroborate with the theoretical analysis presented in this dissertation.publishedVersio

Online Joint Topology Identification and Signal Estimation with Inexact Proximal Online Gradient Descent

Identifying the topology that underlies a set of time series is useful for tasks such as prediction, denoising, and data completion. Vector autoregressive (VAR) model based topologies capture dependencies among time series, and are often inferred from observed spatio-temporal data. When the data are affected by noise and/or missing samples, the tasks of topology identification and signal recovery (reconstruction) have to be performed jointly. Additional challenges arise when i) the underlying topology is time-varying, ii) data become available sequentially, and iii) no delay is tolerated. To overcome these challenges, this paper proposes two online algorithms to estimate the VAR model-based topologies. The proposed algorithms have constant complexity per iteration, which makes them interesting for big data scenarios. They also enjoy complementary merits in terms of complexity and performance. A performance guarantee is derived for one of the algorithms in the form of a dynamic regret bound. Numerical tests are also presented, showcasing the ability of the proposed algorithms to track the time-varying topologies with missing data in an online fashion.Comment: 14 pages including supplementary material, 2 figures, submitted to IEEE Transactions on Signal Processin

Online Machine Learning for Inference from Multivariate Time-series

Inference and data analysis over networks have become significant areas of research due to the increasing prevalence of interconnected systems and the growing volume of data they produce. Many of these systems generate data in the form of multivariate time series, which are collections of time series data that are observed simultaneously across multiple variables. For example, EEG measurements of the brain produce multivariate time series data that record the electrical activity of different brain regions over time. Cyber-physical systems generate multivariate time series that capture the behaviour of physical systems in response to cybernetic inputs. Similarly, financial time series reflect the dynamics of multiple financial instruments or market indices over time. Through the analysis of these time series, one can uncover important details about the behavior of the system, detect patterns, and make predictions. Therefore, designing effective methods for data analysis and inference over networks of multivariate time series is a crucial area of research with numerous applications across various fields. In this Ph.D. Thesis, our focus is on identifying the directed relationships between time series and leveraging this information to design algorithms for data prediction as well as missing data imputation. This Ph.D. thesis is organized as a compendium of papers, which consists of seven chapters and appendices. The first chapter is dedicated to motivation and literature survey, whereas in the second chapter, we present the fundamental concepts that readers should understand to grasp the material presented in the dissertation with ease. In the third chapter, we present three online nonlinear topology identification algorithms, namely NL-TISO, RFNL-TISO, and RFNL-TIRSO. In this chapter, we assume the data is generated from a sparse nonlinear vector autoregressive model (VAR), and propose online data-driven solutions for identifying nonlinear VAR topology. We also provide convergence guarantees in terms of dynamic regret for the proposed algorithm RFNL-TIRSO. Chapters four and five of the dissertation delve into the issue of missing data and explore how the learned topology can be leveraged to address this challenge. Chapter five is distinct from other chapters in its exclusive focus on edge flow data and introduces an online imputation strategy based on a simplicial complex framework that leverages the known network structure in addition to the learned topology. Chapter six of the dissertation takes a different approach, assuming that the data is generated from nonlinear structural equation models. In this chapter, we propose an online topology identification algorithm using a time-structured approach, incorporating information from both the data and the model evolution. The algorithm is shown to have convergence guarantees achieved by bounding the dynamic regret. Finally, chapter seven of the dissertation provides concluding remarks and outlines potential future research directions.publishedVersio