1D1D to nDnD: A Meta Algorithm for Multivariate Global Optimization via Univariate Optimizers

In this work, we propose a meta algorithm that can solve a multivariate global optimization problem using univariate global optimizers. Although the univariate global optimization does not receive much attention compared to the multivariate case, which is more emphasized in academia and industry; we show that it is still relevant and can be directly used to solve problems of multivariate optimization. We also provide the corresponding regret bounds in terms of the time horizon $T$ and the average regret of the univariate optimizer, when it is robust against nonnegative noises with robust regret guarantees.Comment: this article extends arXiv:2108.10859, arXiv:2201.0716

Online change detection techniques in time series: an overview

Time-series change detection has been studied in several fields. From sensor data, engineering systems, medical diagnosis, and financial markets to user actions on a network, huge amounts of temporal data are generated. There is a need for a clear separation between normal and abnormal behaviour of the system in order to investigate causes or forecast change. Characteristics include irregularities, deviations, anomalies, outliers, novelties or surprising patterns. The efficient detection of such patterns is challenging, especially when constraints need to be taken into account, such as the data velocity, volume, limited time for reacting to events, and the details of the temporal sequence.This paper reviews the main techniques for time series change point detection, focusing on online methods. Performance criteria including complexity, time granularity, and robustness is used to compare techniques, followed by a discussion about current challenges and open issue

Two-stage data segmentation permitting multiscale change points, heavy tails and dependence

The segmentation of a time series into piecewise stationary segments, a.k.a. multiple change point analysis, is an important problem both in time series analysis and signal processing. In the presence of multiscale change points with both large jumps over short intervals and small changes over long stationary intervals, multiscale methods achieve good adaptivity in their localisation but at the same time, require the removal of false positives and duplicate estimators via a model selection step. In this paper, we propose a localised application of Schwarz information criterion which, as a generic methodology, is applicable with any multiscale candidate generating procedure fulfilling mild assumptions. We establish the theoretical consistency of the proposed localised pruning method in estimating the number and locations of multiple change points under general assumptions permitting heavy tails and dependence. Further, we show that combined with a MOSUM-based candidate generating procedure, it attains minimax optimality in terms of detection lower bound and localisation for i.i.d. sub-Gaussian errors. A careful comparison with the existing methods by means of (a) theoretical properties such as generality, optimality and algorithmic complexity, (b) performance on simulated datasets and run time, as well as (c) performance on real data applications, confirm the overall competitiveness of the proposed methodology

Data-Driven Quickest Change Detection

The quickest change detection (QCD) problem is to detect abrupt changes in a sensing environment as quickly as possible in real time while limiting the risk of false alarm. Statistical inference about the monitored stochastic process is performed through observations acquired sequentially over time. After each observation, QCD algorithm either stops and declares a change or continues to have a further observation in the next time interval. There is an inherent tradeoff between speed and accuracy in the decision making process. The design goal is to optimally balance the average detection delay and the false alarm rate to have a timely and accurate response to abrupt changes. The objective of this thesis is to investigate effective and scalable QCD approaches for real-world data streams. The classical QCD framework is model-based, that is, statistical data model is assumed to be known for both the pre- and post-change cases. However, real-world data often exhibit significant challenges for data modeling such as high dimensionality, complex multivariate nature, lack of parametric models, unknown post-change (e.g., attack or anomaly) patterns, and complex temporal correlation. Further, in some cases, data is privacy-sensitive and distributed over a system, and it is not fully available to QCD algorithm. This thesis addresses these challenges and proposes novel data-driven QCD approaches that are robust to data model mismatch and hence widely applicable to a variety of practical settings. In Chapter 2, online cyber-attack detection in the smart power grid is formulated as a partially observable Markov decision process (POMDP) problem based on the QCD framework. A universal robust online cyber-attack detection algorithm is proposed using the model-free reinforcement learning (RL) for POMDPs. In Chapter 3, online anomaly detection for big data streams is studied where the nominal (i.e., pre-change) and anomalous (i.e., post-change) high-dimensional statistical data models are unknown. A data-driven solution approach is proposed, where firstly a set of useful univariate summary statistics is computed from a nominal dataset in an offline phase and next, online summary statistics are evaluated for a persistent deviation from the nominal statistics. In Chapter 4, a generic data-driven QCD procedure is proposed, called DeepQCD, that learns the change detection rule directly from the observed raw data via deep recurrent neural networks. With sufficient amount of training data including both pre- and post-change samples, DeepQCD can effectively learn the change detection rule for all complex, high-dimensional, and temporally correlated data streams. Finally, in Chapter 5, online privacy-preserving anomaly detection is studied in a setting where the data is distributed over a network and locally sensitive to each node, and its statistical model is unknown. A data-driven differentially private distributed detection scheme is proposed, which infers network-wide anomalies based on the perturbed and encrypted statistics received from nodes. Furthermore, analytical privacy-security tradeoff in the network-wide anomaly detection problem is investigated

Optimal Bayesian Quickest Detection for Hidden Markov Models and Structured Generalisations

In this paper we consider the problem of quickly detecting changes in hidden Markov models (HMMs) in a Bayesian setting, as well as several structured generalisations including changes in statistically periodic processes, quickest detection of a Markov process across a sensor array, quickest detection of a moving target in a sensor network and quickest change detection (QCD) in multistream data. Our main result establishes an optimal Bayesian HMM QCD rule with a threshold structure. This framework and proof techniques allow us to to elegantly establish optimal rules for several structured generalisations by showing that these problems are special cases of the Bayesian HMM QCD problem. We develop bounds to characterise the performance of our optimal rule and provide an efficient method for computing the test statistic. Finally, we examine the performance of our rule in several simulation examples and propose a technique for calculating the optimal threshold