Fast and Accurate Mining of Correlated Heavy Hitters

The problem of mining Correlated Heavy Hitters (CHH) from a two-dimensional data stream has been introduced recently, and a deterministic algorithm based on the use of the Misra--Gries algorithm has been proposed by Lahiri et al. to solve it. In this paper we present a new counter-based algorithm for tracking CHHs, formally prove its error bounds and correctness and show, through extensive experimental results, that our algorithm outperforms the Misra--Gries based algorithm with regard to accuracy and speed whilst requiring asymptotically much less space

Doctor of Philosophy

dissertationMatrices are essential data representations for many large-scale problems in data analytics; for example, in text analysis under the bag-of-words model, a large corpus of documents are often represented as a matrix. Many data analytic tasks rely on obtaining a summary (a.k.a sketch) of the data matrix. Using this summary in place of the original data matrix saves on space usage and run-time of machine learning algorithms. Therefore, sketching a matrix is often a necessary first step in data reduction, and sometimes has direct relationships to core techniques including PCA, LDA, and clustering. In this dissertation, we study the problem of matrix sketching over data streams. We first describe a deterministic matrix sketching algorithm called FrequentDirections. The algorithm is presented an arbitrary input matrix A∈ Rn&× d one row at a time. It performs O(dl) operations per row and maintains a sketch matrix B ∈ Rl× d such that for any k< l, ||ATA - BTB \|| 2 < ||A - Ak||F2 / (l-k) and ||A - πBk(A)||F2 ≤ (1 + k/l-k)||A-Ak||F2 . Here, Ak stands for the minimizer of ||A - Ak||F over all rank k matrices (similarly Bk), and πBk (A) is the rank k matrix resulting from projecting A on the row span of Bk. We show both of these bounds are the best possible for the space allowed, the sketch is mergeable, and hence trivially parallelizable. We propose several variants of FrequentDirections that improve its error-size tradeoff, and nearly matches the simple heuristic Iterative SVD method in practice. We then describe SparseFrequentDirections for sketching sparse matrices. It resembles the original algorithm in many ways including having the same optimal asymptotic guarantees with respect to the space-accuracy tradeoff in the streaming setting, but unlike FrequentDirections which runs in O(ndl) time, SparseFrequentDirections runs in Õ(nnz(A)l + nl2) time. We then extend our methods to distributed streaming model, where there are m distributed sites each observing a distinct stream of data, and which has a communication channel with a coordinator. The goal is to track an ε-approximation (for ε ∈ (0,1)) to the norm of the matrix along any direction. We present novel algorithms to address this problem. All our methods satisfy an additive error bound that for any unit vector x, | ||A x||2 - ||B x ||2 | ≤ |ε ||A||F2 holds

SQUAD: Combining Sketching and Sampling Is Better than Either for Per-item Quantile Estimation

Latency quantiles measurements are essential as they often capture the user's utility. For example, if a video connection has high tail latency, the perceived quality will suffer, even if the average and median latencies are low. In this work, we consider the problem of approximating the per-item quantiles. Elements in our stream are (ID, latency) tuples, and we wish to track the latency quantiles for each ID. Existing quantile sketches are designed for a single number stream (e.g., containing just the latency). While one could allocate a separate sketch instance for each ID, this may require an infeasible amount of memory. Instead, we consider tracking the quantiles for the heavy hitters (most frequent items), which are often considered particularly important, without knowing them beforehand. We first present a simple sampling algorithm that serves as a benchmark. Then, we design an algorithm that augments a quantile sketch within each entry of a heavy hitter algorithm, resulting in similar space complexity but with a deterministic error guarantee. Finally, we present SQUAD, a method that combines sampling and sketching while improving the asymptotic space complexity. Intuitively, SQUAD uses a background sampling process to capture the behaviour of the latencies of an item before it is allocated with a sketch, thereby allowing us to use fewer samples and sketches. Our solutions are rigorously analyzed, and we demonstrate the superiority of our approach using extensive simulations