Engineering Streaming Algorithms

Streaming algorithms must process a large quantity of small updates quickly to allow queries about the input to be answered from a small summary. Initial work on streaming algorithms laid out theoretical results, and subsequent efforts have involved engineering these for practical use. Informed by experiments, streaming algorithms have been widely implemented and used in practice. This talk will survey this line of work, and identify some lessons learned

XQuery Streaming by Forest Transducers

Streaming of XML transformations is a challenging task and only very few systems support streaming. Research approaches generally define custom fragments of XQuery and XPath that are amenable to streaming, and then design custom algorithms for each fragment. These languages have several shortcomings. Here we take a more principles approach to the problem of streaming XQuery-based transformations. We start with an elegant transducer model for which many static analysis problems are well-understood: the Macro Forest Transducer (MFT). We show that a large fragment of XQuery can be translated into MFTs --- indeed, a fragment of XQuery, that can express important features that are missing from other XQuery stream engines, such as GCX: our fragment of XQuery supports XPath predicates and let-statements. We then rely on a streaming execution engine for MFTs, one which uses a well-founded set of optimizations from functional programming, such as strictness analysis and deforestation. Our prototype achieves time and memory efficiency comparable to the fastest known engine for XQuery streaming, GCX. This is surprising because our engine relies on the OCaml built in garbage collector and does not use any specialized buffer management, while GCX's efficiency is due to clever and explicit buffer management.Comment: Full version of the paper in the Proceedings of the 30th IEEE International Conference on Data Engineering (ICDE 2014

A clustering algorithm for multivariate data streams with correlated components

Common clustering algorithms require multiple scans of all the data to achieve convergence, and this is prohibitive when large databases, with data arriving in streams, must be processed. Some algorithms to extend the popular K-means method to the analysis of streaming data are present in literature since 1998 (Bradley et al. in Scaling clustering algorithms to large databases. In: KDD. p. 9-15, 1998; O'Callaghan et al. in Streaming-data algorithms for high-quality clustering. In: Proceedings of IEEE international conference on data engineering. p. 685, 2001), based on the memorization and recursive update of a small number of summary statistics, but they either don't take into account the specific variability of the clusters, or assume that the random vectors which are processed and grouped have uncorrelated components. Unfortunately this is not the case in many practical situations. We here propose a new algorithm to process data streams, with data having correlated components and coming from clusters with different covariance matrices. Such covariance matrices are estimated via an optimal double shrinkage method, which provides positive definite estimates even in presence of a few data points, or of data having components with small variance. This is needed to invert the matrices and compute the Mahalanobis distances that we use for the data assignment to the clusters. We also estimate the total number of clusters from the data.Comment: title changed, rewritte

Decision Making Analysis of Video Streaming Algorithm for Private Cloud Computing Infrastructure

The issue on how to effectively deliver video streaming contents over cloud computing infrastructures is tackled in this study. Basically, quality of service of video streaming is strongly influenced by bandwidth, jitter and data loss problems. A number of intelligent video streaming algorithms are proposed by using different techniques to deal with such issues. This study aims to propose and demonstrate a novel decision making analysis which combines ISO 9126 (international standard for software engineering) and Analytic Hierarchy Process to help experts selecting the best video streaming algorithm for the case of private cloud computing infrastructure. The given case study concluded that Scalable Streaming algorithm is the best algorithm to be implemented for delivering high quality of service of video streaming over  the private cloud computing infrastructure

Downlink Video Streaming for Users Non-Equidistant from Base Station

We consider multiuser video transmission for users that are non-equidistantly positioned from base station. We propose a greedy algorithm for video streaming in a wireless system with capacity achieving channel coding, that implements the cross-layer principle by partially separating the physical and the application layer. In such a system the parameters at the physical layer are dependent on the packet length and the conditions in the wireless channel and the parameters at the application layer are dependent on the reduction of the expected distortion assuming no packet errors in the system. We also address the fairness in the multiuser video system with non-equidistantly positioned users. Our fairness algorithm is based on modified opportunistic round robin scheduling. We evaluate the performance of the proposed algorithms by simulating the transmission of H.264/AVC video signals in a TDMA wireless system

Evaluation of HTTP/DASH Adaptation Algorithms on Vehicular Networks

Video streaming currently accounts for the majority of Internet traffic. One factor that enables video streaming is HTTP Adaptive Streaming (HAS), that allows the users to stream video using a bit rate that closely matches the available bandwidth from the server to the client. MPEG Dynamic Adaptive Streaming over HTTP (DASH) is a widely used standard, that allows the clients to select the resolution to download based on their own estimations. The algorithm for determining the next segment in a DASH stream is not partof the standard, but it is an important factor in the resulting playback quality. Nowadays vehicles are increasingly equipped with mobile communication devices, and in-vehicle multimedia entertainment systems. In this paper, we evaluate the performance of various DASH adaptation algorithms over a vehicular network. We present detailed simulation results highlighting the advantages and disadvantages of various adaptation algorithms in delivering video content to vehicular users, and we show how the different adaptation algorithms perform in terms of throughput, playback interruption time, and number of interruptions

Applications of Fog Computing in Video Streaming

The purpose of this paper is to show the viability of fog computing in the area of video streaming in vehicles. With the rise of autonomous vehicles, there needs to be a viable entertainment option for users. The cloud fails to address these options due to latency problems experienced during high internet traffic. To improve video streaming speeds, fog computing seems to be the best option. Fog computing brings the cloud closer to the user through the use of intermediary devices known as fog nodes. It does not attempt to replace the cloud but improve the cloud by allowing faster upload and download of information. This paper explores two algorithms that would work well with vehicles and video streaming. This is simulated using a Java application, and then graphically represented. The results showed that the simulation was an accurate model and that the best algorithm for request history maintenance was the variable model