The crowd as a cameraman : on-stage display of crowdsourced mobile video at large-scale events

Recording videos with smartphones at large-scale events such as concerts and festivals is very common nowadays. These videos register the atmosphere of the event as it is experienced by the crowd and offer a perspective that is hard to capture by the professional cameras installed throughout the venue. In this article, we present a framework to collect videos from smartphones in the public and blend these into a mosaic that can be readily mixed with professional camera footage and shown on displays during the event. The video upload is prioritized by matching requests of the event director with video metadata, while taking into account the available wireless network capacity. The proposed framework's main novelty is its scalability, supporting the real-time transmission, processing and display of videos recorded by hundreds of simultaneous users in ultra-dense Wi-Fi environments, as well as its proven integration in commercial production environments. The framework has been extensively validated in a controlled lab setting with up to 1 000 clients as well as in a field trial where 1 183 videos were collected from 135 participants recruited from an audience of 8 050 people. 90 % of those videos were uploaded within 6.8 minutes

Optimal Big Data Aggregation Systems - From Theory to Practical Application

The integration of computers into many facets of our lives has made the collection and storage of staggering amounts of data feasible. However, the data on its own is not so useful to us as the analysis and manipulation which allows manageable descriptive information to be extracted. New tools to extract this information from ever growing repositories of data are required. Some of these analyses can take the form of a two phase problem which is easily distributed to take advantage of available computing power. The first phase involves computing some descriptive partial result from some subset of the original data, and the second phase involves aggregating all the partial results to create a combined output. We formalize this compute-aggregate model for a rigorous performance analysis in an effort to minimize the latency of the aggregation phase with minimal intrusive analysis or modification. Based on our model we find an aggregation overlay attribute which highly affects aggregation latency and its dependence on an easily findable trait of aggregation. We rigorously prove the dependence and find optimal overlays for aggregation. We use the proven optima to create simple heuristics and build a system, NOAH, to take advantage of the findings. NOAH can be used by big data analysis systems. We also study an individual problem, top-k matching, to explore the effects of optimizing the computation phase separately from aggregation and create a complete distributed system to fulfill an economically relevant task

Fast, Expressive Top-k Matching

Top-k matching is a fundamental problem underlying on-line advertising platforms, mobile social networks, etc. Distributed processes (e.g., advertisers) specify predicates, which we call subscriptions, for events (e.g., user actions) they wish to react to. Subscriptions define weights for elementary constraints on individual event attributes and do not require that events match all constraints. An event is multicast only to the processes with the k highest match scores for that event -- this score is the aggregation of the weights of all constraints in a subscription matching the event. However, state-of-the-art approaches to top-k matching support only rigid models of events and subscriptions, which leads to suboptimal matches. We present a novel model of weighted top-k matching which is more expressive than the state-of-the-art, and a corresponding efficient algorithm. Our model supports attributes with intervals, weights specified by producers of events or by subscriptions, negative weights, prorating of matched constraints, and the ability to vary scores dynamically with system parameters. Our algorithm exhibits time and space complexities which are competitive with state-of-the-art algorithms regardless of our added expressiveness -- O(M log N + S log k) and O(M N + k) respectively, with N the number of constraints, M the number of event attributes, and S the number of matching constraints. Through empirical evaluation with both statistically generated and real-world data we demonstrate that our algorithm is (a) equally or more efficient and scalable than the state-of-the art without exploiting our added expressiveness, and it (b) significantly outperforms existing approaches upgraded -- if possible at all -- to match our expressiveness