The crowd as a cameraman : on-stage display of crowdsourced mobile video at large-scale events

Recording videos with smartphones at large-scale events such as concerts and festivals is very common nowadays. These videos register the atmosphere of the event as it is experienced by the crowd and offer a perspective that is hard to capture by the professional cameras installed throughout the venue. In this article, we present a framework to collect videos from smartphones in the public and blend these into a mosaic that can be readily mixed with professional camera footage and shown on displays during the event. The video upload is prioritized by matching requests of the event director with video metadata, while taking into account the available wireless network capacity. The proposed framework's main novelty is its scalability, supporting the real-time transmission, processing and display of videos recorded by hundreds of simultaneous users in ultra-dense Wi-Fi environments, as well as its proven integration in commercial production environments. The framework has been extensively validated in a controlled lab setting with up to 1 000 clients as well as in a field trial where 1 183 videos were collected from 135 participants recruited from an audience of 8 050 people. 90 % of those videos were uploaded within 6.8 minutes

Quality-aware Content Adaptation in Digital Video Streaming

User-generated video has attracted a lot of attention due to the success of Video Sharing Sites such as YouTube and Online Social Networks. Recently, a shift towards live consumption of these videos is observable. The content is captured and instantly shared over the Internet using smart mobile devices such as smartphones. Large-scale platforms arise such as YouTube.Live, YouNow or Facebook.Live which enable the smartphones of users to livestream to the public. These platforms achieve the distribution of tens of thousands of low resolution videos to remote viewers in parallel. Nonetheless, the providers are not capable to guarantee an efficient collection and distribution of high-quality video streams. As a result, the user experience is often degraded, and the needed infrastructure installments are huge. Efficient methods are required to cope with the increasing demand for these video streams; and an understanding is needed how to capture, process and distribute the videos to guarantee a high-quality experience for viewers. This thesis addresses the quality awareness of user-generated videos by leveraging the concept of content adaptation. Two types of content adaptation, the adaptive video streaming and the video composition, are discussed in this thesis. Then, a novel approach for the given scenario of a live upload from mobile devices, the processing of video streams and their distribution is presented. This thesis demonstrates that content adaptation applied to each step of this scenario, ranging from the upload to the consumption, can significantly improve the quality for the viewer. At the same time, if content adaptation is planned wisely, the data traffic can be reduced while keeping the quality for the viewers high. The first contribution of this thesis is a better understanding of the perceived quality in user-generated video and its influencing factors. Subjective studies are performed to understand what affects the human perception, leading to the first of their kind quality models. Developed quality models are used for the second contribution of this work: novel quality assessment algorithms. A unique attribute of these algorithms is the usage of multiple features from different sensors. Whereas classical video quality assessment algorithms focus on the visual information, the proposed algorithms reduce the runtime by an order of magnitude when using data from other sensors in video capturing devices. Still, the scalability for quality assessment is limited by executing algorithms on a single server. This is solved with the proposed placement and selection component. It allows the distribution of quality assessment tasks to mobile devices and thus increases the scalability of existing approaches by up to 33.71% when using the resources of only 15 mobile devices. These three contributions are required to provide a real-time understanding of the perceived quality of the video streams produced on mobile devices. The upload of video streams is the fourth contribution of this work. It relies on content and mechanism adaptation. The thesis introduces the first prototypically evaluated adaptive video upload protocol (LiViU) which transcodes multiple video representations in real-time and copes with changing network conditions. In addition, a mechanism adaptation is integrated into LiViU to react to changing application scenarios such as streaming high-quality videos to remote viewers or distributing video with a minimal delay to close-by recipients. A second type of content adaptation is discussed in the fifth contribution of this work. An automatic video composition application is presented which enables live composition from multiple user-generated video streams. The proposed application is the first of its kind, allowing the in-time composition of high-quality video streams by inspecting the quality of individual video streams, recording locations and cinematographic rules. As a last contribution, the content-aware adaptive distribution of video streams to mobile devices is introduced by the Video Adaptation Service (VAS). The VAS analyzes the video content streamed to understand which adaptations are most beneficial for a viewer. It maximizes the perceived quality for each video stream individually and at the same time tries to produce as little data traffic as possible - achieving data traffic reduction of more than 80%

Collaborative Uploading in Heterogeneous Networks: Optimal and Adaptive Strategies

Collaborative uploading describes a type of crowdsourcing scenario in networked environments where a device utilizes multiple paths over neighboring devices to upload content to a centralized processing entity such as a cloud service. Intermediate devices may aggregate and preprocess this data stream. Such scenarios arise in the composition and aggregation of information, e.g., from smartphones or sensors. We use a queuing theoretic description of the collaborative uploading scenario, capturing the ability to split data into chunks that are then transmitted over multiple paths, and finally merged at the destination. We analyze replication and allocation strategies that control the mapping of data to paths and provide closed-form expressions that pinpoint the optimal strategy given a description of the paths' service distributions. Finally, we provide an online path-aware adaptation of the allocation strategy that uses statistical inference to sequentially minimize the expected waiting time for the uploaded data. Numerical results show the effectiveness of the adaptive approach compared to the proportional allocation and a variant of the join-the-shortest-queue allocation, especially for bursty path conditions.Comment: 15 pages, 11 figures, extended version of a conference paper accepted for publication in the Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), 201

Providing crowd-sourced and real-time media services through a NDN-based platform

International audienceThe diffusion of social networks and broadband technologies is letting emerge large online communities of people that stay always in touch with each other and exchange messages, thoughts, photos, videos, files, and any other type of contents. At the same time, due to the introduction of crowd-sourcing strategies, according to which services and contents can be obtained by soliciting contributions from a group of users, the amount of data generated and exchanged within a social community may experience a radical increment never seen before. In this context, it becomes essential to guarantee resource scalability and load balancing to support real time media delivery. To this end, the present book chapter aims at investigating the design of a network architecture, based on the emerging Named Data Networking (NDN) paradigm, providing crowd-sourced real-time media contents. Such an architecture is composed by four different entities: a very large group of heterogeneous devices that produce media contents to be shared, an equally large group of users interested in them, a distributed Event Management System that creates events and handles the social community, and a NDN communication infrastructure able to efficiently manage users requests and distribute multimedia contents. To demonstrate the effectiveness of the proposed approach, we have evaluate its performance through a simulation campaign using real-world topologies

Automatic Mobile Video Remixing and Collaborative Watching Systems

In the thesis, the implications of combining collaboration with automation for remix creation are analyzed. We first present a sensor-enhanced Automatic Video Remixing System (AVRS), which intelligently processes mobile videos in combination with mobile device sensor information. The sensor-enhanced AVRS system involves certain architectural choices, which meet the key system requirements (leverage user generated content, use sensor information, reduce end user burden), and user experience requirements. Architecture adaptations are required to improve certain key performance parameters. In addition, certain operating parameters need to be constrained, for real world deployment feasibility. Subsequently, sensor-less cloud based AVRS and low footprint sensorless AVRS approaches are presented. The three approaches exemplify the importance of operating parameter tradeoffs for system design. The approaches cover a wide spectrum, ranging from a multimodal multi-user client-server system (sensor-enhanced AVRS) to a mobile application which can automatically generate a multi-camera remix experience from a single video. Next, we present the findings from the four user studies involving 77 users related to automatic mobile video remixing. The goal was to validate selected system design goals, provide insights for additional features and identify the challenges and bottlenecks. Topics studied include the role of automation, the value of a video remix as an event memorabilia, the requirements for different types of events and the perceived user value from creating multi-camera remix from a single video. System design implications derived from the user studies are presented. Subsequently, sport summarization, which is a specific form of remix creation is analyzed. In particular, the role of content capture method is analyzed with two complementary approaches. The first approach performs saliency detection in casually captured mobile videos; in contrast, the second one creates multi-camera summaries from role based captured content. Furthermore, a method for interactive customization of summary is presented. Next, the discussion is extended to include the role of users’ situational context and the consumed content in facilitating collaborative watching experience. Mobile based collaborative watching architectures are described, which facilitate a common shared context between the participants. The concept of movable multimedia is introduced to highlight the multidevice environment of current day users. The thesis presents results which have been derived from end-to-end system prototypes tested in real world conditions and corroborated with extensive user impact evaluation

Systems and Methods for Measuring and Improving End-User Application Performance on Mobile Devices

In today's rapidly growing smartphone society, the time users are spending on their smartphones is continuing to grow and mobile applications are becoming the primary medium for providing services and content to users. With such fast paced growth in smart-phone usage, cellular carriers and internet service providers continuously upgrade their infrastructure to the latest technologies and expand their capacities to improve the performance and reliability of their network and to satisfy exploding user demand for mobile data. On the other side of the spectrum, content providers and e-commerce companies adopt the latest protocols and techniques to provide smooth and feature-rich user experiences on their applications. To ensure a good quality of experience, monitoring how applications perform on users' devices is necessary. Often, network and content providers lack such visibility into the end-user application performance. In this dissertation, we demonstrate that having visibility into the end-user perceived performance, through system design for efficient and coordinated active and passive measurements of end-user application and network performance, is crucial for detecting, diagnosing, and addressing performance problems on mobile devices. My dissertation consists of three projects to support this statement. First, to provide such continuous monitoring on smartphones with constrained resources that operate in such a highly dynamic mobile environment, we devise efficient, adaptive, and coordinated systems, as a platform, for active and passive measurements of end-user performance. Second, using this platform and other passive data collection techniques, we conduct an in-depth user trial of mobile multipath to understand how Multipath TCP (MPTCP) performs in practice. Our measurement study reveals several limitations of MPTCP. Based on the insights gained from our measurement study, we propose two different schemes to address the identified limitations of MPTCP. Last, we show how to provide visibility into the end- user application performance for internet providers and in particular home WiFi routers by passively monitoring users' traffic and utilizing per-app models mapping various network quality of service (QoS) metrics to the application performance.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146014/1/ashnik_1.pd

Assessing Quality of Experience of Video Streaming Applications via Crowdsourcing

Για δεκαετίες, το Quality of Service έχει υπάρξει η κυρίαρχη μετρική με την οποία οι δυνατότητες ενός δικτύου μετριούνται. Πρόσφατα όμως μια νέα μετρική, το Quality of Experience, έχει ανοδική πορεία και σταθερά γίνεται η πρωτεύουσα, καθώς έχει αποδειχτεί ότι απεικονίζει το βαθμό ικανοποίησης των συνδρομητών με πολύ πιο ακριβή τρόπο. Όπως είναι φυσιολογικό, προκλήσεις έχουν προκύψει που αφορούν αυτήν τη νέα μεθοδολογία. Τα μέσα με τα οποία παράγονται αξιόπιστα αποτελέσματα με έναν οικονομικά βιώσιμο τρόπο είναι το κύριο από αυτά, αφού κάποιες παραδοσιακές τεχνικές όπως τα ελεγχόμενα πειράματα σε εργαστήρια μπορούν να χρησιμοποϊηθούν μόνο σε περιορισμένο βαθμό. Μία πιθανή λύση σε αυτό το πρόβλημα είναι το crowdsourcing, που πρακτικά είναι η διαδικασία της εκτέλεσης πειραμάτων σε ειδικές διαδικτυακές πλατφόρμες. Συμμετέχοντες στα πειράματα από όλον τον κόσμο παίρνουν μέρος εθελοντικά για μια μικρή χρηματική αποζημίωση. Αυτή η εργασία επικεντρώνεται στην παρουσίαση των αρχών του Quality of Experience μαζί με τις βασικές διαφορές με το Quality of Sevice, περιγράφει λεπτομερώς τη σωστή χρήση του crowdsourcing κι εξετάζει αν αυτή αποτελεί μια κατάλληλη πηγή ανάδρασης των χρηστών, πραγματοποιώντας διάφορα πειράματα που αφορούν συγκεκριμένα χαρακτηριστικά της ποιότηττας ενός δικτύου, όπως αυτά βιώνονται από τους χρήστες.For decades, Quality of Service has been the dominant metric by which the capabilities of a network are determined. Recently though another metric, Quality of Experience, has gained significant traction and is steadily becoming the standard, as it has been proven to depict the satisfaction rate of the subscribers much more accurately. Naturally, challenges have risen relating to his new methodology. The means to produce credible results in a financially viable way is the main one, as some traditional techniques such as controlled experiments in a laboratory can only be utilized in a limited capacity. A possible solution to this problem is crowdsourcing, which essentially is the process of conducting experiments via special online platforms. Test subjects around the world willingly participate in these experiments for a small monetary compensation. This paper focuses on presenting the principles of Quality of Experience along with the key differences with Quality of Service, describes in detail the proper use of crowdsourcing and examines whether it is a suitable source of user feedback, by conducting several test cases regarding specific aspects of the quality of a network, as experienced by the subscribers

"A Community Unlike Any Other": Incorporating Fansubbers into Corporate Capitalism on Viki.com

Viki.com, founded in 2008, is a streaming site that offers Korean (and other East Asian) television programs with subtitles in a variety of languages. Unlike other K-drama distribution sites that serve audiences outside of South Korea, Viki utilizes fan-volunteers, called fansubbers, as laborers to produce its subtitles. Fan subtitling and distribution of foreign language media in the United States is a rich fan practice dating back to the 1980s, and Viki is the first corporate entity that has harnessed the productive power of fansubbers. In this thesis, I investigate how Viki has been able to capture the enthusiasm and productive capacity of fansubbers. Particularly, I examine how Viki has been able to monetize fansubbing in while still staying competitive with sites who employee trained, professional translators. I argue that Viki has succeeded in courting fansubbers as laborers by co-opting the concept of the “fan community.” I focus on how Viki strategically speaks about the community and builds its site to facilitate the functioning of its community so as to encourage fansubbers to view themselves as semi-professional laborers instead of amateur fans. In reframing the role of the fansubbing community, Viki creates a new image for what being a fansubber means and why fansubbing is valuable that emphasizes creating value for Viki over creating value for the fan community.Plan II Honors Progra