The Effective Transmission and Processing of Mobile Multimedia

Ph.DDOCTOR OF PHILOSOPH

무선랜 비디오 멀티캐스트의 문제 발견 및 성능 향상 기법

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·컴퓨터공학부, 2017. 8. 최성현.Video multicast, streaming real-time videos via multicast, over wireless local area network (WLAN) has been considered a promising solution to share common venue-specific videos. By virtue of the nature of the wireless broadcast medium, video multicast basically enables scale-free video delivery, i.e., it can deliver a common video with the fixed amount of wireless resource regardless of the number of receivers. However, video multicast has not been widely enjoyed in our lives due to three major challenges: (1) power saving-related problem, (2) low reliability and efficiency, and (3) limited coverage. In this dissertation, we consider three research topics, i.e., (1) identification of practical issues with multicast power saving, (2) physical (PHY) rate and forward erasure correction code (FEC) rate adaptation over a single-hop network, and (3) multi-hop multicast, which deal with the three major challenges, respectively. Firstly, video multicast needs to be reliably delivered to power-saving stations, given that many portable devices are battery-powered. Accordingly, we investigate the impact of multicast power saving, and address two practical issues related with the multicast power saving. From the measurement with several commercial WLAN devices, we observe that many devices are not standard compliant, thus making video multicast performance severely degraded. We categorize such standard incompliant malfunctions that can result in significant packet losses. We also figure out a coexistence problem between video multicast and voice over Internet protocol (VoIP) when video receivers runs in power saving mode (PSM). The standard-compliant power save delivery of multicast deteriorates the VoIP performance in the same WLAN. We analyze the VoIP packet losses due to the coexistence problem, and propose a new power save delivery scheme to resolve the problem. We further implement the proposed scheme with an open source device driver, and our measurement results demonstrate that the proposed scheme significantly enhances the VoIP performance without sacrificing the video multicast performance. Second, multi-PHY rate FEC-applied wireless multicast enables reliable and efficient video multicast with intelligent selection of PHY rate and FEC rate. The optimal PHY/FEC rates depend on the cause of the packet losses. However, previous approaches select the PHY/FEC rates by considering only channel errors even when interference is also a major source of packet losses.We propose InFRA, an interference-aware PHY/FEC rate adaptation framework that (1) infers the cause of the packet losses based on received signal strength indicator (RSSI) and cyclic redundancy check (CRC) error notifications, and (2) determines the PHY/FEC rates based on the cause of packet losses. Our prototype implementation with off-the-shelf chipsets demonstrates that InFRA enhances the multicast delivery under various network scenarios. InFRA enables 2.3x and 1.8x more nodes to achieve a target video packet loss rate with a contention interferer and a hidden interferer, respectively, compared with the state-of-theart PHY/FEC rate adaptation scheme. To the best of our knowledge, InFRA is the first work to take the impact of interference into account for the PHY/FEC rate adaptation. Finally, collaborative relaying that enables selected receiver nodes to relay the received packets from source node to other nodes enhances service coverage, reliability, and efficiency of video multicast. The intelligent selection of sender nodes (source and relays) and their transmission parameters (PHY rate and the number of packets to send) is the key to optimize the performance. We propose EV-CAST, an interference and energy-aware video multicast system using collaborative relays, which entails online network management based on interference-aware link characterization, an algorithm for joint determination of sender nodes and transmission parameters, and polling-based relay protocol. In order to select most appropriate set of the relay nodes, EV-CAST considers interference, battery status, and spatial reuse, as well as other factors accumulated over last decades. Our prototype-based measurement results demonstrate that EV-CAST outperforms the state-of-the-art video multicast schemes. In summary, from Chapter 2 to Chapter 4, the aforementioned three pieces of the research work, i.e., identification of power saving-related practical issues, InFRA for interference-resilient single-hop multicast, and EV-CAST for efficient multi-hop multicast, will be presented, respectively.1 Introduction 1 1.1 Video Multicast over WLAN 1 1.2 Overview of Existing Approaches 4 1.2.1 Multicast Power Saving 4 1.2.2 Reliability and Efficiency Enhancement 4 1.2.3 Coverage Extension 5 1.3 Main Contributions 7 1.3.1 Practical Issues with Multicast Power Saving 7 1.3.2 Interference-aware PHY/FEC Rate Adaptation 8 1.3.3 Energy-aware Multi-hop Multicast 9 1.4 Organization of the Dissertation 10 2 Practical Issues with Multicast Power Saving 12 2.1 Introduction 12 2.2 Multicast & Power Management Operation in IEEE 802.11 14 2.3 Inter-operability Issue 15 2.3.1 Malfunctions of Commercial WLAN Devices 17 2.3.2 Performance Evaluation 20 2.4 Coexistence Problem of Video Multicast and VoIP 21 2.4.1 Problem Statement 21 2.4.2 Problem Identification: A Measurement Study 23 2.4.3 Packet Loss Analysis 27 2.4.4 Proposed Scheme 32 2.4.5 Performance Evaluation 33 2.5 Summary 37 3 InFRA: Interference-Aware PHY/FEC Rate Adaptation for Video Multicast over WLAN 39 3.1 Introduction 39 3.2 Related Work 42 3.2.1 Reliable Multicast Protocol 42 3.2.2 PHY/FEC rate adaptation for multicast service 44 3.2.3 Wireless Video Transmission 45 3.2.4 Wireless Loss Differentiation 46 3.3 Impact of Interference on Multi-rate FEC-applied Multicast 46 3.3.1 Measurement Setup 47 3.3.2 Measurement Results 47 3.4 InFRA: Interference-aware PHY/FEC Rate Adaptation Framework 49 3.4.1 Network Model and Objective 49 3.4.2 Overall Architecture 50 3.4.3 FEC Scheme 52 3.4.4 STA-side Operation 53 3.4.5 AP-side Operation 61 3.4.6 Practical Issues 62 3.5 Performance Evaluation 65 3.5.1 Measurement Setup 66 3.5.2 Small Scale Evaluation 67 3.5.3 Large Scale Evaluation 70 3.6 Summary 74 4 EV-CAST: Interference and Energy-aware Video Multicast Exploiting Collaborative Relays 75 4.1 Introduction 75 4.2 Factors for Sender Node and Transmission Parameter Selection 78 4.3 EV-CAST: Interference and Energy-aware Multicast Exploiting Collaborative Relays 80 4.3.1 Network Model and Objective 80 4.3.2 Overview 81 4.3.3 Network Management 81 4.3.4 Interference and Energy-aware Sender Nodes and Transmission Parameter Selection (INFER) Algorithm 87 4.3.5 Assignment, Polling, and Re-selection of Relays 93 4.3.6 Discussion 95 4.4 Evaluation 96 4.4.1 Measurement Setup 96 4.4.2 Micro-benchmark 98 4.4.3 Macro-benchmark 103 4.5 Related Work 105 4.5.1 Multicast Opportunistic Routing 105 4.5.2 Multicast over WLAN 106 4.6 Summary 106 5 Conclusion 108 5.1 Research Contributions 108 5.2 Future Research Directions 109 Abstract (In Korean) 121Docto

QoE on media deliveriy in 5G environments

231 p.5G expandirá las redes móviles con un mayor ancho de banda, menor latencia y la capacidad de proveer conectividad de forma masiva y sin fallos. Los usuarios de servicios multimedia esperan una experiencia de reproducción multimedia fluida que se adapte de forma dinámica a los intereses del usuario y a su contexto de movilidad. Sin embargo, la red, adoptando una posición neutral, no ayuda a fortalecer los parámetros que inciden en la calidad de experiencia. En consecuencia, las soluciones diseñadas para realizar un envío de tráfico multimedia de forma dinámica y eficiente cobran un especial interés. Para mejorar la calidad de la experiencia de servicios multimedia en entornos 5G la investigación llevada a cabo en esta tesis ha diseñado un sistema múltiple, basado en cuatro contribuciones.El primer mecanismo, SaW, crea una granja elástica de recursos de computación que ejecutan tareas de análisis multimedia. Los resultados confirman la competitividad de este enfoque respecto a granjas de servidores. El segundo mecanismo, LAMB-DASH, elige la calidad en el reproductor multimedia con un diseño que requiere una baja complejidad de procesamiento. Las pruebas concluyen su habilidad para mejorar la estabilidad, consistencia y uniformidad de la calidad de experiencia entre los clientes que comparten una celda de red. El tercer mecanismo, MEC4FAIR, explota las capacidades 5G de analizar métricas del envío de los diferentes flujos. Los resultados muestran cómo habilita al servicio a coordinar a los diferentes clientes en la celda para mejorar la calidad del servicio. El cuarto mecanismo, CogNet, sirve para provisionar recursos de red y configurar una topología capaz de conmutar una demanda estimada y garantizar unas cotas de calidad del servicio. En este caso, los resultados arrojan una mayor precisión cuando la demanda de un servicio es mayor

QoE on media deliveriy in 5G environments

231 p.5G expandirá las redes móviles con un mayor ancho de banda, menor latencia y la capacidad de proveer conectividad de forma masiva y sin fallos. Los usuarios de servicios multimedia esperan una experiencia de reproducción multimedia fluida que se adapte de forma dinámica a los intereses del usuario y a su contexto de movilidad. Sin embargo, la red, adoptando una posición neutral, no ayuda a fortalecer los parámetros que inciden en la calidad de experiencia. En consecuencia, las soluciones diseñadas para realizar un envío de tráfico multimedia de forma dinámica y eficiente cobran un especial interés. Para mejorar la calidad de la experiencia de servicios multimedia en entornos 5G la investigación llevada a cabo en esta tesis ha diseñado un sistema múltiple, basado en cuatro contribuciones.El primer mecanismo, SaW, crea una granja elástica de recursos de computación que ejecutan tareas de análisis multimedia. Los resultados confirman la competitividad de este enfoque respecto a granjas de servidores. El segundo mecanismo, LAMB-DASH, elige la calidad en el reproductor multimedia con un diseño que requiere una baja complejidad de procesamiento. Las pruebas concluyen su habilidad para mejorar la estabilidad, consistencia y uniformidad de la calidad de experiencia entre los clientes que comparten una celda de red. El tercer mecanismo, MEC4FAIR, explota las capacidades 5G de analizar métricas del envío de los diferentes flujos. Los resultados muestran cómo habilita al servicio a coordinar a los diferentes clientes en la celda para mejorar la calidad del servicio. El cuarto mecanismo, CogNet, sirve para provisionar recursos de red y configurar una topología capaz de conmutar una demanda estimada y garantizar unas cotas de calidad del servicio. En este caso, los resultados arrojan una mayor precisión cuando la demanda de un servicio es mayor

Real-time video streaming using peer-to-peer for video distribution

The growth of the Internet has led to research and development of several new and useful applications including video streaming. Commercial experiments are underway to determine the feasibility of multimedia broadcasting using packet based data networks alongside traditional over-the-air broadcasting. Broadcasting companies are offering low cost or free versions of video content online to both guage and at the same time generate popularity. In addition to television broadcasting, video streaming is used in a number of application areas including video conferencing, telecommuting and long distance education. Large scale video streaming has not become as widespread or widely deployed as could be expected. The reason for this is the high bandwidth requirement (and thus high cost) associated with video data. Provision of a constant stream of video data on a medium to large scale typically consumes a significant amount of bandwidth. An effect of this is that encoding bit rates are lowered and consequently video quality is degraded resulting in even slower uptake rates for video streaming services. The aim of this dissertation is to investigate peer-to-peer streaming as a potential solution to this bandwidth problem. The proposed peer-to-peer based solution relies on end user co-operation for video data distribution. This approach is highly effective in reducing the outgoing bandwidth requirement for the video streaming server. End users redistribute received video chunks amongst their respective peers and in so doing increase the potential capacity of the entire network for supporting more clients. A secondary effect of such a system is that encoding capabilities (including higher encoding bit rates or encoding of additional sub-channels) can be enhanced. Peer-to-peer distribution enables any regular user to stream video to large streaming networks with many viewers. This research includes a detailed overview of the fields of video streaming and peer-to-peer networking. Techniques for optimal video preparation and data distribution were investigated. A variety of academic and commercial peer-to-peer based multimedia broadcasting systems were analysed as a means to further define and place the proposed implementation in context with respect to other peercasting implementations. A proof-of-concept of the proposed implementation was developed, mathematically analyzed and simulated in a typical deployment scenario. Analysis was carried out to predict simulation performance and as a form of design evaluation and verification. The analysis highlighted some critical areas which resulted in adaptations to the initial design as well as conditions under which performance can be guaranteed. A simulation of the proof-of-concept system was used to determine the extent of bandwidth savings for the video server. The aim of the simulations was to show that it is possible to encode and deliver video data in real time over a peer-to-peer network. The proposed system achieved expectations and showed significant bandwidth savings for a sustantially large video streaming audience. The implementation was able to encode video in real time and continually stream video packets on time to connected peers while continually supporting network growth by connecting additional peers (or stream viewers). The system performed well and showed good performance under typical real world restrictions on available bandwith capacity.Dissertation (MEng)--University of Pretoria, 2009.Electrical, Electronic and Computer Engineeringunrestricte

Adaptive video delivery using semantics

The diffusion of network appliances such as cellular phones, personal digital assistants and hand-held computers has created the need to personalize the way media content is delivered to the end user. Moreover, recent devices, such as digital radio receivers with graphics displays, and new applications, such as intelligent visual surveillance, require novel forms of video analysis for content adaptation and summarization. To cope with these challenges, we propose an automatic method for the extraction of semantics from video, and we present a framework that exploits these semantics in order to provide adaptive video delivery. First, an algorithm that relies on motion information to extract multiple semantic video objects is proposed. The algorithm operates in two stages. In the first stage, a statistical change detector produces the segmentation of moving objects from the background. This process is robust with regard to camera noise and does not need manual tuning along a sequence or for different sequences. In the second stage, feedbacks between an object partition and a region partition are used to track individual objects along the frames. These interactions allow us to cope with multiple, deformable objects, occlusions, splitting, appearance and disappearance of objects, and complex motion. Subsequently, semantics are used to prioritize visual data in order to improve the performance of adaptive video delivery. The idea behind this approach is to organize the content so that a particular network or device does not inhibit the main content message. Specifically, we propose two new video adaptation strategies. The first strategy combines semantic analysis with a traditional frame-based video encoder. Background simplifications resulting from this approach do not penalize overall quality at low bitrates. The second strategy uses metadata to efficiently encode the main content message. The metadata-based representation of object's shape and motion suffices to convey the meaning and action of a scene when the objects are familiar. The impact of different video adaptation strategies is then quantified with subjective experiments. We ask a panel of human observers to rate the quality of adapted video sequences on a normalized scale. From these results, we further derive an objective quality metric, the semantic peak signal-to-noise ratio (SPSNR), that accounts for different image areas and for their relevance to the observer in order to reflect the focus of attention of the human visual system. At last, we determine the adaptation strategy that provides maximum value for the end user by maximizing the SPSNR for given client resources at the time of delivery. By combining semantic video analysis and adaptive delivery, the solution presented in this dissertation permits the distribution of video in complex media environments and supports a large variety of content-based applications

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance

Urban Informatics

This open access book is the first to systematically introduce the principles of urban informatics and its application to every aspect of the city that involves its functioning, control, management, and future planning. It introduces new models and tools being developed to understand and implement these technologies that enable cities to function more efficiently – to become ‘smart’ and ‘sustainable’. The smart city has quickly emerged as computers have become ever smaller to the point where they can be embedded into the very fabric of the city, as well as being central to new ways in which the population can communicate and act. When cities are wired in this way, they have the potential to become sentient and responsive, generating massive streams of ‘big’ data in real time as well as providing immense opportunities for extracting new forms of urban data through crowdsourcing. This book offers a comprehensive review of the methods that form the core of urban informatics from various kinds of urban remote sensing to new approaches to machine learning and statistical modelling. It provides a detailed technical introduction to the wide array of tools information scientists need to develop the key urban analytics that are fundamental to learning about the smart city, and it outlines ways in which these tools can be used to inform design and policy so that cities can become more efficient with a greater concern for environment and equity

Urban Informatics

This open access book is the first to systematically introduce the principles of urban informatics and its application to every aspect of the city that involves its functioning, control, management, and future planning. It introduces new models and tools being developed to understand and implement these technologies that enable cities to function more efficiently – to become ‘smart’ and ‘sustainable’. The smart city has quickly emerged as computers have become ever smaller to the point where they can be embedded into the very fabric of the city, as well as being central to new ways in which the population can communicate and act. When cities are wired in this way, they have the potential to become sentient and responsive, generating massive streams of ‘big’ data in real time as well as providing immense opportunities for extracting new forms of urban data through crowdsourcing. This book offers a comprehensive review of the methods that form the core of urban informatics from various kinds of urban remote sensing to new approaches to machine learning and statistical modelling. It provides a detailed technical introduction to the wide array of tools information scientists need to develop the key urban analytics that are fundamental to learning about the smart city, and it outlines ways in which these tools can be used to inform design and policy so that cities can become more efficient with a greater concern for environment and equity