Ubiquitous Scalable Graphics: An End-to-End Framework using Wavelets

Advances in ubiquitous displays and wireless communications have fueled the emergence of exciting mobile graphics applications including 3D virtual product catalogs, 3D maps, security monitoring systems and mobile games. Current trends that use cameras to capture geometry, material reflectance and other graphics elements means that very high resolution inputs is accessible to render extremely photorealistic scenes. However, captured graphics content can be many gigabytes in size, and must be simplified before they can be used on small mobile devices, which have limited resources, such as memory, screen size and battery energy. Scaling and converting graphics content to a suitable rendering format involves running several software tools, and selecting the best resolution for target mobile device is often done by trial and error, which all takes time. Wireless errors can also affect transmitted content and aggressive compression is needed for low-bandwidth wireless networks. Most rendering algorithms are currently optimized for visual realism and speed, but are not resource or energy efficient on mobile device. This dissertation focuses on the improvement of rendering performance by reducing the impacts of these problems with UbiWave, an end-to-end Framework to enable real time mobile access to high resolution graphics using wavelets. The framework tackles the issues including simplification, transmission, and resource efficient rendering of graphics content on mobile device based on wavelets by utilizing 1) a Perceptual Error Metric (PoI) for automatically computing the best resolution of graphics content for a given mobile display to eliminate guesswork and save resources, 2) Unequal Error Protection (UEP) to improve the resilience to wireless errors, 3) an Energy-efficient Adaptive Real-time Rendering (EARR) heuristic to balance energy consumption, rendering speed and image quality and 4) an Energy-efficient Streaming Technique. The results facilitate a new class of mobile graphics application which can gracefully adapt the lowest acceptable rendering resolution to the wireless network conditions and the availability of resources and battery energy on mobile device adaptively

A Survey on Multimedia-Based Cross-Layer Optimization in Visual Sensor Networks

Visual sensor networks (VSNs) comprised of battery-operated electronic devices endowed with low-resolution cameras have expanded the applicability of a series of monitoring applications. Those types of sensors are interconnected by ad hoc error-prone wireless links, imposing stringent restrictions on available bandwidth, end-to-end delay and packet error rates. In such context, multimedia coding is required for data compression and error-resilience, also ensuring energy preservation over the path(s) toward the sink and improving the end-to-end perceptual quality of the received media. Cross-layer optimization may enhance the expected efficiency of VSNs applications, disrupting the conventional information flow of the protocol layers. When the inner characteristics of the multimedia coding techniques are exploited by cross-layer protocols and architectures, higher efficiency may be obtained in visual sensor networks. This paper surveys recent research on multimedia-based cross-layer optimization, presenting the proposed strategies and mechanisms for transmission rate adjustment, congestion control, multipath selection, energy preservation and error recovery. We note that many multimedia-based cross-layer optimization solutions have been proposed in recent years, each one bringing a wealth of contributions to visual sensor networks

Quality of Experience and Adaptation Techniques for Multimedia Communications

The widespread use of multimedia services on the World Wide Web and the advances in end-user portable devices have recently increased the user demands for better quality. Moreover, providing these services seamlessly and ubiquitously on wireless networks and with user mobility poses hard challenges. To meet these challenges and fulfill the end-user requirements, suitable strategies need to be adopted at both application level and network level. At the application level rate and quality have to be adapted to time-varying bandwidth limitations, whereas on the network side a mechanism for efficient use of the network resources has to be implemented, to provide a better end-user Quality of Experience (QoE) through better Quality of Service (QoS). The work in this thesis addresses these issues by first investigating multi-stream rate adaptation techniques for Scalable Video Coding (SVC) applications aimed at a fair provision of QoE to end-users. Rate Distortion (R-D) models for real-time and non real-time video streaming have been proposed and a rate adaptation technique is also developed to minimize with fairness the distortion of multiple videos with difference complexities. To provide resiliency against errors, the effect of Unequal Error protection (UXP) based on Reed Solomon (RS) encoding with erasure correction has been also included in the proposed R-D modelling. Moreover, to improve the support of QoE at the network level for multimedia applications sensitive to delays, jitters and packet drops, a technique to prioritise different traffic flows using specific QoS classes within an intermediate DiffServ network integrated with a WiMAX access system is investigated. Simulations were performed to test the network under different congestion scenarios

QUALITY-DRIVEN CROSS LAYER DESIGN FOR MULTIMEDIA SECURITY OVER RESOURCE CONSTRAINED WIRELESS SENSOR NETWORKS

The strong need for security guarantee, e.g., integrity and authenticity, as well as privacy and confidentiality in wireless multimedia services has driven the development of an emerging research area in low cost Wireless Multimedia Sensor Networks (WMSNs). Unfortunately, those conventional encryption and authentication techniques cannot be applied directly to WMSNs due to inborn challenges such as extremely limited energy, computing and bandwidth resources. This dissertation provides a quality-driven security design and resource allocation framework for WMSNs. The contribution of this dissertation bridges the inter-disciplinary research gap between high layer multimedia signal processing and low layer computer networking. It formulates the generic problem of quality-driven multimedia resource allocation in WMSNs and proposes a cross layer solution. The fundamental methodologies of multimedia selective encryption and stream authentication, and their application to digital image or video compression standards are presented. New multimedia selective encryption and stream authentication schemes are proposed at application layer, which significantly reduces encryption/authentication complexity. In addition, network resource allocation methodologies at low layers are extensively studied. An unequal error protection-based network resource allocation scheme is proposed to achieve the best effort media quality with integrity and energy efficiency guarantee. Performance evaluation results show that this cross layer framework achieves considerable energy-quality-security gain by jointly designing multimedia selective encryption/multimedia stream authentication and communication resource allocation

Multimedia delivery in the future internet

The term “Networked Media” implies that all kinds of media including text, image, 3D graphics, audio and video are produced, distributed, shared, managed and consumed on-line through various networks, like the Internet, Fiber, WiFi, WiMAX, GPRS, 3G and so on, in a convergent manner [1]. This white paper is the contribution of the Media Delivery Platform (MDP) cluster and aims to cover the Networked challenges of the Networked Media in the transition to the Future of the Internet. Internet has evolved and changed the way we work and live. End users of the Internet have been confronted with a bewildering range of media, services and applications and of technological innovations concerning media formats, wireless networks, terminal types and capabilities. And there is little evidence that the pace of this innovation is slowing. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes1 of content. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens’ quality of life, working conditions, edutainment and safety. In this evolving environment, new transport protocols, new multimedia encoding schemes, cross-layer inthe network adaptation, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P) overlays are expected to generate new models of interaction and cooperation, and be able to support enhanced perceived quality-of-experience (PQoE) and innovative applications “on the move”, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. Based on work that has taken place in a number of EC co-funded projects, in Framework Program 6 (FP6) and Framework Program 7 (FP7), a group of experts and technology visionaries have voluntarily contributed in this white paper aiming to describe the status, the state-of-the art, the challenges and the way ahead in the area of Content Aware media delivery platforms

Energy Optimization for Hybrid ARQ

Hybrid automatic repeat request (HARQ) \cite{costello1983error} plays an important role in providing reliable and efficient data transmission. In wireless communications, the wireless channel may vary fast, due to the mobility of the transmitter/receiver and the channel. Forward error correction (FEC) and automatic repeat request (ARQ) are two basic techniques to control errors. FEC employs error correction coding, by adding parity bits to the information bits, to combat channel errors. ARQ allows the receiver to request a retransmission of the packet when an error is detected in the received packet. HARQ gives protection to the wireless transmission by combining FEC and ARQ. In typical HARQ systems, redundancy is added to the information bits, and a retransmission is performed until either the packet is successfully decoded, or a maximum number of transmissions is reached.The motivation to optimize the energy consumption of HARQ is the high energy consumption of wireless communications on mobile devices. Wireless devices usually have a limited battery life, and wireless communications consume the majority of the battery energy of mobile devices. One example is that 3G and Wifi units consume more than 50\% of the energy for some smart phones \cite{tawalbeh2016studying}. Another example is that battery depletion has been identified as one of the primary factors that limit the lifetime of wireless sensor networks \cite{verdone2010wireless}.Previous works on HARQ mainly use information-theoretic approach, which assumes that the number of bits in each transmission round is sufficiently large. This assumption does not necessarily hold for actual codes with finite length. Therefore, in this dissertation, we consider HARQ with actual codes. We use turbo-coded HARQ, since turbo codes are well-known capacity-approaching codes \cite{berrou1993near} and widely used in standards such as 3GPP Long-Term Evolution (LTE) \cite{3gpp2007mulltiplexing}. We study the energy optimization for HARQ in two scenarios: the energy optimization for incremental redundancy (IR) HARQ, and the energy optimization for HARQ in wireless video transmission. For IR HARQ, each retransmission contains additional parity bits beyond those of the previous transmissions. For the first scenario, we consider different cases of channel state information (CSI) at the transmitter: the transmitter has no knowledge of any CSI, or knows the CSI in previous transmission rounds through a perfect feedback channel, or knows both current and previous CSI. The transmitter decides the forward error correction code rate based on the CSI it has. We minimize the energy consumption of turbo-coded HARQ, subject to a packet loss rate constraint. Numerical results show that the energy consumption of HARQ decreases when more CSI information is available at the transmitter. We also compare IR combining with both Chase combining and the system without combining, and IR combining yields the least energy consumption.For the second scenario, we formulate the problem as maximizing the video quality, subject to a constraint on the wireless transmission energy consumption. We consider multiple parameters in multiple layers in a wireless video transmission system: transmit power, alphabet size, FEC code rate, maximum number of transmissions and unequal video data importance. An analytical framework is proposed to include these parameters, which allows us to divide this problem into two sub-problems: data transmission and unequal error protection (UEP) for video content. The problem is tackled by solving the two sub-problems, which are done by exhaustive search and convex optimization, respectively. Simulations of different videos show that the proposed scheme outperforms methods using conventional data transmission and/or unequal error protection. For example, in the low SNR region, there is a total gain of 4.8 to 5.6dB on the peak signal-to-noise ratio of the received video compared to video transmission using conventional HARQ without any video UEP