Multimedia Standards

The aim of this paper is to review some of the standards, connected with multimedia and their metadata. We start with MPEG family. MPEG-21 provides an open framework for multimedia delivery and consumption. MPEG- 7 is a multimedia content description standard. With the Internet grow several format were proposed for media scenes description. Some of them are open standards such as: VRML1, X3D2, SMIL3, SVG4, MPEG-4 BIFS, MPEG-4, XMT, MPEG-4, LaSER, COLLADA5, published by ISO, W3C, etc. Television has become the most important mass medium. Standards such as MHEG, DAVIC, Java TV, MHP, GEM, OCAP and ACAP have been developed. Efficient video-streaming is presented. There exist a large number of standards for representing audiovisual metadata. We cover the Material Exchange Format (MXF), the Digital Picture Exchange (DPX), and the Digital Cinema Package (DCP)

Authoring objektbasierter AV-Anwendungen

This thesis is focused on the authoring process of object-based AV applications according to the object and scene concept of MPEG-4. The object-based approach embraces the extended interactive opportunities of multimedia applications as well as the distribution opportunities of audiovisual media. Producing of object-based AV applications has profound effects on the whole digital media chain. The existence of efficient authoring tools and systems is an important requirement for the success of such applications. The goal of this work was the development of concepts and components for an authoring system for object-based AV applications that supports collaborative work of several authors. The development was divided into three fields: authoring formats, authoring servers, and authoring tools. Authoring formats store all of the resulting information for the description of object-based AV applications during the authoring process. They describe a scene on a well adapted abstraction level and enable the exchange of scene data with other authors and systems. Some special authoring formats were developed which fulfill the requirements of concrete applications regarding abstraction level and function range. Authoring tools are the interfaces of the authoring system to the authors. In focus are interactive graphic tools for the support of an intuitive work during the authoring process. The authoring server is the technical basis for the collaborative work of several authors in the production of object-based AV applications. A flexible data management was developed on the basis of a native XML database. It controls the access at the level of nodes within the scene graph and enables the collaboration of many authors in a novel way. The authoring server is also the interface between the producers of media objects and the authors. Furthermore, it enables the reusability of scenes and elements in different projects. The developed concepts are oriented towards the possibilities of MPEG-4, but they are transferable to other multimedia applications which are based on a scene graph. With these concepts and components both can be realized, universally applicable as well as specialized authoring systems and tools. Several exemplary applications show the operability of the developed components.Die vorliegende Dissertation beschäftigt sich mit dem Authoring-Prozess objektbasierter AV-Anwendungen auf Basis des Objekt- und Szenenkonzeptes von MPEG-4. Diese moderne Beschreibungsform vereint die interaktiven Nutzungsmöglichkeiten digitaler Medien mit den Distributionsmöglichkeiten audiovisueller Medien. Die Umsetzung des Objekt- und Szenenkonzeptes hat tief greifende Auswirkungen auf die gesamte digitale Medienkette. Die Schaffung leistungsfähiger Autorensysteme ist eine wichtige Voraussetzung für die Verbreitung solcher Anwendungen. Das Ziel der Arbeit war die Entwicklung von Konzepten und Komponenten für ein Autorensystem mit Unterstützung eines auf mehrere Autoren verteilten Authoring-Prozesses. Authoring-Formate speichern alle anfallenden Informationen zur Beschreibung einer objektbasierten AV-Anwendung. Es wurden Authoring-Formate entwickelt, welche an die Anforderungen konkreter Anwendungen hinsichtlich Abstraktionsebene und Funktionsumfang angepasst sind. Autorenwerkzeuge sind die Schnittstellen des Autorensystems zu den Autoren. Im Fokus stehen grafisch-interaktive Werkzeuge zur Unterstützung eines intuitiven Arbeitens während des Authoring-Prozesses. Der Authoring-Server ist die technische Grundlage des Autorensystems für die verteilte Erstellung objektbasierter AV-Anwendungen. Er verwaltet alle anfallenden Daten und stellt diese den Autoren unter Berücksichtigung ihrer individuellen Berechtigungen zur Verfügung. Der Authoring-Server bildet die Schnittstelle zwischen den Produzenten der Medienobjekte und den Autoren. Er ermöglicht eine Wiederverwendung von Szenen und Szenenelementen über Produktionsgrenzen hinweg. Der Authoring-Server erlaubt es Autoren und auch Medienproduzenten, gemeinsam an der Erstellung einer AV-Anwendung zu arbeiten. Dafür wurde ein flexibles Datenmanagement auf Basis einer XML-Datenbank entworfen. Die entwickelten Konzepte orientieren sich an den Möglichkeiten von MPEG-4, sind aber auch auf andere multimediale Anwendungen übertragbar, die auf einem Szenengraphen beruhen. Auf dieser Basis können sowohl universell einsetzbare als auch spezialisierte Autorensysteme und Werkzeuge realisiert werden. Mehrere exemplarische Umsetzungen belegen die Funktionsfähigkeit der entwickelten Komponenten

Extensions to the SMIL multimedia language

The goal of this work has been to extend the Synchronized Multimedia Integration Language (SMIL) to study the capabilities and possibilities of declarative multimedia languages for the World Wide Web (Web). The work has involved design and implementation of several extensions to SMIL. A novel approach to include 3D audio in SMIL was designed and implemented. This involved extending the SMIL 2D spatial model with an extra dimension to support a 3D space. New audio elements and a listening point were positioned in the 3D space. The extension was designed to be modular so that it was possible to use it in conjunction with other XML languages, such as XHTML and Scalable Vector Graphics (SVG) language. Web forms are one of the key features in the Web, as they offer a way to send user data to a server. A similar feature is therefore desirable in SMIL, which currently lacks forms. The XForms language, due to its modular approach, was used to add this feature to SMIL. An evaluation of this integration was carried out as part of this work. Furthermore, the SMIL player was designed to play out dynamic SMIL documents, which can be modified at run-time and the result is immediately reflected in the presentation. Dynamic SMIL enables execution of scripts to modify the presentation. XML Events and ECMAScript were chosen to provide the scripting functionality. In addition, generic methods to extend SMIL were studied based on the previous extensions. These methods include ways to attach new input and output capabilities to SMIL. To experiment with the extensions, a Synchronized Multimedia Integration Language (SMIL) player was developed. The current final version can play out SMIL 2.0 Basic profile documents with a few additional SMIL modules, such as event timing, basic animations, and brush media modules. The player includes all above-mentioned extensions. The SMIL player has been designed to work within an XML browser called X-Smiles. X-Smiles is intended for various embedded devices, such as mobile phones, Personal Digital Assistants (PDA), and digital television set-top boxes. Currently, the browser supports XHTML, SMIL, and XForms, which are developed by the current research group. The browser also supports other XML languages developed by 3rd party open-source projects. The SMIL player can also be run as a standalone player without the browser. The standalone player is portable and has been run on a desktop PC, PDA, and digital television set-top box. The core of the SMIL player is platform-independent, only media renderers require platform-dependent implementation.reviewe

The specification and design of a prototype 2-D MPEG-4 authoring tool

The purpose of this project was the specification, design and implementation of a prototype 2-D MPEG-4 authoring tool. A literature study was conducted of the MPEG-4 standard and multimedia authoring tools to determine the specification and design of a prototype 2- D MPEG-4 authoring tool. The specification and design was used as a basis for the implementation of a prototype 2-D MPEG-4 authoring tool that complies with the Complete 2-D Scene Graph Profile. The need for research into MPEG-4 authoring tools arose from the reported lack of knowledge of the MPEG-4 standard and the limited implementations of MPEG-4 authoring tools available to content authors. In order for MPEG-4 to reach its full potential, it will require authoring tools and content players that satisfy the needs of its users. The theoretical component of this dissertation included a literature study of the MPEG-4 standard and an investigation of relevant multimedia authoring systems. MPEG-4 was introduced as a standard that allows for the creation and streaming of interactive multimedia content at variable bit rates over high and low bandwidth connections. The requirements for the prototype 2-D MPEG-4 authoring system were documented and a prototype system satisfying the requirements was designed, implemented and evaluated. The evaluation of the prototype system showed that the system successfully satisfied all its requirements and that it provides the user with an easy to use and intuitive authoring tool. MPEG-4 has the potential to satisfy the increasing demand for innovative multimedia content on low bandwidth networks, including the Internet and mobile networks, as well as the need expressed by users to interact with multimedia content. This dissertation makes an important contribution to the understanding of the MPEG-4 standard, its functionality and the design of a 2-D MPEG-4 Authoring tool. Keywords: MPEG-4; MPEG-4 authoring; Binary Format for Scenes

Enhancing Productivity and Performance Portability of General-Purpose Parallel Programming

This work focuses on compiler and run-time techniques for improving the productivity and the performance portability of general-purpose parallel programming. More specifically, we focus on shared-memory task-parallel languages, where the programmer explicitly exposes parallelism in the form of short tasks that may outnumber the cores by orders of magnitude. The compiler, the run-time, and the platform (henceforth the system) are responsible for harnessing this unpredictable amount of parallelism, which can vary from none to excessive, towards efficient execution. The challenge arises from the aspiration to support fine-grained irregular computations and nested parallelism. This work is even more ambitious by also aspiring to lay the foundations to efficiently support declarative code, where the programmer exposes all available parallelism, using high-level language constructs such as parallel loops, reducers or futures. The appeal of declarative code is twofold for general-purpose programming: it is often easier for the programmer who does not have to worry about the granularity of the exposed parallelism, and it achieves better performance portability by avoiding overfitting to a small range of platforms and inputs for which the programmer is coarsening. Furthermore, PRAM algorithms, an important class of parallel algorithms, naturally lend themselves to declarative programming, so supporting it is a necessary condition for capitalizing on the wealth of the PRAM theory. Unfortunately, declarative codes often expose such an overwhelming number of fine-grained tasks that existing systems fail to deliver performance. Our contributions can be partitioned into three components. First, we tackle the issue of coarsening, which declarative code leaves to the system. We identify two goals of coarsening and advocate tackling them separately, using static compiler transformations for one and dynamic run-time approaches for the other. Additionally, we present evidence that the current practice of burdening the programmer with coarsening either leads to codes with poor performance-portability, or to a significantly increased programming effort. This is a ``show-stopper'' for general-purpose programming. To compare the performance portability among approaches, we define an experimental framework and two metrics, and we demonstrate that our approaches are preferable. We close the chapter on coarsening by presenting compiler transformations that automatically coarsen some types of very fine-grained codes. Second, we propose Lazy Scheduling, an innovative run-time scheduling technique that infers the platform load at run-time, using information already maintained. Based on the inferred load, Lazy Scheduling adapts the amount of available parallelism it exposes for parallel execution and, thus, saves parallelism overheads that existing approaches pay. We implement Lazy Scheduling and present experimental results on four different platforms. The results show that Lazy Scheduling is vastly superior for declarative codes and competitive, if not better, for coarsened codes. Moreover, Lazy Scheduling is also superior in terms of performance-portability, supporting our thesis that it is possible to achieve reasonable efficiency and performance portability with declarative codes. Finally, we also implement Lazy Scheduling on XMT, an experimental manycore platform developed at the University of Maryland, which was designed to support codes derived from PRAM algorithms. On XMT, we manage to harness the existing hardware support for scheduling flat parallelism to compose it with Lazy Scheduling, which supports nested parallelism. In the resulting hybrid scheduler, the hardware and software work in synergy to overcome each other's weaknesses. We show the performance composability of the hardware and software schedulers, both in an abstract cost model and experimentally, as the hybrid always performs better than the software scheduler alone. Furthermore, the cost model is validated by using it to predict if it is preferable to execute a code sequentially, with outer parallelism, or with nested parallelism, depending on the input, the available hardware parallelism and the calling context of the parallel code

A graphics software architecture for high-end interactive TV terminals

This thesis proposes a graphics architecture for next-generation digital television receivers. The starting assumption is that in the future, a number of multimedia terminals will have access through a number of networks to a variety of content and services. One example of such a device is a media station capable of integrating different kinds of multimedia objects such as 2D/3D graphics and video, reacting to user interaction, and supporting the temporal dimension of applications. Some of the services intended for these devices include, for example, games and enhanced information over broadcasted video. First, this thesis provides an overview of the digital television environment, focusing on the limitations of current receivers and hints at future directions. In addition, this thesis compares different solutions from regional standardisation bodies such as DVB, CableLabs, and ARIB. It proposes the adoption of the most relevant initiative, GEM by DVB. Unfortunately, GEM software middleware only considers Java language as an authoring format, meaning that the declarative environment and advanced functionalities (e.g., 3D graphics support) remain to be standardised. Because in the future different user groups will have different demands with regard to television, this thesis identifies two major extensions to the GEM standard. First, it proposes a declarative environment for GEM that takes into account W3C standardisation efforts. This environment is divided into two configurations: one capable of rendering limited interactive applications such as information services, and another intended for more demanding applications, for example a distance learning portal that synchronises videos of lecturers and slides. Second, this thesis proposes to extend the procedural environment of GEM with 3D graphics support. The potential services of this new profile, High-End Interactive, include games and commercials. Then, based on the requirements the proposed profiles should meet, this thesis defines a graphics architecture model composed of five layers. The hardware abstraction layer is in charge of rendering the final graphics output. The graphical context is a cross-platform abstraction of the rendering region and provides graphics primitives (e.g., rectangles and images). The graphical environment provides the means to control different graphical contexts. The GUI toolkit is a set of ready-made user interface widgets and layout schemes. Finally, high-level languages are easy-to-use tools for developing simple services. The thesis concludes with a report of my experience implementing a digital television receiver based on the proposals described. In addition to testing the application of the proposed graphics architecture to the design and implementation of a next-generation digital television receiver, the implementation permits the analysis of the requirements of such receivers and of the services they can provide.reviewe

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

DESIGN OF MOBILE DATA COLLECTOR BASED CLUSTERING ROUTING PROTOCOL FOR WIRELESS SENSOR NETWORKS

Wireless Sensor Networks (WSNs) consisting of hundreds or even thousands of nodes, canbe used for a multitude of applications such as warfare intelligence or to monitor the environment. A typical WSN node has a limited and usually an irreplaceable power source and the efficient use of the available power is of utmost importance to ensure maximum lifetime of eachWSNapplication. Each of the nodes needs to transmit and communicate sensed data to an aggregation point for use by higher layer systems. Data and message transmission among nodes collectively consume the largest amount of energy available in WSNs. The network routing protocols ensure that every message reaches thedestination and has a direct impact on the amount of transmissions to deliver messages successfully. To this end, the transmission protocol within the WSNs should be scalable, adaptable and optimized to consume the least possible amount of energy to suite different network architectures and application domains. The inclusion of mobile nodes in the WSNs deployment proves to be detrimental to protocol performance in terms of nodes energy efficiency and reliable message delivery. This thesis which proposes a novel Mobile Data Collector based clustering routing protocol for WSNs is designed that combines cluster based hierarchical architecture and utilizes three-tier multi-hop routing strategy between cluster heads to base station by the help of Mobile Data Collector (MDC) for inter-cluster communication. In addition, a Mobile Data Collector based routing protocol is compared with Low Energy Adaptive Clustering Hierarchy and A Novel Application Specific Network Protocol for Wireless Sensor Networks routing protocol. The protocol is designed with the following in mind: minimize the energy consumption of sensor nodes, resolve communication holes issues, maintain data reliability, finally reach tradeoff between energy efficiency and latency in terms of End-to-End, and channel access delays. Simulation results have shown that the Mobile Data Collector based clustering routing protocol for WSNs could be easily implemented in environmental applications where energy efficiency of sensor nodes, network lifetime and data reliability are major concerns