Face modeling and animation language for MPEG-4 XMT framework

This paper proposes FML, an XML-based face modeling and animation language. FML provides a structured content description method for multimedia presentations based on face animation. The language can be used as direct input to compatible players, or be compiled within MPEG-4 XMT framework to create MPEG-4 presentations. The language allows parallel and sequential action description, decision-making and dynamic event-based scenarios, model configuration, and behavioral template definition. Facial actions include talking, expressions, head movements, and low-level MPEG-4 FAPs. The ShowFace and iFACE animation frameworks are also reviewed as example FML-based animation systems

HANDLING MULTILINGUAL CONTENT IN DIGITAL MEDIA: A CRITICAL ANALYSIS

This document expresses and analyzes the need to define a generic method for representing multilingual information in multimedia data. It describes the basic requirements that would bear upon such representations and establishes the potential link with ISO committee TC 37/SC 4 (Language Resource Management) and with XMT (eXtended MPEG-4 Textual format)

Authoring objektbasierter AV-Anwendungen

This thesis is focused on the authoring process of object-based AV applications according to the object and scene concept of MPEG-4. The object-based approach embraces the extended interactive opportunities of multimedia applications as well as the distribution opportunities of audiovisual media. Producing of object-based AV applications has profound effects on the whole digital media chain. The existence of efficient authoring tools and systems is an important requirement for the success of such applications. The goal of this work was the development of concepts and components for an authoring system for object-based AV applications that supports collaborative work of several authors. The development was divided into three fields: authoring formats, authoring servers, and authoring tools. Authoring formats store all of the resulting information for the description of object-based AV applications during the authoring process. They describe a scene on a well adapted abstraction level and enable the exchange of scene data with other authors and systems. Some special authoring formats were developed which fulfill the requirements of concrete applications regarding abstraction level and function range. Authoring tools are the interfaces of the authoring system to the authors. In focus are interactive graphic tools for the support of an intuitive work during the authoring process. The authoring server is the technical basis for the collaborative work of several authors in the production of object-based AV applications. A flexible data management was developed on the basis of a native XML database. It controls the access at the level of nodes within the scene graph and enables the collaboration of many authors in a novel way. The authoring server is also the interface between the producers of media objects and the authors. Furthermore, it enables the reusability of scenes and elements in different projects. The developed concepts are oriented towards the possibilities of MPEG-4, but they are transferable to other multimedia applications which are based on a scene graph. With these concepts and components both can be realized, universally applicable as well as specialized authoring systems and tools. Several exemplary applications show the operability of the developed components.Die vorliegende Dissertation beschäftigt sich mit dem Authoring-Prozess objektbasierter AV-Anwendungen auf Basis des Objekt- und Szenenkonzeptes von MPEG-4. Diese moderne Beschreibungsform vereint die interaktiven Nutzungsmöglichkeiten digitaler Medien mit den Distributionsmöglichkeiten audiovisueller Medien. Die Umsetzung des Objekt- und Szenenkonzeptes hat tief greifende Auswirkungen auf die gesamte digitale Medienkette. Die Schaffung leistungsfähiger Autorensysteme ist eine wichtige Voraussetzung für die Verbreitung solcher Anwendungen. Das Ziel der Arbeit war die Entwicklung von Konzepten und Komponenten für ein Autorensystem mit Unterstützung eines auf mehrere Autoren verteilten Authoring-Prozesses. Authoring-Formate speichern alle anfallenden Informationen zur Beschreibung einer objektbasierten AV-Anwendung. Es wurden Authoring-Formate entwickelt, welche an die Anforderungen konkreter Anwendungen hinsichtlich Abstraktionsebene und Funktionsumfang angepasst sind. Autorenwerkzeuge sind die Schnittstellen des Autorensystems zu den Autoren. Im Fokus stehen grafisch-interaktive Werkzeuge zur Unterstützung eines intuitiven Arbeitens während des Authoring-Prozesses. Der Authoring-Server ist die technische Grundlage des Autorensystems für die verteilte Erstellung objektbasierter AV-Anwendungen. Er verwaltet alle anfallenden Daten und stellt diese den Autoren unter Berücksichtigung ihrer individuellen Berechtigungen zur Verfügung. Der Authoring-Server bildet die Schnittstelle zwischen den Produzenten der Medienobjekte und den Autoren. Er ermöglicht eine Wiederverwendung von Szenen und Szenenelementen über Produktionsgrenzen hinweg. Der Authoring-Server erlaubt es Autoren und auch Medienproduzenten, gemeinsam an der Erstellung einer AV-Anwendung zu arbeiten. Dafür wurde ein flexibles Datenmanagement auf Basis einer XML-Datenbank entworfen. Die entwickelten Konzepte orientieren sich an den Möglichkeiten von MPEG-4, sind aber auch auf andere multimediale Anwendungen übertragbar, die auf einem Szenengraphen beruhen. Auf dieser Basis können sowohl universell einsetzbare als auch spezialisierte Autorensysteme und Werkzeuge realisiert werden. Mehrere exemplarische Umsetzungen belegen die Funktionsfähigkeit der entwickelten Komponenten

The specification and design of a prototype 2-D MPEG-4 authoring tool

The purpose of this project was the specification, design and implementation of a prototype 2-D MPEG-4 authoring tool. A literature study was conducted of the MPEG-4 standard and multimedia authoring tools to determine the specification and design of a prototype 2- D MPEG-4 authoring tool. The specification and design was used as a basis for the implementation of a prototype 2-D MPEG-4 authoring tool that complies with the Complete 2-D Scene Graph Profile. The need for research into MPEG-4 authoring tools arose from the reported lack of knowledge of the MPEG-4 standard and the limited implementations of MPEG-4 authoring tools available to content authors. In order for MPEG-4 to reach its full potential, it will require authoring tools and content players that satisfy the needs of its users. The theoretical component of this dissertation included a literature study of the MPEG-4 standard and an investigation of relevant multimedia authoring systems. MPEG-4 was introduced as a standard that allows for the creation and streaming of interactive multimedia content at variable bit rates over high and low bandwidth connections. The requirements for the prototype 2-D MPEG-4 authoring system were documented and a prototype system satisfying the requirements was designed, implemented and evaluated. The evaluation of the prototype system showed that the system successfully satisfied all its requirements and that it provides the user with an easy to use and intuitive authoring tool. MPEG-4 has the potential to satisfy the increasing demand for innovative multimedia content on low bandwidth networks, including the Internet and mobile networks, as well as the need expressed by users to interact with multimedia content. This dissertation makes an important contribution to the understanding of the MPEG-4 standard, its functionality and the design of a 2-D MPEG-4 Authoring tool. Keywords: MPEG-4; MPEG-4 authoring; Binary Format for Scenes

Einsatz von XMT und MPEG-4 zur Erstellung von RichMedia

Diese Diplomarbeit beschäftigt sich mit dem Einsatz von XMT und MPEG-4 im Allgemeinen und unter spezieller Betrachtung im RichMedia-Umfeld. Es wird versucht aufzuzeigen, welche Vorteile und Nachteile die Realisierung solcher Anwendungen mittels MPEG-4 in Verbindung mit XMT bringen können. Die Arbeit beginnt in den ersten zwei Kapiteln mit einem generellen Überblick über MPEG-4 und XMT. Dieser Teil vermittelt dem Leser - unabhängig von der später folgenden Betrachtung bezogen auf RichMedia - allgemein die Materie und Technologie, welche sich hinter diesen beiden Schlagwörtern versteckt. Auf einzelne technische Details wird, sofern sie keine besondere Bedeutung im Rahmen dieser Diplomarbeit darstellen, nicht näher eingegangen und bleiben den jeweiligen Spezifikationen vorbehalten. Im dritten Teil wird das Thema RichMedia zunächst unter technischem Aspekt aufgegriffen und versucht, ein Prototyp einer RichMedia-Anwendung praktisch umzusetzen. Schwerpunkte sind dabei unter anderem die Erstellung und Konvertierung von Inhalten sowie deren Distribution und Konsum. Der vierte Abschnitt versucht die beiden Technologien mit anderen, bereits bestehenden Standards und Lösungsansätzen zu vergleichen und gegenüber zustellen. Das letzte Kapitel bietet schließlich eine gesamtheitliche Zusammenfassung der Technologien XMT und MPEG-4 und ein Fazit hinsichtlich der Verwendung dieser beiden Technologien für RichMedia sowie einen Ausblick

HANDLING MULTILINGUAL CONTENT IN DIGITAL MEDIA: A CRITICAL ANALYSIS

This document expresses and analyzes the need to define a generic method for representing multilingual information in multimedia data. It describes the basic requirements that would bear upon such representations and establishes the potential link with ISO committee TC 37/SC 4 (Language Resource Management) and with XMT (eXtended MPEG-4 Textual format)

XATA 2006: XML: aplicações e tecnologias associadas

Esta é a quarta conferência sobre XML e Tecnologias Associadas. Este evento tem-se tornado um ponto de encontro para quem se interessa pela temática e tem sido engraçado observar que os participantes gostam e tentam voltar nos anos posteriores. O grupo base de trabalho, a comissão científica, também tem vindo a ser alargada e todos os que têm colaborado com vontade e com uma qualidade crescente ano após ano. Pela quarta vez estou a redigir este prefácio e não consigo evitar a redacção de uma descrição da evolução da XATA ao longo destes quatro anos: 2003 Nesta "reunião", houve uma vintena de trabalhos submetidos, maioritariamente da autoria ou da supervisão dos membros que integravam a comissão organizadora o que não envalidou uma grande participação e acesas discussões. 2004 Houve uma participação mais forte da comunidade portuguesa mas ainda com números pouco expressivos. Nesta altura, apostou-se também numa forte participação da indústria, o que se traduziu num conjunto apreciável de apresentações de casos reais. Foi introduzido o processo de revisão formal dos trabalhos submetidos. 2005 Houve uma forte adesão nacional e internacional (Espanha e Brasil, o que para um evento onde se pretende privilegiar a língua portuguesa é ainda mais significativo). A distribuição geográfica em Portugal também aumentou, havendo mais instituições participantes. Automatizaram-se várias tarefas como o processo de submissão e de revisão de artigos. 2006 Nesta edição actual, e contrariamente ao que acontece no plano nacional, houve um crescimento significativo. Em todas as edições, tem sido objectivo da comissão organizadora, previlegiar a produção científica e dar voz ao máximo número de participantes. Nesse sentido, este ano, não haverá oradores convidados, sendo o programa integralmente preenchido com as apresentações dos trabalhos seleccionados. Apesar disso ainda houve uma taxa significativa de rejeições, principalmente devido ao elevado número de submissões. Foi introduzido também, nesta edição, um dia de tutoriais com o objectivo de fornecer competências mínimas a quem quer começar a trabalhar na área e também poder assistir de uma forma mais informada à conferência. Se analisarmos as temáticas, abordadas nas quatro conferências, percebemos que também aqui há uma evolução no sentido de uma maior maturidade. Enquanto que no primeiro encontro, os trabalhos abordavam problemas emergentes na utilização da tecnologia, no segundo encontro a grande incidência foi nos Web Services, uma nova tecnologia baseada em XML, no terceiro, a maior incidência foi na construção de repositórios, motores de pesquisa e linguagens de interrogação, nesta quarta edição há uma distribuição quase homogénea por todas as áreas temáticas tendo mesmo aparecido trabalhos que abordam aspectos científicos e tecnológicos da base da tecnologia XML. Desta forma, podemos concluir que a tecnologia sob o ponto de vista de utilização e aplicação está dominada e que a comunidade portuguesa começa a fazer contributos para a ciência de base.Microsoft

TREE-D-SEEK: A Framework for Retrieving Three-Dimensional Scenes

In this dissertation, a strategy and framework for retrieving 3D scenes is proposed. The strategy is to retrieve 3D scenes based on a unified approach for indexing content from disparate information sources and information levels. The TREE-D-SEEK framework implements the proposed strategy for retrieving 3D scenes and is capable of indexing content from a variety of corpora at distinct information levels. A semantic annotation model for indexing 3D scenes in the TREE-D-SEEK framework is also proposed. The semantic annotation model is based on an ontology for rapid prototyping of 3D virtual worlds. With ongoing improvements in computer hardware and 3D technology, the cost associated with the acquisition, production and deployment of 3D scenes is decreasing. As a consequence, there is a need for efficient 3D retrieval systems for the increasing number of 3D scenes in corpora. An efficient 3D retrieval system provides several benefits such as enhanced sharing and reuse of 3D scenes and 3D content. Existing 3D retrieval systems are closed systems and provide search solutions based on a predefined set of indexing and matching algorithms Existing 3D search systems and search solutions cannot be customized for specific requirements, type of information source and information level. In this research, TREE-D-SEEK—an open, extensible framework for retrieving 3D scenes—is proposed. The TREE-D-SEEK framework is capable of retrieving 3D scenes based on indexing low level content to high-level semantic metadata. The TREE-D-SEEK framework is discussed from a software architecture perspective. The architecture is based on a common process flow derived from indexing disparate information sources. Several indexing and matching algorithms are implemented. Experiments are conducted to evaluate the usability and performance of the framework. Retrieval performance of the framework is evaluated using benchmarks and manually collected corpora. A generic, semantic annotation model is proposed for indexing a 3D scene. The primary objective of using the semantic annotation model in the TREE-D-SEEK framework is to improve retrieval relevance and to support richer queries within a 3D scene. The semantic annotation model is driven by an ontology. The ontology is derived from a 3D rapid prototyping framework. The TREE-D-SEEK framework supports querying by example, keyword based and semantic annotation based query types for retrieving 3D scenes

Extensions to the SMIL multimedia language

The goal of this work has been to extend the Synchronized Multimedia Integration Language (SMIL) to study the capabilities and possibilities of declarative multimedia languages for the World Wide Web (Web). The work has involved design and implementation of several extensions to SMIL. A novel approach to include 3D audio in SMIL was designed and implemented. This involved extending the SMIL 2D spatial model with an extra dimension to support a 3D space. New audio elements and a listening point were positioned in the 3D space. The extension was designed to be modular so that it was possible to use it in conjunction with other XML languages, such as XHTML and Scalable Vector Graphics (SVG) language. Web forms are one of the key features in the Web, as they offer a way to send user data to a server. A similar feature is therefore desirable in SMIL, which currently lacks forms. The XForms language, due to its modular approach, was used to add this feature to SMIL. An evaluation of this integration was carried out as part of this work. Furthermore, the SMIL player was designed to play out dynamic SMIL documents, which can be modified at run-time and the result is immediately reflected in the presentation. Dynamic SMIL enables execution of scripts to modify the presentation. XML Events and ECMAScript were chosen to provide the scripting functionality. In addition, generic methods to extend SMIL were studied based on the previous extensions. These methods include ways to attach new input and output capabilities to SMIL. To experiment with the extensions, a Synchronized Multimedia Integration Language (SMIL) player was developed. The current final version can play out SMIL 2.0 Basic profile documents with a few additional SMIL modules, such as event timing, basic animations, and brush media modules. The player includes all above-mentioned extensions. The SMIL player has been designed to work within an XML browser called X-Smiles. X-Smiles is intended for various embedded devices, such as mobile phones, Personal Digital Assistants (PDA), and digital television set-top boxes. Currently, the browser supports XHTML, SMIL, and XForms, which are developed by the current research group. The browser also supports other XML languages developed by 3rd party open-source projects. The SMIL player can also be run as a standalone player without the browser. The standalone player is portable and has been run on a desktop PC, PDA, and digital television set-top box. The core of the SMIL player is platform-independent, only media renderers require platform-dependent implementation.reviewe