Extensions to the SMIL multimedia language

The goal of this work has been to extend the Synchronized Multimedia Integration Language (SMIL) to study the capabilities and possibilities of declarative multimedia languages for the World Wide Web (Web). The work has involved design and implementation of several extensions to SMIL. A novel approach to include 3D audio in SMIL was designed and implemented. This involved extending the SMIL 2D spatial model with an extra dimension to support a 3D space. New audio elements and a listening point were positioned in the 3D space. The extension was designed to be modular so that it was possible to use it in conjunction with other XML languages, such as XHTML and Scalable Vector Graphics (SVG) language. Web forms are one of the key features in the Web, as they offer a way to send user data to a server. A similar feature is therefore desirable in SMIL, which currently lacks forms. The XForms language, due to its modular approach, was used to add this feature to SMIL. An evaluation of this integration was carried out as part of this work. Furthermore, the SMIL player was designed to play out dynamic SMIL documents, which can be modified at run-time and the result is immediately reflected in the presentation. Dynamic SMIL enables execution of scripts to modify the presentation. XML Events and ECMAScript were chosen to provide the scripting functionality. In addition, generic methods to extend SMIL were studied based on the previous extensions. These methods include ways to attach new input and output capabilities to SMIL. To experiment with the extensions, a Synchronized Multimedia Integration Language (SMIL) player was developed. The current final version can play out SMIL 2.0 Basic profile documents with a few additional SMIL modules, such as event timing, basic animations, and brush media modules. The player includes all above-mentioned extensions. The SMIL player has been designed to work within an XML browser called X-Smiles. X-Smiles is intended for various embedded devices, such as mobile phones, Personal Digital Assistants (PDA), and digital television set-top boxes. Currently, the browser supports XHTML, SMIL, and XForms, which are developed by the current research group. The browser also supports other XML languages developed by 3rd party open-source projects. The SMIL player can also be run as a standalone player without the browser. The standalone player is portable and has been run on a desktop PC, PDA, and digital television set-top box. The core of the SMIL player is platform-independent, only media renderers require platform-dependent implementation.reviewe

A VISUAL DESIGN METHOD AND ITS APPLICATION TO HIGH RELIABILITY HYPERMEDIA SYSTEMS

This work addresses the problem of the production of hypermedia documentation for applications that require high reliability, particularly technical documentation in safety critical industries. One requirement of this application area is for the availability of a task-based organisation, which can guide and monitor such activities as maintenance and repair. In safety critical applications there must be some guarantee that such sequences are correctly presented. Conventional structuring and design methods for hypermedia systems do not allow such guarantees to be made. A formal design method that is based on a process algebra is proposed as a solution to this problem. Design methods of this kind need to be accessible to information designers. This is achieved by use of a technique already familiar to them: the storyboard. By development of a storyboard notation that is syntactically equivalent to a process algebra a bridge is made between information design and computer science, allowing formal analysis and refinement of the specification drafted by information designers. Process algebras produce imperative structures that do not map easily into the declarative formats used for some hypermedia systems, but can be translated into concurrent programs. This translation process, into a language developed by the author, called ClassiC, is illustrated and the properties that make ClassiC a suitable implementation target discussed. Other possible implementation targets are evaluated, and a comparative illustration given of translation into another likely target, Java

Propuesta de una guía para el modelado y desarrollo de aplicaciones web multimodales basada en web semántica

En este trabajo se propone una guía para el desarrollo de aplicaciones web que permitan interacción multimodal, y presenten un modelo de datos basado en web semántica. Para lograrlo se realizó un estudio del estado del arte de los lenguajes de marcado para el desarrollo de aplicaciones multimodales, y un estudio aplicado de las capas ontológica y lógica de la web semántica, además se probó la guía a través de la construcción de un prototipo de portal de información sobre el problema de desplazamiento forzado en la ciudad de Cartagena. Dicha guía no pretende convertirse en una metodología de desarrollo de software, sino más bien complementar las metodologías actuales, específicamente en los aspectos relacionados con la interacción multimodal y el modelado semántico. Para la construcción de la guía se tuvo en cuenta el patrón de desarrollo de software denominado: Modelo Vista Controlador, teniendo en cuenta que su principal finalidad consiste en separar las capas de presentación, lógica y datos. Descriptores del trabajo: Interacción Multimodal, Web Semántica, Aplicaciones Web, Lenguajes de Marcado, Ontología, Marco de Descripción de Recursos, Arquitectura de software.Incluye bibliografí

Proposal for a guide for the modeling and development of multimodal web applications based on semantic web

En este trabajo se propone una guía para el desarrollo de aplicaciones web que permitan interacción multimodal, y presenten un modelo de datos basado en web semántica. Para lograrlo se realizó un estudio del estado del arte de los lenguajes de marcado para el desarrollo de aplicaciones multimodales, y un estudio aplicado de las capas ontológica y lógica de la web semántica, además se probó la guía a través de la construcción de un prototipo de portal de información sobre el problema de desplazamiento forzado en la ciudad de Cartagena. Dicha guía no pretende convertirse en una metodología de desarrollo de software, sino más bien complementar las metodologías actuales, específicamente en los aspectos relacionados con la interacción multimodal y el modelado semántico. Para la construcción de la guía se tuvo en cuenta el patrón de desarrollo de software denominado: Modelo Vista Controlador, teniendo en cuenta que su principal finalidad consiste en separar las capas de presentación, lógica y datos.Instituto Tecnológico de Estudios Superiores de Monterrey ITESM1. INTRODUCCIÓN.................................................................................................7 1.1. ANTECEDENTES 7 1.2. OBJETIVOS 7 1.3 PLANTEAMIENTO DEL PROBLEMA Y JUSTIFICACIÓN 8 1.4 TRABAJOS RELACIONADOS 9 1.5 MÉTODO DE INVESTIGACIÓN Y DESCRIPCIÓN DE RESULTADOS 11 2. MARCO TEÓRICO (ESTADO DEL ARTE)........................................................13 2.1 INTERACCIÓN MULTIMODAL 13 2.1.1 Acceso Multimodal........................................................................................13 2.1.2 Requerimientos del acceso multimodal........................................................15 2.1.3 Interface de usuario multimodal....................................................................18 2.1.4 Plataforma de interacción multimodal..........................................................18 2.1.5 Arquitectura de interacción Multimodal.........................................................22 2.1.6 Herramientas................................................................................................23 2.2 WEB SEMÁNTICA 28 2.2.1 Capas de la Web Semántica.........................................................................29 2.2.2 Marco de Descripción de Recursos..............................................................30 2.2.3 Ontología......................................................................................................34 2.2.4 Lenguajes para la Construcción de Ontologías............................................39 2.2.5 Lenguaje de reglas para la Web Semántica.................................................41 2.2.6 Herramientas................................................................................................42 2.3 ESTADO DEL ARTE DE LOS LENGUAJES DE MARCADO PARA EL DESARROLLO DE APLICACIONES MULTIMODALES 45 2.3.1 InkML (Ink Markup Language)......................................................................46 2.3.2 VoiceXML......................................................................................................49 2.3.3 XHTML + Voice.............................................................................................54 2.3.4 SALT (Speech Application Language Tags).................................................56 2.3.5 TalkML..........................................................................................................61 2.3.6 VoxML...........................................................................................................63 2.3.7 SSML (Speech Synthesis Markup Language)..............................................68 2.3.8 XForms.........................................................................................................70 2.3.9 EMMA (Extensible MultiModal Annotation markup language)......................74 2.3.10 SMIL (Synchronized Multimedia Integration Language).............................76 2.4 ESTUDIO SOBRE LA CAPA LÓGICA Y ONTOLÓGICA DE LA WEB SEMÁNTICA 78 2.4.1 Capa RDF + RDFSchema............................................................................79 2.4.2 Capa Ontológica...........................................................................................81 2.4.3 Capa Lógica..................................................................................................90 4 3. GUÍA PARA EL MODELADO Y DESARROLLO DE APLICACIONES WEB MULTIMODALES QUE INCLUYAN MODELADO DE DATOS BASADO EN WEB SEMÁNTICA...........................................................................................94 3.1 MODELADO DE LA INTERACCIÓN MULTIMODAL. 95 3.1.1 Aspectos generales:.....................................................................................96 3.1.2 Funciones de la aplicación............................................................................97 3.1.3 Componente de sesión.................................................................................97 3.1.4 Sistema y Entorno.........................................................................................98 3.1.5 Administrador de interacción.........................................................................98 3.1.6 Componente de entrada:..............................................................................99 3.1.7 Componente de salida:...............................................................................102 3.2 MODELADO SEMÁNTICO 103 3.2.1 Premisas.....................................................................................................105 3.2.2 Pasos..........................................................................................................106 4. MODELADO DEL PROTOTIPO DE PORTAL DE INFORMACIÓN SOBRE EL PROBLEMA DE DESPLAZAMIENTO FORZADO EN LA CIUDAD DE CARTAGENA.................................................................................................112 4.1 MODELO DEL NEGOCIO 112 4.2 IDENTIFICACIÓN DE REQUERIMIENTOS 114 4.3 MODELO DE DISEÑO 114 4.4 MODELO DE INTERACCIÓN MULTIMODAL 120 4.4.1 Aspectos generales:...................................................................................120 4.4.2 Funciones de la aplicación..........................................................................121 4.4.3 Componente de sesión...............................................................................121 4.4.4 Administrador de interacción.......................................................................122 4.4.5 Componente de entrada:............................................................................126 4.4.6 Componente de salida:...............................................................................129 4.5 MODELO SEMÁNTICO 130 4.5.1 Diseño de la ontología................................................................................130 4.5.2 Diseño de la interacción lógica...................................................................137 4.6 MODELO DE IMPLEMENTACIÓN 139 4.7 EVALUACIÓN DEL PROTOTIPO..................................................................140 CONCLUSIONES................................................................................................141 TRABAJOS FUTUROS........................................................................................143 BIBLIOGRAFÍA....................................................................................................144 ANEXOS..............................................................................................................153 CRONOGRAMA DE ACTIVIDADES....................................................................154 PRESUPUESTO..................................................................................................155 RECURSOS NECESARIOS................................................................................156MaestríaThis paper proposes a guide for the development of web applications that allow multimodal interaction, and present a data model based on semantic web. To achieve this, a study of the state of the art of markup languages ​​for the development of multimodal applications was carried out, and an applied study of the ontological and logical layers of the semantic web, in addition, the guide was tested through the construction of a prototype information portal on the problem of forced displacement in the city of Cartagena. This guide is not intended to become a software development methodology, but rather to complement current methodologies, specifically in aspects related to multimodal interaction and semantic modeling. For the construction of the guide, the software development pattern called: Controller View Model was taken into account, taking into account that its main purpose is to separate the presentation, logic and data layers.Modalidad Presencia

A Simple, Intuitive Hypermedia Synchronization Model and its Realization in the Browser/Java Environment

This paper presents a simple and intuitive hypermedia synchronization model – the Media Relation Graph (MRG), and an alternative implementation of the Hypermedia Presentation and Authoring System (HPAS), which is the testbed for MRG. Our model combines the power of both interval-based and point-based synchronization mechanisms. The new implementation exploits many rich features of commercial web browsers and reuses existing browser components, such as plugins and Java applets. (An overview of HPAS and its original Unix/C implementation can be found elsewhere [18].)