Language-based multimedia information retrieval

This paper describes various methods and approaches for language-based multimedia information retrieval, which have been developed in the projects POP-EYE and OLIVE and which will be developed further in the MUMIS project. All of these project aim at supporting automated indexing of video material by use of human language technologies. Thus, in contrast to image or sound-based retrieval methods, where both the query language and the indexing methods build on non-linguistic data, these methods attempt to exploit advanced text retrieval technologies for the retrieval of non-textual material. While POP-EYE was building on subtitles or captions as the prime language key for disclosing video fragments, OLIVE is making use of speech recognition to automatically derive transcriptions of the sound tracks, generating time-coded linguistic elements which then serve as the basis for text-based retrieval functionality

LAYLAB : a constraint-based layout manager for multimedia presentations

When developing advanced intelligent user interfaces composing text, graphics, animation, hypermedia etc., the question of automatically designing the graphical layout of such multimedia presentations in an appropriate format plays a crucial role. This paper introduces the task, the functionality and the architecture of the constraint-based multimedia layout manager LayLab

CHORUS Deliverable 4.5: Report of the 3rd CHORUS Conference

The third and last CHORUS conference on Multimedia Search Engines took place from the 26th to the 27th of May 2009 in Brussels, Belgium. About 100 participants from 15 European countries, the US, Japan and Australia learned about the latest developments in the domain. An exhibition of 13 stands presented 16 research projects currently ongoing around the world

Automatic design of multimodal presentations

We describe our attempt to integrate multiple AI components such as planning, knowledge representation, natural language generation, and graphics generation into a functioning prototype called WIP that plans and coordinates multimodal presentations in which all material is generated by the system. WIP allows the generation of alternate presentations of the same content taking into account various contextual factors such as the user\u27s degree of expertise and preferences for a particular output medium or mode. The current prototype of WIP generates multimodal explanations and instructions for assembling, using, maintaining or repairing physical devices. This paper introduces the task, the functionality and the architecture of the WIP system. We show that in WIP the design of a multimodal document is viewed as a non-monotonic process that includes various revisions of preliminary results, massive replanning and plan repairs, and many negotiations between design and realization components in order to achieve an optimal division of work between text and graphics. We describe how the plan-based approach to presentation design can be exploited so that graphics generation influences the production of text and vice versa. Finally, we discuss the generation of cross-modal expressions that establish referential relationships between text and graphics elements

Using graphical style and visibility constraints for a meaningful layout in visual programming interfaces

In the expanding field of visual applications, layout design and graphical editing tasks are crucial points. In this paper, we address the incorporation of AI aspects in the visual software design process and the automatic layout and beautification of informational graphics such as visual programs and chart diagrams. Since layout in dynamic settings frequently requires a direct manipulation responsiveness, an incremental redesign of the generated visual material is necessary. Following our previous work on constraint-based multimodal design, we show how powerful constraint processing techniques, such as constraint hierarchies and dynamic constraint satisfaction, can be applied to visual programming environments in order to maintain graphical style and consistency for a meaningful layout. We describe InLay, a system for constraint-based presenting and editing visual programs. Finally, we will have a short look at some extensions with regard to advanced interaction and visualization techniques

PPP-Persona : ein objektorientierter Multimedia-Präsentationsagent

In zukünftigen Benutzerschnittstellen werden zunehmend animierte Präsentationsagenten eingesetzt. Der am DFKI entwickelte Präsentationsagent PPP-Persona dient dazu, den Benutzer durch eine Hypermedia-Präsentation zu führen. Dank seiner objektorientierten Konzeption ist er jedoch auch für eine Vielzahl weiterer Anwendungen einsetzbar, wie beispielsweise Online-Hilfesysteme, Home-Shopping und Tele-Banking. Dem Paradigma der Client/Server-Architektur folgend, empfängt der PPP-Persona-Server Präsentationsaufgaben von einem Anwendungsprogramm, die dieser dann selbständig ausführt. Zur visuellen Realisierung des Präsentationsagenten kann sowohl auf Cartoon-Zeichungen als auch auf Videobildmaterial zurückgegriffen werden. Um die Präsentation lebendiger zu gestalten, kann PPP-Persona selbständig Animationssequenzen initiieren

Perspektiven zur Kombination von automatischem Animationsdesign und planbasierter Hilfe

Aktuelle Themen auf dem Gebiet der intelligenten Benutzerschnittstellen behandeln derzeit die automatische Planung multimodaler Präsentationen. Hierbei stand bisher im wesentlichen die koordinierte Generierung von Text und Graphik im Vordergrund. In Zukunft wird hier aufgrund der Komplexität der zu präsentierenden Information zunehmend auch die Einbeziehung realistischer animierter 3D-Graphiken gefordert sein. Einen anderen wichtigen Forschungsschwerpunkt bildet der Einsatz graphischer Ausgabekomponenten für planbasierte Hilfesysteme. Die vorliegende Arbeit hat zum Ziel zunächst einen Überblick über den derzeitigen Stand der Forschung in diesen beiden Bereichen zu geben, als auch neue Anforderungen an die automatische Animationsgenerierung und an Systeme zur planbasierten graphischen Hilfe zu formulieren. Anschließend wollen wir, basierend auf Ergebnissen und Erfahrungen aus WIP und PLUS, Perspektiven für eine mögliche Weiterentwicklung und Integration von Techniken der Animationsplanung und graphischen Hilfe präsentieren

Verbmobil : translation of face-to-face dialogs

Verbmobil is a long-term project on the translation of spontaneous language in negotiation dialogs. We describe the goals of the project, the chosen discourse domains and the initial project schedule. We discuss some of the distinguishing features of Verbmobil and introduce the notion of translation on demand and variable depth of processing in speech translation. Finally, the role of anytime modules for efficient dialog translation in close to real time is described