Actas da 10ª Conferência sobre Redes de Computadores

Universidade do MinhoCCTCCentro AlgoritmiCisco SystemsIEEE Portugal Sectio

Semantics of video shots for content-based retrieval

Content-based video retrieval research combines expertise from many different areas, such as signal processing, machine learning, pattern recognition, and computer vision. As video extends into both the spatial and the temporal domain, we require techniques for the temporal decomposition of footage so that specific content can be accessed. This content may then be semantically classified - ideally in an automated process - to enable filtering, browsing, and searching. An important aspect that must be considered is that pictorial representation of information may be interpreted differently by individual users because it is less specific than its textual representation. In this thesis, we address several fundamental issues of content-based video retrieval for effective handling of digital footage. Temporal segmentation, the common first step in handling digital video, is the decomposition of video streams into smaller, semantically coherent entities. This is usually performed by detecting the transitions that separate single camera takes. While abrupt transitions - cuts - can be detected relatively well with existing techniques, effective detection of gradual transitions remains difficult. We present our approach to temporal video segmentation, proposing a novel algorithm that evaluates sets of frames using a relatively simple histogram feature. Our technique has been shown to range among the best existing shot segmentation algorithms in large-scale evaluations. The next step is semantic classification of each video segment to generate an index for content-based retrieval in video databases. Machine learning techniques can be applied effectively to classify video content. However, these techniques require manually classified examples for training before automatic classification of unseen content can be carried out. Manually classifying training examples is not trivial because of the implied ambiguity of visual content. We propose an unsupervised learning approach based on latent class modelling in which we obtain multiple judgements per video shot and model the users' response behaviour over a large collection of shots. This technique yields a more generic classification of the visual content. Moreover, it enables the quality assessment of the classification, and maximises the number of training examples by resolving disagreement. We apply this approach to data from a large-scale, collaborative annotation effort and present ways to improve the effectiveness for manual annotation of visual content by better design and specification of the process. Automatic speech recognition techniques along with semantic classification of video content can be used to implement video search using textual queries. This requires the application of text search techniques to video and the combination of different information sources. We explore several text-based query expansion techniques for speech-based video retrieval, and propose a fusion method to improve overall effectiveness. To combine both text and visual search approaches, we explore a fusion technique that combines spoken information and visual information using semantic keywords automatically assigned to the footage based on the visual content. The techniques that we propose help to facilitate effective content-based video retrieval and highlight the importance of considering different user interpretations of visual content. This allows better understanding of video content and a more holistic approach to multimedia retrieval in the future

SUMMARIZATION AND VISUALIZATION OF DIGITAL CONVERSATIONS

Digital conversations are all around us: recorded meetings, television debates, instant messaging, blogs, and discussion forums. With this work, we present some solutions for the condensation and distillation of content from digital conversation based on advanced language technology. At the core of this technology we have argumentative analysis, which allow us to produce high-quality text summaries and intuitive graphical visualizations of conversational content enabling easier and faster access to digital conversations

Output constraints in multimedia database systems

Zusammenfassung Semantische Fehler treten bei jeder Art von Datenverwaltung auf. Herkömmliche Datenbanksysteme verwenden eine Integritätskontrolle, um semantische Fehler zu vermeiden. Um die Integrität der Daten zu gewährleisten werden Integritätsregeln benutzt. Diese Regeln können allerdings nur die Konsistenz einfach strukturierter Daten überprüfen. Multimedia Datenbanksystem verwalten neben einfachen alphanumerischen Daten auch komplexe Mediendaten wie Videos. Um die Konsistenz dieser Daten zu sichern, bedarf es einer erheblichen Erweiterung des bestehenden Integritätskonzeptes. Dabei muss besonders auf die konsistente Datenausgabe geachtet werden. Im Gegensatz zu alphanumerischen Daten können Mediendaten während der Ausgabe verfälscht werden. Dieser Fall kann eintreten, wenn eine geforderte Datenqualität bei der Ausgabe nicht erreicht werden kann oder wenn Synchronisationsbedingungen zwischen Medienobjekten nicht eingehalten werden können. Es besteht daher die Notwendigkeit, Ouptut Constraints einzuführen. Mit ihrer Hilfe kann definiert werden, wann die Ausgabe von Mediendaten semantisch korrekt ist. Das Datenbanksystem kann diese Bedingungen überprüfen und so gewährleisten, dass der Nutzer semantisch einwandfreie Daten erhält. In dieser Arbeit werden alle Aspekte betrachtet, die notwendig sind, um Ausgabebedingungen in ein Multimedia Datenbanksystem zu integrieren. Im einzelnen werden die Modellierung der Bedingungen, deren datenbankinterne Repräsentation sowie die Bedingungsüberprüfung betrachtet. Für die Bedingungsmodellierung wird eine Constraint Language auf Basis der Prädikatenlogik eingeführt. Um die Definition von zeitlichen und räumlichen Synchronisationen zu ermöglichen, verwenden wir Allen-Relationen. Für die effiziente Überprüfung der Ausgabebedingungen müssen diese aus der Spezifikationssprache in eine datenbankinterne Darstellung überführt werden. Für die datenbankinterne Darstellung werden Difference Constraints verwendet. Diese erlauben eine sehr effiziente Bedingungsüberprüfung. Wir haben Algorithmen entwickelt, die eine effiziente Überprüfung von Ausgabebedingungen erlauben und dies anhand von Experimenten nachgewiesen. Neben der Überprüfung der Bedingungen müssen Mediendaten so synchronisiert werden, dass dies den Ausgabebedingungen entspricht. Wir haben dazu das Konzept des Output Schedules entwickelt. Dieser wird aufgrund der definierten Ausgabebedingungen generiert. Durch die Ausgabebedingungen, die in dieser Arbeit eingeführt werden, werden semantische Fehler bei der Verwaltung von Mediendaten erheblich reduziert. Die Arbeit stellt daher einen Beitrag zur qualitativen Verbesserung der Verwaltung von Mediendaten dar.Semantic errors exist as long as data are managed. Traditional database systems try to prevent this errors by proposing integrity concepts for stored data. Integrity constraints are used to implement these integrity concepts. However, integrity constraints can only detect semantic errors in elementary data. Multimedia database systems manage elementary data as well as complex media data, like videos. Considering these media data we need a much wider consistency concept as traditional database systems provide. Especially, data output of media data must be taken into account. In contrast to alphanumeric data the semantics of media data can be falsified during data output if data quality or synchronization of data are not suitable. Thus, we need a concept for output constraints that allow for preventing semantic errors in case of data output. For integrating output constraints into a multimedia database system we have to consider modelling, representation and checking of output constraints. For modelling output constraints we have introduced a constraint language which uses the same principles as traditional constraint languages. Our constraint specification language must support temporal and spatial synchronization constraints. However, it is desired to support both kinds of synchronization in almost the same manner. Therefore, we use Allen-Relations for defining temporal synchronization constraints as well as for defining spatial synchronization constraints. We need a database internal representation of output constraints that makes efficient constraint checking possible. The Allen-Relations used in the constraint language cannot be checked efficiently. However, difference constraints are a class of constraints that allows an very efficient checking. Therefore, we use difference constraints as database internal representation of output constraints. As methods for checking consistency of output constraints we use an approach based on graph theory as well as an analytical approach. Both approaches require a constraint graph as data structure. For data output we need an output order that is adequate to the defined output constraints. This output schedule can be produced based on the output constraints. With output constraints, proposed in this thesis, semantical correctness of media data considering the data output can be supported.Thus, the contribution of this work is an qualitative improvement of managing media data by database systems

The XFM view adaptation mechanism: An essential component for XML data warehouses

In the past few years, with many organisations providing web services for business and communication purposes, large volumes of XML transactions take place on a daily basis. In many cases, organisations maintain these transactions in their native XML format due to its flexibility for xchanging data between heterogeneous systems. This XML data provides an important resource for decision support systems. As a consequence, XML technology has slowly been included within decision support systems of data warehouse systems. The problem encountered is that existing native XML database systems suffer from poor performance in terms of managing data volume and response time for complex analytical queries. Although materialised XML views can be used to improve the performance for XML data warehouses, update problems then become the bottleneck of using materialised views. Specifically, synchronising materialised views in the face of changing view definitions, remains a significant issue. In this dissertation, we provide a method for XML-based data warehouses to manage updates caused by the change of view definitions (view redefinitions), which is referred to as the view adaptation problem. In our approach, views are defined using XPath and then modelled using a set of novel algebraic operators and fragments. XPath views are integrated into a single view graph called the XML Fragment Materialisation (XFM) View Graph, where common parts between different views are shared and appear only once in the graph. Fragments within the view graph can be selected for materialisation to facilitate the view adaptation process. While changes are applied, our view adaptation algorithms can quickly determine what part of the XFM view graph is affected. The adaptation algorithms then perform a structural adaptation to update the view graph, followed by data adaptation to update materialised fragments

Efficient Change Management of XML Documents

XML-based documents play a major role in modern information architectures and their corresponding work-flows. In this context, the ability to identify and represent differences between two versions of a document is essential. A second important aspect is the merging of document versions, which becomes crucial in parallel editing processes. Many different approaches exist that meet these challenges. Most rely on operational transformation or document annotation. In both approaches, the operations leading to changes are tracked, which requires corresponding editing applications. In the context of software development, however, a state-based approach is common. Here, document versions are compared and merged using external tools, called diff and patch. This allows users for freely editing documents without being tightened to special tools. Approaches exist that are able to compare XML documents. A corresponding merge capability is still not available. In this thesis, I present a comprehensive framework that allows for comparing and merging of XML documents using a state-based approach. Its design is based on an analysis of XML documents and their modification patterns. The heart of the framework is a context-oriented delta model. I present a diff algorithm that appears to be highly efficient in terms of speed and delta quality. The patch algorithm is able to merge document versions efficiently and reliably. The efficiency and the reliability of my approach are verified using a competitive test scenario

Photo searching on small screen devices

Word processed copy.Includes bibliographical references (p. 221-239).The aim of this thesis is to improve HCI (Human-Computer Interaction) knowledge in the design of the next generation of photo search tools on small display devices. Today, these devices have all the ingredients for a truly mobile photo collection, such as large storage, multiple networking capabilities and high resolution screens. However, they lack the tools for searching through large collections of photographs. This is particularly important as users have expressed a desire to store images on mobile devices in the long term. No substantial research has looked at addressing users searching needs. Few researchers have considered the importance of supporting both searching and browsing to cater for user needs. None that we could find have assessed the potential impact of adding desktop-based annotation capabilities. Consequently, this thesis seeks to address these challenges and provide an empirical foundation for the design of photo search tools. To achieve these objectives an iterative user-centered design methodology was employed. The end practical result was a single photo search interface that incorporates the best traits of a variety of tools to support search. The thesis reflects on each cycle in the iterative design process. The first major area of contribution to the field of HCI improves existing knowledge on photo searching behavior by providing a number of empirically grounded findings about searching behavior. It identifies some of the core factors that influence search strategies and outlines a conceptual framework to guide the design of future systems. The second area of contribution is a single photo searching tool for small display devices that is based on iterative studies of various user interface designs. It integrates multiple search methods within a single user interface. In contrast to previous research in this area, the design is centered on locating events rather than individual images as we found that people naturally associate photographs with events when searching