Towards an All-Purpose Content-Based Multimedia Information Retrieval System

The growth of multimedia collections - in terms of size, heterogeneity, and variety of media types - necessitates systems that are able to conjointly deal with several forms of media, especially when it comes to searching for particular objects. However, existing retrieval systems are organized in silos and treat different media types separately. As a consequence, retrieval across media types is either not supported at all or subject to major limitations. In this paper, we present vitrivr, a content-based multimedia information retrieval stack. As opposed to the keyword search approach implemented by most media management systems, vitrivr makes direct use of the object's content to facilitate different types of similarity search, such as Query-by-Example or Query-by-Sketch, for and, most importantly, across different media types - namely, images, audio, videos, and 3D models. Furthermore, we introduce a new web-based user interface that enables easy-to-use, multimodal retrieval from and browsing in mixed media collections. The effectiveness of vitrivr is shown on the basis of a user study that involves different query and media types. To the best of our knowledge, the full vitrivr stack is unique in that it is the first multimedia retrieval system that seamlessly integrates support for four different types of media. As such, it paves the way towards an all-purpose, content-based multimedia information retrieval system

Integrating memory context into personal information re-finding

Personal information archives are emerging as a new challenge for information retrieval (IR) techniques. The user’s memory plays a greater role in retrieval from person archives than from other more traditional types of information collection (e.g. the Web), due to the large overlap of its content and individual human memory of the captured material. This paper presents a new analysis on IR of personal archives from a cognitive perspective. Some existing work on personal information management (PIM) has begun to employ human memory features into their IR systems. In our work we seek to go further, we assume that for IR in PIM system terms can be weighted not only by traditional IR methods, but also taking the user’s recall reliability into account. We aim to develop algorithms that combine factors from both the system side and the user side to achieve more effective searching. In this paper, we discuss possible applications of human memory theories for this algorithm, and present results from a pilot study and a proposed model of data structure for the HDMs achieves

An information-driven framework for image mining

[Abstract]: Image mining systems that can automatically extract semantically meaningful information (knowledge) from image data are increasingly in demand. The fundamental challenge in image mining is to determine how low-level, pixel representation contained in a raw image or image sequence can be processed to identify high-level spatial objects and relationships. To meet this challenge, we propose an efficient information-driven framework for image mining. We distinguish four levels of information: the Pixel Level, the Object Level, the Semantic Concept Level, and the Pattern and Knowledge Level. High-dimensional indexing schemes and retrieval techniques are also included in the framework to support the flow of information among the levels. We believe this framework represents the first step towards capturing the different levels of information present in image data and addressing the issues and challenges of discovering useful patterns/knowledge from each level

Quantum theory-inspired search

With the huge number and diversity of the users, the advertising products and services, the rapid growth of online multimedia resources, the context of information needs are even more broad and complex. Although research in search engine technology has led to various models over the past three decades, the investigation for effectively integrating the dimensions of context to deploy advanced search technology has been limited due to the lack of a unified modeling and evaluation framework. Quantum Theory (QT) has created new and unprecedented means for communicating and computing. Besides computer science, optics, electronics, physics, QT and search engine technology can be combined: interference in user interaction; entanglement in cognition; superposition in word meaning; non-classical probability in information ranking; complex vector spaces in multimedia search. This paper highlights our recent results on QT-inspired search engine technology

LDEF: A bibliography with abstracts

The Long Duration Exposure Facility (LDEF) was a free-flying cylindrical structure that housed self-contained experiments in trays mounted on the exterior of the structure. Launched into orbit from the Space Shuttle Challenger in 1984, the LDEF spent almost six years in space before being recovered in 1990. The 57 experiments investigated the effects of the low earth orbit environment on materials, coatings, electronics, thermal systems, seeds, and optics. It also carried experiments that measured crystals growth, cosmic radiation, and micrometeoroids. This bibliography contains 435 selected records from the NASA aerospace database covering the years 1973 through June of 1992. The citations are arranged within subject categories by author and date of publication