Multimedia search without visual analysis: the value of linguistic and contextual information

This paper addresses the focus of this special issue by analyzing the potential contribution of linguistic content and other non-image aspects to the processing of audiovisual data. It summarizes the various ways in which linguistic content analysis contributes to enhancing the semantic annotation of multimedia content, and, as a consequence, to improving the effectiveness of conceptual media access tools. A number of techniques are presented, including the time-alignment of textual resources, audio and speech processing, content reduction and reasoning tools, and the exploitation of surface features

Gabor Barcodes for Medical Image Retrieval

In recent years, advances in medical imaging have led to the emergence of massive databases, containing images from a diverse range of modalities. This has significantly heightened the need for automated annotation of the images on one side, and fast and memory-efficient content-based image retrieval systems on the other side. Binary descriptors have recently gained more attention as a potential vehicle to achieve these goals. One of the recently introduced binary descriptors for tagging of medical images are Radon barcodes (RBCs) that are driven from Radon transform via local thresholding. Gabor transform is also a powerful transform to extract texture-based information. Gabor features have exhibited robustness against rotation, scale, and also photometric disturbances, such as illumination changes and image noise in many applications. This paper introduces Gabor Barcodes (GBCs), as a novel framework for the image annotation. To find the most discriminative GBC for a given query image, the effects of employing Gabor filters with different parameters, i.e., different sets of scales and orientations, are investigated, resulting in different barcode lengths and retrieval performances. The proposed method has been evaluated on the IRMA dataset with 193 classes comprising of 12,677 x-ray images for indexing, and 1,733 x-rays images for testing. A total error score as low as $351$ ($\approx 80\%$ accuracy for the first hit) was achieved.Comment: To appear in proceedings of The 2016 IEEE International Conference on Image Processing (ICIP 2016), Sep 25-28, 2016, Phoenix, Arizona, US

Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed

Digital archiving of manuscripts and other heritage items for conservation and information retrieval

Expression of cultural heritage looking from the informatics angle falls into text, images, video and sound categories. ICT can be used to conserve all these heritage items like; the text information consisting of palm leaf manuscripts, stone tablets, handwritten paper documents, old printed records, books, microfilms, fiche etc, images including paintings, drawings, photographs and the like, sound items which includes musical concerts, poetry recitations, chanting of mantras, talks of important persons etc, and video items like archival films historical importance. To retrieve required information from such a large mass of materials in different formats and to transmit them across space and time, there are several limitations. Digital technology allows hitherto unavailable facilities for durable storage and speedy and efficient transmission / retrieval of information contained in all the above formats. Hypertext and hypermedia features of digital media enable integrating text with graphics, sound, video and animation. This paper discusses the international and national efforts for digitizing heritage items, digital archiving solutions available, the possibilities of the media, and the need to follow standards prescribed by organizations like UNESCO to enable easy exchange and pooling of information and documents generated in digital archiving systems at national and international level. The need to develop language technology for local scripts for organizing and preserving our cultural heritage is also stressed