A Four-Factor User Interaction Model for Content-Based Image Retrieval

In order to bridge the “Semantic gap”, a number of relevance feedback (RF) mechanisms have been applied to content-based image retrieval (CBIR). However current RF techniques in most existing CBIR systems still lack satisfactory user interaction although some work has been done to improve the interaction as well as the search accuracy. In this paper, we propose a four-factor user interaction model and investigate its effects on CBIR by an empirical evaluation. Whilst the model was developed for our research purposes, we believe the model could be adapted to any content-based search system

Interactive retrieval of video using pre-computed shot-shot similarities

A probabilistic framework for content-based interactive video retrieval is described. The developed indexing of video fragments originates from the probability of the user's positive judgment about key-frames of video shots. Initial estimates of the probabilities are obtained from low-level feature representation. Only statistically significant estimates are picked out, the rest are replaced by an appropriate constant allowing efficient access at search time without loss of search quality and leading to improvement in most experiments. With time, these probability estimates are updated from the relevance judgment of users performing searches, resulting in further substantial increases in mean average precision

Быстрая классификация JPEG-изображений

Среди различных видов классификаций мультимедиа изображений наиболее значимой и наиболее сложной является задача их семантической классификации [1,2]. По статистическим оценкам в мультимедийных базах данных большинство изображений (до 80%) представлены в JPEG формате или производных от него (JFIF, SPIFF, JBIG, JPEG-EXIF, MPEG). Формат сжатия JPEG обеспечивает значительную экономию ресурсов, как при хранении, так и при обработке зрительных образов, что и определяет его широкую распространенность. Поэтому разработка эффективных методов семантической классификации JPEG-изображений является в настоящее время весьма актуальной задачей. В предлагаемом подходе используются три основные идеи: 1) для классификации изображений используется спектральное признаковое пространство, формируемое стандартной процедурой блочного кодирования JPEG-формата, что позволяет производить классификацию без восстановления изображения. 2) Семантика полного изображения является производной от семантики сегментов изображения, что позволяет реализовать экономную иерархическую процедуру классификации. 3) Исключаются какие-либо априорные предположения о конфигурации семантического класса в пространстве признаков, классификация выполняются по достоверным прецедентам базы данных, что позволяет ограничиться достаточно простыми метрическими методами классификации. В результате выполнения проекта были разработаны алгоритмы адаптивной сегментации изображений, алгоритмы информативной оценки системы первичных признаков и формирования сложных вторичных признаков, алгоритмы нечеткой метрической классификации сегментов изображения, алгоритмы нечеткой семантической классификации видеообразов по результатам сегментной классификации. Алгоритмы реализованы в программной среде МАТЛАБ и представляют собой интерактивную программу накопления, обучения и классификации видеоданных представленных в JPEG-формате. Проведенные тестовые испытания в целом подтверждают эффективность предложенных подходов. Разработанные алгоритмы могут быть использованы для семантической классификации при достаточной представительности базы данных. Надежность распознавания может быть улучшена добавлением уровня онтологий и расширением вторичного признакового пространства.Between different kinds of classifications the most complex task is the semantic classification of video patterns [1,2]. Accordingly to statistic estimations in multimedia data bases the majority of images (until 80 percents) are represented in JPEG format or in derivations from one. JPEG compressing format provides the significant resource economy the both storing and processing of video images and it determinates its widely propagation. Therefore the developing of effective methods of semantic classification for JPEG images is presentday task. In the project the three main ideas are used. 1) The spectral indication space formed with standard JPEG procedure is used for image classification. It permits to make classification without full image restoration. 2) The semantic of a total image is carried out from image segment semantics that permits to realize the economic hierarchical classification procedure. 3) Any a priory suppositions about point locations of a semantic class are excluded. It permits to use the enough simple metric classification methods. In the project there were developed the algorithms of adaptive image segmentation, the algorithms of informative estimation of primary indications and the algorithms of forming complex second indications, the algorithms of fuzzy metric classification for image segments, the algorithms fuzzy image semantic classification on base of the image segment classification. All algorithms are realized in program MATHLAB media and ones represent the interactive program for accumulation, teaching and classification of JPEG-images. The carried out experiments confirm the effectiveness of supposed methods. Developed algorithms may be used for semantic classification in condition of representative data base. The reliability of image recognition may be increased with adding of ontology level and extending the space of second indications

Image Retrieval: History, Current Approaches, and Promising Framework

Abstract Today, by dominant use of the world computer networks, the volume of image database is increased and retrieving the required image similar with the image is a serious need. Here having a dynamic and flexible framework can help considerably in the design of an image retrieval system with high accuracy. In this study, by the investigation and analysis of three systems of current famous systems of retrieving and emphasis on weaknesses and strengths of the systems, presented a general framework for image retrieval systems. The important issue is that an ideal image retrieval system should be able to automatically extract semantic content and make the images indexing

Content Based Image Retrieval (CBIR) in Remote Clinical Diagnosis and Healthcare

Content-Based Image Retrieval (CBIR) locates, retrieves and displays images alike to one given as a query, using a set of features. It demands accessible data in medical archives and from medical equipment, to infer meaning after some processing. A problem similar in some sense to the target image can aid clinicians. CBIR complements text-based retrieval and improves evidence-based diagnosis, administration, teaching, and research in healthcare. It facilitates visual/automatic diagnosis and decision-making in real-time remote consultation/screening, store-and-forward tests, home care assistance and overall patient surveillance. Metrics help comparing visual data and improve diagnostic. Specially designed architectures can benefit from the application scenario. CBIR use calls for file storage standardization, querying procedures, efficient image transmission, realistic databases, global availability, access simplicity, and Internet-based structures. This chapter recommends important and complex aspects required to handle visual content in healthcare.Comment: 28 pages, 6 figures, Book Chapter from "Encyclopedia of E-Health and Telemedicine

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Semantic image retrieval using relevance feedback and transaction logs

Due to the recent improvements in digital photography and storage capacity, storing large amounts of images has been made possible, and efficient means to retrieve images matching a user’s query are needed. Content-based Image Retrieval (CBIR) systems automatically extract image contents based on image features, i.e. color, texture, and shape. Relevance feedback methods are applied to CBIR to integrate users’ perceptions and reduce the gap between high-level image semantics and low-level image features. The precision of a CBIR system in retrieving semantically rich (complex) images is improved in this dissertation work by making advancements in three areas of a CBIR system: input, process, and output. The input of the system includes a mechanism that provides the user with required tools to build and modify her query through feedbacks. Users behavioral in CBIR environments are studied, and a new feedback methodology is presented to efficiently capture users’ image perceptions. The process element includes image learning and retrieval algorithms. A Long-term image retrieval algorithm (LTL), which learns image semantics from prior search results available in the system’s transaction history, is developed using Factor Analysis. Another algorithm, a short-term learner (STL) that captures user’s image perceptions based on image features and user’s feedbacks in the on-going transaction, is developed based on Linear Discriminant Analysis. Then, a mechanism is introduced to integrate these two algorithms to one retrieval procedure. Finally, a retrieval strategy that includes learning and searching phases is defined for arranging images in the output of the system. The developed relevance feedback methodology proved to reduce the effect of human subjectivity in providing feedbacks for complex images. Retrieval algorithms were applied to images with different degrees of complexity. LTL is efficient in extracting the semantics of complex images that have a history in the system. STL is suitable for query and images that can be effectively represented by their image features. Therefore, the performance of the system in retrieving images with visual and conceptual complexities was improved when both algorithms were applied simultaneously. Finally, the strategy of retrieval phases demonstrated promising results when the query complexity increases