A Literature Review on the Development of Multimedia Information Retrieval (MIR) and the Futere Challenges

 Abstrak Multimedia information retrieval (MIR) adalah proses pencarian dan pengambilan informasi (information retrieval/IR) dalam content berbentuk multimedia, seperti suara, gambar, video, dan animasi. Penelitian ini menggunakan metode kajian literatur (literature review) terhadap perkembangan MIR saat ini dan tantangan yang akan dihadapi di masa depan bagi para periset di bidang IR. Berbagai penelitian MIR saat ini meliputi komputasi yang berpusat pada manusia (aktor) terhadap pencarian informasi, memungkinkan mesin melakukan pembelajaran (semantik), memungkinkan mesin meminta koreksi (umpan balik), penambahan fitur atau faktor baru, penelitian pada media baru, perangkuman informasi dari content multimedia, pengindeksan dengan performa tinggi, dan mekanisme terhadap teknik evaluasi. Di masa yang akan datang, tantangan yang menjadi potensi penelitian MIR meliputi peran manusia yang tetap menjadi pusat (aktor) terhadap pencarian informasi, kolaborasi konten multimedia yang lebih beragam, dan penggunaan kata kunci sederhana (folksonomi). Kata kunci: multimedia information retrieval, multimedia, komputasi, semantik, pencarian informasi  Abstract Multimedia information retrieval (MIR) is the process of searching and retrieving information (information retrieval/IR) in multimedia content, such as audio, image, video, and animation. This study uses literature review method against current MIR conditions and what challenges to be faced in the future for researchers in the field of IR. Various studies of MIR currently include human centered computation for IR, allowing machine to do the learning (semantics); allowing machine to request feedback, add new features or factors, research on new media, summarize information from multimedia content, high-performance indexing, and evaluation techniques. In the future, the potential of MIR research includes the human-centered role for information retrieval, more diverse collaborative multimedia content, and the use of simple keyword (folksonomy). Keywords: multimedia information retrieval, multimedia, computation, semantics, information searchÂ

¿Cómo puede ayudar la bioinformática a los Museos de Historia Natural?

We propose the founding of a Natural History bioinformatics framework, which would solve one of the main problems in Natural History: data which is scattered around in many incompatible systems (not only computer systems, but also paper ones). This framework consists of computer resources (hardware and software), methodologies that ease the circulation of data, and staff expert in dealing with computers, who will develop software solutions to the problems encountered by naturalists. This system is organized in three layers: acquisition, data and analysis. Each layer is described, and an account of the elements that constitute it given.Se presentan las bases de una estructura bioinformática para Historia Natural, que trata de resolver uno de los principales problemas en ésta: la presencia de datos distribuidos a lo largo de muchos sistemas incompatibles entre sí (y no sólo hablamos de sistemas informáticos, sino también en papel). Esta estructura se sustenta en recursos informáticos (en sus dos vertientes: hardware y software), en metodologías que permitan la fácil circulación de los datos, y personal experto en el uso de ordenadores que se encargue de desarrollar soluciones software a los problemas que plantean los naturalistas. Este sistema estaría organizado en tres capas: de adquisición, de datos y de análisis. Cada una de estas capas se describe, indicando los elementos que la componen

Deficient Human Aspects in Current Multimedia Indexing and Retrieval (MIR) of Large Social Networks Databases

An inside look at the contents of social networks databases shows a significant diversion from traditional database contents and functionality. There is also enormous evidences that Social networks are changing the way multimedia content is shared on the web, by allowing users to upload their photos, videos, and audio content, produced by any means of digital recorders such as mobile/smart-phones, and web/digital cameras. In this article, an overview of multimedia indexing and searching algorithms, following the data growth curve is presented in detail. This paper concludes with the social aspects and new, interesting views on multimedia retrieval in the large social media databases.Keywords: multimedia, indexing, social media, algorithms social networks, databases, retrieva

Color image quality measures and retrieval

The focus of this dissertation is mainly on color image, especially on the images with lossy compression. Issues related to color quantization, color correction, color image retrieval and color image quality evaluation are addressed. A no-reference color image quality index is proposed. A novel color correction method applied to low bit-rate JPEG image is developed. A novel method for content-based image retrieval based upon combined feature vectors of shape, texture, and color similarities has been suggested. In addition, an image specific color reduction method has been introduced, which allows a 24-bit JPEG image to be shown in the 8-bit color monitor with 256-color display. The reduction in download and decode time mainly comes from the smart encoder incorporating with the proposed color reduction method after color space conversion stage. To summarize, the methods that have been developed can be divided into two categories: one is visual representation, and the other is image quality measure. Three algorithms are designed for visual representation: (1) An image-based visual representation for color correction on low bit-rate JPEG images. Previous studies on color correction are mainly on color image calibration among devices. Little attention was paid to the compressed image whose color distortion is evident in low bit-rate JPEG images. In this dissertation, a lookup table algorithm is designed based on the loss of PSNR in different compression ratio. (2) A feature-based representation for content-based image retrieval. It is a concatenated vector of color, shape, and texture features from region of interest (ROI). (3) An image-specific 256 colors (8 bits) reproduction for color reduction from 16 millions colors (24 bits). By inserting the proposed color reduction method into a JPEG encoder, the image size could be further reduced and the transmission time is also reduced. This smart encoder enables its decoder using less time in decoding. Three algorithms are designed for image quality measure (IQM): (1) A referenced IQM based upon image representation in very low-dimension. Previous studies on IQMs are based on high-dimensional domain including spatial and frequency domains. In this dissertation, a low-dimensional domain IQM based on random projection is designed, with preservation of the IQM accuracy in high-dimensional domain. (2) A no-reference image blurring metric. Based on the edge gradient, the degree of image blur can be measured. (3) A no-reference color IQM based upon colorfulness, contrast and sharpness

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Event-centric management of personal photos

Since the last decade we have been observing a tremendous growth in the size of personal photo collections. For this reason, and due to the lack of proper automatic classification and annotation in standard album-centric photo software, users find it increasingly difficult to organise and make use of their photos. Although automatic annotation of media content can work to achieve more sophisticated multimedia classification and retrieval if its used in combination with rich knowledge representations, it still requires the availability of well-annotated training sets to produce the type of higher-level descriptions that would be of interest to casual users. Thus, the applicability of this approach is highly unlikely in the broad domain of personal photography. Recent developments in the media industry show an interest towards the organisation and structuring of media collections using an event-centric metaphor. This event-centric approach is inspired by strong research in psychology on how our autobiographical memory works to organise, recollect and share our life experiences. While this metaphor is backed by some early user studies, these were led before the large adoption of social media sharing services and there has been little recent research on how users actually use events digitally to organise and share their media. In this work we first present an updated study on what users are doing with their photos in current online platforms to support the suitability of an event-centric approach. Next, we introduce a simple framework for event-centric personal photo management focused on temporal and spatial aspects and through it we describe our techniques for automatic photo organisation and sharing. Finally, we propose a platform for personal photo management that makes use of these automatic techniques and present an evaluation of a prototypical implementation