Bi-Modal Conceptual Indexing for Medical Image Retrieval

International audienceTo facilitate the automatic indexing and retrieval of large medical image databases, images and associated texts are indexed us- ing concepts from the Unified Medical Language System (UMLS) meta- thesaurus.We propose a structured learning framework for learning med- ical semantics from images. Two complementary global and local visual indexing approaches are presented. Two fusion approaches are also used to improve textual retrieval using the UMLS-based image indexing: a simple post-query fusion and a visual modality filtering to remove visu- ally aberrant images according to the query modality concepts. Using the ImageCLEFmed database, we demonstrate that our framework is superior when compared to the automatic runs evaluated in 2005 on the same Medical Image Retrieval task