An architecture for semantic integration of data and medical images

Resumen En las organizaciones prestadoras de servicios de salud, existen diferentes fuentes de datos (e.g. historia clínica, datos demográficos, archivos DICOM) de naturaleza distinta, que están dispersas en medios de almacenamiento y que provienen de fuentes heterogéneas. Adicionalmente, el formato DICOM brinda la posibilidad de almacenar información del paciente y de las imágenes médicas. Estos archivos son administrados en PACS, sin embargo los PACs no brinda herramientas de apoyo al diagnóstico. En este artículo se presenta una arquitectura de integración de datos orientada a enriquecer imágenes médicas mediante metadatos extraídos de un vocabulario controlado. La arquitectura fue instanciada en un prototipo que ofrece mecanismos de anotación – manual y automática – de imágenes y estrategias de búsqueda y recuperación de datos e imágenes diferentes a los tradicionales usando palabras claves o descriptores MPEG-7. La anotación se basa en un vocabulario controlado que forma parte de una taxonomía de conceptos y términos médicos. Abstract In organisations providing health services, there exist different data sources (e.g. clinical history, demographics data, DICOM files) of diverse nature, which are scattered storage and come from heterogeneous sources. Additionally, the DICOM format stores patient information and medical images. These files are managed in PACS, however PACs does not provide diagnostic support tools. In this paper, a data integration architecture oriented to enrich medical images using metadata extracted from a controlled vocabulary is presented. The architecture was instantiated in a prototype that provides image annotation mechanisms - manual and automatic – and strategies for searching and retrieving data and images using keywords or descriptors MPEG-7, which are different from traditional ones. The annotation is based on a controlled vocabulary that is part of a taxonomy of concepts and medical terms

Knowledge graph applications in medical imaging analysis : a scoping review

There is an increasing trend to represent domain knowledge in structured graphs, which provide efficient knowledge representations for many downstream tasks. Knowledge graphs are widely used to model prior knowledge in the form of nodes and edges to represent semantically connected knowledge entities, which several works have adopted into different medical imaging applications. We systematically search over five databases to find relevant articles that apply knowledge graphs to medical imaging analysis. After screening, evaluating, and reviewing the selected articles, we performed a systematic analysis. We look at four applications in medical imaging analysis, including disease classification, disease localization and segmentation, report generation, and image retrieval. We also identify limitations of current work, such as the limited amount of available annotated data and weak generalizability to other tasks. We further identify the potential future directions according to the identified limitations, including employing semi-supervised frameworks to alleviate the need for annotated data and exploring task-agnostic models to provide better generalizability. We hope that our article will provide the readers with aggregated documentation of the state-of-the-art knowledge graph applications for medical imaging.Electrical and Computer Engineerin

A semantic fusion approach between medical images and reports using umls

Abstract. One of the main challenges in content-based image retrieval still remains to bridge the gap between low-level features and semantic information. In this paper, we present our first results concerning a medical image retrieval approach using a semantic medical image and report indexing within a fusion framework, based on the Unified Medical Language System (UMLS) metathesaurus. We propose a structured learning framework based on Support Vector Machines to facilitate modular design and extract medical semantics from images. We developed two complementary visual indexing approaches within this framework: a global indexing to access image modality, and a local indexing to access semantic local features. Visual indexes and textual indexes- extracted from medical reports using MetaMap software application- constitute the input of the late fusion module. A weighted vectorial norm fusion algorithm allows the retrieval system to increase its meaningfulness, efficiency and robustness. First results on the CLEF medical database are presented. The important perspectives of this approach in terms of semantic query expansion and data-mining are discussed.