Automatic Segmentation of Subfigure Image Panels for Multimodal Biomedical Document Retrieval

Biomedical images are often referenced for clinical decision support (CDS), educational purposes, and research. The task of automatically finding the images in a scientific article that are most useful for the purpose of determining relevance to a clinical situation is traditionally done using text and is quite challenging. We propose to improve this by associating image features from the entire image and from relevant regions of interest with biomedical concepts described in the figure caption or discussion in the article. However, images used in scientific article figures are often composed of multiple panels where each sub-figure (panel) is referenced in the caption using alphanumeric labels, e.g. Figure 1(a), 2(c), etc. It is necessary to separate individual panels from a multi-panel figure as a first step toward automatic annotation of images. In this work we present methods that add make robust our previous efforts reported here. Specifically, we address the limitation in segmenting figures that do not exhibit explicit inter-panel boundaries, e.g. illustrations, graphs, and charts. We present a novel hybrid clustering algorithm based on particle swarm optimization (PSO) with fuzzy logic controller (FLC) to locate related figure components in such images. Results from our evaluation are very promising with 93.64% panel detection accuracy for regular (non-illustration) figure images and 92.1% accuracy for illustration images. A computational complexity analysis also shows that PSO is an optimal approach with relatively low computation time. The accuracy of separating these two type images is 98.11% and is achieved using decision tree

Performance evaluation of deep neural ensembles toward malaria parasite detection in thin-blood smear images

Background Malaria is a life-threatening disease caused by Plasmodium parasites that infect the red blood cells (RBCs). Manual identification and counting of parasitized cells in microscopic thick/thin-film blood examination remains the common, but burdensome method for disease diagnosis. Its diagnostic accuracy is adversely impacted by inter/intra-observer variability, particularly in large-scale screening under resource-constrained settings. Introduction State-of-the-art computer-aided diagnostic tools based on data-driven deep learning algorithms like convolutional neural network (CNN) has become the architecture of choice for image recognition tasks. However, CNNs suffer from high variance and may overfit due to their sensitivity to training data fluctuations. Objective The primary aim of this study is to reduce model variance, improve robustness and generalization through constructing model ensembles toward detecting parasitized cells in thin-blood smear images. Methods We evaluate the performance of custom and pretrained CNNs and construct an optimal model ensemble toward the challenge of classifying parasitized and normal cells in thin-blood smear images. Cross-validation studies are performed at the patient level to ensure preventing data leakage into the validation and reduce generalization errors. The models are evaluated in terms of the following performance metrics: (a) Accuracy; (b) Area under the receiver operating characteristic (ROC) curve (AUC); (c) Mean squared error (MSE); (d) Precision; (e) F-score; and (f) Matthews Correlation Coefficient (MCC). Results It is observed that the ensemble model constructed with VGG-19 and SqueezeNet outperformed the state-of-the-art in several performance metrics toward classifying the parasitized and uninfected cells to aid in improved disease screening. Conclusions Ensemble learning reduces the model variance by optimally combining the predictions of multiple models and decreases the sensitivity to the specifics of training data and selection of training algorithms. The performance of the model ensemble simulates real-world conditions with reduced variance, overfitting and leads to improved generalization

Vertebra Shape Classification using MLP for Content-Based Image Retrieval

A desirable content-based image retrieval (CBIR) system would classify extracted image features to support some form of semantic retrieval. The Lister Hill National Center for Biomedical Communications, an intramural R&D division of the National Library for Medicine (NLM), maintains an archive of digitized X-rays of the cervical and lumbar spine taken as part of the second national health and nutrition examination survey (NHANES II). It is our goal to provide shape-based access to digitized X-rays including retrieval on automatically detected and classified pathology, e.g., anterior osteophytes. This is done using radius of curvature analysis along the anterior portion, and morphological analysis for quantifying protrusion regions along the vertebra boundary. Experimental results are presented for the classification of 704 cervical spine vertebrae by evaluating the features using a multi-layer perceptron (MLP) based approach. In this paper, we describe the design and current status of the content-based image retrieval (CBIR) system and the role of neural networks in the design of an effective multimedia information retrieval system

Graphical Image Classification Combining an Evolutionary Algorithm and Binary Particle Swarm Optimization

Biomedical journal articles contain a variety of image types that can be broadly classified into two categories: regular images, and graphical images. Graphical images can be further classified into four classes: diagrams, statistical figures, flow charts, and tables. Automatic figure type identification is an important step toward improved multimodal (text + image) information retrieval and clinical decision support applications. This paper describes a feature-based learning approach to automatically identify these four graphical figure types. We apply Evolutionary Algorithm (EA), Binary Particle Swarm Optimization (BPSO) and a hybrid of EA and BPSO (EABPSO) methods to select an optimal subset of extracted image features that are then classified using a Support Vector Machine (SVM) classifier. Evaluation performed on 1038 figure images extracted from ten BioMedCentral® journals with the features selected by EABPSO yielded classification accuracy as high as 87.5%

Image Analysis Techniques for the Automated Evaluation of Subaxial Subluxation in Cervical Spine X-Ray Images

Rheumatoid arthritis is a chronic inflammatory disease affecting synovial joints of the body, especially the hands and feet, spine, knees and hips. For many patients, the cervical spine is associated with rheumatoid arthritis. Subluxation is the abnormal movement of one of the bones that comprise a joint. In this research, image analysis techniques have been investigated for the recognition of cervical spine x-ray images with one or more instances of subaxial subluxation. Receiver operating characteristic curve results are presented, showing potential for subaxial subluxation discrimination on an image-by-image basis

Detection and visualization of abnormality in chest radiographs using modality-specific convolutional neural network ensembles

Convolutional neural networks (CNNs) trained on natural images are extremely successful in image classification and localization due to superior automated feature extraction capability. In extending their use to biomedical recognition tasks, it is important to note that visual features of medical images tend to be uniquely different than natural images. There are advantages offered through training these networks on large scale medical common modality image collections pertaining to the recognition task. Further, improved generalization in transferring knowledge across similar tasks is possible when the models are trained to learn modality-specific features and then suitably repurposed for the target task. In this study, we propose modality-specific ensemble learning toward improving abnormality detection in chest X-rays (CXRs). CNN models are trained on a large-scale CXR collection to learn modality-specific features and then repurposed for detecting and localizing abnormalities. Model predictions are combined using different ensemble strategies toward reducing prediction variance and sensitivity to the training data while improving overall performance and generalization. Class-selective relevance mapping (CRM) is used to visualize the learned behavior of the individual models and their ensembles. It localizes discriminative regions of interest (ROIs) showing abnormal regions and offers an improved explanation of model predictions. It was observed that the model ensembles demonstrate superior localization performance in terms of Intersection of Union (IoU) and mean Average Precision (mAP) metrics than any individual constituent model

A Prototype Content-based Image Retrieval system for Spine X-rays

Abstract At the Lister Hill National Center for Biomedical Communications, an R&D division of the National Library of Medicine, we are engaged in an effort in content-based image retrieval (CBIR) for biomedical image databases. Toward the goal of developing a functional and significant CBIR capability, we have created a prototype system for image indexing and retrieval which operates on a collection of spine x-rays and associated health survey data. In this paper, we present our prototype system functionality, performance results, ongoing research, and outstanding technical issues. 1. Content-based image retrieval (CBIR) Our work in CBIR is the latest phase of research and development into the use of technology for the dissemination of biomedical multimedia information; this work has previously resulted in the development of a biomedical multimedia database system, a digital atlas of the cervical and lumbar spine, and an Internet archive of digitized x-ray images Indexing -the computer-assisted data reduction of images into mathematical features. For the spine images, the features capture the shape information for vertebrae. Indexing consists of the steps of segmentation of the objects of interest (the vertebrae) and extraction of feature vectors (shape representation, in a data-reduced fashion) from the raw segmentations. An implicit requirement for indexing is that the feature vectors are organized for efficient search and retrieval. A step that we also propose to carry out at indexing time is the classification of the shape data (raw segmentations or feature vectors) into categories of interest at a semantic level: namely, the categories of "normal" or "abnormal" for particular biomedical characteristics associated with osteoarthritis and degenerative disk disease, such as anterior osteophytes, disc space narrowing, subluxation, and spondylolisthesis. Finally, we propose to store any text data that may be associated with our images as additional indexing information. Retrieval -the user interaction to obtain desired images from the database. We break retrieval into the steps of user query formulation, user query feature vector extraction, query search, and similarity matching. At retrieval time, a feature vector q is derived from the user's query, and the database of feature vectors is navigated to locate feature vectors similar to q. Efficient organization of the database is required to avoid searches that are prohibitively expensive in search time. For example, if the database is organized as a tree, an efficient organization will allow a search to quickly rule out nodes too distant from q, and to localize the search to nodes that are computed to lie within an acceptable search radius, with respect to the similarity metric being used. A characteristic that we also desire for our retrieval system is th