A novel augmented deep transfer learning for classification of COVID-19 and other thoracic diseases from X-rays

Deep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification

Pneumonia Detection on X-Ray Imaging using Softmax Output in Multilevel Meta Ensemble Algorithm of Deep Convolutional Neural Network Transfer Learning Models

Pneumonia is the leading cause of death from a single infection worldwide in children. A proven clinical method for diagnosing pneumonia is through a chest X-ray. However, the resulting X-ray images often need clarification, resulting in subjective judgments. In addition, the process of diagnosis requires a longer time. One technique can be applied by applying advanced deep learning, namely, Transfer Learning with Deep Convolutional Neural Network (Deep CNN) and modified Multilevel Meta Ensemble Learning using Softmax. The purpose of this research was to improve the accuracy of the pneumonia classification model. This study proposes a classification model with a meta-ensemble approach using five classification algorithms: Xception, Resnet 15V2, InceptionV3, VGG16, and VGG19. The ensemble stage used two different concepts, where the first level ensemble combined the output of the Xception, ResNet15V2, and InceptionV3 algorithms. Then the output from the first ensemble level is reused for the following learning process, combined with the output from other algorithms, namely VGG16 and VGG19. This process is called ensemble level two. The classification algorithm used at this stage is the same as the previous stage, using KNN as a classification model. Based on experiments, the model proposed in this study has better accuracy than the others, with a test accuracy value of 98.272%. The benefit of this research could help doctors as a recommendation tool to make more accurate and timely diagnoses, thus speeding up the treatment process and reducing the risk of complications

DermAI 1.0: A Robust, Generalized, and Novel Attention-Enabled Ensemble-Based Transfer Learning Paradigm for Multiclass Classification of Skin Lesion Images

Skin lesion classification plays a crucial role in dermatology, aiding in the early detection, diagnosis, and management of life-threatening malignant lesions. However, standalone transfer learning (TL) models failed to deliver optimal performance. In this study, we present an attention-enabled ensemble-based deep learning technique, a powerful, novel, and generalized method for extracting features for the classification of skin lesions. This technique holds significant promise in enhancing diagnostic accuracy by using seven pre-trained TL models for classification. Six ensemble-based DL (EBDL) models were created using stacking, softmax voting, and weighted average techniques. Furthermore, we investigated the attention mechanism as an effective paradigm and created seven attention-enabled transfer learning (aeTL) models before branching out to construct three attention-enabled ensemble-based DL (aeEBDL) models to create a reliable, adaptive, and generalized paradigm. The mean accuracy of the TL models is 95.30%, and the use of an ensemble-based paradigm increased it by 4.22%, to 99.52%. The aeTL models' performance was superior to the TL models in accuracy by 3.01%, and aeEBDL models outperformed aeTL models by 1.29%. Statistical tests show significant p-value and Kappa coefficient along with a 99.6% reliability index for the aeEBDL models. The approach is highly effective and generalized for the classification of skin lesions

A Deep Learning Model Compression and Ensemble Approach for Weed Detection

Site-specific weed management is an important practice in precision agriculture. Current advances in artificial intelligence have resulted in the use of large deep convolutional neural networks for weed detection. In this paper, a transfer learning, model compression, and ensemble learning approach is introduced that is suitable for resource-limited hardware such as mobile and embedded devices. The resulting ensemble model achieves 91.2% classification accuracy which is comparable to the performance of state-of-the-art deep learning models (such as the vanilla VGG16, DenseNet, and ResNet) while being about 62.22% smaller in size than DenseNet (the smallest-sized full-sized model). The approach used in this study is beneficial for further development of deep convolutional neural networks on smaller resource-limited hardware typically used in agriculture, as well as other industries such as healthcare and telecommunication

Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule

As a modern ensemble technique, Deep Forest (DF) employs a cascading structure to construct deep models, providing stronger representational power compared to traditional decision forests. However, its greedy multi-layer learning procedure is prone to overfitting, limiting model effectiveness and generalizability. This paper presents an optimized Deep Forest, featuring learnable, layerwise data augmentation policy schedules. Specifically, We introduce the Cut Mix for Tabular data (CMT) augmentation technique to mitigate overfitting and develop a population-based search algorithm to tailor augmentation intensity for each layer. Additionally, we propose to incorporate outputs from intermediate layers into a checkpoint ensemble for more stable performance. Experimental results show that our method sets new state-of-the-art (SOTA) benchmarks in various tabular classification tasks, outperforming shallow tree ensembles, deep forests, deep neural network, and AutoML competitors. The learned policies also transfer effectively to Deep Forest variants, underscoring its potential for enhancing non-differentiable deep learning modules in tabular signal processing

Transfer and Ensemble Approach for Breast Cancer Detection and Classification Using Deep Learning

Breast cancer is a serious disease that can cause significant health problems for women worldwide. It is crucial to detect and classify breast cancer early stage so that doctors can promptly treat it and aid patients in their recovery. Many investigators have used various deep learning (DL) strategies to detect and classify breast cancer. However, due to the complexity of the problem, relying on a single DL model may not suffice to achieve high accuracy. Therefore, this study suggests a transfer and ensemble deep model for breast cancer detection and classification. The suggested model involves using pre-trained models such as Sequential, Xception, DenseNet201, VGG16, and InceptionResNetV2. The top three models are selected to collaborate and deliver the most accurate results. The proposed DL model was tested on publicly available breast BUSI datasets, demonstrating its superiority over single DL models, achieving an accuracy of 87.9% on the BUSI dataset. Additionally, the model proved to be adapTABLE to different amounts of data, making it potentially valuable in hospitals and clinics

Multi-categorical deep learning neural network to classify retinal images: A pilot study employing small database

Deep learning emerges as a powerful tool for analyzing medical images. Retinal disease detection by using computer-aided diagnosis from fundus image has emerged as a new method. We applied deep learning convolutional neural network by using MatConvNet for an automated detection of multiple retinal diseases with fundus photographs involved in STructured Analysis of the REtina (STARE) database. Dataset was built by expanding data on 10 categories, including normal retina and nine retinal diseases. The optimal outcomes were acquired by using a random forest transfer learning based on VGG-19 architecture. The classification results depended greatly on the number of categories. As the number of categories increased, the performance of deep learning models was diminished. When all 10 categories were included, we obtained results with an accuracy of 30.5%, relative classifier information (RCI) of 0.052, and Cohen's kappa of 0.224. Considering three integrated normal, background diabetic retinopathy, and dry age-related macular degeneration, the multi-categorical classifier showed accuracy of 72.8%, 0.283 RCI, and 0.577 kappa. In addition, several ensemble classifiers enhanced the multi-categorical classification performance. The transfer learning incorporated with ensemble classifier of clustering and voting approach presented the best performance with accuracy of 36.7%, 0.053 RCI, and 0.225 kappa in the 10 retinal diseases classification problem. First, due to the small size of datasets, the deep learning techniques in this study were ineffective to be applied in clinics where numerous patients suffering from various types of retinal disorders visit for diagnosis and treatment. Second, we found that the transfer learning incorporated with ensemble classifiers can improve the classification performance in order to detect multi-categorical retinal diseases. Further studies should confirm the effectiveness of algorithms with large datasets obtained from hospitals.ope