FiLM-Ensemble: Probabilistic Deep Learning via Feature-wise Linear Modulation

The ability to estimate epistemic uncertainty is often crucial when deploying machine learning in the real world, but modern methods often produce overconfident, uncalibrated uncertainty predictions. A common approach to quantify epistemic uncertainty, usable across a wide class of prediction models, is to train a model ensemble. In a naive implementation, the ensemble approach has high computational cost and high memory demand. This challenges in particular modern deep learning, where even a single deep network is already demanding in terms of compute and memory, and has given rise to a number of attempts to emulate the model ensemble without actually instantiating separate ensemble members. We introduce FiLM-Ensemble, a deep, implicit ensemble method based on the concept of Feature-wise Linear Modulation (FiLM). That technique was originally developed for multi-task learning, with the aim of decoupling different tasks. We show that the idea can be extended to uncertainty quantification: by modulating the network activations of a single deep network with FiLM, one obtains a model ensemble with high diversity, and consequently well-calibrated estimates of epistemic uncertainty, with low computational overhead in comparison. Empirically, FiLM-Ensemble outperforms other implicit ensemble methods, and it and comes very close to the upper bound of an explicit ensemble of networks (sometimes even beating it), at a fraction of the memory cost.Comment: accepted at NeurIPS 202

Preventing Unauthorized AI Over-Analysis by Medical Image Adversarial Watermarking

The advancement of deep learning has facilitated the integration of Artificial Intelligence (AI) into clinical practices, particularly in computer-aided diagnosis. Given the pivotal role of medical images in various diagnostic procedures, it becomes imperative to ensure the responsible and secure utilization of AI techniques. However, the unauthorized utilization of AI for image analysis raises significant concerns regarding patient privacy and potential infringement on the proprietary rights of data custodians. Consequently, the development of pragmatic and cost-effective strategies that safeguard patient privacy and uphold medical image copyrights emerges as a critical necessity. In direct response to this pressing demand, we present a pioneering solution named Medical Image Adversarial watermarking (MIAD-MARK). Our approach introduces watermarks that strategically mislead unauthorized AI diagnostic models, inducing erroneous predictions without compromising the integrity of the visual content. Importantly, our method integrates an authorization protocol tailored for legitimate users, enabling the removal of the MIAD-MARK through encryption-generated keys. Through extensive experiments, we validate the efficacy of MIAD-MARK across three prominent medical image datasets. The empirical outcomes demonstrate the substantial impact of our approach, notably reducing the accuracy of standard AI diagnostic models to a mere 8.57% under white box conditions and 45.83% in the more challenging black box scenario. Additionally, our solution effectively mitigates unauthorized exploitation of medical images even in the presence of sophisticated watermark removal networks. Notably, those AI diagnosis networks exhibit a meager average accuracy of 38.59% when applied to images protected by MIAD-MARK, underscoring the robustness of our safeguarding mechanism

Overview of the ImageCLEFmed 2019 concept detection task

This paper describes the ImageCLEF 2019 Concept Detection Task. This is the 3rd edition of the medical caption task, after it was first proposed in ImageCLEF 2017. Concept detection from medical images remains a challenging task. In 2019, the format changed to a single subtask and it is part of the medical tasks, alongside the tuberculosis and visual question and answering tasks. To reduce noisy labels and limit variety, the data set focuses solely on radiology images rather than biomedical figures, extracted from the biomedical open access literature (PubMed Central). The development data consists of 56,629 training and 14,157 validation images, with corresponding Unified Medical Language System (UMLSR) concepts, extracted from the image captions. In 2019 the participation is higher, regarding the number of participating teams as well as the number of submitted runs. Several approaches were used by the teams, mostly deep learning techniques. Long short-term memory (LSTM) recurrent neural networks (RNN), adversarial auto-encoder, convolutional neural networks (CNN) image encoders and transfer learning-based multi-label classification models were the frequently used approaches. Evaluation uses F1-scores computed per image and averaged across all 10,000 test images

Deep Learning Tools for Yield and Price Forecasting Using Satellite Images

The ability to forecast crop yields and prices is vital to secure global food availability and provide farmers, retailers, and consumers with valuable information to maximize effectiveness. Conventional approaches used to tackle this often use localized methods that are expensive and limited in generalizability. To tackle some of these known issues and to benefit from recently developed advanced tools of machine learning, this thesis explores the use of deep learning models as well as satellite images to forecast various crop yields and prices across the USA. The special case of the USA was chosen given the abundance of datasets pertaining to weather and agricultural information. Moreover, the thesis explores Transfer Learning (TL) and incremental learning applications in the field for generalizability. In addition, a web application along with a user-friendly interface are designed and implemented to facilitate the ease of user application of the proposed models and approaches. Multiple machine learning models, specifically those based on artificial neural networks, are deployed and tested, along with several voting regressor ensembles. The models are tested using satellite images for California and the Midwest in USA to predict soybean yield and forecast strawberry and raspberry yield and price. Dimensionality reduction is applied by converting those satellite images into histograms that represent the pixel value frequency count. To gauge the performance of the deployed models, several evaluations metrics are used including Mean Absolute Error (MAE), Root Mean-Squared Error (RMSE), R-Squared Coefficient (R^2), as well as Aggregated Measure (AGM) and their Average Aggregated Measure (AAGM). The potential of using deep learning based models in real-life applications which provides crucial insight for all stakeholders in the field of agriculture is demonstrated in this work. The deployed multi-module based models and voting regressors ensembles proved to have higher performance compared to the single module models. The proposed CNN-LSTM is found to outperform Convolutional Neural Network (CNN) models proposed in the literature by an average RMSE percentage improvement of 31% while the inclusion of the satellite images of surface and subsurface moisture levels enhances the prediction performance. In addition, it is observed that all deployed models consistently lose forecasting performance the further they forecast in the future, with the CNN-LSTM Ensemble outperforming each of its components as well as the LSTM in yield forecasting while the CNN-LSTM outperforms the LSTM in price forecasting. Moreover, the proposed CNN-LSTM-SAE Ensemble outperforms the deployed CNN-LSTM, VAE, and SAE models including the literature CNN model by 70% AGM improvement for yield forecasting and 66% for price forecasting. The deployment of incremental learning with the CNN-LSTM Ensemble for yield forecasting without drastic loss in performance is achieved. Finally, based on the AGM metric, it is found that the TL CNN-LSTM outperforms the non-TL CNN-LSTM model by almost 28% AGM with reduction of 49% in computational time. For future work, there is potential in expanding the utilized datasets and models to verify and improve the obtained results as well as investigating the performance on additional fresh produce and counties to better gauge and enhance the effectiveness of the models and application

Learning with Scalability and Compactness

Artificial Intelligence has been thriving for decades since its birth. Traditional AI features heuristic search and planning, providing good strategy for tasks that are inherently search-based problems, such as games and GPS searching. In the meantime, machine learning, arguably the hottest subfield of AI, embraces data-driven methodology with great success in a wide range of applications such as computer vision and speech recognition. As a new trend, the applications of both learning and search have shifted toward mobile and embedded devices which entails not only scalability but also compactness of the models. Under this general paradigm, we propose a series of work to address the issues of scalability and compactness within machine learning and its applications on heuristic search. We first focus on the scalability issue of memory-based heuristic search which is recently ameliorated by Maximum Variance Unfolding (MVU), a manifold learning algorithm capable of learning state embeddings as effective heuristics to speed up $A^*$ search. Though achieving unprecedented online search performance with constraints on memory footprint, MVU is notoriously slow on offline training. To address this problem, we introduce Maximum Variance Correction (MVC), which finds large-scale feasible solutions to MVU by post-processing embeddings from any manifold learning algorithm. It increases the scale of MVU embeddings by several orders of magnitude and is naturally parallel. We further propose Goal-oriented Euclidean Heuristic (GOEH), a variant to MVU embeddings, which preferably optimizes the heuristics associated with goals in the embedding while maintaining their admissibility. We demonstrate unmatched reductions in search time across several non-trivial $A^*$ benchmark search problems. Through these work, we bridge the gap between the manifold learning literature and heuristic search which have been regarded as fundamentally different, leading to cross-fertilization for both fields. Deep learning has made a big splash in the machine learning community with its superior accuracy performance. However, it comes at a price of huge model size that might involves billions of parameters, which poses great challenges for its use on mobile and embedded devices. To achieve the compactness, we propose HashedNets, a general approach to compressing neural network models leveraging feature hashing. At its core, HashedNets randomly group parameters using a low-cost hash function, and share parameter value within the group. According to our empirical results, a neural network could be 32x smaller with little drop in accuracy performance. We further introduce Frequency-Sensitive Hashed Nets (FreshNets) to extend this hashing technique to convolutional neural network by compressing parameters in the frequency domain. Compared with many AI applications, neural networks seem not graining as much popularity as it should be in traditional data mining tasks. For these tasks, categorical features need to be first converted to numerical representation in advance in order for neural networks to process them. We show that a na\ {i}ve use of the classic one-hot encoding may result in gigantic weight matrices and therefore lead to prohibitively expensive memory cost in neural networks. Inspired by word embedding, we advocate a compellingly simple, yet effective neural network architecture with category embedding. It is capable of directly handling both numerical and categorical features as well as providing visual insights on feature similarities. At the end, we conduct comprehensive empirical evaluation which showcases the efficacy and practicality of our approach, and provides surprisingly good visualization and clustering for categorical features