Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment

We present a deep neural network-based approach to image quality assessment (IQA). The network is trained end-to-end and comprises ten convolutional layers and five pooling layers for feature extraction, and two fully connected layers for regression, which makes it significantly deeper than related IQA models. Unique features of the proposed architecture are that: 1) with slight adaptations it can be used in a no-reference (NR) as well as in a full-reference (FR) IQA setting and 2) it allows for joint learning of local quality and local weights, i.e., relative importance of local quality to the global quality estimate, in an unified framework. Our approach is purely data-driven and does not rely on hand-crafted features or other types of prior domain knowledge about the human visual system or image statistics. We evaluate the proposed approach on the LIVE, CISQ, and TID2013 databases as well as the LIVE In the wild image quality challenge database and show superior performance to state-of-the-art NR and FR IQA methods. Finally, cross-database evaluation shows a high ability to generalize between different databases, indicating a high robustness of the learned features

Bayesian Importance of Features (BIF)

We introduce a simple and intuitive framework that provides quantitative explanations of statistical models through the probabilistic assessment of input feature importance. The core idea comes from utilizing the Dirichlet distribution to define the importance of input features and learning it via approximate Bayesian inference. The learned importance has probabilistic interpretation and provides the relative significance of each input feature to a model's output, additionally assessing confidence about its importance quantification. As a consequence of using the Dirichlet distribution over the explanations, we can define a closed-form divergence to gauge the similarity between learned importance under different models. We use this divergence to study the feature importance explainability tradeoffs with essential notions in modern machine learning, such as privacy and fairness. Furthermore, BIF can work on two levels: global explanation (feature importance across all data instances) and local explanation (individual feature importance for each data instance). We show the effectiveness of our method on a variety of synthetic and real datasets, taking into account both tabular and image datasets. The code is available at https://github.com/kamadforge/featimp_dp

Insights and approaches using deep learning to classify wildlife.

The implementation of intelligent software to identify and classify objects and individuals in visual fields is a technology of growing importance to operatives in many fields, including wildlife conservation and management. To non-experts, the methods can be abstruse and the results mystifying. Here, in the context of applying cutting edge methods to classify wildlife species from camera-trap data, we shed light on the methods themselves and types of features these methods extract to make efficient identifications and reliable classifications. The current state of the art is to employ convolutional neural networks (CNN) encoded within deep-learning algorithms. We outline these methods and present results obtained in training a CNN to classify 20 African wildlife species with an overall accuracy of 87.5% from a dataset containing 111,467 images. We demonstrate the application of a gradient-weighted class-activation-mapping (Grad-CAM) procedure to extract the most salient pixels in the final convolution layer. We show that these pixels highlight features in particular images that in some cases are similar to those used to train humans to identify these species. Further, we used mutual information methods to identify the neurons in the final convolution layer that consistently respond most strongly across a set of images of one particular species. We then interpret the features in the image where the strongest responses occur, and present dataset biases that were revealed by these extracted features. We also used hierarchical clustering of feature vectors (i.e., the state of the final fully-connected layer in the CNN) associated with each image to produce a visual similarity dendrogram of identified species. Finally, we evaluated the relative unfamiliarity of images that were not part of the training set when these images were one of the 20 species "known" to our CNN in contrast to images of the species that were "unknown" to our CNN

Characterizing bladder cancer cells by comparing general machine learning methods to convolutional neural network

Recently, deep learning techniques from the computer science field have dramatically improved the ability of computers to recognize objects in images. This raised the possibility of fully automated computer-aided diagnosis in the medical field. Among all the machine learning models, convolutional neural network (CNN) is one of the most studied and validated artificial neural networks in image recognition. Not only that it has great performance, but the design of most modern CNN hidden layers also allows the model to extract meaningful features without the needs of prior knowledge. Thus, the pathology community is showing increasing interests in comparing CNN to human judgments. As demonstrated in a number of studies reporting various image analysis models that can accurately localize and characterize cells into different cell types and predict patient outcome, the pathological field is incorporating artificial intelligence technologies into their diagnosis. Although using the deep neural network on recognizing pathological slides is not a new idea and is showing promising results, its requirement of a large quantity of data for training can be a big obstacle for many unpopular histopathological cases. In the bladder cancer field, the Tumor-Node-Metastasis (TNM) system defines T1 bladder cancer as the invasion of tumor cells into the lamina propria (LP). However, pathologists often struggle to confirm LP and/or muscularis mucosae invasion using hematoxylin & eosin (H&E) stains from bladder biopsies. Accurately reporting the presence of tumor invasion, which is associated with worse clinical outcomes, is critical for adequate patient management. In this thesis, we have developed various traditional machine learning models and compared their performances to 2 convolutional neural networks (CNN), VGG16 and VGG19, on histology image classification in distinguishing non-invasive versus invasive bladder tumors. By using approximately 1,200 H&E images from non-invasive and invasive bladder cancer tissues, our results showed the traditional machine learning methods with the human-directed features outperformed the fully automated CNN model as much as 12%. For 2-class classification task to distinguish non-invasive and invasive bladder cancer tissues, we achieved around 91~96% accuracy by using classic machine learning classifiers such as random forest, logistic regression, and probabilistic neural network. Whereas, CNN with VGG16 as hidden layers only achieved around 84%. In addition to performance, because of the transparency of features extraction in the pipeline, we were able to evaluate and rank the patterns in the bladder histological images. As based on their relative importance in prediction, classic machine learning methods provided a well-rounded approach under limited data size

Interpretation of Neural Networks is Fragile

In order for machine learning to be deployed and trusted in many applications, it is crucial to be able to reliably explain why the machine learning algorithm makes certain predictions. For example, if an algorithm classifies a given pathology image to be a malignant tumor, then the doctor may need to know which parts of the image led the algorithm to this classification. How to interpret black-box predictors is thus an important and active area of research. A fundamental question is: how much can we trust the interpretation itself? In this paper, we show that interpretation of deep learning predictions is extremely fragile in the following sense: two perceptively indistinguishable inputs with the same predicted label can be assigned very different interpretations. We systematically characterize the fragility of several widely-used feature-importance interpretation methods (saliency maps, relevance propagation, and DeepLIFT) on ImageNet and CIFAR-10. Our experiments show that even small random perturbation can change the feature importance and new systematic perturbations can lead to dramatically different interpretations without changing the label. We extend these results to show that interpretations based on exemplars (e.g. influence functions) are similarly fragile. Our analysis of the geometry of the Hessian matrix gives insight on why fragility could be a fundamental challenge to the current interpretation approaches.Comment: Published as a conference paper at AAAI 201