4,963 research outputs found
Dissecting Deep Learning Networks—Visualizing Mutual Information
Deep Learning (DL) networks are recent revolutionary developments in artificial intelligence research. Typical networks are stacked by groups of layers that are further composed of many convolutional kernels or neurons. In network design, many hyper-parameters need to be defined heuristically before training in order to achieve high cross-validation accuracies. However, accuracy evaluation from the output layer alone is not sufficient to specify the roles of the hidden units in associated networks. This results in a significant knowledge gap between DL’s wider applications and its limited theoretical understanding. To narrow the knowledge gap, our study explores visualization techniques to illustrate the mutual information (MI) in DL networks. The MI is a theoretical measurement, reflecting the relationship between two sets of random variables even if their relationship is highly non-linear and hidden in high-dimensional data. Our study aims to understand the roles of DL units in classification performance of the networks. Via a series of experiments using several popular DL networks, it shows that the visualization of MI and its change patterns between the input/output with the hidden layers and basic units can facilitate a better understanding of these DL units’ roles. Our investigation on network convergence suggests a more objective manner to potentially evaluate DL networks. Furthermore, the visualization provides a useful tool to gain insights into the network performance, and thus to potentially facilitate the design of better network architectures by identifying redundancy and less-effective network units
Recommended from our members
MELANOMA DETECTION BASED ON DEEP LEARNING NETWORKS
Our main objective is to develop a method for identifying melanoma enabling accurate assessments of patient’s health. Skin cancer, such as melanoma can be extremely dangerous if not detected and treated early. Detecting skin cancer accurately and promptly can greatly increase the chances of survival. To achieve this, it is important to develop a computer-aided diagnostic support system. In this study a research team introduces a sophisticated transfer learning model that utilizes Resnet50 to classify melanoma. Transfer learning is a machine learning technique that takes advantage of trained models, for similar tasks resulting in time saving and enhanced accuracy by avoiding the need to train from scratch. The Resnet50 is a type of network that can distinguish between cancerous skin lesions in each sample. To evaluate its performance, we used data from the melanoma cancer dataset. However, the dataset has a percentage of samples which creates an imbalance between the classes. We addressed this issue by making the dataset more diverse through data augmentation techniques. In our project we implemented the Resnet50 pretrained model with learning rates and weight decay. This model consists of 50 layers organized into blocks that include batch normalization and skip connections (known as connections). We adjusted the depth of the model to improve its accuracy. Our experimental results demonstrate that our proposed deep learning technique performs better in terms of accuracy compared to state of the art algorithms in this field. iii
The model achieves an accuracy of 91.70%, with a learning rate of 0.0001 and a model depth of 34. By tuning hyperparameters using RESNET 50 we can further enhance the accuracy of our trained models
- …