Guided Dropout

Dropout is often used in deep neural networks to prevent over-fitting. Conventionally, dropout training invokes \textit{random drop} of nodes from the hidden layers of a Neural Network. It is our hypothesis that a guided selection of nodes for intelligent dropout can lead to better generalization as compared to the traditional dropout. In this research, we propose "guided dropout" for training deep neural network which drop nodes by measuring the strength of each node. We also demonstrate that conventional dropout is a specific case of the proposed guided dropout. Experimental evaluation on multiple datasets including MNIST, CIFAR10, CIFAR100, SVHN, and Tiny ImageNet demonstrate the efficacy of the proposed guided dropout.Comment: Accepted in AAAI201

Excitation Dropout: Encouraging Plasticity in Deep Neural Networks

We propose a guided dropout regularizer for deep networks based on the evidence of a network prediction defined as the firing of neurons in specific paths. In this work, we utilize the evidence at each neuron to determine the probability of dropout, rather than dropping out neurons uniformly at random as in standard dropout. In essence, we dropout with higher probability those neurons which contribute more to decision making at training time. This approach penalizes high saliency neurons that are most relevant for model prediction, i.e. those having stronger evidence. By dropping such high-saliency neurons, the network is forced to learn alternative paths in order to maintain loss minimization, resulting in a plasticity-like behavior, a characteristic of human brains too. We demonstrate better generalization ability, an increased utilization of network neurons, and a higher resilience to network compression using several metrics over four image/video recognition benchmarks

Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’ meta-analysis

Background. It is well known that web-based interventions can be effective treatments for depression. However, dropout rates in web-based interventions are typically high, especially in self-guided web-based interventions. Rigorous empirical evidence regarding factors influencing dropout in self-guided web-based interventions is lacking due to small study sample sizes. In this paper we examined predictors of dropout in an individual patient data meta-analysis to gain a better understanding of who may benefit from these interventions. Method. A comprehensive literature search for all randomized controlled trials (RCTs) of psychotherapy for adults with depression from 2006 to January 2013 was conducted. Next, we approached authors to collect the primary data of the selected studies. Predictors of dropout, such as socio-demographic, clinical, and intervention characteristics were examined. Results. Data from 2705 participants across ten RCTs of self-guided web-based interventions for depression were analysed. The multivariate analysis indicated that male gender [relative risk (RR) 1.08], lower educational level (primary education, RR 1.26) and co-morbid anxiety symptoms (RR 1.18) significantly increased the risk of dropping out, while for every additional 4 years of age, the risk of dropping out significantly decreased (RR 0.94). Conclusions. Dropout can be predicted by several variables and is not randomly distributed. This knowledge may inform tailoring of online self-help interventions to prevent dropout in identified groups at ris

Psychotherapy dropout : the influence of ethnic identity and stigma on early termination.

Although clients continue to drop out of psychotherapy, researchers have made few inroads into understanding the dropout phenomenon. Clinical studies have reported client dropout rates based on demographic factors (e.g., race, socioeconomic status, educational status). However, few studies have investigated the cultural correlates that may underlie these demographic factors and affect client termination. Furthermore, none have provided an empirically driven explanation as to why some clients drop out more often than others. In two studies, I explore dropout in two settings: a community mental health center and a university counseling center. Guided by current theory and research, I will examine the association between clients’ ethnic identity, the stigma they experience, and their decision to terminate therapy. Through logistic regression, I will seek to explore these cultural influences on the dropout process. It is expected that the study will contribute to current research and assist practitioners in preventing dropout

Guided Proofreading of Automatic Segmentations for Connectomics

Automatic cell image segmentation methods in connectomics produce merge and split errors, which require correction through proofreading. Previous research has identified the visual search for these errors as the bottleneck in interactive proofreading. To aid error correction, we develop two classifiers that automatically recommend candidate merges and splits to the user. These classifiers use a convolutional neural network (CNN) that has been trained with errors in automatic segmentations against expert-labeled ground truth. Our classifiers detect potentially-erroneous regions by considering a large context region around a segmentation boundary. Corrections can then be performed by a user with yes/no decisions, which reduces variation of information 7.5x faster than previous proofreading methods. We also present a fully-automatic mode that uses a probability threshold to make merge/split decisions. Extensive experiments using the automatic approach and comparing performance of novice and expert users demonstrate that our method performs favorably against state-of-the-art proofreading methods on different connectomics datasets.Comment: Supplemental material available at http://rhoana.org/guidedproofreading/supplemental.pd