Data Dropout: Optimizing Training Data for Convolutional Neural Networks

Deep learning models learn to fit training data while they are highly expected to generalize well to testing data. Most works aim at finding such models by creatively designing architectures and fine-tuning parameters. To adapt to particular tasks, hand-crafted information such as image prior has also been incorporated into end-to-end learning. However, very little progress has been made on investigating how an individual training sample will influence the generalization ability of a model. In other words, to achieve high generalization accuracy, do we really need all the samples in a training dataset? In this paper, we demonstrate that deep learning models such as convolutional neural networks may not favor all training samples, and generalization accuracy can be further improved by dropping those unfavorable samples. Specifically, the influence of removing a training sample is quantifiable, and we propose a Two-Round Training approach, aiming to achieve higher generalization accuracy. We locate unfavorable samples after the first round of training, and then retrain the model from scratch with the reduced training dataset in the second round. Since our approach is essentially different from fine-tuning or further training, the computational cost should not be a concern. Our extensive experimental results indicate that, with identical settings, the proposed approach can boost performance of the well-known networks on both high-level computer vision problems such as image classification, and low-level vision problems such as image denoising

Two Polyak-Type Step Sizes for Mirror Descent

We propose two Polyak-type step sizes for mirror descent and prove their convergences for minimizing convex locally Lipschitz functions. Both step sizes, unlike the original Polyak step size, do not need the optimal value of the objective function.Comment: 13 page

On the pointwise domination of a function by its maximal function

We show that under rather general circumstances, the almost everywhere pointwise inequality $|f|(x) \le Mf (x)$ is equivalent to a weak form of the Lebesgue density theorem, for totally bounded closed sets. We derive both positive and negative results from this characterization.Comment: 13 pages, incorporates suggestions by a refere

Steered Molecular Dynamics Simulations Reveal the Likelier Dissociation Pathway of Imatinib from Its Targeting Kinases c-Kit and Abl

Development of small molecular kinase inhibitors has recently been the central focus in drug discovery. And type II kinase inhibitors that target inactive conformation of kinases have attracted particular attention since their potency and selectivity are thought to be easier to achieve compared with their counterpart type I inhibitors that target active conformation of kinases. Although mechanisms underlying the interactions between type II inhibitors and their targeting kinases have been widely studied, there are still some challenging problems, for example, how type II inhibitors associate with or dissociate from their targeting kinases. In this investigation, steered molecular dynamics simulations have been carried out to explore the possible dissociation pathways of typical type II inhibitor imatinib from its targeting protein kinases c-Kit and Abl. The simulation results indicate that the most favorable pathway for imatinib dissociation corresponds to the ATP-channel rather than the relatively wider allosteric-pocket-channel, which is mainly due to the different van der Waals interaction that the ligand suffers during dissociation. Nevertheless, the direct reason comes from the fact that the residues composing the ATP-channel are more flexible than that forming the allosteric-pocket-channel. The present investigation suggests that a bulky hydrophobic head is unfavorable, but a large polar tail is allowed for a potent type II inhibitor. The information obtained here can be used to direct the discovery of type II kinase inhibitors