138,085 research outputs found
BlockDrop: Dynamic Inference Paths in Residual Networks
Very deep convolutional neural networks offer excellent recognition results,
yet their computational expense limits their impact for many real-world
applications. We introduce BlockDrop, an approach that learns to dynamically
choose which layers of a deep network to execute during inference so as to best
reduce total computation without degrading prediction accuracy. Exploiting the
robustness of Residual Networks (ResNets) to layer dropping, our framework
selects on-the-fly which residual blocks to evaluate for a given novel image.
In particular, given a pretrained ResNet, we train a policy network in an
associative reinforcement learning setting for the dual reward of utilizing a
minimal number of blocks while preserving recognition accuracy. We conduct
extensive experiments on CIFAR and ImageNet. The results provide strong
quantitative and qualitative evidence that these learned policies not only
accelerate inference but also encode meaningful visual information. Built upon
a ResNet-101 model, our method achieves a speedup of 20\% on average, going as
high as 36\% for some images, while maintaining the same 76.4\% top-1 accuracy
on ImageNet.Comment: CVPR 201
Unbounded Human Learning: Optimal Scheduling for Spaced Repetition
In the study of human learning, there is broad evidence that our ability to
retain information improves with repeated exposure and decays with delay since
last exposure. This plays a crucial role in the design of educational software,
leading to a trade-off between teaching new material and reviewing what has
already been taught. A common way to balance this trade-off is spaced
repetition, which uses periodic review of content to improve long-term
retention. Though spaced repetition is widely used in practice, e.g., in
electronic flashcard software, there is little formal understanding of the
design of these systems. Our paper addresses this gap in three ways. First, we
mine log data from spaced repetition software to establish the functional
dependence of retention on reinforcement and delay. Second, we use this memory
model to develop a stochastic model for spaced repetition systems. We propose a
queueing network model of the Leitner system for reviewing flashcards, along
with a heuristic approximation that admits a tractable optimization problem for
review scheduling. Finally, we empirically evaluate our queueing model through
a Mechanical Turk experiment, verifying a key qualitative prediction of our
model: the existence of a sharp phase transition in learning outcomes upon
increasing the rate of new item introductions.Comment: Accepted to the ACM SIGKDD Conference on Knowledge Discovery and Data
Mining 201
- …