1,192 research outputs found
Inconsistency, Instability, and Generalization Gap of Deep Neural Network Training
As deep neural networks are highly expressive, it is important to find
solutions with small generalization gap (the difference between the performance
on the training data and unseen data). Focusing on the stochastic nature of
training, we first present a theoretical analysis in which the bound of
generalization gap depends on what we call inconsistency and instability of
model outputs, which can be estimated on unlabeled data. Our empirical study
based on this analysis shows that instability and inconsistency are strongly
predictive of generalization gap in various settings. In particular, our
finding indicates that inconsistency is a more reliable indicator of
generalization gap than the sharpness of the loss landscape. Furthermore, we
show that algorithmic reduction of inconsistency leads to superior performance.
The results also provide a theoretical basis for existing methods such as
co-distillation and ensemble
Pseudorehearsal in value function approximation
Catastrophic forgetting is of special importance in reinforcement learning,
as the data distribution is generally non-stationary over time. We study and
compare several pseudorehearsal approaches for Q-learning with function
approximation in a pole balancing task. We have found that pseudorehearsal
seems to assist learning even in such very simple problems, given proper
initialization of the rehearsal parameters
- …