43,664 research outputs found
Symmetry constrained machine learning
Symmetry, a central concept in understanding the laws of nature, has been
used for centuries in physics, mathematics, and chemistry, to help make
mathematical models tractable. Yet, despite its power, symmetry has not been
used extensively in machine learning, until rather recently. In this article we
show a general way to incorporate symmetries into machine learning models. We
demonstrate this with a detailed analysis on a rather simple real world machine
learning system - a neural network for classifying handwritten digits, lacking
bias terms for every neuron. We demonstrate that ignoring symmetries can have
dire over-fitting consequences, and that incorporating symmetry into the model
reduces over-fitting, while at the same time reducing complexity, ultimately
requiring less training data, and taking less time and resources to train
Noisy regression and classification with continuous multilayer networks
We investigate zero temperature Gibbs learning for two classes of
unrealizable rules which play an important role in practical applications of
multilayer neural networks with differentiable activation functions:
classification problems and noisy regression problems. Considering one step of
replica symmetry breaking, we surprisingly find that for sufficiently large
training sets the stable state is replica symmetric even though the target rule
is unrealizable. Further, the classification problem is shown to be formally
equivalent to the noisy regression problem.Comment: 7 pages, including 2 figure
- …