2 research outputs found
Node-By-Node Greedy Deep Learning for Interpretable Features
Multilayer networks have seen a resurgence under the umbrella of deep
learning. Current deep learning algorithms train the layers of the network
sequentially, improving algorithmic performance as well as providing some
regularization. We present a new training algorithm for deep networks which
trains \emph{each node in the network} sequentially. Our algorithm is orders of
magnitude faster, creates more interpretable internal representations at the
node level, while not sacrificing on the ultimate out-of-sample performance
Forward Thinking: Building and Training Neural Networks One Layer at a Time
We present a general framework for training deep neural networks without
backpropagation. This substantially decreases training time and also allows for
construction of deep networks with many sorts of learners, including networks
whose layers are defined by functions that are not easily differentiated, like
decision trees. The main idea is that layers can be trained one at a time, and
once they are trained, the input data are mapped forward through the layer to
create a new learning problem. The process is repeated, transforming the data
through multiple layers, one at a time, rendering a new data set, which is
expected to be better behaved, and on which a final output layer can achieve
good performance. We call this forward thinking and demonstrate a proof of
concept by achieving state-of-the-art accuracy on the MNIST dataset for
convolutional neural networks. We also provide a general mathematical
formulation of forward thinking that allows for other types of deep learning
problems to be considered