Explicit Computation of Input Weights in Extreme Learning Machines

We present a closed form expression for initializing the input weights in a multi-layer perceptron, which can be used as the first step in synthesis of an Extreme Learning Ma-chine. The expression is based on the standard function for a separating hyperplane as computed in multilayer perceptrons and linear Support Vector Machines; that is, as a linear combination of input data samples. In the absence of supervised training for the input weights, random linear combinations of training data samples are used to project the input data to a higher dimensional hidden layer. The hidden layer weights are solved in the standard ELM fashion by computing the pseudoinverse of the hidden layer outputs and multiplying by the desired output values. All weights for this method can be computed in a single pass, and the resulting networks are more accurate and more consistent on some standard problems than regular ELM networks of the same size.Comment: In submission for the ELM 2014 Conferenc

Review of Advances in Neural Networks: Neural Design Technology Stack

This review provides a high-level synthesis of significant recent advances in artificial neural network research, as well as multi-disciplinary concepts connected to the far-reaching goal of obtaining intelligent systems. We assume that a global outlook of these interconnected fields can benefit researchers by providing alternative viewpoints. Therefore, we present different network and neuron models, we discuss model parameters and the means to obtain them, and we draw a quick outline of information encoding, before proceeding to an overview of the relevant learning mechanisms, ranging from established approaches to novel ideas. We specifically focus on comparing the classical artificial model with the biologically-feasible spiking neuron, and we take this comparison further into a discussion on the biological plausibility of various learning approaches