3 research outputs found
Multi-Activation Hidden Units for Neural Networks with Random Weights
Single layer feedforward networks with random weights are successful in a
variety of classification and regression problems. These networks are known for
their non-iterative and fast training algorithms. A major drawback of these
networks is that they require a large number of hidden units. In this paper, we
propose the use of multi-activation hidden units. Such units increase the
number of tunable parameters and enable formation of complex decision surfaces,
without increasing the number of hidden units. We experimentally show that
multi-activation hidden units can be used either to improve the classification
accuracy, or to reduce computations.Comment: 4 pages, 4 figures. arXiv admin note: substantial text overlap with
arXiv:2008.1042
Efficient Design of Neural Networks with Random Weights
Single layer feedforward networks with random weights are known for their
non-iterative and fast training algorithms and are successful in a variety of
classification and regression problems. A major drawback of these networks is
that they require a large number of hidden units. In this paper, we propose a
technique to reduce the number of hidden units substantially without affecting
the accuracy of the networks significantly. We introduce the concept of primary
and secondary hidden units. The weights for the primary hidden units are chosen
randomly while the secondary hidden units are derived using pairwise
combinations of the primary hidden units. Using this technique, we show that
the number of hidden units can be reduced by at least one order of magnitude.
We experimentally show that this technique leads to significant drop in
computations at inference time and has only a minor impact on network accuracy.
A huge reduction in computations is possible if slightly lower accuracy is
acceptable.Comment: 5 pages, 8 figure
Evolutionary Cost-sensitive Extreme Learning Machine
Conventional extreme learning machines solve a Moore-Penrose generalized
inverse of hidden layer activated matrix and analytically determine the output
weights to achieve generalized performance, by assuming the same loss from
different types of misclassification. The assumption may not hold in
cost-sensitive recognition tasks, such as face recognition based access control
system, where misclassifying a stranger as a family member may result in more
serious disaster than misclassifying a family member as a stranger. Though
recent cost-sensitive learning can reduce the total loss with a given cost
matrix that quantifies how severe one type of mistake against another, in many
realistic cases the cost matrix is unknown to users. Motivated by these
concerns, this paper proposes an evolutionary cost-sensitive extreme learning
machine (ECSELM), with the following merits: 1) to our best knowledge, it is
the first proposal of ELM in evolutionary cost-sensitive classification
scenario; 2) it well addresses the open issue of how to define the cost matrix
in cost-sensitive learning tasks; 3) an evolutionary backtracking search
algorithm is induced for adaptive cost matrix optimization. Experiments in a
variety of cost-sensitive tasks well demonstrate the effectiveness of the
proposed approaches, with about 5%~10% improvements.Comment: This paper has been accepted for publication in IEEE Transactions on
Neural Networks and Learning System