Self-organizing cooperative neural network experts

Neural networks are generally considered as function approximation models that map a set of input features to their target outputs. Their approximation capability can be improved through “ensemble learning”. An ensemble of neural networks decreases the error correlation of the group by having each network in the ensemble compensate for the performance of one another. One ensembling technique is the Mixture-of-Experts model, which consists of a set of independently-trained expert neural networks that specialize on their own subset of the dataset, and a gating network that manages the specialization of the expert neural networks. In this model, all the neural networks are trained concurrently, but the expert neural networks are only trained on cases in which they perform well. Some major components of the proposed architecture for this thesis are the Cooperative Ensemble, which trains its neural networks concurrently instead of independently, and the k-Winners-Take-All activation function to drive the specialization among neural network experts on a subset of the input features. This way, there is no longer a need for a centralized gating network to manage the specialization of the neural network experts. We further improve upon the k-Winners-Take-All ensemble neural network by training another neural network with the designated task of learning useful feature representations for the neural networks in the ensemble. To learn such representations, the neural network uses the Soft Nearest Neighbor Loss which engenders a simpler function approximation task for the neural networks in the ensemble. We call the resulting full architecture “Self-Organizing Cooperative Neural Network Experts” (SOCONNE), in which a set of neural networks gain the right to specialize on their own subsets of the dataset without the use of a centralized gating neural network. Numerous experiments on a variety of test datasets show that the novel architecture (1) takes advantage of the learned representations for the set of input features by learning their underlying structure, and (2) uses these learned representations to simplify the task of the neural networks in a cooperative ensemble set-up

Use of word and character N-grams for low-resourced local languages

Language identification is a text classification task for identifying the language of a given text. Several works use this as a preprocessing technique prior to sentiment analysis, mood analysis, and named entity recognition among others. Thus, building an accurate language identification engine is important given that the Philippines is home to more than 170 languages, and is scarce of language documents and resources. We compare machine learning algorithms such as Naive Bayes, Linear Support Vector Machines (SVM), and Random Forest for classification of Philippine languages. Results show that the Linear SVM model had the best performance with 0.97 Fl-score. © 2018 IEEE