Training for Fast Sequential Prediction Using Dynamic Feature Selection

We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components. This is accomplished by partitioning the features into a sequence of templates which are ordered such that high confidence can often be reached using only a small fraction of all features. Parameter estimation is arranged to maximize accuracy and early confidence in this sequence. We present experiments in left-to-right part-of-speech tagging on WSJ, demonstrating that we can preserve accuracy above 97% with over a five-fold reduction in run-time.Comment: 5 pages, NIPS Modern ML + NLP Workshop 201

Learning Dynamic Feature Selection for Fast Sequential Prediction

We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components. This is accomplished by partitioning the features into a sequence of templates which are ordered such that high confidence can often be reached using only a small fraction of all features. Parameter estimation is arranged to maximize accuracy and early confidence in this sequence. Our approach is simpler and better suited to NLP than other related cascade methods. We present experiments in left-to-right part-of-speech tagging, named entity recognition, and transition-based dependency parsing. On the typical benchmarking datasets we can preserve POS tagging accuracy above 97% and parsing LAS above 88.5% both with over a five-fold reduction in run-time, and NER F1 above 88 with more than 2x increase in speed.Comment: Appears in The 53rd Annual Meeting of the Association for Computational Linguistics, Beijing, China, July 201

Application of EOS-ELM with binary Jaya-based feature selection to real-time transient stability assessment using PMU data

Recent studies show that pattern-recognition-based transient stability assessment (PRTSA) is a promising approach for predicting the transient stability status of power systems. However, many of the current well-known PRTSA methods suffer from excessive training time and complex tuning of parameters, resulting in inefficiency for real-time implementation and lacking the online model updating ability. In this paper, a novel PRTSA approach based on an ensemble of OS-extreme learning machine (EOSELM) with binary Jaya (BinJaya)-based feature selection is proposed with the use of phasor measurement units (PMUs) data. After briefly describing the principles of OS-ELM, an EOS-ELM-based PRTSA model is built to predict the post-fault transient stability status of power systems in real time by integrating OS-ELM and an online boosting algorithm, respectively, as a weak classifier and an ensemble learning algorithm. Furthermore, a BinJaya-based feature selection approach is put forward for selecting an optimal feature subset from the entire feature space constituted by a group of system-level classification features extracted from PMU data. The application results on the IEEE 39-bus system and a real provincial system show that the proposal has superior computation speed and prediction accuracy than other state-of-the-art sequential learning algorithms. In addition, without sacrificing the classification performance, the dimension of the input space has been reduced to about one-third of its initial value.Comment: Accepted by IEEE Acces