Continual Learning with Deep Streaming Regularized Discriminant Analysis

Continual learning is increasingly sought after in real world machine learning applications, as it enables learning in a more human-like manner. Conventional machine learning approaches fail to achieve this, as incrementally updating the model with non-identically distributed data leads to catastrophic forgetting, where existing representations are overwritten. Although traditional continual learning methods have mostly focused on batch learning, which involves learning from large collections of labeled data sequentially, this approach is not well-suited for real-world applications where we would like new data to be integrated directly. This necessitates a paradigm shift towards streaming learning. In this paper, we propose a streaming version of regularized discriminant analysis as a solution to this challenge. We combine our algorithm with a convolutional neural network and demonstrate that it outperforms both batch learning and existing streaming learning algorithms on the ImageNet ILSVRC-2012 dataset

Feature Selection and Non-Euclidean Dimensionality Reduction: Application to Electrocardiology.

Heart disease has been the leading cause of human death for decades. To improve treatment of heart disease, algorithms to perform reliable computer diagnosis using electrocardiogram (ECG) data have become an area of active research. This thesis utilizes well-established methods from cluster analysis, classification, and localization to cluster and classify ECG data, and aims to help clinicians diagnose and treat heart diseases. The power of these methods is enhanced by state-of-the-art feature selection and dimensionality reduction. The specific contributions of this thesis are as follows. First, a unique combination of ECG feature selection and mixture model clustering is introduced to classify the sites of origin of ventricular tachycardias. Second, we apply a restricted Boltzmann machine (RBM) to learn sparse representations of ECG signals and to build an enriched classifier from patient data. Third, a novel manifold learning algorithm is introduced, called Quaternion Laplacian Information Maps (QLIM), and is applied to visualize high-dimensional ECG signals. These methods are applied to design of an automated supervised classification algorithm to help a physician identify the origin of ventricular arrhythmias (VA) directed from a patient's ECG data. The algorithm is trained on a large database of ECGs and catheter positions collected during the electrophysiology (EP) pace-mapping procedures. The proposed algorithm is demonstrated to have a correct classification rate of over 80% for the difficult task of classifying VAs having epicardial or endocardial origins.PhDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/113303/1/dyjung_1.pd

Towards Deep Learning Interpretability: A Topic Modeling Approach

The recent development of deep learning has achieved the state-of-the-art performance in various machine learning tasks. The IS research community has started to leveraged deep learning-based text mining for analyzing textual documents. The lack of interpretability is endemic among the state-of-the-art deep learning models, constraining model improvement, limiting additional insights, and prohibiting adoption. In this study, we propose a novel text mining research framework, Neural Topic Embedding, capable of extracting useful and interpretable representations of texts through deep neural networks. Specifically, we leverage topic modeling to enrich deep learning data representations with meaning. To demonstrate the effectiveness of our proposed framework, we conducted a preliminary evaluation experiment on a testbed of fake review detection and our interpretable representations improves the state-of-the-art by almost 8 percent as measured by F1 score. Our study contributes to the IS community by opening the gate for future adoption of the state-of-the-art deep learning methods

Ordering-sensitive and Semantic-aware Topic Modeling

Topic modeling of textual corpora is an important and challenging problem. In most previous work, the "bag-of-words" assumption is usually made which ignores the ordering of words. This assumption simplifies the computation, but it unrealistically loses the ordering information and the semantic of words in the context. In this paper, we present a Gaussian Mixture Neural Topic Model (GMNTM) which incorporates both the ordering of words and the semantic meaning of sentences into topic modeling. Specifically, we represent each topic as a cluster of multi-dimensional vectors and embed the corpus into a collection of vectors generated by the Gaussian mixture model. Each word is affected not only by its topic, but also by the embedding vector of its surrounding words and the context. The Gaussian mixture components and the topic of documents, sentences and words can be learnt jointly. Extensive experiments show that our model can learn better topics and more accurate word distributions for each topic. Quantitatively, comparing to state-of-the-art topic modeling approaches, GMNTM obtains significantly better performance in terms of perplexity, retrieval accuracy and classification accuracy.Comment: To appear in proceedings of AAAI 201

Topological Feature Based Classification

There has been a lot of interest in developing algorithms to extract clusters or communities from networks. This work proposes a method, based on blockmodelling, for leveraging communities and other topological features for use in a predictive classification task. Motivated by the issues faced by the field of community detection and inspired by recent advances in Bayesian topic modelling, the presented model automatically discovers topological features relevant to a given classification task. In this way, rather than attempting to identify some universal best set of clusters for an undefined goal, the aim is to find the best set of clusters for a particular purpose. Using this method, topological features can be validated and assessed within a given context by their predictive performance. The proposed model differs from other relational and semi-supervised learning models as it identifies topological features to explain the classification decision. In a demonstration on a number of real networks the predictive capability of the topological features are shown to rival the performance of content based relational learners. Additionally, the model is shown to outperform graph-based semi-supervised methods on directed and approximately bipartite networks.Comment: Awarded 3rd Best Student Paper at 14th International Conference on Information Fusion 201

LogisticLDA: Regularizing Latent Dirichlet Allocation by Logistic Regression

PACLIC 23 / City University of Hong Kong / 3-5 December 200