60,470 research outputs found
Semi-supervised Learning with the EM Algorithm: A Comparative Study between Unstructured and Structured Prediction
Semi-supervised learning aims to learn prediction models from both labeled
and unlabeled samples. There has been extensive research in this area. Among
existing work, generative mixture models with Expectation-Maximization (EM) is
a popular method due to clear statistical properties. However, existing
literature on EM-based semi-supervised learning largely focuses on unstructured
prediction, assuming that samples are independent and identically distributed.
Studies on EM-based semi-supervised approach in structured prediction is
limited. This paper aims to fill the gap through a comparative study between
unstructured and structured methods in EM-based semi-supervised learning.
Specifically, we compare their theoretical properties and find that both
methods can be considered as a generalization of self-training with soft class
assignment of unlabeled samples, but the structured method additionally
considers structural constraint in soft class assignment. We conducted a case
study on real-world flood mapping datasets to compare the two methods. Results
show that structured EM is more robust to class confusion caused by noise and
obstacles in features in the context of the flood mapping application
Network modeling of patients' biomolecular profiles for clinical phenotype/outcome prediction
Methods for phenotype and outcome prediction are largely based on inductive supervised models that use selected biomarkers to make predictions, without explicitly considering the functional relationships between individuals. We introduce a novel network-based approach named Patient-Net (P-Net) in which biomolecular profiles of patients are modeled in a graph-structured space that represents gene expression relationships between patients. Then a kernel-based semi-supervised transductive algorithm is applied to the graph to explore the overall topology of the graph and to predict the phenotype/clinical outcome of patients. Experimental tests involving several publicly available datasets of patients afflicted with pancreatic, breast, colon and colorectal cancer show that our proposed method is competitive with state-of-the-art supervised and semi-supervised predictive systems. Importantly, P-Net also provides interpretable models that can be easily visualized to gain clues about the relationships between patients, and to formulate hypotheses about their stratification
Bethe Projections for Non-Local Inference
Many inference problems in structured prediction are naturally solved by
augmenting a tractable dependency structure with complex, non-local auxiliary
objectives. This includes the mean field family of variational inference
algorithms, soft- or hard-constrained inference using Lagrangian relaxation or
linear programming, collective graphical models, and forms of semi-supervised
learning such as posterior regularization. We present a method to
discriminatively learn broad families of inference objectives, capturing
powerful non-local statistics of the latent variables, while maintaining
tractable and provably fast inference using non-Euclidean projected gradient
descent with a distance-generating function given by the Bethe entropy. We
demonstrate the performance and flexibility of our method by (1) extracting
structured citations from research papers by learning soft global constraints,
(2) achieving state-of-the-art results on a widely-used handwriting recognition
task using a novel learned non-convex inference procedure, and (3) providing a
fast and highly scalable algorithm for the challenging problem of inference in
a collective graphical model applied to bird migration.Comment: minor bug fix to appendix. appeared in UAI 201
Learning to Make Predictions on Graphs with Autoencoders
We examine two fundamental tasks associated with graph representation
learning: link prediction and semi-supervised node classification. We present a
novel autoencoder architecture capable of learning a joint representation of
both local graph structure and available node features for the multi-task
learning of link prediction and node classification. Our autoencoder
architecture is efficiently trained end-to-end in a single learning stage to
simultaneously perform link prediction and node classification, whereas
previous related methods require multiple training steps that are difficult to
optimize. We provide a comprehensive empirical evaluation of our models on nine
benchmark graph-structured datasets and demonstrate significant improvement
over related methods for graph representation learning. Reference code and data
are available at https://github.com/vuptran/graph-representation-learningComment: Published as a conference paper at IEEE DSAA 201
Well-M³N: A Maximum-Margin Approach to Unsupervised Structured Prediction
Unsupervised structured prediction is of fundamental importance for the clustering and classification of unannotated structured data. To date, its most common approach still relies on the use of structural probabilistic models and the expectation-maximization (EM) algorithm. Conversely, structural maximum-margin approaches, despite their extensive success in supervised and semi-supervised classification, have not raised equivalent attention in the unsupervised case. For this reason, in this paper we propose a novel approach that extends the maximum-margin Markov networks (M3N) to an unsupervised training framework. The main contributions of our extension are new formulations for the feature map and loss function of M3N that decouple the labels from the measurements and support multiple ground-truth training. Experiments on two challenging segmentation datasets have achieved competitive accuracy and generalization compared to other unsupervised algorithms such as k-means, EM and unsupervised structural SVM, and comparable performance to a contemporary deep learning-based approach
- …