7,373 research outputs found
Model and Feature Selection in Hidden Conditional Random Fields with Group Regularization
Proceedings of: 8th International Conference on Hybrid Artificial Intelligence Systems (HAIS 2013). Salamanca, September 11-13, 2013.Sequence classification is an important problem in computer vision, speech analysis or computational biology. This paper presents a new training strategy for the Hidden Conditional Random Field sequence classifier incorporating model and feature selection. The standard Lasso regularization employed in the estimation of model parameters is replaced by overlapping group-L1 regularization. Depending on the configuration of the overlapping groups, model selection, feature selection,or both are performed. The sequence classifiers trained in this way have better predictive performance. The application of the proposed method in a human action recognition task confirms that fact.This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485)Publicad
Efficient Learning of Sparse Conditional Random Fields for Supervised Sequence Labelling
Conditional Random Fields (CRFs) constitute a popular and efficient approach
for supervised sequence labelling. CRFs can cope with large description spaces
and can integrate some form of structural dependency between labels. In this
contribution, we address the issue of efficient feature selection for CRFs
based on imposing sparsity through an L1 penalty. We first show how sparsity of
the parameter set can be exploited to significantly speed up training and
labelling. We then introduce coordinate descent parameter update schemes for
CRFs with L1 regularization. We finally provide some empirical comparisons of
the proposed approach with state-of-the-art CRF training strategies. In
particular, it is shown that the proposed approach is able to take profit of
the sparsity to speed up processing and hence potentially handle larger
dimensional models
Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior
In recent years a number of methods have been developed for automatically
learning the (sparse) connectivity structure of Markov Random Fields. These
methods are mostly based on L1-regularized optimization which has a number of
disadvantages such as the inability to assess model uncertainty and expensive
cross-validation to find the optimal regularization parameter. Moreover, the
model's predictive performance may degrade dramatically with a suboptimal value
of the regularization parameter (which is sometimes desirable to induce
sparseness). We propose a fully Bayesian approach based on a "spike and slab"
prior (similar to L0 regularization) that does not suffer from these
shortcomings. We develop an approximate MCMC method combining Langevin dynamics
and reversible jump MCMC to conduct inference in this model. Experiments show
that the proposed model learns a good combination of the structure and
parameter values without the need for separate hyper-parameter tuning.
Moreover, the model's predictive performance is much more robust than L1-based
methods with hyper-parameter settings that induce highly sparse model
structures.Comment: Accepted in the Conference on Uncertainty in Artificial Intelligence
(UAI), 201
Neural networks versus Logistic regression for 30 days all-cause readmission prediction
Heart failure (HF) is one of the leading causes of hospital admissions in the
US. Readmission within 30 days after a HF hospitalization is both a recognized
indicator for disease progression and a source of considerable financial burden
to the healthcare system. Consequently, the identification of patients at risk
for readmission is a key step in improving disease management and patient
outcome. In this work, we used a large administrative claims dataset to
(1)explore the systematic application of neural network-based models versus
logistic regression for predicting 30 days all-cause readmission after
discharge from a HF admission, and (2)to examine the additive value of
patients' hospitalization timelines on prediction performance. Based on data
from 272,778 (49% female) patients with a mean (SD) age of 73 years (14) and
343,328 HF admissions (67% of total admissions), we trained and tested our
predictive readmission models following a stratified 5-fold cross-validation
scheme. Among the deep learning approaches, a recurrent neural network (RNN)
combined with conditional random fields (CRF) model (RNNCRF) achieved the best
performance in readmission prediction with 0.642 AUC (95% CI, 0.640-0.645).
Other models, such as those based on RNN, convolutional neural networks and CRF
alone had lower performance, with a non-timeline based model (MLP) performing
worst. A competitive model based on logistic regression with LASSO achieved a
performance of 0.643 AUC (95%CI, 0.640-0.646). We conclude that data from
patient timelines improve 30 day readmission prediction for neural
network-based models, that a logistic regression with LASSO has equal
performance to the best neural network model and that the use of administrative
data result in competitive performance compared to published approaches based
on richer clinical datasets
- …