Conditional Random Field Autoencoders for Unsupervised Structured Prediction

We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observable data using a feature-rich conditional random field. Then a reconstruction of the input is (re)generated, conditional on the latent structure, using models for which maximum likelihood estimation has a closed-form. Our autoencoder formulation enables efficient learning without making unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. We show competitive results with instantiations of the model for two canonical NLP tasks: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines

On The Smoothness of Cross-Validation-Based Estimators Of Classifier Performance

Many versions of cross-validation (CV) exist in the literature; and each version though has different variants. All are used interchangeably by many practitioners; yet, without explanation to the connection or difference among them. This article has three contributions. First, it starts by mathematical formalization of these different versions and variants that estimate the error rate and the Area Under the ROC Curve (AUC) of a classification rule, to show the connection and difference among them. Second, we prove some of their properties and prove that many variants are either redundant or "not smooth". Hence, we suggest to abandon all redundant versions and variants and only keep the leave-one-out, the $K$-fold, and the repeated $K$-fold. We show that the latter is the only among the three versions that is "smooth" and hence looks mathematically like estimating the mean performance of the classification rules. However, empirically, for the known phenomenon of "weak correlation", which we explain mathematically and experimentally, it estimates both conditional and mean performance almost with the same accuracy. Third, we conclude the article with suggesting two research points that may answer the remaining question of whether we can come up with a finalist among the three estimators: (1) a comparative study, that is much more comprehensive than those available in literature and conclude no overall winner, is needed to consider a wide range of distributions, datasets, and classifiers including complex ones obtained via the recent deep learning approach. (2) we sketch the path of deriving a rigorous method for estimating the variance of the only "smooth" version, repeated $K$-fold CV, rather than those ad-hoc methods available in the literature that ignore the covariance structure among the folds of CV.Comment: The paper is currently under review in Pattern Recognition Letters (PRL

A Review of Statistical Learning Machines from ATR to DNA Microarrays: design, assessment, and advice for practitioners

Statistical Learning is the process of estimating an unknown probabilistic input-output relationship of a system using a limited number of observations; and a statistical learning machine (SLM) is the machine that learned such a process. While their roots grow deeply in Probability Theory, SLMs are ubiquitous in the modern world. Automatic Target Recognition (ATR) in military applications, Computer Aided Diagnosis (CAD) in medical imaging, DNA microarrays in Genomics, Optical Character Recognition (OCR), Speech Recognition (SR), spam email filtering, stock market prediction, etc., are few examples and applications for SLM; diverse fields but one theory. The field of Statistical Learning can be decomposed to two basic subfields, Design and Assessment. Three main groups of specializations-namely statisticians, engineers, and computer scientists (ordered ascendingly by programming capabilities and descendingly by mathematical rigor)-exist on the venue of this field and each takes its elephant bite. Exaggerated rigorous analysis of statisticians sometimes deprives them from considering new ML techniques and methods that, yet, have no "complete" mathematical theory. On the other hand, immoderate add-hoc simulations of computer scientists sometimes derive them towards unjustified and immature results. A prudent approach is needed that has the enough flexibility to utilize simulations and trials and errors without sacrificing any rigor. If this prudent attitude is necessary for this field it is necessary, as well, in other fields of Engineering.Comment: This manuscript was composed in 2006 as part of a the author's Ph.D. dissertatio

Ligand reduction in variously substituted cerium (IV) tetrakis acetylacetone complexes by electrochemistry technique

In this work, substantial evidence was obtained for ligand reduction in cerium tetrakis acac complexes. Also, this ligand reduction of a negatively charged ligand proved to depend far less on the nature central metal than neutral ligands does. It is supposed that in Mz(acac)z complexes the charge is distributed evenly over the whole molecule. In this work these complexes were prepared and characterized by IR and CHN analysis to indicate the purities of these complexes. The electrochemistry techniques were shown as obtained for ligand reduction. This research was carried out at School of Chemistry and Molecular Science, Sussex University, U.K

Comparison of Albuterol Delivery between High Frequency Oscillatory Ventilation and Conventional Mechanical Ventilation in a Simulated Adult Lung Model using Different Compliance Levels

COMPARISON OF ALBUTEROL DELIVERY BETWEEN HIGH FREQUENCY OSCILLATORY VENTILATION AND CONVENTIONAL MECHANICAL VENTILATION IN A SIMULATED ADULT LUNG MODEL USING DIFFERENT COMPLIANCE LEVELS By Waleed A. Alzahrani, BSRT BACKGROUND: Delivery of aerosol by pMDI has been described with conventional mechanical ventilation (CMV) but not with high frequency oscillatory ventilation (HFOV). The purpose of this study was to compare aerosol delivery to a simulated 75 kg adult with low compliance during both CMV and HFOV. Since actuation of pMDI with inspiration is not feasible with HFOV, we investigated the impact of actuation timing only during CMV. METHOD: CMV (Respironics Esprit) and HFOV (Sensor Medics 3100B) ventilators with passover humidifiers and heated circuits were connected by 8 mm ID ETT and filter (Respirgard II, Vital Signs) to a test lung (TTL) with compliance settings of 20 and 40 ml/cm H2O in order to simulate a non compliant lung. Settings for CMV (VT 6 ml/kg, I:E 1:1, PEEP 20 cm H2O, and RR 25/min), and HFOV (RR 5 Hz, IT 33%, ∆P 80 cm H2O and mPaw 35 cm H2O) were used, with similar mPaw on CMV and HFOV. Parameters were selected based on ARDSnet protective lung strategy (Fessler and Hess, Respiratory Care 2007) Eight actuations of albuterol from pMDI (ProAir HFA, Teva Medical) with double nozzle small volume spacer (Mini Spacer, Thayer Medical) placed between the “Y” adapter and ETT at more than 15 sec intervals for each condition (n=3). During CMV, pMDI actuations were synchronized (SYNC) with the start of inspiration at more than 15 s, and nonsynchronized (NONSYNC) with actuations at 15 s intervals. Drug was eluted from the filter and analyzed by spectrophotometry (276 nm). Repeated measures ANOVA, pairwise comparisons and independent t- tests were performed at the significance level of 0.05. RESULTS: In all cases, aerosol delivery was greater with HFOV than CMV (p\u3c0.05). Synchronizing pMDI actuations with the beginning of inspiration increased aerosol deposition significantly at compliance levels 20 ml/cm H2O and 40 ml/cm H2O (p=0.011 and p=0.02, respectively). Lung compliance and aerosol delivery are directly related. Increasing lung compliance to 40 ml/cmH2O improved aerosol delivery during CMV and HFOV (p\u3c0.05). CONCLUSION: Albuterol deposition with pMDI was more than two fold greater with HFOV than CMV in this in-vitro lung model. Changing lung compliance has almost 2 fold impact on aerosol delivery during both modes of ventilation. Furthermore, synchronizing pMDI actuations during CMV improved aerosol delivery up to 4 fold

AUC: Nonparametric Estimators and Their Smoothness

Nonparametric estimation of a statistic, in general, and of the error rate of a classification rule, in particular, from just one available dataset through resampling is well mathematically founded in the literature using several versions of bootstrap and influence function. This article first provides a concise review of this literature to establish the theoretical framework that we use to construct, in a single coherent framework, nonparametric estimators of the AUC (a two-sample statistic) other than the error rate (a one-sample statistic). In addition, the smoothness of some of these estimators is well investigated and explained. Our experiments show that the behavior of the designed AUC estimators confirms the findings of the literature for the behavior of error rate estimators in many aspects including: the weak correlation between the bootstrap-based estimators and the true conditional AUC; and the comparable accuracy of the different versions of the bootstrap estimators in terms of the RMS with little superiority of the .632+ bootstrap estimator

What Aspects of Emotional Intelligence Help Former Prisoners Make Decisions to Desist Crime?

Making good post-incarceration decisions are important for helping formerly incarcerated individuals avoid a return to prison. This is the first study to look at emotional intelligence (EI) components formerly incarcerated men considered important for post-prison criminal desistance. This study explored six former New York state male prisoner’s individual experiences developing EI competencies, and how those EI skills contributed to their post-release decisions to desist crime. Research participants spent an average 7.5 years in a New York state prison, and have been out of prison and living in the community for an average 3.6 years without recommitting a criminal offense. Research interviews revealed that the internal process of self-reflection instigates an increased state of self-awareness. Self-awareness is the foundation for developing responsible decision-making skills and the motivation to desist crime. In addition, participants’ decisions to desist crime were also mediated by external factors including pro-social relationships with family members and friends, and having gainful employment

Individually Perceived Supports and Barriers to Successful Community Reentry After Serving a Prison Sentence

Many ex-offenders face a myriad of challenges to community re-entry after serving a prison sentence that may contribute to recidivism. This qualitative research study explored individually experienced supports and perceived barriers that contributed to a successful reentry experience, and how individuals learned to effectively manage and meet the various challenges of living in the community after being released from prison