Unsupervised Neural Hidden Markov Models

In this work, we present the first results for neuralizing an Unsupervised Hidden Markov Model. We evaluate our approach on tag in- duction. Our approach outperforms existing generative models and is competitive with the state-of-the-art though with a simpler model easily extended to include additional context.Comment: accepted at EMNLP 2016, Workshop on Structured Prediction for NLP. Oral presentatio

DAC: The Double Actor-Critic Architecture for Learning Options

We reformulate the option framework as two parallel augmented MDPs. Under this novel formulation, all policy optimization algorithms can be used off the shelf to learn intra-option policies, option termination conditions, and a master policy over options. We apply an actor-critic algorithm on each augmented MDP, yielding the Double Actor-Critic (DAC) architecture. Furthermore, we show that, when state-value functions are used as critics, one critic can be expressed in terms of the other, and hence only one critic is necessary. We conduct an empirical study on challenging robot simulation tasks. In a transfer learning setting, DAC outperforms both its hierarchy-free counterpart and previous gradient-based option learning algorithms.Comment: NeurIPS 201

Experiment-friendly kinetic analysis of single molecule data in and out of equilibrium

We present a simple and robust technique to extract kinetic rate models and thermodynamic quantities from single molecule time traces. SMACKS (Single Molecule Analysis of Complex Kinetic Sequences) is a maximum likelihood approach that works equally well for long trajectories as for a set of short ones. It resolves all statistically relevant rates and also their uncertainties. This is achieved by optimizing one global kinetic model based on the complete dataset, while allowing for experimental variations between individual trajectories. In particular, neither a priori models nor equilibrium have to be assumed. The power of SMACKS is demonstrated on the kinetics of the multi-domain protein Hsp90 measured by smFRET (single molecule F\"orster resonance energy transfer). Experiments in and out of equilibrium are analyzed and compared to simulations, shedding new light on the role of Hsp90's ATPase function. SMACKS pushes the boundaries of single molecule kinetics far beyond current methods.Comment: 11 pages, 8 figure