Domain adaptation for sequence labeling using hidden Markov models

Most natural language processing systems based on machine learning are not robust to domain shift. For example, a state-of-the-art syntactic dependency parser trained on Wall Street Journal sentences has an absolute drop in performance of more than ten points when tested on textual data from the Web. An efficient solution to make these methods more robust to domain shift is to first learn a word representation using large amounts of unlabeled data from both domains, and then use this representation as features in a supervised learning algorithm. In this paper, we propose to use hidden Markov models to learn word representations for part-of-speech tagging. In particular, we study the influence of using data from the source, the target or both domains to learn the representation and the different ways to represent words using an HMM.Comment: New Directions in Transfer and Multi-Task: Learning Across Domains and Tasks (NIPS Workshop) (2013

Graphical Models in Characterizing the Dependency Relationship in Wireless Networks and Social Networks

Semi-Markov processes have become increasingly important in probability and statistical modeling, which have found applications in traffic analysis, reliability and maintenance, survival analysis, performance evaluation, biology, DNA analysis, risk processes, insurance and finance, earthquake modeling, etc. In the first part of this thesis, our focus is on applying semi-Markov processes to modeling the on-off duty cycles of different nodes in wireless networks. More specifically, we are interested in restoration of statistics of individual occupancy patterns of specific users based on wireless RF observation traces. In particular, we present a novel approach to finding the statistics of several operations, namely down-sampling, superposition and mislabelling, of a discrete time semi-Markov process in terms of the sojourn time distributions and states transition matrix of the resulting process. The resulting process, after those operations, is also a semi-Markov processes or a Markov renewal process. We show that the statistics of the original sequence before the superposition operation of two semi Markov processes can be generally recovered. However the statistics of the original sequence cannot be recovered under the down-sampling operation, namely the probability transition matrix and the sojourn time distribution properties are distorted after the down-sampling. Simulation and numerical results further demonstrate the validity of our theoretical findings. Our results thus provide a more profound understanding on the limitation of applying semi-Markov models in characterizing and learning the dynamics of nodes\u27 activities in wireless networks. In the second portion of the thesis a review is provided about several graphical models that have been widely used in literature recently to characterize the relationships between different users in social networks, the influence of the neighboring nodes in the networks or the semantic similarity in different contexts

A Markovian approach to distributional semantics with application to semantic compositionality

International audienceIn this article, we describe a new approach to distributional semantics. This approach relies on a generative model of sentences with latent variables, which takes the syntax into account by using syntactic dependency trees. Words are then represented as posterior distributions over those latent classes, and the model allows to naturally obtain in-context and out-of-context word representations, which are comparable. We train our model on a large corpus and demonstrate the compositionality capabilities of our approach on different datasets

Hidden Markov tree models for semantic class induction

International audienceIn this paper, we propose a new method for semantic class induction. First, we introduce a generative model of sentences, based on dependency trees and which takes into account homonymy. Our model can thus be seen as a generalization of Brown clustering. Second, we describe an efficient algorithm to perform inference and learning in this model. Third, we apply our proposed method on two large datasets ($10^8$ tokens, $10^5$ words types), and demonstrate that classes induced by our algorithm improve performance over Brown clustering on the task of semi-supervised supersense tagging and named entity recognition