Position Models and Language Modeling

International audienceIn statistical language modelling the classic model used is $n$-gram. This model is not able however to capture long term dependencies, \emph{i.e.} dependencies larger than $n$. An alternative to this model is the probabilistic automaton. Unfortunately, it appears that preliminary experiments on the use of this model in language modelling is not yet competitive, partly because it tries to model too long term dependencies. We propose here to improve the use of this model by restricting the dependency to a more reasonable value. Experiments shows an improvement of 45\% reduction in the perplexity obtained on the Wall Street Journal language modeling task

A compact statistical model of the song syntax in Bengalese finch

Songs of many songbird species consist of variable sequences of a finite number of syllables. A common approach for characterizing the syntax of these complex syllable sequences is to use transition probabilities between the syllables. This is equivalent to the Markov model, in which each syllable is associated with one state, and the transition probabilities between the states do not depend on the state transition history. Here we analyze the song syntax in a Bengalese finch. We show that the Markov model fails to capture the statistical properties of the syllable sequences. Instead, a state transition model that accurately describes the statistics of the syllable sequences includes adaptation of the self-transition probabilities when states are repeatedly revisited, and allows associations of more than one state to the same syllable. Such a model does not increase the model complexity significantly. Mathematically, the model is a partially observable Markov model with adaptation (POMMA). The success of the POMMA supports the branching chain network hypothesis of how syntax is controlled within the premotor song nucleus HVC, and suggests that adaptation and many-to-one mapping from neural substrates to syllables are important features of the neural control of complex song syntax

The Photodetector Array Camera and Spectrometer (PACS) on the Herschel Space Observatory

The Photodetector Array Camera and Spectrometer (PACS) is one of the three science instruments on ESA's far infrared and submillimetre observatory. It employs two Ge:Ga photoconductor arrays (stressed and unstressed) with 16x25 pixels, each, and two filled silicon bolometer arrays with 16x32 and 32x64 pixels, respectively, to perform integral-field spectroscopy and imaging photometry in the 60-210\mu\ m wavelength regime. In photometry mode, it simultaneously images two bands, 60-85\mu\ m or 85-125\mu\m and 125-210\mu\ m, over a field of view of ~1.75'x3.5', with close to Nyquist beam sampling in each band. In spectroscopy mode, it images a field of 47"x47", resolved into 5x5 pixels, with an instantaneous spectral coverage of ~1500km/s and a spectral resolution of ~175km/s. We summarise the design of the instrument, describe observing modes, calibration, and data analysis methods, and present our current assessment of the in-orbit performance of the instrument based on the Performance Verification tests. PACS is fully operational, and the achieved performance is close to or better than the pre-launch predictions

Learning Hidden Markov Models to Fit Long-Term Dependencies

this report a novel approach to the induction of the structure of Hidden Markov Models (HMMs). The notion of partially observable Markov models (POMMs) is introduced. POMMs form a particular case of HMMs where any state emits a single letter with probability one, but several states can emit the same letter. It is shown that any HMM can be represented by an equivalent POMM. The proposed induction algorithm aims at finding a POMM fitting the dynamics of the target machine, that is to best approximate the stationary distribution and the mean first passage times observed in the sample. The induction relies on non-linear optimization and iterative state splitting from an initial order one Markov chain. Experimental results illustrate the advantages of the proposed approach as compared to Baum-Welch HMM estimation or back-o# smoothed Ngrams equivalent to variable order Markov chain

walks in

subgraph extraction from rando

F/sub beta / support vector machines

We introduce in this paper F/sub beta / SVMs, a new parametrization of support vector machines. It allows to optimize a SVM in terms of F/sub beta /, a classical information retrieval criterion, instead of the usual classification rate. Experiments illustrate the advantages of this approach with respect to the traditional 2-norm soft-margin SVM when precision and recall are of unequal importance. An automatic model selection procedure based on the generalization F/sub beta / score is introduced. It relies on the results of Chapelle, Vapnjk et al. (2002) about the use of gradient-based techniques in SVM model selection. The derivatives of a F /sub beta / loss function with respect to the hyperparameters C and the width sigma of a gaussian kernel are formally defined. The model is then selected by performing a gradient descent of the F/sub beta / loss function over the set of hyperparameters. Experiments on artificial and real-life data show the benefits of this method when the F/sub beta / score is considered.Anglai

Inducing Hidden Markov Models to model long-term dependencies

We propose in this paper a novel approach to the induction of the structure of Hidden Markov Models. The induced model is seen as a lumped process of a Markov chain. It is constructed to fit the dynamics of the target machine, that is to best approximate the stationary distribution and the mean first passage times observed in the sample. The induction relies on non-linear optimization and iterative state splitting from an initial order one Markov chain

Within-Network Classification Using Local Structure Similarity

Abstract. Within-network classification, where the goal is to classify the nodes of a partly labeled network, is a semi-supervised learning problem that has applications in several important domains like image processing, the classification of documents, and the detection of malicious activities. While most methods for this problem infer the missing labels collectively based on the hypothesis that linked or nearby nodes are likely to have the same labels, there are many types of networks for which this assumption fails, e.g., molecular graphs, trading networks, etc. In this paper, we present a collective classification method, based on relaxation labeling, that classifies entities of a network using their local structure. This method uses a marginalized similarity kernel that compares the local structure of two nodes with parallel random walks in the network. Through experimentation on different datasets, we show our method to be more accurate than several state-of-the-art approaches for this problem. Key words: Network, semi-supervised learning, random walk