Variational Bayesian Model Selection for Mixture Distributions

Mixture models, in which a probability distribu-tion is represented as a linear superposition of component distributions, are widely used in sta-tistical modeling and pattern recognition. One of the key tasks in the application of mixture models is the determination of a suitable number of components. Conventional approaches based on cross-validation are computationally expen-sive, are wasteful of data, and give noisy esti-mates for the optimal number of components. A fully Bayesian treatment, based on Markov chain Monte Carlo methods for instance, will re-turn a posterior distribution over the number of components. However, in practical applications it is generally convenient, or even computation-ally essential, to select a single, most appropri-ate model. Recently it has been shown, in the context of linear latent variable models, that the use of hierarchical priors governed by continuous hyperparameters whose values are set by type-II maximum likelihood, can be used to optimize model complexity. In this paper we extend this framework to mixture distributions by consider-ing the classical task of density estimation us-ing mixtures of Gaussians. We show that, by setting the mixing coefficients to maximize the marginal log-likelihood, unwanted components can be suppressed, and the appropriate number of components for the mixture can be determined in a single training run without recourse to cross-validation. Our approach uses a variational treat-ment based on a factorized approximation to the posterior distribution.

Frame-semantic parsing

Frame semantics is a linguistic theory that has been instantiated for English in the FrameNet lexicon. We solve the problem of frame-semantic parsing using a two-stage statistical model that takes lexical targets (i.e., content words and phrases) in their sentential contexts and predicts frame-semantic structures. Given a target in context, the first stage disambiguates it to a semantic frame. This model uses latent variables and semi-supervised learning to improve frame disambiguation for targets unseen at training time. The second stage finds the target's locally expressed semantic arguments. At inference time, a fast exact dual decomposition algorithm collectively predicts all the arguments of a frame at once in order to respect declaratively stated linguistic constraints, resulting in qualitatively better structures than naïve local predictors. Both components are feature-based and discriminatively trained on a small set of annotated frame-semantic parses. On the SemEval 2007 benchmark data set, the approach, along with a heuristic identifier of frame-evoking targets, outperforms the prior state of the art by significant margins. Additionally, we present experiments on the much larger FrameNet 1.5 data set. We have released our frame-semantic parser as open-source software.United States. Defense Advanced Research Projects Agency (DARPA grant NBCH-1080004)National Science Foundation (U.S.) (NSF grant IIS-0836431)National Science Foundation (U.S.) (NSF grant IIS-0915187)Qatar National Research Fund (NPRP 08-485-1-083

Learning spatially-variable filters for super-resolution of text

Images magnified by standard methods display a degradation of detail that is particularly noticeable in the blurry edges of text. Current super-resolution algorithms address the lack of sharpness by filling in the image with probable details. These algorithms break the outlines of text. Our novel algorithm for super-resolution of text magnifies images in real-time by interpolation with a variable linear filter. The coefficients of the filter are determined nonlinearly from the neighborhood to which it is applied. We train the mapping that defines the coefficients to specifically enhance edges of text, producing a conservative algorithm that infers the detail of magnified text. Possible applications include resizing web page layouts or other interfaces, and enhancing low resolution camera captures of text. In general, learning spatially-variable filters is applicable to other image filtering tasks. 1

Stable mixing of complete and incomplete information

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2002.Includes bibliographical references (p. 74-76).by Adrian Corduneanu.S.M