Conditional Restricted Boltzmann Machines for Structured Output Prediction

Conditional Restricted Boltzmann Machines (CRBMs) are rich probabilistic models that have recently been applied to a wide range of problems, including collaborative filtering, classification, and modeling motion capture data. While much progress has been made in training non-conditional RBMs, these algorithms are not applicable to conditional models and there has been almost no work on training and generating predictions from conditional RBMs for structured output problems. We first argue that standard Contrastive Divergence-based learning may not be suitable for training CRBMs. We then identify two distinct types of structured output prediction problems and propose an improved learning algorithm for each. The first problem type is one where the output space has arbitrary structure but the set of likely output configurations is relatively small, such as in multi-label classification. The second problem is one where the output space is arbitrarily structured but where the output space variability is much greater, such as in image denoising or pixel labeling. We show that the new learning algorithms can work much better than Contrastive Divergence on both types of problems

A description of the software analysis from flight and simulation data of the course cut limiter in the TCV b-737 area navigation computer

During automatic horizontal path captures, the (Terminal Configured Vehicle) B-737 airplane maintained smaller than designed path intercept angles and experienced a sawtooth bank angle oscillation during its turn towards the path. From flight data, it was determined that these anomalies were caused by the improper output of the course cut limiter in the horizontal path control law. The output from the course cut limiter did not obtain its full value and it was calculated stepwise discontinuously. The automatic horizontal path captures were then conducted on the TCV B-737 airplane real-time simulation. The path intercept angles were maintained properly and no bank angle oscillation was encountered. Data showed that the course cut limiter was calculated at its full value in a continuous manner. The intermediate calculations of the course cut limiter in the airplane's navigation computer were rewritten and rescaled in such a manner that truncation errors could be minimized. The horizontal path capture tests were then reflown. The airplane maintained the proper path intercept angle and no bank angle oscillations occurred on any of the tests

Modeling Documents with Deep Boltzmann Machines

We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents. We overcome the apparent difficulty of training a DBM with judicious parameter tying. This parameter tying enables an efficient pretraining algorithm and a state initialization scheme that aids inference. The model can be trained just as efficiently as a standard Restricted Boltzmann Machine. Our experiments show that the model assigns better log probability to unseen data than the Replicated Softmax model. Features extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classification tasks.Comment: Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013

Gamma-ray emission associated with Cluster-scale AGN Outbursts

Recent observations have revealed the existence of enormously energetic ~10^61 erg AGN outbursts in three relatively distant galaxy clusters. These outbursts have produced bubbles in the intra-cluster medium, apparently supported by pressure from relativistic particles and/or magnetic fields. Here we argue that if > GeV particles are responsible then these particles are very likely protons and nuclei, rather than electrons, and that the gamma-ray emission from these objects, arising from the interactions of these hadrons in the intra-cluster medium, may be marginally detectable with instruments such as GLAST and HESS.Comment: 8 pages, 4 figures, accepted by MNRA

Learning generative texture models with extended Fields-of-Experts

We evaluate the ability of the popular Field-of-Experts (FoE) to model structure in images. As a test case we focus on modeling synthetic and natural textures. We find that even for modeling single textures, the FoE provides insufficient flexibility to learn good generative models – it does not perform any better than the much simpler Gaussian FoE. We propose an extended version of the FoE (allowing for bimodal potentials) and demonstrate that this novel formulation, when trained with a better approximation of the likelihood gradient, gives rise to a more powerful generative model of specific visual structure that produces significantly better results for the texture task