Relaxations for inference in restricted Boltzmann machines

We propose a relaxation-based approximate inference algorithm that samples near-MAP configurations of a binary pairwise Markov random field. We experiment on MAP inference tasks in several restricted Boltzmann machines. We also use our underlying sampler to estimate the log-partition function of restricted Boltzmann machines and compare against other sampling-based methods.Comment: ICLR 2014 workshop track submissio

Uniform sampling of steady states in metabolic networks: heterogeneous scales and rounding

The uniform sampling of convex polytopes is an interesting computational problem with many applications in inference from linear constraints, but the performances of sampling algorithms can be affected by ill-conditioning. This is the case of inferring the feasible steady states in models of metabolic networks, since they can show heterogeneous time scales . In this work we focus on rounding procedures based on building an ellipsoid that closely matches the sampling space, that can be used to define an efficient hit-and-run (HR) Markov Chain Monte Carlo. In this way the uniformity of the sampling of the convex space of interest is rigorously guaranteed, at odds with non markovian methods. We analyze and compare three rounding methods in order to sample the feasible steady states of metabolic networks of three models of growing size up to genomic scale. The first is based on principal component analysis (PCA), the second on linear programming (LP) and finally we employ the lovasz ellipsoid method (LEM). Our results show that a rounding procedure is mandatory for the application of the HR in these inference problem and suggest that a combination of LEM or LP with a subsequent PCA perform the best. We finally compare the distributions of the HR with that of two heuristics based on the Artificially Centered hit-and-run (ACHR), gpSampler and optGpSampler. They show a good agreement with the results of the HR for the small network, while on genome scale models present inconsistencies.Comment: Replacement with major revision

A discriminative view of MRF pre-processing algorithms

While Markov Random Fields (MRFs) are widely used in computer vision, they present a quite challenging inference problem. MRF inference can be accelerated by pre-processing techniques like Dead End Elimination (DEE) or QPBO-based approaches which compute the optimal labeling of a subset of variables. These techniques are guaranteed to never wrongly label a variable but they often leave a large number of variables unlabeled. We address this shortcoming by interpreting pre-processing as a classification problem, which allows us to trade off false positives (i.e., giving a variable an incorrect label) versus false negatives (i.e., failing to label a variable). We describe an efficient discriminative rule that finds optimal solutions for a subset of variables. Our technique provides both per-instance and worst-case guarantees concerning the quality of the solution. Empirical studies were conducted over several benchmark datasets. We obtain a speedup factor of 2 to 12 over expansion moves without preprocessing, and on difficult non-submodular energy functions produce slightly lower energy.Comment: ICCV 201

Combinatorial persistency criteria for multicut and max-cut

In combinatorial optimization, partial variable assignments are called persistent if they agree with some optimal solution. We propose persistency criteria for the multicut and max-cut problem as well as fast combinatorial routines to verify them. The criteria that we derive are based on mappings that improve feasible multicuts, respectively cuts. Our elementary criteria can be checked enumeratively. The more advanced ones rely on fast algorithms for upper and lower bounds for the respective cut problems and max-flow techniques for auxiliary min-cut problems. Our methods can be used as a preprocessing technique for reducing problem sizes or for computing partial optimality guarantees for solutions output by heuristic solvers. We show the efficacy of our methods on instances of both problems from computer vision, biomedical image analysis and statistical physics

Modeling Perceptual Aliasing in SLAM via Discrete-Continuous Graphical Models

Perceptual aliasing is one of the main causes of failure for Simultaneous Localization and Mapping (SLAM) systems operating in the wild. Perceptual aliasing is the phenomenon where different places generate a similar visual (or, in general, perceptual) footprint. This causes spurious measurements to be fed to the SLAM estimator, which typically results in incorrect localization and mapping results. The problem is exacerbated by the fact that those outliers are highly correlated, in the sense that perceptual aliasing creates a large number of mutually-consistent outliers. Another issue stems from the fact that most state-of-the-art techniques rely on a given trajectory guess (e.g., from odometry) to discern between inliers and outliers and this makes the resulting pipeline brittle, since the accumulation of error may result in incorrect choices and recovery from failures is far from trivial. This work provides a unified framework to model perceptual aliasing in SLAM and provides practical algorithms that can cope with outliers without relying on any initial guess. We present two main contributions. The first is a Discrete-Continuous Graphical Model (DC-GM) for SLAM: the continuous portion of the DC-GM captures the standard SLAM problem, while the discrete portion describes the selection of the outliers and models their correlation. The second contribution is a semidefinite relaxation to perform inference in the DC-GM that returns estimates with provable sub-optimality guarantees. Experimental results on standard benchmarking datasets show that the proposed technique compares favorably with state-of-the-art methods while not relying on an initial guess for optimization.Comment: 13 pages, 14 figures, 1 tabl

Efficient Decomposed Learning for Structured Prediction

Structured prediction is the cornerstone of several machine learning applications. Unfortunately, in structured prediction settings with expressive inter-variable interactions, exact inference-based learning algorithms, e.g. Structural SVM, are often intractable. We present a new way, Decomposed Learning (DecL), which performs efficient learning by restricting the inference step to a limited part of the structured spaces. We provide characterizations based on the structure, target parameters, and gold labels, under which DecL is equivalent to exact learning. We then show that in real world settings, where our theoretical assumptions may not completely hold, DecL-based algorithms are significantly more efficient and as accurate as exact learning.Comment: ICML201

Bayesian inference on compact binary inspiral gravitational radiation signals in interferometric data

Presented is a description of a Markov chain Monte Carlo (MCMC) parameter estimation routine for use with interferometric gravitational radiational data in searches for binary neutron star inspiral signals. Five parameters associated with the inspiral can be estimated, and summary statistics are produced. Advanced MCMC methods were implemented, including importance resampling and prior distributions based on detection probability, in order to increase the efficiency of the code. An example is presented from an application using realistic, albeit fictitious, data.Comment: submitted to Classical and Quantum Gravity. 14 pages, 5 figure

Detection and classification of neurodegenerative diseases: a spatially informed bayesian deep learning approach

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesNeurodegenerative diseases comprise a group of chronic and irreversible conditions characterized by the progressive degeneration of the structure and function of the central nervous system. The detection and classification of patients according to the underlying disease are crucial for developing oriented treatments and enriching prognosis. In this context, Magnetic resonance imaging (MRI) data can provide meaningful insights into neurodegeneration by detecting the physiological manifestations in the brain caused by the disease processes. One field of extensive clinical use of MRI is the accurate and automated classification of neurodegenerative disorders. Most studies distinguish patients from healthy subjects or stages within the same disease. Such distinction does not mirror clinical practice, as a patient may not show all symptoms, especially if the disease is in an early stage, or show, due to comorbidities, other symptoms as well. Likewise, automated classifiers are partly suited for medical diagnosis since they cannot produce probabilistic predictions nor account for uncertainty. Also, existent studies ignore the spatial heterogeneity of the brain alterations caused by neurodegenerative processes. The spatial configuration of the neuronal loss is a characteristic hallmark for each disorder. To fill these gaps, this thesis aims to develop a classification technique that incorporates uncertainty and spatial information for distinguishing four neurodegenerative diseases, Alzheimer’s disease, Mild cognitive impairment, Parkinson’s disease and Multiple Sclerosis, and healthy subjects. This technique will produce automated, contingent, and accurate predictions to support clinical diagnosis. To quantify prediction uncertainty and improve classification accuracy, this study introduces a Bayesian neural network with a spatially informed input. A convolutional neural network (CNN) is developed to identify a neurodegenerative condition based on T1weighted MRI scans from patients and healthy controls. Bayesian inference is incorporated into the CNN to measure uncertainty and produce probabilistic predictions. Also, a spatially informed MRI scan is added to the CNN to improve feature detection and classification accuracy. The Spatially informed Bayesian Neural Network (SBNN) proposed in this work demonstrates that classification accuracy can be increased up to 25% by including the spatially informed MRI scan. Furthermore, the SBNN provides robust probabilistic diagnosis that resembles clinical decision-making and accounts for atypical, numerous, and early presentations of neurodegenerative disorders