32,478 research outputs found
Inference and Model Parameter Learning for Image Labeling by Geometric Assignment
Image labeling is a fundamental problem in the area of low-level image analysis. In this work, we present novel approaches to maximum a posteriori (MAP) inference and model
parameter learning for image labeling, respectively. Both approaches are formulated in a smooth geometric setting, whose respective solution space is a simple Riemannian manifold. Optimization
consists of multiplicative updates that geometrically integrate the resulting Riemannian gradient flow.
Our novel approach to MAP inference is based on discrete graphical models. By utilizing local Wasserstein distances for coupling assignment measures across edges of the
underlying graph, we smoothly approximate a given discrete objective function and restrict it to the
assignment manifold. A corresponding update scheme combines geometric integration of the resulting gradient flow, and rounding to integral solutions that represent
valid labelings. This formulation constitutes an inner relaxation of the discrete labeling problem, i.e. throughout this process local marginalization constraints known from the established linear programming relaxation are satisfied.
Furthermore, we study the inverse problem of model parameter learning using the linear assignment flow and training data with ground truth. This is accomplished by a Riemannian gradient flow on the manifold of parameters that determine the regularization properties of the assignment flow. This smooth formulation enables us to tackle the model parameter learning problem from the perspective of parameter estimation of dynamical systems. By using symplectic partitioned Runge--Kutta methods for numerical integration, we show that deriving the sensitivity conditions of the parameter learning problem and its discretization commute. A favorable property of our approach is that learning is based on exact inference
High-dimensional learning of linear causal networks via inverse covariance estimation
We establish a new framework for statistical estimation of directed acyclic
graphs (DAGs) when data are generated from a linear, possibly non-Gaussian
structural equation model. Our framework consists of two parts: (1) inferring
the moralized graph from the support of the inverse covariance matrix; and (2)
selecting the best-scoring graph amongst DAGs that are consistent with the
moralized graph. We show that when the error variances are known or estimated
to close enough precision, the true DAG is the unique minimizer of the score
computed using the reweighted squared l_2-loss. Our population-level results
have implications for the identifiability of linear SEMs when the error
covariances are specified up to a constant multiple. On the statistical side,
we establish rigorous conditions for high-dimensional consistency of our
two-part algorithm, defined in terms of a "gap" between the true DAG and the
next best candidate. Finally, we demonstrate that dynamic programming may be
used to select the optimal DAG in linear time when the treewidth of the
moralized graph is bounded.Comment: 41 pages, 7 figure
Learning Probabilistic Graphical Models for Image Segmentation
Probabilistic graphical models provide a powerful framework for representing image structures. Based on this concept, many inference and learning algorithms have been
developed. However, both algorithm classes are NP-hard combinatorial problems in the general case. As a consequence, relaxation methods were developed to
approximate the original problems but with the benefit of being computationally efficient. In this work we consider the learning problem on binary graphical models and their relaxations. Two novel methods for determining the model parameters in discrete energy functions from training data are proposed. Learning the model parameters overcomes the problem of heuristically determining them.
Motivated by common learning methods which aim at minimizing the training error measured by a loss function we develop a new learning method similar in fashion to structured SVM. However, computationally more efficient. We term this method
as linearized approach (LA) as it is restricted to linearly dependent potentials. The linearity of LA is crucial to come up with a tight convex relaxation, which allows
to use off-the-shelf inference solvers to approach subproblems which emerge from solving the overall problem.
However, this type of learning methods almost never yield optimal solutions or perfect performance on the training data set. So what happens if the learned graphical
model on the training data would lead to exact ground segmentation? Will this give a benefit when predicting?
Motivated by the idea of inverse optimization, we take advantage of inverse linear programming to develop a learning approach, referred to as inverse linear
programming approach (invLPA). It further refines the graphical models trained, using the previously introduced methods and is capable to perfectly predict ground
truth on training data. The empirical results from implementing invLPA give answers to our questions posed before. LA is able to learn both unary and pairwise potentials jointly while with invLPA
this is not possible due to the representation we use. On the other hand, invLPA does not rely on a certain form for the potentials and thus is flexible in the choice of
the fitting method. Although the corrected potentials with invLPA always result in ground truth
segmentation of the training data, invLPA is able to find corrections on the foreground segments only. Due to the relaxed problem formulation this does not affect the final
segmentation result. Moreover, as long as we initialize invLPA with model parameters of a learning method performing sufficiently well, this drawback of invLPA does not
significantly affect the final prediction result.
The performance of the proposed learning methods is evaluated on both synthetic and real world datasets. We demonstrate that LA is competitive in comparison to other parameter learning methods using loss functions based on Maximum a Posteriori Marginal (MPM) and Maximum Likelihood Estimation (MLE). Moreover, we illustrate the benefits of learning with inverse linear programming. In a further experiment we demonstrate the versatility of our learning methods by applying LA to learning motion segmentation in video sequences and comparing it to state-of-the-art segmentation algorithms
Conic Optimization Theory: Convexification Techniques and Numerical Algorithms
Optimization is at the core of control theory and appears in several areas of
this field, such as optimal control, distributed control, system
identification, robust control, state estimation, model predictive control and
dynamic programming. The recent advances in various topics of modern
optimization have also been revamping the area of machine learning. Motivated
by the crucial role of optimization theory in the design, analysis, control and
operation of real-world systems, this tutorial paper offers a detailed overview
of some major advances in this area, namely conic optimization and its emerging
applications. First, we discuss the importance of conic optimization in
different areas. Then, we explain seminal results on the design of hierarchies
of convex relaxations for a wide range of nonconvex problems. Finally, we study
different numerical algorithms for large-scale conic optimization problems.Comment: 18 page
- …