On Correcting Inputs: Inverse Optimization for Online Structured Prediction

Algorithm designers typically assume that the input data is correct, and then proceed to find "optimal" or "sub-optimal" solutions using this input data. However this assumption of correct data does not always hold in practice, especially in the context of online learning systems where the objective is to learn appropriate feature weights given some training samples. Such scenarios necessitate the study of inverse optimization problems where one is given an input instance as well as a desired output and the task is to adjust the input data so that the given output is indeed optimal. Motivated by learning structured prediction models, in this paper we consider inverse optimization with a margin, i.e., we require the given output to be better than all other feasible outputs by a desired margin. We consider such inverse optimization problems for maximum weight matroid basis, matroid intersection, perfect matchings, minimum cost maximum flows, and shortest paths and derive the first known results for such problems with a non-zero margin. The effectiveness of these algorithmic approaches to online learning for structured prediction is also discussed.Comment: Conference version to appear in FSTTCS, 201

Cross-layer Congestion Control, Routing and Scheduling Design in Ad Hoc Wireless Networks

This paper considers jointly optimal design of crosslayer congestion control, routing and scheduling for ad hoc wireless networks. We first formulate the rate constraint and scheduling constraint using multicommodity flow variables, and formulate resource allocation in networks with fixed wireless channels (or single-rate wireless devices that can mask channel variations) as a utility maximization problem with these constraints. By dual decomposition, the resource allocation problem naturally decomposes into three subproblems: congestion control, routing and scheduling that interact through congestion price. The global convergence property of this algorithm is proved. We next extend the dual algorithm to handle networks with timevarying channels and adaptive multi-rate devices. The stability of the resulting system is established, and its performance is characterized with respect to an ideal reference system which has the best feasible rate region at link layer. We then generalize the aforementioned results to a general model of queueing network served by a set of interdependent parallel servers with time-varying service capabilities, which models many design problems in communication networks. We show that for a general convex optimization problem where a subset of variables lie in a polytope and the rest in a convex set, the dual-based algorithm remains stable and optimal when the constraint set is modulated by an irreducible finite-state Markov chain. This paper thus presents a step toward a systematic way to carry out cross-layer design in the framework of “layering as optimization decomposition” for time-varying channel models

A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching

We propose a combinatorial solution for the problem of non-rigidly matching a 3D shape to 3D image data. To this end, we model the shape as a triangular mesh and allow each triangle of this mesh to be rigidly transformed to achieve a suitable matching to the image. By penalising the distance and the relative rotation between neighbouring triangles our matching compromises between image and shape information. In this paper, we resolve two major challenges: Firstly, we address the resulting large and NP-hard combinatorial problem with a suitable graph-theoretic approach. Secondly, we propose an efficient discretisation of the unbounded 6-dimensional Lie group SE(3). To our knowledge this is the first combinatorial formulation for non-rigid 3D shape-to-image matching. In contrast to existing local (gradient descent) optimisation methods, we obtain solutions that do not require a good initialisation and that are within a bound of the optimal solution. We evaluate the proposed method on the two problems of non-rigid 3D shape-to-shape and non-rigid 3D shape-to-image registration and demonstrate that it provides promising results.Comment: 10 pages, 7 figure

Globally minimal surfaces by continuous maximal flows

In this paper we address the computation of globally minimal curves and surfaces for image segmentation and stereo reconstruction. We present a solution, simulating a continuous maximal flow by a novel system of partial differential equations. Existing methods are either grid-biased (graph-based methods) or sub-optimal (active contours and surfaces). The solution simulates the flow of an ideal fluid with isotropic velocity constraints. Velocity constraints are defined by a metric derived from image data. An auxiliary potential function is introduced to create a system of partial differential equations. It is proven that the algorithm produces a globally maximal continuous flow at convergence, and that the globally minimal surface may be obtained trivially from the auxiliary potential. The bias of minimal surface methods toward small objects is also addressed. An efficient implementation is given for the flow simulation. The globally minimal surface algorithm is applied to segmentation in 2D and 3D as well as to stereo matching. Results in 2D agree with an existing minimal contour algorithm for planar images. Results in 3D segmentation and stereo matching demonstrate that the new algorithm is robust and free from grid bias

Using Incomplete Information for Complete Weight Annotation of Road Networks -- Extended Version

We are witnessing increasing interests in the effective use of road networks. For example, to enable effective vehicle routing, weighted-graph models of transportation networks are used, where the weight of an edge captures some cost associated with traversing the edge, e.g., greenhouse gas (GHG) emissions or travel time. It is a precondition to using a graph model for routing that all edges have weights. Weights that capture travel times and GHG emissions can be extracted from GPS trajectory data collected from the network. However, GPS trajectory data typically lack the coverage needed to assign weights to all edges. This paper formulates and addresses the problem of annotating all edges in a road network with travel cost based weights from a set of trips in the network that cover only a small fraction of the edges, each with an associated ground-truth travel cost. A general framework is proposed to solve the problem. Specifically, the problem is modeled as a regression problem and solved by minimizing a judiciously designed objective function that takes into account the topology of the road network. In particular, the use of weighted PageRank values of edges is explored for assigning appropriate weights to all edges, and the property of directional adjacency of edges is also taken into account to assign weights. Empirical studies with weights capturing travel time and GHG emissions on two road networks (Skagen, Denmark, and North Jutland, Denmark) offer insight into the design properties of the proposed techniques and offer evidence that the techniques are effective.Comment: This is an extended version of "Using Incomplete Information for Complete Weight Annotation of Road Networks," which is accepted for publication in IEEE TKD

An Algorithmic Theory of Dependent Regularizers, Part 1: Submodular Structure

We present an exploration of the rich theoretical connections between several classes of regularized models, network flows, and recent results in submodular function theory. This work unifies key aspects of these problems under a common theory, leading to novel methods for working with several important models of interest in statistics, machine learning and computer vision. In Part 1, we review the concepts of network flows and submodular function optimization theory foundational to our results. We then examine the connections between network flows and the minimum-norm algorithm from submodular optimization, extending and improving several current results. This leads to a concise representation of the structure of a large class of pairwise regularized models important in machine learning, statistics and computer vision. In Part 2, we describe the full regularization path of a class of penalized regression problems with dependent variables that includes the graph-guided LASSO and total variation constrained models. This description also motivates a practical algorithm. This allows us to efficiently find the regularization path of the discretized version of TV penalized models. Ultimately, our new algorithms scale up to high-dimensional problems with millions of variables

Approximating the Permanent with Fractional Belief Propagation

We discuss schemes for exact and approximate computations of permanents, and compare them with each other. Specifically, we analyze the Belief Propagation (BP) approach and its Fractional Belief Propagation (FBP) generalization for computing the permanent of a non-negative matrix. Known bounds and conjectures are verified in experiments, and some new theoretical relations, bounds and conjectures are proposed. The Fractional Free Energy (FFE) functional is parameterized by a scalar parameter $\gamma\in[-1;1]$, where $\gamma=-1$ corresponds to the BP limit and $\gamma=1$ corresponds to the exclusion principle (but ignoring perfect matching constraints) Mean-Field (MF) limit. FFE shows monotonicity and continuity with respect to $\gamma$. For every non-negative matrix, we define its special value $\gamma_*\in[-1;0]$ to be the $\gamma$ for which the minimum of the $\gamma$-parameterized FFE functional is equal to the permanent of the matrix, where the lower and upper bounds of the $\gamma$-interval corresponds to respective bounds for the permanent. Our experimental analysis suggests that the distribution of $\gamma_*$ varies for different ensembles but $\gamma_*$ always lies within the $[-1;-1/2]$ interval. Moreover, for all ensembles considered the behavior of $\gamma_*$ is highly distinctive, offering an emprirical practical guidance for estimating permanents of non-negative matrices via the FFE approach.Comment: 42 pages, 14 figure