Finding Non-overlapping Clusters for Generalized Inference Over Graphical Models

Graphical models use graphs to compactly capture stochastic dependencies amongst a collection of random variables. Inference over graphical models corresponds to finding marginal probability distributions given joint probability distributions. In general, this is computationally intractable, which has led to a quest for finding efficient approximate inference algorithms. We propose a framework for generalized inference over graphical models that can be used as a wrapper for improving the estimates of approximate inference algorithms. Instead of applying an inference algorithm to the original graph, we apply the inference algorithm to a block-graph, defined as a graph in which the nodes are non-overlapping clusters of nodes from the original graph. This results in marginal estimates of a cluster of nodes, which we further marginalize to get the marginal estimates of each node. Our proposed block-graph construction algorithm is simple, efficient, and motivated by the observation that approximate inference is more accurate on graphs with longer cycles. We present extensive numerical simulations that illustrate our block-graph framework with a variety of inference algorithms (e.g., those in the libDAI software package). These simulations show the improvements provided by our framework.Comment: Extended the previous version to include extensive numerical simulations. See http://www.ima.umn.edu/~dvats/GeneralizedInference.html for code and dat

A nearly-mlogn time solver for SDD linear systems

We present an improved algorithm for solving symmetrically diagonally dominant linear systems. On input of an $n\times n$ symmetric diagonally dominant matrix $A$ with $m$ non-zero entries and a vector $b$ such that $A\bar{x} = b$ for some (unknown) vector $\bar{x}$, our algorithm computes a vector $x$ such that $||{x}-\bar{x}||_A < \epsilon ||\bar{x}||_A$ {$||\cdot||_A$ denotes the A-norm} in time $${\tilde O}(m\log n \log (1/\epsilon)).$$ The solver utilizes in a standard way a `preconditioning' chain of progressively sparser graphs. To claim the faster running time we make a two-fold improvement in the algorithm for constructing the chain. The new chain exploits previously unknown properties of the graph sparsification algorithm given in [Koutis,Miller,Peng, FOCS 2010], allowing for stronger preconditioning properties. We also present an algorithm of independent interest that constructs nearly-tight low-stretch spanning trees in time $\tilde{O}(m\log{n})$, a factor of $O(\log{n})$ faster than the algorithm in [Abraham,Bartal,Neiman, FOCS 2008]. This speedup directly reflects on the construction time of the preconditioning chain.Comment: to appear in FOCS1

Essential Constraints of Edge-Constrained Proximity Graphs

Given a plane forest $F = (V, E)$ of $|V| = n$ points, we find the minimum set $S \subseteq E$ of edges such that the edge-constrained minimum spanning tree over the set $V$ of vertices and the set $S$ of constraints contains $F$. We present an $O(n \log n )$-time algorithm that solves this problem. We generalize this to other proximity graphs in the constraint setting, such as the relative neighbourhood graph, Gabriel graph, $\beta$-skeleton and Delaunay triangulation. We present an algorithm that identifies the minimum set $S\subseteq E$ of edges of a given plane graph $I=(V,E)$ such that $I \subseteq CG_\beta(V, S)$ for $1 \leq \beta \leq 2$, where $CG_\beta(V, S)$ is the constraint $\beta$-skeleton over the set $V$ of vertices and the set $S$ of constraints. The running time of our algorithm is $O(n)$, provided that the constrained Delaunay triangulation of $I$ is given.Comment: 24 pages, 22 figures. A preliminary version of this paper appeared in the Proceedings of 27th International Workshop, IWOCA 2016, Helsinki, Finland. It was published by Springer in the Lecture Notes in Computer Science (LNCS) serie

W-types in setoids

W-types and their categorical analogue, initial algebras for polynomial endofunctors, are an important tool in predicative systems to replace transfinite recursion on well-orderings. Current arguments to obtain W-types in quotient completions rely on assumptions, like Uniqueness of Identity Proofs, or on constructions that involve recursion into a universe, that limit their applicability to a specific setting. We present an argument, verified in Coq, that instead uses dependent W-types in the underlying type theory to construct W-types in the setoid model. The immediate advantage is to have a proof more type-theoretic in flavour, which directly uses recursion on the underlying W-type to prove initiality. Furthermore, taking place in intensional type theory and not requiring any recursion into a universe, it may be generalised to various categorical quotient completions, with the aim of finding a uniform construction of extensional W-types.Comment: 17 pages, formalised in Coq; v2: added reference to formalisatio