A Simple Parallel and Distributed Sampling Technique: Local Glauber Dynamics

Sampling constitutes an important tool in a variety of areas: from machine learning and combinatorial optimization to computational physics and biology. A central class of sampling algorithms is the Markov Chain Monte Carlo method, based on the construction of a Markov chain with the desired sampling distribution as its stationary distribution. Many of the traditional Markov chains, such as the Glauber dynamics, do not scale well with increasing dimension. To address this shortcoming, we propose a simple local update rule based on the Glauber dynamics that leads to efficient parallel and distributed algorithms for sampling from Gibbs distributions. Concretely, we present a Markov chain that mixes in O(log n) rounds when Dobrushin\u27s condition for the Gibbs distribution is satisfied. This improves over the LubyGlauber algorithm by Feng, Sun, and Yin [PODC\u2717], which needs O(Delta log n) rounds, and their LocalMetropolis algorithm, which converges in O(log n) rounds but requires a considerably stronger mixing condition. Here, n denotes the number of nodes in the graphical model inducing the Gibbs distribution, and Delta its maximum degree. In particular, our method can sample a uniform proper coloring with alpha Delta colors in O(log n) rounds for any alpha >2, which almost matches the threshold of the sequential Glauber dynamics and improves on the alpha>2 + sqrt{2} threshold of Feng et al

On Derandomizing Local Distributed Algorithms

The gap between the known randomized and deterministic local distributed algorithms underlies arguably the most fundamental and central open question in distributed graph algorithms. In this paper, we develop a generic and clean recipe for derandomizing LOCAL algorithms. We also exhibit how this simple recipe leads to significant improvements on a number of problem. Two main results are: - An improved distributed hypergraph maximal matching algorithm, improving on Fischer, Ghaffari, and Kuhn [FOCS'17], and giving improved algorithms for edge-coloring, maximum matching approximation, and low out-degree edge orientation. The first gives an improved algorithm for Open Problem 11.4 of the book of Barenboim and Elkin, and the last gives the first positive resolution of their Open Problem 11.10. - An improved distributed algorithm for the Lov\'{a}sz Local Lemma, which gets closer to a conjecture of Chang and Pettie [FOCS'17], and moreover leads to improved distributed algorithms for problems such as defective coloring and $k$-SAT.Comment: 37 page

Improved Distributed Algorithms for Random Colorings

Markov Chain Monte Carlo (MCMC) algorithms are a widely-used algorithmic tool for sampling from high-dimensional distributions, a notable example is the equilibirum distribution of graphical models. The Glauber dynamics, also known as the Gibbs sampler, is the simplest example of an MCMC algorithm; the transitions of the chain update the configuration at a randomly chosen coordinate at each step. Several works have studied distributed versions of the Glauber dynamics and we extend these efforts to a more general family of Markov chains. An important combinatorial problem in the study of MCMC algorithms is random colorings. Given a graph $G$ of maximum degree $\Delta$ and an integer $k\geq\Delta+1$, the goal is to generate a random proper vertex $k$-coloring of $G$. Jerrum (1995) proved that the Glauber dynamics has $O(n\log{n})$ mixing time when $k>2\Delta$. Fischer and Ghaffari (2018), and independently Feng, Hayes, and Yin (2018), presented a parallel and distributed version of the Glauber dynamics which converges in $O(\log{n})$ rounds for $k>(2+\varepsilon)\Delta$ for any $\varepsilon>0$. We improve this result to $k>(11/6-\delta)\Delta$ for a fixed $\delta>0$. This matches the state of the art for randomly sampling colorings of general graphs in the sequential setting. Whereas previous works focused on distributed variants of the Glauber dynamics, our work presents a parallel and distributed version of the more general flip dynamics presented by Vigoda (2000) (and refined by Chen, Delcourt, Moitra, Perarnau, and Postle (2019)), which recolors local maximal two-colored components in each step.Comment: 25 pages, 2 figure

Theoretically Efficient Parallel Graph Algorithms Can Be Fast and Scalable

There has been significant recent interest in parallel graph processing due to the need to quickly analyze the large graphs available today. Many graph codes have been designed for distributed memory or external memory. However, today even the largest publicly-available real-world graph (the Hyperlink Web graph with over 3.5 billion vertices and 128 billion edges) can fit in the memory of a single commodity multicore server. Nevertheless, most experimental work in the literature report results on much smaller graphs, and the ones for the Hyperlink graph use distributed or external memory. Therefore, it is natural to ask whether we can efficiently solve a broad class of graph problems on this graph in memory. This paper shows that theoretically-efficient parallel graph algorithms can scale to the largest publicly-available graphs using a single machine with a terabyte of RAM, processing them in minutes. We give implementations of theoretically-efficient parallel algorithms for 20 important graph problems. We also present the optimizations and techniques that we used in our implementations, which were crucial in enabling us to process these large graphs quickly. We show that the running times of our implementations outperform existing state-of-the-art implementations on the largest real-world graphs. For many of the problems that we consider, this is the first time they have been solved on graphs at this scale. We have made the implementations developed in this work publicly-available as the Graph-Based Benchmark Suite (GBBS).Comment: This is the full version of the paper appearing in the ACM Symposium on Parallelism in Algorithms and Architectures (SPAA), 201

Improved Distributed Fractional Coloring Algorithms

We prove new bounds on the distributed fractional coloring problem in the LOCAL model. Fractional $c$-colorings can be understood as multicolorings as follows. For some natural numbers $p$ and $q$ such that $p/q\leq c$, each node $v$ is assigned a set of at least $q$ colors from $\{1,\dots,p\}$ such that adjacent nodes are assigned disjoint sets of colors. The minimum $c$ for which a fractional $c$-coloring of a graph $G$ exists is called the fractional chromatic number $\chi_f(G)$ of $G$. Recently, [Bousquet, Esperet, and Pirot; SIROCCO '21] showed that for any constant $\epsilon>0$, a fractional $(\Delta+\epsilon)$-coloring can be computed in $\Delta^{O(\Delta)} + O(\Delta\cdot\log^* n)$ rounds. We show that such a coloring can be computed in only $O(\log^2 \Delta)$ rounds, without any dependency on $n$. We further show that in $O\big(\frac{\log n}{\epsilon}\big)$ rounds, it is possible to compute a fractional $(1+\epsilon)\chi_f(G)$-coloring, even if the fractional chromatic number $\chi_f(G)$ is not known. That is, this problem can be approximated arbitrarily well by an efficient algorithm in the LOCAL model. For the standard coloring problem, it is only known that an $O\big(\frac{\log n}{\log\log n}\big)$-approximation can be computed in polylogarithmic time in the LOCAL model. We also show that our distributed fractional coloring approximation algorithm is best possible. We show that in trees, which have fractional chromatic number $2$, computing a fractional $(2+\epsilon)$-coloring requires at least $\Omega\big(\frac{\log n}{\epsilon}\big)$ rounds. We finally study fractional colorings of regular grids. In [Bousquet, Esperet, and Pirot; SIROCCO '21], it is shown that in regular grids of bounded dimension, a fractional $(2+\epsilon)$-coloring can be computed in time $O(\log^* n)$. We show that such a coloring can even be computed in $O(1)$ rounds in the LOCAL model

Extremal Optimization at the Phase Transition of the 3-Coloring Problem

We investigate the phase transition of the 3-coloring problem on random graphs, using the extremal optimization heuristic. 3-coloring is among the hardest combinatorial optimization problems and is closely related to a 3-state anti-ferromagnetic Potts model. Like many other such optimization problems, it has been shown to exhibit a phase transition in its ground state behavior under variation of a system parameter: the graph's mean vertex degree. This phase transition is often associated with the instances of highest complexity. We use extremal optimization to measure the ground state cost and the ``backbone'', an order parameter related to ground state overlap, averaged over a large number of instances near the transition for random graphs of size $n$ up to 512. For graphs up to this size, benchmarks show that extremal optimization reaches ground states and explores a sufficient number of them to give the correct backbone value after about $O(n^{3.5})$ update steps. Finite size scaling gives a critical mean degree value $\alpha_{\rm c}=4.703(28)$. Furthermore, the exploration of the degenerate ground states indicates that the backbone order parameter, measuring the constrainedness of the problem, exhibits a first-order phase transition.Comment: RevTex4, 8 pages, 4 postscript figures, related information available at http://www.physics.emory.edu/faculty/boettcher

Optimal (Degree+1)-Coloring in Congested Clique

We consider the distributed complexity of the (degree+1)-list coloring problem, in which each node u of degree d(u) is assigned a palette of d(u)+1 colors, and the goal is to find a proper coloring using these color palettes. The (degree+1)-list coloring problem is a natural generalization of the classical (?+1)-coloring and (?+1)-list coloring problems, both being benchmark problems extensively studied in distributed and parallel computing. In this paper we settle the complexity of the (degree+1)-list coloring problem in the Congested Clique model by showing that it can be solved deterministically in a constant number of rounds