On Symbolic Ultrametrics, Cotree Representations, and Cograph Edge Decompositions and Partitions

Symbolic ultrametrics define edge-colored complete graphs K_n and yield a simple tree representation of K_n. We discuss, under which conditions this idea can be generalized to find a symbolic ultrametric that, in addition, distinguishes between edges and non-edges of arbitrary graphs G=(V,E) and thus, yielding a simple tree representation of G. We prove that such a symbolic ultrametric can only be defined for G if and only if G is a so-called cograph. A cograph is uniquely determined by a so-called cotree. As not all graphs are cographs, we ask, furthermore, what is the minimum number of cotrees needed to represent the topology of G. The latter problem is equivalent to find an optimal cograph edge k-decomposition {E_1,...,E_k} of E so that each subgraph (V,E_i) of G is a cograph. An upper bound for the integer k is derived and it is shown that determining whether a graph has a cograph 2-decomposition, resp., 2-partition is NP-complete

Random Graph Coloring - a Statistical Physics Approach

The problem of vertex coloring in random graphs is studied using methods of statistical physics and probability. Our analytical results are compared to those obtained by exact enumeration and Monte-Carlo simulations. We critically discuss the merits and shortcomings of the various methods, and interpret the results obtained. We present an exact analytical expression for the 2-coloring problem as well as general replica symmetric approximated solutions for the thermodynamics of the graph coloring problem with p colors and K-body edges.Comment: 17 pages, 9 figure

Optimization hardness as transient chaos in an analog approach to constraint satisfaction

Boolean satisfiability [1] (k-SAT) is one of the most studied optimization problems, as an efficient (that is, polynomial-time) solution to k-SAT (for $k\geq 3$) implies efficient solutions to a large number of hard optimization problems [2,3]. Here we propose a mapping of k-SAT into a deterministic continuous-time dynamical system with a unique correspondence between its attractors and the k-SAT solution clusters. We show that beyond a constraint density threshold, the analog trajectories become transiently chaotic [4-7], and the boundaries between the basins of attraction [8] of the solution clusters become fractal [7-9], signaling the appearance of optimization hardness [10]. Analytical arguments and simulations indicate that the system always finds solutions for satisfiable formulae even in the frozen regimes of random 3-SAT [11] and of locked occupation problems [12] (considered among the hardest algorithmic benchmarks); a property partly due to the system's hyperbolic [4,13] character. The system finds solutions in polynomial continuous-time, however, at the expense of exponential fluctuations in its energy function.Comment: 27 pages, 14 figure

Backprojection for Training Feedforward Neural Networks in the Input and Feature Spaces

After the tremendous development of neural networks trained by backpropagation, it is a good time to develop other algorithms for training neural networks to gain more insights into networks. In this paper, we propose a new algorithm for training feedforward neural networks which is fairly faster than backpropagation. This method is based on projection and reconstruction where, at every layer, the projected data and reconstructed labels are forced to be similar and the weights are tuned accordingly layer by layer. The proposed algorithm can be used for both input and feature spaces, named as backprojection and kernel backprojection, respectively. This algorithm gives an insight to networks with a projection-based perspective. The experiments on synthetic datasets show the effectiveness of the proposed method.Comment: Accepted (to appear) in International Conference on Image Analysis and Recognition (ICIAR) 2020, Springe

User-friendly tail bounds for sums of random matrices

This paper presents new probability inequalities for sums of independent, random, self-adjoint matrices. These results place simple and easily verifiable hypotheses on the summands, and they deliver strong conclusions about the large-deviation behavior of the maximum eigenvalue of the sum. Tail bounds for the norm of a sum of random rectangular matrices follow as an immediate corollary. The proof techniques also yield some information about matrix-valued martingales. In other words, this paper provides noncommutative generalizations of the classical bounds associated with the names Azuma, Bennett, Bernstein, Chernoff, Hoeffding, and McDiarmid. The matrix inequalities promise the same diversity of application, ease of use, and strength of conclusion that have made the scalar inequalities so valuable.Comment: Current paper is the version of record. The material on Freedman's inequality has been moved to a separate note; other martingale bounds are described in Caltech ACM Report 2011-0

String Matching and 1d Lattice Gases

We calculate the probability distributions for the number of occurrences $n$ of a given $l$ letter word in a random string of $k$ letters. Analytical expressions for the distribution are known for the asymptotic regimes (i) $k \gg r^l \gg 1$ (Gaussian) and $k,l \to \infty$ such that $k/r^l$ is finite (Compound Poisson). However, it is known that these distributions do now work well in the intermediate regime $k \gtrsim r^l \gtrsim 1$. We show that the problem of calculating the string matching probability can be cast into a determining the configurational partition function of a 1d lattice gas with interacting particles so that the matching probability becomes the grand-partition sum of the lattice gas, with the number of particles corresponding to the number of matches. We perform a virial expansion of the effective equation of state and obtain the probability distribution. Our result reproduces the behavior of the distribution in all regimes. We are also able to show analytically how the limiting distributions arise. Our analysis builds on the fact that the effective interactions between the particles consist of a relatively strong core of size $l$, the word length, followed by a weak, exponentially decaying tail. We find that the asymptotic regimes correspond to the case where the tail of the interactions can be neglected, while in the intermediate regime they need to be kept in the analysis. Our results are readily generalized to the case where the random strings are generated by more complicated stochastic processes such as a non-uniform letter probability distribution or Markov chains. We show that in these cases the tails of the effective interactions can be made even more dominant rendering thus the asymptotic approximations less accurate in such a regime.Comment: 44 pages and 8 figures. Major revision of previous version. The lattice gas analogy has been worked out in full, including virial expansion and equation of state. This constitutes the main part of the paper now. Connections with existing work is made and references should be up to date now. To be submitted for publicatio

People Efficiently Explore the Solution Space of the Computationally Intractable Traveling Salesman Problem to Find Near-Optimal Tours

Humans need to solve computationally intractable problems such as visual search, categorization, and simultaneous learning and acting, yet an increasing body of evidence suggests that their solutions to instantiations of these problems are near optimal. Computational complexity advances an explanation to this apparent paradox: (1) only a small portion of instances of such problems are actually hard, and (2) successful heuristics exploit structural properties of the typical instance to selectively improve parts that are likely to be sub-optimal. We hypothesize that these two ideas largely account for the good performance of humans on computationally hard problems. We tested part of this hypothesis by studying the solutions of 28 participants to 28 instances of the Euclidean Traveling Salesman Problem (TSP). Participants were provided feedback on the cost of their solutions and were allowed unlimited solution attempts (trials). We found a significant improvement between the first and last trials and that solutions are significantly different from random tours that follow the convex hull and do not have self-crossings. More importantly, we found that participants modified their current better solutions in such a way that edges belonging to the optimal solution (“good” edges) were significantly more likely to stay than other edges (“bad” edges), a hallmark of structural exploitation. We found, however, that more trials harmed the participants' ability to tell good from bad edges, suggesting that after too many trials the participants “ran out of ideas.” In sum, we provide the first demonstration of significant performance improvement on the TSP under repetition and feedback and evidence that human problem-solving may exploit the structure of hard problems paralleling behavior of state-of-the-art heuristics

How to share underground reservoirs

Many resources, such as oil, gas, or water, are extracted from porous soils and their exploration is often shared among different companies or nations. We show that the effective shares can be obtained by invading the porous medium simultaneously with various fluids. Partitioning a volume in two parts requires one division surface while the simultaneous boundary between three parts consists of lines. We identify and characterize these lines, showing that they form a fractal set consisting of a single thread spanning the medium and a surrounding cloud of loops. While the spanning thread has fractal dimension ${1.55\pm0.03}$, the set of all lines has dimension ${1.69\pm0.02}$. The size distribution of the loops follows a power law and the evolution of the set of lines exhibits a tricritical point described by a crossover with a negative dimension at criticality