Why solutions can be hard to find: a featural theory of cost for a local search algorithm on random satisfiability instances

The local search algorithm WSat is one of the most successful algorithms for solving the archetypal NP-complete problem of satisfiability (SAT). It is notably effective at solving Random-3-SAT instances near the so-called 'satisfiability threshold', which are thought to be universally hard. However, WSat still shows a peak in search cost near the threshold and large variations in cost over different instances. Why are solutions to the threshold instances so hard to find using WSat? What features characterise threshold instances which make them difficult for WSat to solve? We make a number of significant contributions to the analysis of WSat on these high-cost random instances, using the recently-introduced concept of the backbone of a SAT instance. The backbone is the set of literals which are implicates of an instance. We find that the number of solutions predicts the cost well for small-backbone instances but is much less relevant for the large-backbone instances which appear near the threshold and dominate in the overconstrained region. We undertake a detailed study of the behaviour of the algorithm during search and uncover some interesting patterns. These patterns lead us to introduce a measure of the backbone fragility of an instance, which indicates how persistent the backbone is as clauses are removed. We propose that high-cost random instances for WSat are those with large backbones which are also backbone-fragile. We suggest that the decay in cost for WSat beyond the satisfiability threshold, which has perplexed a number of researchers, is due to the decreasing backbone fragility. Our hypothesis makes three correct predictions. First, that a measure of the backbone robustness of an instance (the opposite to backbone fragility) is negatively correlated with the WSat cost when other factors are controlled for. Second, that backbone-minimal instances (which are 3-SAT instances altered so as to be more backbone-fragile) are unusually hard for WSat. Third, that the clauses most often unsatisfied during search are those whose deletion has the most effect on the backbone. Our analysis of WSat on random-3-SAT threshold instances can be seen as a featural theory of WSat cost, predicting features of cost behaviour from structural features of SAT instances. In this thesis, we also present some initial studies which investigate whether the scope of this featural theory can be broadened to other kinds of random SAT instance. random-2+p-SAT interpolates between the polynomial-time problem Random-2-SAT when p = 0 and Random-3-SAT when p = 1. At some value p ~ pq ~ 0.41, a dramatic change in the structural nature of instances is predicted by statistical mechanics methods, which may imply the appearance of backbone fragile instances. We tested NovELTY+, a recent variant of WSat, on rand o m- 2 +p-SAT and find some evidence that growth of its median cost changes from polynomial to superpolynomial between p = 0.3 and p = 0.5. We also find evidence that it is the onset of backbone fragility which is the cause of this change in cost scaling: typical instances at p — 0.5 are more backbone-fragile than their counterparts at p — 0.3. Not-All-Equal (NAE) 3-SAT is a variant of the SAT problem which is similar to it in most respects. However, for NAE 3-SAT instances no implicate literals are possible. Hence the backbone for NAE 3-SAT must be redefined. We show that under a redefinition of the backbone, the pattern of factors influencing WSat cost at the NAE Random-3-SAT threshold is much the same as in Random-3-SAT, including the role of backbone fragility

Integration of constraint programming and linear programming techniques for constraint satisfaction problem and general constrained optimization problem.

Wong Siu Ham.Thesis (M.Phil.)--Chinese University of Hong Kong, 2001.Includes bibliographical references (leaves 131-138).Abstracts in English and Chinese.Abstract --- p.iiAcknowledgments --- p.viChapter 1 --- Introduction --- p.1Chapter 1.1 --- Motivation for Integration --- p.2Chapter 1.2 --- Thesis Overview --- p.4Chapter 2 --- Preliminaries --- p.5Chapter 2.1 --- Constraint Programming --- p.5Chapter 2.1.1 --- Constraint Satisfaction Problems (CSP's) --- p.6Chapter 2.1.2 --- Satisfiability (SAT) Problems --- p.10Chapter 2.1.3 --- Systematic Search --- p.11Chapter 2.1.4 --- Local Search --- p.13Chapter 2.2 --- Linear Programming --- p.17Chapter 2.2.1 --- Linear Programming Problems --- p.17Chapter 2.2.2 --- Simplex Method --- p.19Chapter 2.2.3 --- Mixed Integer Programming Problems --- p.27Chapter 3 --- Integration of Constraint Programming and Linear Program- ming --- p.29Chapter 3.1 --- Problem Definition --- p.29Chapter 3.2 --- Related works --- p.30Chapter 3.2.1 --- Illustrating the Performances --- p.30Chapter 3.2.2 --- Improving the Searching --- p.33Chapter 3.2.3 --- Improving the representation --- p.36Chapter 4 --- A Scheme of Integration for Solving Constraint Satisfaction Prob- lem --- p.37Chapter 4.1 --- Integrated Algorithm --- p.38Chapter 4.1.1 --- Overview of the Integrated Solver --- p.38Chapter 4.1.2 --- The LP Engine --- p.44Chapter 4.1.3 --- The CP Solver --- p.45Chapter 4.1.4 --- Proof of Soundness and Completeness --- p.46Chapter 4.1.5 --- Compared with Previous Work --- p.46Chapter 4.2 --- Benchmarking Results --- p.48Chapter 4.2.1 --- Comparison with CLP solvers --- p.48Chapter 4.2.2 --- Magic Squares --- p.51Chapter 4.2.3 --- Random CSP's --- p.52Chapter 5 --- A Scheme of Integration for Solving General Constrained Opti- mization Problem --- p.68Chapter 5.1 --- Integrated Optimization Algorithm --- p.69Chapter 5.1.1 --- Overview of the Integrated Optimizer --- p.69Chapter 5.1.2 --- The CP Solver --- p.74Chapter 5.1.3 --- The LP Engine --- p.75Chapter 5.1.4 --- Proof of the Optimization --- p.77Chapter 5.2 --- Benchmarking Results --- p.77Chapter 5.2.1 --- Weighted Magic Square --- p.77Chapter 5.2.2 --- Template design problem --- p.78Chapter 5.2.3 --- Random GCOP's --- p.79Chapter 6 --- Conclusions and Future Work --- p.97Chapter 6.1 --- Conclusions --- p.97Chapter 6.2 --- Future work --- p.98Chapter 6.2.1 --- Detection of implicit equalities --- p.98Chapter 6.2.2 --- Dynamical variable selection --- p.99Chapter 6.2.3 --- Analysis on help of linear constraints --- p.99Chapter 6.2.4 --- Local Search and Linear Programming --- p.99Appendix --- p.101Proof of Soundness and Completeness --- p.101Proof of the optimization --- p.126Bibliography --- p.13

Simplest random K-satisfiability problem

We study a simple and exactly solvable model for the generation of random satisfiability problems. These consist of $\gamma N$ random boolean constraints which are to be satisfied simultaneously by $N$ logical variables. In statistical-mechanics language, the considered model can be seen as a diluted p-spin model at zero temperature. While such problems become extraordinarily hard to solve by local search methods in a large region of the parameter space, still at least one solution may be superimposed by construction. The statistical properties of the model can be studied exactly by the replica method and each single instance can be analyzed in polynomial time by a simple global solution method. The geometrical/topological structures responsible for dynamic and static phase transitions as well as for the onset of computational complexity in local search method are thoroughly analyzed. Numerical analysis on very large samples allows for a precise characterization of the critical scaling behaviour.Comment: 14 pages, 5 figures, to appear in Phys. Rev. E (Feb 2001). v2: minor errors and references correcte

When Gravity Fails: Local Search Topology

Local search algorithms for combinatorial search problems frequently encounter a sequence of states in which it is impossible to improve the value of the objective function; moves through these regions, called plateau moves, dominate the time spent in local search. We analyze and characterize plateaus for three different classes of randomly generated Boolean Satisfiability problems. We identify several interesting features of plateaus that impact the performance of local search algorithms. We show that local minima tend to be small but occasionally may be very large. We also show that local minima can be escaped without unsatisfying a large number of clauses, but that systematically searching for an escape route may be computationally expensive if the local minimum is large. We show that plateaus with exits, called benches, tend to be much larger than minima, and that some benches have very few exit states which local search can use to escape. We show that the solutions (i.e., global minima) of randomly generated problem instances form clusters, which behave similarly to local minima. We revisit several enhancements of local search algorithms and explain their performance in light of our results. Finally we discuss strategies for creating the next generation of local search algorithms.Comment: See http://www.jair.org/ for any accompanying file

Glassy Behavior and Jamming of a Random Walk Process for Sequentially Satisfying a Constraint Satisfaction Formula

Random $K$-satisfiability ($K$-SAT) is a model system for studying typical-case complexity of combinatorial optimization. Recent theoretical and simulation work revealed that the solution space of a random $K$-SAT formula has very rich structures, including the emergence of solution communities within single solution clusters. In this paper we investigate the influence of the solution space landscape to a simple stochastic local search process {\tt SEQSAT}, which satisfies a $K$-SAT formula in a sequential manner. Before satisfying each newly added clause, {\tt SEQSAT} walk randomly by single-spin flips in a solution cluster of the old subformula. This search process is efficient when the constraint density $\alpha$ of the satisfied subformula is less than certain value $\alpha_{cm}$; however it slows down considerably as $\alpha > \alpha_{cm}$ and finally reaches a jammed state at $\alpha \approx \alpha_{j}$. The glassy dynamical behavior of {\tt SEQSAT} for $\alpha \geq \alpha_{cm}$ probably is due to the entropic trapping of various communities in the solution cluster of the satisfied subformula. For random 3-SAT, the jamming transition point $\alpha_j$ is larger than the solution space clustering transition point $\alpha_d$, and its value can be predicted by a long-range frustration mean-field theory. For random $K$-SAT with $K\geq 4$, however, our simulation results indicate that $\alpha_j = \alpha_d$. The relevance of this work for understanding the dynamic properties of glassy systems is also discussed.Comment: 10 pages, 6 figures, 1 table, a mistake of numerical simulation corrected, and new results adde

Minimizing energy below the glass thresholds

Focusing on the optimization version of the random K-satisfiability problem, the MAX-K-SAT problem, we study the performance of the finite energy version of the Survey Propagation (SP) algorithm. We show that a simple (linear time) backtrack decimation strategy is sufficient to reach configurations well below the lower bound for the dynamic threshold energy and very close to the analytic prediction for the optimal ground states. A comparative numerical study on one of the most efficient local search procedures is also given.Comment: 12 pages, submitted to Phys. Rev. E, accepted for publicatio