Glassy Behavior and Jamming of a Random Walk Process for Sequentially Satisfying a Constraint Satisfaction Formula

Random $K$-satisfiability ($K$-SAT) is a model system for studying typical-case complexity of combinatorial optimization. Recent theoretical and simulation work revealed that the solution space of a random $K$-SAT formula has very rich structures, including the emergence of solution communities within single solution clusters. In this paper we investigate the influence of the solution space landscape to a simple stochastic local search process {\tt SEQSAT}, which satisfies a $K$-SAT formula in a sequential manner. Before satisfying each newly added clause, {\tt SEQSAT} walk randomly by single-spin flips in a solution cluster of the old subformula. This search process is efficient when the constraint density $\alpha$ of the satisfied subformula is less than certain value $\alpha_{cm}$; however it slows down considerably as $\alpha > \alpha_{cm}$ and finally reaches a jammed state at $\alpha \approx \alpha_{j}$. The glassy dynamical behavior of {\tt SEQSAT} for $\alpha \geq \alpha_{cm}$ probably is due to the entropic trapping of various communities in the solution cluster of the satisfied subformula. For random 3-SAT, the jamming transition point $\alpha_j$ is larger than the solution space clustering transition point $\alpha_d$, and its value can be predicted by a long-range frustration mean-field theory. For random $K$-SAT with $K\geq 4$, however, our simulation results indicate that $\alpha_j = \alpha_d$. The relevance of this work for understanding the dynamic properties of glassy systems is also discussed.Comment: 10 pages, 6 figures, 1 table, a mistake of numerical simulation corrected, and new results adde

Modeling scale-dependent bias on the baryonic acoustic scale with the statistics of peaks of Gaussian random fields

Models of galaxy and halo clustering commonly assume that the tracers can be treated as a continuous field locally biased with respect to the underlying mass distribution. In the peak model pioneered by BBKS, one considers instead density maxima of the initial, Gaussian mass density field as an approximation to the formation site of virialized objects. In this paper, the peak model is extended in two ways to improve its predictive accuracy. Firstly, we derive the two-point correlation function of initial density peaks up to second order and demonstrate that a peak-background split approach can be applied to obtain the k-independent and k-dependent peak bias factors at all orders. Secondly, we explore the gravitational evolution of the peak correlation function within the Zel'dovich approximation. We show that the local (Lagrangian) bias approach emerges as a special case of the peak model, in which all bias parameters are scale-independent and there is no statistical velocity bias. We apply our formulae to study how the Lagrangian peak biasing, the diffusion due to large scale flows and the mode-coupling due to nonlocal interactions affect the scale dependence of bias from small separations up to the baryon acoustic oscillation (BAO) scale. For 2-sigma density peaks collapsing at z=0.3, our model predicts a ~ 5% residual scale-dependent bias around the acoustic scale that arises mostly from first-order Lagrangian peak biasing (as opposed to second-order gravity mode-coupling). We also search for a scale dependence of bias in the large scale auto-correlation of massive halos extracted from a very large N-body simulation provided by the MICE collaboration. For halos with mass M>10^{14}Msun/h, our measurements demonstrate a scale-dependent bias across the BAO feature which is very well reproduced by a prediction based on the peak model.Comment: (v1): 23 pages text, 8 figures + appendix (v2): typos fixed, references added, accepted for publication in PR

Solution space heterogeneity of the random K-satisfiability problem: Theory and simulations

The random K-satisfiability (K-SAT) problem is an important problem for studying typical-case complexity of NP-complete combinatorial satisfaction; it is also a representative model of finite-connectivity spin-glasses. In this paper we review our recent efforts on the solution space fine structures of the random K-SAT problem. A heterogeneity transition is predicted to occur in the solution space as the constraint density alpha reaches a critical value alpha_cm. This transition marks the emergency of exponentially many solution communities in the solution space. After the heterogeneity transition the solution space is still ergodic until alpha reaches a larger threshold value alpha_d, at which the solution communities disconnect from each other to become different solution clusters (ergodicity-breaking). The existence of solution communities in the solution space is confirmed by numerical simulations of solution space random walking, and the effect of solution space heterogeneity on a stochastic local search algorithm SEQSAT, which performs a random walk of single-spin flips, is investigated. The relevance of this work to glassy dynamics studies is briefly mentioned.Comment: 11 pages, 4 figures. Final version as will appear in Journal of Physics: Conference Series (Proceedings of the International Workshop on Statistical-Mechanical Informatics, March 7-10, 2010, Kyoto, Japan

A directed isoperimetric inequality with application to Bregman near neighbor lower bounds

Bregman divergences $D_\phi$ are a class of divergences parametrized by a convex function $\phi$ and include well known distance functions like $\ell_2^2$ and the Kullback-Leibler divergence. There has been extensive research on algorithms for problems like clustering and near neighbor search with respect to Bregman divergences, in all cases, the algorithms depend not just on the data size $n$ and dimensionality $d$, but also on a structure constant $\mu \ge 1$ that depends solely on $\phi$ and can grow without bound independently. In this paper, we provide the first evidence that this dependence on $\mu$ might be intrinsic. We focus on the problem of approximate near neighbor search for Bregman divergences. We show that under the cell probe model, any non-adaptive data structure (like locality-sensitive hashing) for $c$-approximate near-neighbor search that admits $r$ probes must use space $\Omega(n^{1 + \frac{\mu}{c r}})$. In contrast, for LSH under $\ell_1$ the best bound is $\Omega(n^{1+\frac{1}{cr}})$. Our new tool is a directed variant of the standard boolean noise operator. We show that a generalization of the Bonami-Beckner hypercontractivity inequality exists "in expectation" or upon restriction to certain subsets of the Hamming cube, and that this is sufficient to prove the desired isoperimetric inequality that we use in our data structure lower bound. We also present a structural result reducing the Hamming cube to a Bregman cube. This structure allows us to obtain lower bounds for problems under Bregman divergences from their $\ell_1$ analog. In particular, we get a (weaker) lower bound for approximate near neighbor search of the form $\Omega(n^{1 + \frac{1}{cr}})$ for an $r$-query non-adaptive data structure, and new cell probe lower bounds for a number of other near neighbor questions in Bregman space.Comment: 27 page

On the use of biased-randomized algorithms for solving non-smooth optimization problems

Soft constraints are quite common in real-life applications. For example, in freight transportation, the fleet size can be enlarged by outsourcing part of the distribution service and some deliveries to customers can be postponed as well; in inventory management, it is possible to consider stock-outs generated by unexpected demands; and in manufacturing processes and project management, it is frequent that some deadlines cannot be met due to delays in critical steps of the supply chain. However, capacity-, size-, and time-related limitations are included in many optimization problems as hard constraints, while it would be usually more realistic to consider them as soft ones, i.e., they can be violated to some extent by incurring a penalty cost. Most of the times, this penalty cost will be nonlinear and even noncontinuous, which might transform the objective function into a non-smooth one. Despite its many practical applications, non-smooth optimization problems are quite challenging, especially when the underlying optimization problem is NP-hard in nature. In this paper, we propose the use of biased-randomized algorithms as an effective methodology to cope with NP-hard and non-smooth optimization problems in many practical applications. Biased-randomized algorithms extend constructive heuristics by introducing a nonuniform randomization pattern into them. Hence, they can be used to explore promising areas of the solution space without the limitations of gradient-based approaches, which assume the existence of smooth objective functions. Moreover, biased-randomized algorithms can be easily parallelized, thus employing short computing times while exploring a large number of promising regions. This paper discusses these concepts in detail, reviews existing work in different application areas, and highlights current trends and open research lines

Clustering Phase Transitions and Hysteresis: Pitfalls in Constructing Network Ensembles

Ensembles of networks are used as null models in many applications. However, simple null models often show much less clustering than their real-world counterparts. In this paper, we study a model where clustering is enhanced by means of a fugacity term as in the Strauss (or "triangle") model, but where the degree sequence is strictly preserved -- thus maintaining the quenched heterogeneity of nodes found in the original degree sequence. Similar models had been proposed previously in [R. Milo et al., Science 298, 824 (2002)]. We find that our model exhibits phase transitions as the fugacity is changed. For regular graphs (identical degrees for all nodes) with degree k > 2 we find a single first order transition. For all non-regular networks that we studied (including Erdos - Renyi and scale-free networks) we find multiple jumps resembling first order transitions, together with strong hysteresis. The latter transitions are driven by the sudden emergence of "cluster cores": groups of highly interconnected nodes with higher than average degrees. To study these cluster cores visually, we introduce q-clique adjacency plots. We find that these cluster cores constitute distinct communities which emerge spontaneously from the triangle generating process. Finally, we point out that cluster cores produce pitfalls when using the present (and similar) models as null models for strongly clustered networks, due to the very strong hysteresis which effectively leads to broken ergodicity on realistic time scales.Comment: 13 pages, 11 figure

A Probabilistic Embedding Clustering Method for Urban Structure Detection

Urban structure detection is a basic task in urban geography. Clustering is a core technology to detect the patterns of urban spatial structure, urban functional region, and so on. In big data era, diverse urban sensing datasets recording information like human behaviour and human social activity, suffer from complexity in high dimension and high noise. And unfortunately, the state-of-the-art clustering methods does not handle the problem with high dimension and high noise issues concurrently. In this paper, a probabilistic embedding clustering method is proposed. Firstly, we come up with a Probabilistic Embedding Model (PEM) to find latent features from high dimensional urban sensing data by learning via probabilistic model. By latent features, we could catch essential features hidden in high dimensional data known as patterns; with the probabilistic model, we can also reduce uncertainty caused by high noise. Secondly, through tuning the parameters, our model could discover two kinds of urban structure, the homophily and structural equivalence, which means communities with intensive interaction or in the same roles in urban structure. We evaluated the performance of our model by conducting experiments on real-world data and experiments with real data in Shanghai (China) proved that our method could discover two kinds of urban structure, the homophily and structural equivalence, which means clustering community with intensive interaction or under the same roles in urban space.Comment: 6 pages, 7 figures, ICSDM201