We study a variation of the graph colouring problem on random graphs of
finite average connectivity. Given the number of colours, we aim to maximise
the number of different colours at neighbouring vertices (i.e. one edge
distance) of any vertex. Two efficient algorithms, belief propagation and
Walksat are adapted to carry out this task. We present experimental results
based on two types of random graphs for different system sizes and identify the
critical value of the connectivity for the algorithms to find a perfect
solution. The problem and the suggested algorithms have practical relevance
since various applications, such as distributed storage, can be mapped onto
this problem.Comment: 10 pages, 10 figure

B. Selman

D. J. C. MacKay

D. J. Patterson

D. McAllester

D. Saad

E. Aurell

J. Kubiatowicz

J. Pearl

J. S. Yedidia

J. van Mourik

S. Bounkong

T. R. Jensen

English

arXiv

Crossref

Coloring random graphs and maximizing local diversity

We study a variation of the graph coloring problem on random graphs of finite average connectivity. Given the number of colors, we aim to maximize the number of different colors at neighboring vertices (i.e. one edge distance) of any vertex. Two efficient algorithms, belief propagation and Walksat are adapted to carry out this task. We present experimental results based on two types of random graphs for different system sizes and identify the critical value of the connectivity for the algorithms to find a perfect solution. The problem and the suggested algorithms have practical relevance since various applications, such as distributed storage, can be mapped onto this problem

Bounkong, S.

van Mourik, Jort

Saad, David

Aston Publications Explorer

Coloring Random Graphs and Maximizing Local DiversityS. Bounkong, J. van Mourik, and D. SaadNeural Computing Research Group, Aston University, Birmingham B4 7ET, UK(Dated: February 2, 2006)We study a variation of the graph coloring problem on random graphs of ﬁnite average connectiv-ity. Given the number of colors, we aim to maximize the number of diﬀerent colors at neighboringvertices (i.e. one edge distance) of any vertex. Two eﬃcient algorithms, belief propagation andWalksat are adapted to carry out this task. We present experimental results based on two typesof random graphs for diﬀerent system sizes and identify the critical value of the connectivity forthe algorithms to ﬁnd a perfect solution. The problem and the suggested algorithms have practicalrelevance since various applications, such as distributed storage, can be mapped onto this problem.PACS numbers: 89.75.-k, 02.60.Pn, 75.10.NrI. INTRODUCTIONThe graph coloring problem [1, 2] has received a signif-icant level of attention. Much of this interest stems fromthe fact many real-world optimization problems can berepresented as coloring problems. In the original formu-lation, given q colors, one aims at ﬁnding a coloring solu-tion such that any two connected vertices have diﬀerentcolors. In the current work, the aim is to maximize thenumber of colors at one edge distance of any vertex.One application can be found in the ﬁeld of logistics,where each vertex represents a storage unit. The prob-lem is then to ﬁnd how to distribute the diﬀerent types ofgoods such that, at each site, any type can be retrievedeither from the given unit or from directly adjacent stor-age units. The problem that got us interested in thisproblem is that of distributed data storage where ﬁlesare divided to a number of segments, which are then dis-tributed over the graph representing the network. Nodesrequesting a particular ﬁle collect the required number ofﬁle segments from neighboring nodes to retrieve the origi-nal information. Distributed storage is used in many realworld applications such as OceanStore [3].It should be emphasized that typical properties of themain problem we are interested in should be taken intoaccount when a color assignment algorithm is considered:1) The problem is characterized by a ﬁnite number (of theorder of the graph connectivity) of diﬀerent ﬁle segments.2) An adaptive assignment of colors may be required asthe topology continuously changes due to the emergenceand disappearance of nodes. 3) The networks consideredare of moderate size, 102-103 nodes.Although this problem has not yet been shown to beNP-complete, it seems nonetheless intractable for a largesystem size. Since no research has been carried out onthis speciﬁc problem, no dedicated tools exist either[4].However, as we report in this paper, existing optimiza-tion algorithms can be adapted quite easily to solve thisand similar problems. In particular, we investigate twowell established techniques: belief propagation (BP) anda variant Walksat (WSAT) for this purpose.In this paper, we show how BP and Walksat can beused to solve this particular problem. For a given numberof colors q we identify the transition points in terms ofthe critical connectivity λqc above which the algorithmstypically ﬁnd a perfect coloring. We also calculate theaverage minimum measure of unsatisfaction Eq(λ) as afunction of the connectivity λ. The latter is deﬁned asEq(λ) =∑ni=1 Eqi (λ) where for each vertex i with localconnectivity λiEqi (λ) = min(q, λi + 1)− qi (1)is the diﬀerence between the number of actually avail-able colors at that node qi, and the maximal numberof available colors (at the vertex and its nearest neigh-bors min(q, λi + 1)). In this paper, we only considergraphs with local connectivities λi ≥ q − 1, such thatEqi (λ) = q − qi just counts the number of missing colors.One should note that, contrary to the original graph col-oring problem, the problem of ﬁnding a coloring for ourproblem actually becomes easier with increasing connec-tivity.The main goal of this paper is to introduce the prob-lem, and investigate the performance of the two algo-rithms on realistic system sizes. The full analysis of themodel in the inﬁnite system size is a separate issue thatis currently being investigated.II. THE ALGORITHMSBelief propagation: BP, also called the sum-productalgorithm, relies on iterative message passing to pro-vide near optimal performance at low computationalcost [6, 7]. It is based on conditional probabilistic mes-sages passed from the immediate neighborhood to ﬁndthe most probable assignment of states to variables givenconstraints. In our problem, the constraints correspondto the retrieval of min(q, λi + 1) colors per vertex, fromthe vertex itself and its ﬁrst order neighbors.These constraints can be represented by clusters of ver-tices on a graph as in Fig. 1a, where ‘A’, ‘B1’, ‘B2’,‘C11’ and ‘C12’ correspond to the vertices, while ‘ZA’and ‘ZB1 ’ are check variables corresponding to the con-straints. Checks variables relate to the unsatisfaction of(a) (b)A BB11CC121 2ZAZB1ABBCC111212ZZZZZABCBC112211FIG. 1: (a) A graph representing the successive local con-straints. (b) A bi-partite graph representation.given color assignments for the corresponding node andits direct neighbors. For instance, for the node A withqA available colors (β = 20 for computational reasons)P (ZA|A, {Bi}) = e−β(q−qA) . (2)The graph can be transformed into a bipartite graph,shown in Fig. 1b, which separates the vertices from thechecks. The following update rules can be easily obtainedby naively adapting the original BP rules [6, 7]. Thus,the messages from a check to a vertex are given byP (ZA|A) =∑{Bi}P (ZA|A, {Bi})P ({Bi}|{ZBi}, {ZCi·}) ,∑{Bi}P (ZA|A, {Bi})∏iP (Bi|ZBi , {ZCi·}) ,(3)while the message from a vertex to a check is given byP (A|{ZBi}) = αA∏{ZBi}P (ZBi |A) , (4)Finally, the node pseudo-posterior is given byP (A|ZA, {ZBi}) = αP (ZA|A)∏{ZBi}P (ZBi |A) , (5)where αA and α are normalization coeﬃcients. Note thatthe factorization in (3) is a relatively crude approxima-tion even in the large system limit, as the {Bi} nodes arecorrelated. To deal with this properly, a more advancedanalysis using a cluster expansion [8] is currently beenundertaken. Nevertheless, as we will see these approxi-mations work remarkably well.If convergence of the BP algorithm is reached, the col-ors of vertices whose (pseudo) posterior is greater thana pre-deﬁned threshold set at 0.9 in our experiments canbe ﬁxed. If no such high posterior exists then the vertexwith the highest posterior value has its color ﬁxed. Then,the update rules are re-iterated and the decimation pro-cess repeated until a global coloring is reached.A major drawback of the BP algorithm is that con-vergence is not guaranteed for graphs with loops due tofragmentation of the solution space. Random initializa-tion results in the emergence of competing local solutionsand conﬂicting messages, leading to non convergence.Time averaging [9] is a way of getting around the prob-lem by carrying out the decimation and color ﬁxing pro-cess according to the average posterior (over time i.e. anumber of iterations) instead of instantaneous posterior.In the case of non-convergence, this method decimatesthe vertex with the strongest average coloring probabil-ity over all competing solutions and thus reduce the ﬂuc-tuations due to the competition. After several trials, atime window of 30 iterations was chosen for all numeri-cal data presented here. We have opted for BP combinedwith time averaging due to its improved performance androbustness (e.g., in the distributed storage applicationnodes may suddenly switch oﬀ and on).Walksat: Walksat is a local search algorithm, originallydesigned to maximize the number of satisﬁable clausesin problems that assume a conjunctive normal form [10].Although Walksat may seem to be suboptimal at ﬁrstsight, studies have shown it to be a powerful tool [11].Many variants of the original algorithm exist [10, 12, 13].In this study, we have adapted the variant referred to asSKC; it uses the notion of variable breakcount, deﬁnedas the number of clauses that are currently satisﬁed, butwould become unsatisﬁed if the variable assignment wereto be changed. The SKC variable selection is as follows:1. If there are variables with breakcount equal to 0,randomly select one such variable.2. Otherwise• with probability p randomly select a variable.• with probability 1− p randomly select a vari-able with minimal breakcount.3. Flip the selected variable.4. Repeat until all clauses are satisﬁed or until themax-iterations is reached.In our problem, the breakcount of a variable is given bythe number of vertices for which the change of assignmentwould decrease qi. Henceforth, the breakcount dependson the replacement color. In step (1) of the SKC pro-cedure, the selected replacement color is the one whichleads to a breakcount equal to 0 (if more than one, chooserandomly). In the step (3), a replacement color is selectedat random. In our ﬁrst few attempts, this adaptation ofthe Walksat algorithm showed mixed results, which wereup to 50% worse than those obtained with BP. We there-fore adapted another local search algorithm [14] relatedto Walksat. This algorithm is also iterative and basedon a mixture of gradient and “noisy” descent. At eachiteration, one of these two descents is chosen at random,with some probability. Similarly to the Walksat algo-rithm, this step is repeated until all checks are satisﬁed,or the maximal number of iterations is reached.The gradient descent is operated by the GSAT algo-rithm [15], which changes at each an iteration the vari-able assignment that leads to the greatest decrease in thenumber of unsatisﬁed clauses. In our problem, changeswill correspond to the greatest decrease in unsatisfaction(a) (b)3 3.5 4 4.50123456λUnsatisfaction (%)125K250K500K12M120M3 3.5 4 4.5020406080100λPerfect colouring (%)125K250K500K12M120M(c) (d)3 3.5 4 4.5 50123456λUnsatisfaction (%)125K250K500K12M3 3.5 4 4.5 5020406080100λPerfect colouring (%)125K250K500K12MFIG. 2: Walksat performance on linear and Poissonian graphs(n=100) for nbit from 125K to 120M iterations and connec-tivity λ. (a) Unsatisfaction measure - linear. (b) Percentageof perfect coloring - linear.(c) Unsatisfaction measure - Pois-sonian. (d) Percentage of perfect coloring - Poissonian.as deﬁned in (1). “Noisy” moves in the original algo-rithm [14] are replaced here by the SKC heuristic. Theresulting algorithm is a mixture between SKC and GSAT,which is parameterized by a probability pm, set to 0.5.Our experiments show that the combined algorithm per-forms signiﬁcantly better than SKC with no added cost.If not all checks are satisﬁed at the maximal numberof iterations (the choice of which is discussed in the nextsection) the GSAT algorithm is iterated until a local min-imum is reached. The combined algorithm, referred tohere as Walksat, shows similar results to those obtainedusing BP both in terms of unsatisfaction Eq(λ) and thenumber of perfect coloring solutions found.III. SIMULATIONSExperiments were carried out for q =4 colors for twosystem sizes (n) of 100 and 1000 vertices. The two typesof graphs studied have an average connectivity λ:• (cut-) Poissonian: where vertices have local con-nectivities λi given byλi = λmin + zλ−λmin = q − 1 + zλ−q+1 , (6)where zλ−q+1 is randomly drawn from a Poissondistribution with parameter λ− q + 1.• Linear: where vertices have local connectivitiesλi = λ+ zλ−λ (7)where λ is the largest integer smaller or equal toλ, and zλ−λ = 1 with probability λ−λ and 0otherwise.We study the most interesting range of average connec-tivities from λ = 3 to λ = 5 with a step of 0.1. For eachλ, 1000 graphs of each type were randomly generated andthen colored by both the BP and Walksat algorithms.Graph characteristics: Both graphs and constraintsare born from the original problem we have set to solve,namely distributed storage. We point out two observa-tions that may help in getting insight into the characteris-tics of the problem and solutions found by the algorithms:1) The number of checks is always equal to the numberof vertices as each vertex is associated with a check, thatconnects it to all vertices at one edge distance. Thischeck is obeyed when the vertex can retrieve all possi-ble colors from vertices at one edge distance. 2) Edgesare undirected: if vertex ‘B’ is connected to the checkof ‘A’, then vertex ‘A’ is also connected to the check of‘B’. Hence, there are always nλ2 short loops, which corre-spond to the number of edges, in the belief network evenin the large system limit. When the connectivity valueλ increases, the number of loops increases as well, but italso becomes easier to get a lower value for the averageunsatisfaction. Therefore, it is unclear whether the inﬂu-ence of the presence of loops on the performance of the(current) BP algorithm will increase or decrease with λ.Walksat performance: In the Walksat algorithm, themaximal number of iterations nbit is an important pa-rameter. A greater value increases performance, but alsocomputational cost. Unfortunately, the relation betweenperformance and cost is not linear and it is therefore dif-ﬁcult to estimate the optimal number of iterations. Inorder to understand this relation, we carried out severalsimulations with diﬀerent values of nbit for the two sys-tems sizes and all connectivity values.Figure 2 show the results obtained for a system sizen=100 and a range of limits on the number of iterations.One notices that improvements in terms of unsatisfactionand perfect coloring are negligible for λ≤3.8 and λ≤4.1in linear and Poissonian graphs, respectively. In theseregions, no perfect solutions are found and the Walksatalgorithm stops when the maximum number of iterationsis reached and returns a suboptimal solution, even forlarger nbit values. Perfect coloring solutions exist andare found for λ  3.8 and λ  4.1 in linear and Poisso-nian graphs, respectively. However, a larger number ofvertices will also require an exponentially larger numberof iterations to achieve the same performance.To compare the performance of the Walksat and BPalgorithms, we take the results achieved by Walksat forroughly the same computational time to the one usedby BP. This means nbit = 500K and nbit = 12M itera-tions for systems sizes of 100 and 1000 vertices, respec-tively. We also modify the Walksat algorithm describedin Sec. II such that the unsatisfaction returned is the low-est value over all examined color assignments and not theone corresponding to the nearest local minimum.BP vs. Walksat - a comparison: Figure 3 shows thatfor small graphs (100 nodes), and far away from the crit-ical connectivity, BP is generally outperformed by the(a) (b)3 3.5 4 4.50123456λUnsatisfaction (%)BP 100BP 1000WSAT 100WSAT 10003 3.5 4 4.5020406080100λPerfect colouring (%)BP 100BP 1000WSAT 100WSAT 1000(c) (d)3 3.5 4 4.5 50123456λUnsatisfaction (%)BP 100BP 1000WSAT 100WSAT 10003 3.5 4 4.5 5020406080100λPerfect colouring (%)BP 100BP 1000WSAT 100WSAT 1000FIG. 3: Comparison of BP and Walksat algorithms for lin-ear and Poissonian graphs. (a) Unsatisfaction measure - lin-ear.(b) Percentage of perfect coloring- linear. (c) Unsatisfac-tion measure - Poissonian. (d) Percentage of perfect coloring- Poissonian.Walksat algorithm. We believe this is partially due tothe presence of small loops, as discussed earlier, and theuse of generalized BP [16] is currently being investigatedto improve performance; nevertheless one should notethat even the approximative BP algorithm works sur-prisingly well considering the crude approximation madein (3). In addition, while Walksat clearly outperformsBP for 100 nodes systems, this is deﬁnitely not the casefor 1000 nodes systems close to the critical connectivity,where results obtained by BP are better both in terms ofthe percentage of perfectly colored cases and in terms ofthe minimal average unsatisfaction, both for linear andPoissonian graphs. For increasing system sizes and givencomputing resources BP is likely to outperform Walksatin the relevant regions.IV. DISCUSSIONWe study a variation of graph coloring on randomgraphs with ﬁnite average connectivity, aimed at maxi-mizing the number of colors accessible by a vertex withinone edge distance. The problem is of practical relevance,especially in the area of distributed storage. The BP andWalksat algorithms were adapted to perform the task.We present experimental results for two types of ran-dom graphs and system sizes, and identify critical con-nectivity values above which the algorithms ﬁnd a per-fect solution. For q=4 colors, the critical connectivitiesfound are around 4 and 4.4 for linear and graphs andPoissonian graphs, respectively. In principle, the meth-ods presented are applicable for random graphs of anyconnectivity proﬁle and number of colors.We have found that both algorithms give qualita-tively very similar results with similar computing costs.The relative eﬃciency of both algorithms, in terms ofthe quality of obtained solutions and computing time,does however depend on the combination of parameters(λ, q, n) and graph characteristics. A more detailedanalysis of this will be the subject of a separate study, aswill be the thermodynamic phase diagram for this model.Further research will focus on improving the messagepassing approach by using the exact cluster expansion inthe large system limit (i.e. focusing on stars and edgesas our fundamental clusters instead of stars and nodes),combined with generalised BP, which will also provide uswith a phase diagram for the model. It is assumed thatthis approach will remove the inﬂuence of short loopsand therefore improve the performance of the algorithm,especially at low connectivity values.[1] T R Jensen and B Toft, Graph Coloring Problems, WileyInterscience (1995)[2] J van Mourik and D Saad, Phys Rev E 66 056120 (2002)[3] J Kubiatowicz et al, OceanStore: An Extremely Wide-Area Storage System, UCB//CSD-00-1102 U.C. Berkeley(1999)[4] It should be mentioned that diﬀerent systems of similartopology have been investigated in [5][5] M Me´zard and G Biroli Phys Rev Lett 88 025501 (2002)[6] J Pearl, Probabilistic Reasoning in Intelligent SystemsMorgan Kauﬀmann Publishers (1988)[7] D J C MacKay, Information Theory, Inference, andLearning Algorithms, Cambridge University Press (2003)[8] G An, Jour of Stat Phys, 52 727 (1988)[9] J van Mourik, Time averaged belief propagation, inpreperation (2005); B Fabre, Algorithms for coloring ran-dom graphs, MSc thesis, Aston University (2003); YFortune, Algorithms for solving optimization problemon large random graphs, MSc thesis, Aston University(2004).[10] B Selman, H Kautz and B Cohen, DIMACS Series in Dis-crete Mathematics and Theoretical Computer Science,26 521 (1996)[11] E Aurell, U Gordon and S Kirkpatrick, NIPS 2005 18(2005)[12] D McAllester, B Selman and H Kautz Proc. AAAI-97,321 (1997)[13] D J Patterson and H Kautz, Electronic Notes in DiscreteMathematics (ENDM)9, 1 (2001)[14] B Selman, H Kautz and B Cohen Proc. AAAI-94, 337(1994)[15] B Selman, H J Levesque and D Mitchell, Proc. AAAI-92,440 (1992)[16] J S Yedidia, W T Freeman and Y Weiss, NIPS 2000 13689 (2000)

https://publications.aston.ac.uk/id/eprint/1390/1/NCRG_2006_004.pdf

Colouring random graphs and maximising local diversity

Colouring random graphs and maximising local diversity

Abstract

Similar works

Full text

Available Versions

Crossref

Aston Publications Explorer