In this paper we present an analytic study of sampled networks in the case of
some important shortest-path sampling models. We present analytic formulas for
the probability of edge discovery in the case of an evolving and a static
network model. We also show that the number of discovered edges in a finite
network scales much slower than predicted by earlier mean field models.
Finally, we calculate the degree distribution of sampled networks, and we
demonstrate that they are analogous to a destructed network obtained by
randomly removing edges from the original network.Comment: 10 pages, 4 figure

A. Lakhina

Attila Fekete

Gábor Vattay

J. Leskovec

Márton Pósfai

English

arXiv

Crossref

Shortest path discovery of complex networks

In this Rapid Communication we present an analytic study of sampled networks in the case of some important shortest-path sampling models. We present analytic formulas for the probability of edge discovery in the case of an evolving and a static network model. We also show that the number of discovered edges in a finite network scales much more slowly than predicted by earlier mean-field models. Finally, we calculate the degree distribution of sampled networks and we demonstrate that they are analogous to a destroyed network obtained by randomly removing edges from the original network

Fekete, A

Vattay, G

Posfai, M

ELTE Digital Institutional Repository (EDIT)

Shortest path discovery of complex networksAttila Fekete,* Gábor Vattay,† and Márton Pósfai‡Department of Physics of Complex Systems, Eötvös University, Pázmány P. sétány 1/A., H-1117 Budapest, HungaryReceived 8 October 2008; revised manuscript received 14 May 2009; published 23 June 2009In this Rapid Communication we present an analytic study of sampled networks in the case of someimportant shortest-path sampling models. We present analytic formulas for the probability of edge discovery inthe case of an evolving and a static network model. We also show that the number of discovered edges in afinite network scales much more slowly than predicted by earlier mean-field models. Finally, we calculate thedegree distribution of sampled networks and we demonstrate that they are analogous to a destroyed networkobtained by randomly removing edges from the original network.DOI: 10.1103/PhysRevE.79.065101 PACS numbers: 89.75.Hc, 64.60.aq, 89.20.HhComplex networks have attracted significant interest inrecent years 1,2. In most cases, the entire structure of thenetwork is unknown and one is left with statistical samplesof the original network 3,4. The sampling of Internet topol-ogy is one of the greatest challenges due to its enormous sizeand decentralized structure. It motivated numerous studieson the relationship between the original and the samplednetwork, including the degree distribution 5–7 and the ex-pected size of the network 8. Recently, Internet samplingmethods have emerged that rely on the measurement tooltraceroute, which returns the sequence of IP addresses of thenetwork nodes along the path between the measurement hostand a given destination host. An abstraction of the networkdiscovery process consists of selecting a set of source andtarget nodes and finding the shortest paths between sourceand destination pairs. A node or an edge of the network isdiscovered if it belongs to one of those shortest paths. Thestatistical properties of the discovered network have beenstudied extensively by Dall’ Asta et al. 9. The mean-fieldapproximation has been developed in the limit of low sourceand target density ST1 by neglecting the correlation ofdifferent shortest paths.In this Rapid Communication we present exact results forcertain networks. A surprising finding is that the networkdiscovery process is slower in these systems than it is pre-dicted by the mean-field theory. While in mean-field approxi-mation the number of discovered links scales with the prod-uct of the number of the source and target nodes, ourapproach predicts a scaling only with their sum. The lowernumber of discovered edges is a result of the high degree ofoverlapping between shortest paths. Our other importantfinding concerns the degree distribution of the discoverednetwork. We will show that it is analogous with a destroyednetwork where a fraction of the edges of the original networkhas been randomly removed.We investigate two main discovery strategies. In peer-to-peer P2P sampling each node is selected simultaneously forboth source and target with probability . Computer applica-tions using the peer-to-peer principle discover the networkthis way, hence the name. In disjunct DI sampling eachnode is selected for source or target but not for both withprobabilities S and T. This strategy is used in Internet map-ping projects, where source computers belong to the mea-surement infrastructure, while a large number of random ad-dresses are selected as targets.We start our analysis with the discovery of a tree. Themost important observation permitting exact calculations inthis case is that an edge separates the tree into two sides. Anedge is discovered only if the source and the target nodesreside on different sides of the edge. Let us denote the eventthat a node is selected as a source or target by S and T,respectively. Furthermore, we denote the event that at leastone source or target node resides on the “left” or “right” sideof the edge by SL,R and TL,R, respectively. The event that alink is discovered, D, provided that its two sides L and R areknown, is clearly D= SLTR+ SRTL. Therefore, we can ex-press the conditional probabilityPDL,R = PSLL,RPTRL,R + PSRL,RPTLL,R− PSLTLL,RPSRTRL,R .The probabilities arising in this expression can be calculatedeasily: PS L ,R=1− PNS¯, PT L ,R=1− PNT¯  andPST L ,R=1− PNS¯− PNT¯ + PNS¯T¯ , where =L orR, NL and NR are the number of nodes on the two sides of thelink, and the overlines denote complement events.Let us consider an evolving network where each newnode is attached randomly to one of the nodes of the existingnetwork. The structure of this network will be a tree. Sincethe network is connected the cluster sizes NL and NR mustsatisfy the relation NL+NR=N, where N is the size of thewhole network. In the thermodynamic limit N→ we obtainPD NL=1−NL, where we have introduced = PS¯T¯ . Theprobability  in the different sampling models is related tothe source and target densities in a simple way: = 1 −  P2P1 − S − T DI,  1where  ,S ,T 0,1, S+T1. If S+T1 in theDI sampling model, then we can write PD NL1−exp− S+TN be, where be=NLN−NL is the number ofshortest paths that traverse a given link called betweenness*fekete@complex.elte.hu†vattay@elte.hu‡posfai@complex.elte.huPHYSICAL REVIEW E 79, 065101R 2009RAPID COMMUNICATIONS1539-3755/2009/796/0651014 ©2009 The American Physical Society065101-1centrality. Compare this result with the mean-field model ofDall’ Asta et al. 9: PDmf be1−exp−STbe.The probability of finding an arbitrary edge by tracerouteprobes can be given now straightforwardly:d = 	NL=0PDNLPNL = 1 − H1 , 2where H1z=	NLPNLzNL is the generating function of thecluster size distribution PNL.Expression 2 has been tested on the Dorogovtsev-Mendez DM network growth model 10, a generalizationof the Barabási-Albert BA model 11, where new nodeswith m new links are attached to old nodes with degree-dependent probability ki=ki−m+am	iki−m+am , where a	0. Thegrowing tree corresponds to m=1. We calculated the distri-bution PNL for this model analytically in Ref. 12. Thegenerating function can be expressed in terms of hypergeo-metric functions H1z=z2F11− ,1 ,2− ;z−z1−2−2F12− ,1 ,3− ;z and = 11+a . At a=1 we recover the originalBA preferential attachment model with scale-free degree dis-tribution and at a=+ we obtain uniform attachment prob-ability with exponential degree distribution. In these cases dcan be expressed with elementary functionsd = −1 − ln1 −  if a = + i.e.,  = 0 ,1 − 2ln1 + 1 − if a = 1i.e.,  = 1/2 .  3Figure 1 shows simulations for the P2P sampling model at=0 and 1/2. The analytic results 3, plotted with dashedlines, fit the simulation data excellently.From the point of view of the efficiency of the discoveryprocess, it is important to calculate how many edges can bediscovered with a given number of source nS and targetnodes nT. For the Internet discovery the disjunct samplingmodel is relevant, where T+S= nT+nS /N=n /N=1−1. The series expansion of Eq. 2 yields d=1−	NLPNL1−nN NL. We can rearrange the series by addingand subtracting the terms 1−n NLN and averaging them sepa-rately d=nNLN −	NLPNL1−nN NL−1+n NLN .Several authors have pointed out that the distribution ofbe=NLN−NL follows a universal power-law tail in treeswith exponent −2 12–14. It also implies that asymptoticallyPNLcNL−2 in an arbitrary tree for NL1. Specifically, c=1− in the DM model. Using this asymptotic form we cancalculate the leading behavior in the N→ limit d= nNLN−cLi21−n /N+c26 −cnN ln N−, where Li2x is thedilogarithm function and 0.5772 is the Euler constant.For small argument Li21−x can be expanded by using Eu-ler’s reflection formula Li21−x=−Li2x+26 − lnxln1−x−x+ 26 +x lnx+¯. Finally we get d= nNLN +c nN−cnN ln n−cnN.To process this further, let us express the term NL morestraightforwardly. The sum of be for all edges clearly equalsthe total length of the shortest paths between all possiblepairings of nodes: 	eEbe=	i,jVli,j. Since b=1N−1	eEbeand l= 2NN−1	i,jVli,j we can write lN /2= b. Therefore,the average branch size can be given as NL= l /2+ NL2 /N, where NL2 /N= 1N	NL=1N cNL2NL2=c. For a large, butfinite network the average number of discovered edges isnd= N−1d, that isnd  n l2 − c ln n + 2c − c 4in the limit 1n=nS+nTN, The above result shows thatnd depends on the sum of nS and nT. This is in contrast tothe mean-field model, which predicts that nd scales with theproduct of nS and nT. The logarithmic term of Eq. 4 ac-counts for the possibility that a new measurement node isplaced at a node discovered by previous measurement nodes.The inset of Fig. 1 displays simulation results and the for-mula corresponding to the P2P sampling.We continue with the analysis of a static model wherenodes are randomly connected with a prescribed degree dis-tribution pk. This “configuration model” is a generalizationof the Erdős-Rényi ER model 15, where the degree dis-tribution is Poissonian. It has been shown in 16 that thegenerating function of branch sizes H1z satisfies the im-plicit equation H1z=zG0H1z / k, where G0z=	kpkzkis the generating function of the degree distribution. In theconfiguration model loops become irrelevant in the thermo-dynamic limit N→+ and each edge is a part of a tree. Here,NL and NR are independent and the joint probability functionhas a product form PNL ,NR= PNLPNR. The summationin d can be carried out separately for NL and NR, whichyields00.20.40.60.810 0.2 0.4 0.6 0.8 1πd(ρ)ρα = 0α = 1/20501001502002500 10 20 30 40 50ndnFIG. 1. Color online Discovery probability of edges d asthe function of the measurement node density  for P2P sampling ofevolving trees. Data points are averaged over 100 realizations ofN=104 node BA trees with =0 and 1/2. Solid lines show thecorresponding analytic solution 3 with =1−. Inset: the numberof discovered edges nd as the function of the number of the mea-surement nodes n. The solid line represents Eq. 4, whereas thedotted line shows its leading term ln /2 with l=15.48 and 9.045for =0 and 1/2, respectively.FEKETE, VATTAY, AND PÓSFAI PHYSICAL REVIEW E 79, 065101R 2009RAPID COMMUNICATIONS065101-2d=21−H1PS¯ 1−H1PT¯−1−H1PS¯ −H1PT¯ +H1PS¯T¯ 2. 5In the case of P2P discovery this can be reduced tod=1−H11− 2. 6This formula can be tested on the ER model, withG0z=ekz−1. The cluster size distribution can be given by theLambertWfunctionH1z=−W−ke−kz/k. Simulationresults are presented in Fig.2a. The analytic result6isalso shown for comparison. One can see that it is discontinu-ous at zero density if k 1, when a giant componentemerges in the network. The simulation data deviate from theanalytic solution around the discontinuity due to finite-scaleefects. The size of the jump isP0=1−H112, which isprecisely the probability of infinitely large branches beingatached to both sides of an edge. IfP0is regarded as anorder parameter, the observed phenomenon resembles aphase transition atk=kc=1.We also generated networks with power-law degree dis-tribution using the hidden-variable model introduced in17–20. Simulations are shown in Fig.2bwith degree ex-ponent = 3. Note that the analytic solution is discontinuousat zero density, i.e.,P0 0, for al k 0. The phase transi-tion can be observed again, since the analytic solution—andP0—is independent of k below a critical pointkc= −1. Indeed, data points almost colapse atk= 0.5 and 1which are belowkc =3 1.3684. The phenomenon occurswhen the degree generating functionG0zdepends linearlyonk. This is characteristic of pure power-law distributionsuntilkis below the critical valuekc.Now we turn our atention to the degree distributionPdk of the discovered nodes. In our analysis we consideronly the contribution of those shortest paths tok, whichtraversea given node. We wil show thatPdk is analogousto the degree distribution of a partialy severed network ob-tained by random edge pruning21,22. This duality betweenthe sampling and the destruction of networks is very surpris-ing considering the striking diferences between the two pro-cesses, e.g., the explored network is surely connected in con-trast to the destroyed one.Let us consider a nodevwith original degreek. If everylink is removed independently with probabilityp, thenk,the degree of the node after random edge removal, wil fol-low a binomial distribution:Pkk= kk 1−pkpk−k. Con-sequently,Pprunedk =k=kkk 1−pkpk−kP0k. 7Regarding the sampling process we examine a randomlyselected node of the discovered networkv Vdin the staticmodel first. Let us suppose that the sizes of the branches withoriginal degreekareN1,N2,...,Nksee Fig.3. For the sakeof simplicity we discuss only the P2P sampling model, wherethe probability of placing a measurement node in branchiissimply1−Ni. Since branch sizes are independent we canaverage overNiseparately. The results we obtain indicatethat measurement nodes can be found in diferent brancheswith probability 1 −H1 .We can see from Fig. 3that the degree of a discoverednodekequals the number of branches where measurementnodes can be found in. It folows that Pdkk= 1Pv Vdk kk 1−H1 kH1k−k , where 2 k k. Thesubscript ofPdrefers to the probability distribution restrictedto the discovered network. In order to obtain the distributionofk one should average this probability overPdk, thedistribution ofthe original degrees of the discovered nodes.This distribution can be obtained byPdk=Pv VdkP0kPv Vd ,soPdk =k=kkk 1−H1kH1k−k P0kPv Vd , 8where k 2 and Pv Vd=1−G0H1 −1−H1 G0H1 . It is evident from Eqs.7and8thatPdk equalsPprunedk—normalized properly fork 2—ifp=H1 . In other words the discovered network is equiva-lent with an edge destroyed one.In the case of an evolving network at least one of thebranches, sayNk, tends to infinity asN→ , so the probabil-00.20.40.60.810 0.20.40.60.8 1π d(ρ)ρ(a)k=4.0k=2.0k=1.0k=0.50 0.20.40.60.8 100.20.40.60.81π d(ρ)ρ(b)k=4.0k=2.0k=1.0k=0.5FIG. 2. Color online Discovery probability of edges as thefunction of the measurement node density for staticaPoissonianandbpower-law networks. 100 P2P samplings were averaged inN=104size networks with average degreesk= 0.5, 1, 2, and 4.Solid lines show analytic formula6. Note that the analytic resultsfork= 0.5 and 1 are the same in the case of the power-law model.This result is confirmed by the numerical simulations, where datapoints colapse almost completely.N3N4N5N6Nk N1 N2vFIG. 3. Sketch of an arbitrary vertexvwith degreekand theemerging branches with sizesN1,N2,...,Nk. Shaded circles repre-sent branches where measurement nodes can be found in. Thicklines symbolize the discovered edges of nodev.SHORTEST PATH DISCOVERY OF COMPLEX NETWORKS PHYSICAL REVIEW E79, 065101R 2009RAPID COMMUNICATIONS065101-3ity that a measurement node can be found in the kth branchtends to 1. In order to circumvent this effect let us redefinethe network in such a way that every link should be directedtoward the gigantic side of the network. Let q=k−1 denotethe in-degree of nodes in this directed network. It is easy tosee that the discovered in-degree q will be equal to thenumber of branches where measurement nodes can be foundin. We can follow the same procedure as in the case of thestatic model. We only need to replace k and k in Eq. 8 withthe corresponding in-degrees q and q, and the normalizationconstant with PvVd=1−G0inH1.Simulation results are shown for both static and evolvingnetworks in Fig. 4. Note that we have assumed above thatH1 is independent of q. This is only an approximation inthe case of the evolving network model. However, H1 qcan be calculated exactly for the DM model, which is shownwith dotted lines 23.In conclusion we presented a study of network discoveryprocesses. We derived analytically the probability of findingan arbitrary link of the network via shortest-path networkdiscovery. We considered both static and evolving randomnetworks with various sampling scenarios. We also demon-strated an important duality between the discovery of net-works by shortest paths and the destruction of the same net-work by edge removal.The authors thank the partial support of the National Sci-ence Fund Hungary Grant No. OTKA 77779, the NationalOffice for Research and Technology Grant No. NAP 00635/2005, and the EU IST FET Onelab2 Project Grant No.FP7-224263.1 R. Albert and A.-L. Barabási, Rev. Mod. Phys. 74, 47 2002.2 S. N. Dorogovtsev and J. F. F. Mendes, Adv. Phys. 51, 10792002.3 J. Leskovec and C. Faloutsos, in Proceedings of the SIGKDDACM, New York, NY, 2006, pp. 631–636.4 S. H. Lee, P.-J. Kim, and H. Jeong, Phys. Rev. E 73, 0161022006.5 A. Lakhina, J. W. Byers, M. Crovella, and P. Xie, INFO-COM’03 IEEE, New York, NY, 2003, Vol. 1, pp. 332–341.6 T. Petermann and P. De Los Rios, Eur. Phys. J. B 38, 2012004.7 A. Clauset and C. Moore, Phys. Rev. Lett. 94, 018701 2005.8 F. Viger, A. Barrat, L. Dall’Asta, C.-H. Zhang, and E. D.Kolaczyk, Phys. Rev. E 75, 056111 2007.9 L. Dall’Asta, I. Alvarez-Hamelin, A. Barrat, A. Vazquez, andA. Vespignani, Phys. Rev. E 71, 036135 2005.10 S. N. Dorogovtsev, J. F. F. Mendes, and A. N. Samukhin, Phys.Rev. Lett. 85, 4633 2000.11 A.-L. Barabási and R. Albert, Science 286, 509 1999.12 A. Fekete, G. Vattay, and L. Kocarev, Phys. Rev. E 73,046102 2006.13 G. Szabó, M. Alava, and J. Kertész, Phys. Rev. E 66, 0261012002.14 D.-H. Kim, J. D. Noh, and H. Jeong, Phys. Rev. E 70, 0461262004.15 P. Erdős and A. Rényi, Publ. Math. Inst. Hung. Acad. Sci. 5, 171960.16 M. E. J. Newman, S. H. Strogatz, and D. J. Watts, Phys. Rev.E 64, 026118 2001.17 K.-I. Goh, B. Kahng, and D. Kim, Phys. Rev. Lett. 87, 2787012001.18 G. Caldarelli, A. Capocci, P. De Los Rios, and M. A. Muñoz,Phys. Rev. Lett. 89, 258702 2002.19 B. Söderberg, Phys. Rev. E 66, 066121 2002.20 M. Boguñá and R. Pastor-Satorras, Phys. Rev. E 68, 0361122003.21 R. Cohen, K. Erez, D. ben-Avraham, and S. Havlin, Phys. Rev.Lett. 85, 4626 2000.22 D. S. Callaway, M. E. J. Newman, S. H. Strogatz, and D. J.Watts, Phys. Rev. Lett. 85, 5468 2000.23 A. Fekete and G. Vattay unpublished.10-710-610-510-410-310-210-11000 0.2 0.4 0.6 0.8 1Pdkρ(a)k = 2k = 3k = 4k = 5k = 6k = 70 0.2 0.4 0.6 0.8 110-310-210-1100Pdqρ(b)q = 1q = 2q = 3q = 4q = 5q = 6FIG. 4. Color online The probability of discovered degreePdq and in-degree Pdq as the function of  in P2P samplingmodel. The original networks are N=104 node a static ER and bevolving BA graphs. Data points are averaged for ten networks withten samplings in each realization. Solid lines consist of analyticsolution 8. Exact solution for the evolving model is shown withdotted lines for comparison.FEKETE, VATTAY, AND PÓSFAI PHYSICAL REVIEW E 79, 065101R 2009RAPID COMMUNICATIONS065101-4

https://edit.elte.hu/xmlui/bitstream/10831/43284/1/PhysRevE.79.065101.pdf

Shortest path discovery of complex networks

Abstract

Similar works

Full text

Available Versions

Crossref

ELTE Digital Institutional Repository (EDIT)