Bestavros, Azer

English

Boston University Institutional Repository (OpenBU)

Boston UniversityOpenBU http://open.bu.eduBU Open Access Articles BU Open Access Articles2002-05-02Practical and efficient constructionof network caricaturesThis work was made openly accessible by BU Faculty. Please share how this access benefits you.Your story matters.VersionCitation (published version): A Bestavros. 2002. "Practical and Efficient Construction of NetworkCaricatures."https://hdl.handle.net/2144/26101Boston University1http://www.cs.bu.edu/groups/wingPractical and Efficient Construction of Network CaricaturesAzer BestavrosJoint work withKhaled Harfoush & John ByersAzer Bestavros Practical and Efficient Construction of Network Caricatures 2Q: What does the net look like??SenderReceiversUsing only end-to-end observationsAzer Bestavros Practical and Efficient Construction of Network Caricatures 3A: A bunch of “independent” pipesSenderReceiversThat’s how TCP views itAzer Bestavros Practical and Efficient Construction of Network Caricatures 4Q: Why is that not good enough?A motivating example:? How to construct an overlay network to move 100MB of data from grid node A to grid nodes B and C?AB C100MBAzer Bestavros Practical and Efficient Construction of Network Caricatures 5Time = 10sec1012208CAB20sec100MBA: The devil is in the details!A10 108CBTime = 12.5sec100MBAzer Bestavros Practical and Efficient Construction of Network Caricatures 6Other motivating examples…Aggregate congestion control:? How to partition a set of flows into “congestion-equivalent” classes?Parallel downloads from multiple servers:? Which m (out of n) servers to select to maximize aggregate download bandwidth?Request scheduling at busy media servers:? How to prioritize requests to avoid competition for shared network resources?2Azer Bestavros Practical and Efficient Construction of Network Caricatures 7Q: Why not use physical net maps?Azer Bestavros Practical and Efficient Construction of Network Caricatures 8A: It’s overkill!SenderReceiversAzer Bestavros Practical and Efficient Construction of Network Caricatures 9Q: How much detail is enough?? Need to determine the level of sharingbetween a sender and any subset of receivers w.r.t. a metric of interest. ? Example metrics: ? Number of hops (e.g., router or AS hops) ? Delay (e.g., transmission or queuing delays)? Bandwidth (e.g., available or bottleneck)? Packet loss rateAzer Bestavros Practical and Efficient Construction of Network Caricatures 10A EDCBSLogical Topology(link = path segment)From routing to logical topologiesA EDCBSRouting Topology(link = hop)Azer Bestavros Practical and Efficient Construction of Network Caricatures 11Labeled logical topologies? Labeling a logical topology quantifies the sharing among subsets of endpoints (w.r.t. a metric)A EDCBS3%0%1%2%0% 0%1%2%A EDCBS1001010062210 10015510Packet loss rate Bottleneck bandwidthAzer Bestavros Practical and Efficient Construction of Network Caricatures 12Hiding uninteresting structures? Uninteresting links in the logical topology are those with metrical values above or below certain thresholds of interest? Examples:? Hide links with losses below 2%? Hide links with bandwidth above 100 Mbps? Hide links with delays below 1msec3Azer Bestavros Practical and Efficient Construction of Network Caricatures 13Loss-Induced Topology(labels = loss rate)312120 0SA B C D EHiding unobservable segmentsLogical Topology(labels = loss rate)3012120 0SA B C D EAzer Bestavros Practical and Efficient Construction of Network Caricatures 14BW-Induced Topology(labels = bandwidth)100101551010010100SA B C D EHiding over-provisioned segmentsLogical Topology(labels = bandwidth)100101551010062210 100SA B C D EAzer Bestavros Practical and Efficient Construction of Network Caricatures 15MINT(loss > 4%)MINT(loss > 3%)Metric-Induced Network TopologyPhysical LogicalMINT = a summary of the level of sharing between a sender and any subset of receivers w.r.t. a metric of interestAzer Bestavros Practical and Efficient Construction of Network Caricatures 16Problem statement? Given: ? A set of senders and receivers? A metric and associated  sensitivity threshold? Do the following:? Infer the MINT observable at a sender? Label the inferred MINT? Integrate MINTs obtained at different times? Integrate MINTs obtained at different sendersAzer Bestavros Practical and Efficient Construction of Network Caricatures 17Two approaches:? Prior Work:Solve the problem using “metric-specific” or “technology-specific” techniques, e.g.:? Determine if paths share a bottleneck [RKT’00]? Quantify shared losses [CDLPT’99] [HBB’00]? Quantify shared bottleneck bandwidth [HBB’01]? Our Work:Provide generic solutions that work on a variety of metrics satisfying certain propertiesAzer Bestavros Practical and Efficient Construction of Network Caricatures 18MINT Framework [BBH:Infocom02]? Assumes the availability of an “oracle” that quantifies the sharing f(LS) between a sender and any two receivers A and B.SA BLs?Depending on the properties of the metric of interest, provides sound constructions that enable MINT inference, labeling, and integration.4Azer Bestavros Practical and Efficient Construction of Network Caricatures 19Metric properties? Monotonicity:f(Lij) ≤ f(Lik) or f(Lij) ≥ f(Lik)? Separability:f(Ljk) = g(f(Lik), f(Lij))? Symmetry:f(Lij) = f(Lji) i j kAzer Bestavros Practical and Efficient Construction of Network Caricatures 20Metric Properties: Examples Hop CountLoss RateJitterBottleneck Bandwidth2-way Prop Delay1-way Prop DelayQueuing DelaySymmetricSeparableMonotonicAzer Bestavros Practical and Efficient Construction of Network Caricatures 21From properties to constructions? If f(LAB) > f(LAC) then monotonicity implies ability to order A, B, C in terms of where they join the topology.? Theorem: Metric monotonicityenables inference of partially (prefix) labeled MINTs.SABCLACLABAzer Bestavros Practical and Efficient Construction of Network Caricatures 22From properties to constructions? Theorem: Metric monotonicity enables integration of MINTs observed over time.A EDCBS@ t1+ =@ t2 @ t1 & t2Azer Bestavros Practical and Efficient Construction of Network Caricatures 23From properties to constructions? If f() is separable, then given LAB and LAC, we can infer L*.? Theorem: Metric separabilityenables complete labeling of MINTs.SABCLACLABL*Azer Bestavros Practical and Efficient Construction of Network Caricatures 24From properties to constructions? Theorem: Metric symmetry enables the  merger of MINTs observed from multiple vantage pointsS1S0S3S2 S2S1S0S3 S0S2S1S3 S0S3S2S1S0S1S2S3+ ++ =5Azer Bestavros Practical and Efficient Construction of Network Caricatures 25MINT Framework: ConstructionsIs f()Monotonic?Is f()Separable?Is f()Symmetric?YesYesYes? Inference? Partial labeling? Integration over time? Labeling? Integration over spaceGiven an oracle that computes f(Ls)NoNoNoResort to metric-specific constructionsAzer Bestavros Practical and Efficient Construction of Network Caricatures 26How about that “Oracle”?? The oracle is simply a procedure that enables a sender to establish the level of sharing between any pair of receivers.? Many MINT instantiations; each one is associated with a metric and an oracle.? A MINT instantiation is as good as the oracle it relies on for that instantiation!Azer Bestavros Practical and Efficient Construction of Network Caricatures 27MINT example: Hop topologiesMetric: Hop count (h) is monotonic and separable, but not necessarily symmetric.Oracle: Given receivers A and B, find number of hops shared between paths S?A and S?B. MINT: Using our constructions, we can efficiently (better than linear in diameter of network) infer and label the hop topology between a sender and n receivers.Azer Bestavros Practical and Efficient Construction of Network Caricatures 28MINT example: Loss TopologiesMetric: Packet loss probability (p) is monotonic and separable, but not symmetric.Oracle: Given receivers A and B, find p for the shared segment of paths S?A and S?B. Many methods available.† Take your pick!MINT: Using our constructions, we can infer and label loss topologies between a sender and n receivers.† Multicast [CDLPT’99], Poisson [RKT’00], and Bayesian probing [HBB’00]Azer Bestavros Practical and Efficient Construction of Network Caricatures 29Loss Oracle? Many to chose from!? Bayesian Probing [HBB:icnp’00]? Send two packets back-to-back, each to a different client ? Measure probability of different outcomes? Relate outcome probabilities to analytical prediction (using a simple queuing model) to estimate shared loss rate? Has good accuracy and convergence characteristicsRSA BAzer Bestavros Practical and Efficient Construction of Network Caricatures 30Loss Oracle: EvaluationTime in secondsSuccess %6Azer Bestavros Practical and Efficient Construction of Network Caricatures 31Loss Topology: ns Experiments50 receivers organized in random trees of 400+ nodes and 8+ levels deep with bursty cross-traffic along each link.Azer Bestavros Practical and Efficient Construction of Network Caricatures 32MINT example: Bottleneck B/WMetric: Bottleneck Bandwidth (BB) is monotonic, but neither separable nor symmetric.Oracle: Given receivers A and B, find bb for the shared segment of paths S?A and S?B using Cartouche probing [HBB:Infocom’03].MINT: Using our constructions, we can infer and partially label the BB topology between a sender and n receivers. To completely label that topology, we need BB-specific techniques (because bb is not separable) [HBB:Infocom’03].Azer Bestavros Practical and Efficient Construction of Network Caricatures 33Bottleneck Bandwidth Estimation? BB is the speed (capacity) of the slowest physical link along a sequence of links? Existing path BB estimation techniques:? End-to-end BB [K’91][P’96][CC’96][DRM’01]? Hop-by-hop BB [D’99][LB’01]Path BBSegment BBAzer Bestavros Practical and Efficient Construction of Network Caricatures 3460mph90mphSeparation as a Measure of speed1 mile 1 mile75mphfn(speed of slowest railroad segment)Length of trainDelay =How far behind will the 2nd train be when the 1st arrives to station? ∆t = ?Azer Bestavros Practical and Efficient Construction of Network Caricatures 35Leveraging the Packet Pair Technique? How could we estimate the BB of the shared segment S?R ?? Send a packet-pair [mm] from S ? Use separation ∆ = s(m)/BB at R to estimate BBRSA Bmms(m)BB@ S @ R---------Azer Bestavros Practical and Efficient Construction of Network Caricatures 36Towards a BB Oracle? … but how can we measure the separation at R from endpoints?? Need to “preserve” separation so we may measure it at an endpointRSA BLemma:Separation ∆ is preserved if BB(R?A) > s(m)/∆? s(m) is the size of the probing packets? Cannot make s(m) arbitrarily small? ∆ is a function of BB(S?R)? Need a function that yields a large ∆7Azer Bestavros Practical and Efficient Construction of Network Caricatures 37@ S - - - - - @ R @ A- - - - -Magnifying ∆: Packet Pair Trains? Use overlapped packet pairs to make ∆large enough at R. All but first and last packets are diverted (or dropped) at R.RSA Br=4Azer Bestavros Practical and Efficient Construction of Network Caricatures 38Magnifying ∆: Packet Pair Trains? If BB(S?R) = 64 * BB(R?A), we would need to send 65 back-to-back packets to ensure a large enough ∆ at R. ? Not practical!RSA B600Mbps10MbpsHow many ?Azer Bestavros Practical and Efficient Construction of Network Caricatures 39pmpqpm@ S - - - - - @ R @ A- - - - -Cartouche Probing [HBB’01] RSA B?Recognizes difference in function between “marker” packets and “filler” packets.Azer Bestavros Practical and Efficient Construction of Network Caricatures 40Cartouche Probing [HBB’01] ? Filler packets are used to “produce” ∆? larger is better? Marker packets are used to “measure” ∆? smaller is better? Make marker packets as small as possible and filler packets as large as possibles(m) = 40 bytess(p) = 1,500 bytesAzer Bestavros Practical and Efficient Construction of Network Caricatures 41Cartouche Probing [HBB’01] ? From S, send a probe [pm {pq}r-1 pm]? “Filler” packets (p & q) go to B? “Marker” packets (m) go to A? s(p)>> s(m) = s(q)? At A, measure the separation ∆ between markers and calculate the shared BB:BB=r [s(p)+s(m)]∆Azer Bestavros Practical and Efficient Construction of Network Caricatures 42Cartouche Size ? Recall that to survive trip from R to A, the separation ∆ must be > s(m)/BB(R?A)r[s(p)+s(m)]/BB(S?R) > s(m)/BB(R?A)? For s(m)=40 and s(p)=1500 ? k=38.5? If BB(S?R) = 50 * BB(R?A), we would need to send a cartouche of size r=2, or a total of 6 packetsBB(S?R)BB(R?A)1kr >8Azer Bestavros Practical and Efficient Construction of Network Caricatures 43Path BB estimation: Summary38.5 rr1Tolerable BB(S?R):BB(R?A)r[s(m)+s(p)]/BBr s(m)/BBs(m)/BBSeparationCartouchePacket Pair trainPacket PairProbe Structure[pm{pq}r-1pm][m{m}r-1m][mm]Azer Bestavros Practical and Efficient Construction of Network Caricatures 44Labeling BB MINTs? MINT + Cartouche Probing allow us to:? Infer BB topology between a set of endpoints? Partial (i.e. prefix) labeling of BB topology? Need BB-specific techniques to label non-prefix subpaths? Conjecture: If we can measure the BB of a path suffix, we can in effect measure the BB of any path segment!Azer Bestavros Practical and Efficient Construction of Network Caricatures 45Path Suffix BB Estimation? Use packet pairs to measure BB(S?A)? Use Cartouche to measure BB(S?j)? If BB(S?A) < BB(S?j)Then BB(j?A) = BB(S?A)Else BB(j?A) > BB(S?j) S j Ai1Azer Bestavros Practical and Efficient Construction of Network Caricatures 46Path Suffix BB EstimationConsider a Cartouche train of the form:pm{pq}r-1pm{pq}r-1pm{pq}r-1pm…drops out at j drops out at j+1 drops out at j+2...S j Ai1Azer Bestavros Practical and Efficient Construction of Network Caricatures 47Cartouche Train: Illustration (r=1)@j @j+1pppmmm∆j+s(p)-s(m)bj+1∆j∆j∆jAzer Bestavros Practical and Efficient Construction of Network Caricatures 48Cartouche Train: Illustration (r=1)@j @j+2@j+1pppmmm∆j+∆j+ s(p)-s(m)bj+2s(p)-s(m)bj+1∆j∆j9Azer Bestavros Practical and Efficient Construction of Network Caricatures 49Path Suffix BB Estimation? Lemma: The largest ∆ between markers of the Cartouche train corresponds to the suffix BB link? Corollary: If BB(R?A) > BB(S?A) then using a Cartouche train, we can estimate BB(R?A)Azer Bestavros Practical and Efficient Construction of Network Caricatures 50Subpath BB Estimation? We need to preserve the spacing ∆ of markers of a Cartouche train at j? Recall that preserving ∆ is a matter of sizing the Cartouche probes; i.e.S j Ai1BB(S?j)BB(j?A)1kr >Azer Bestavros Practical and Efficient Construction of Network Caricatures 51Subpath BB Estimation? Theorem: Using an appropriately size Cartouche train, we can measure the bottleneck bandwidth of an arbitrary segment i?j on a given path S?A? Corollary: Using Cartouche probes we can infer and label BB MINTsS j Ai1Azer Bestavros Practical and Efficient Construction of Network Caricatures 52PeriScope†: Linux API [HBB:Pam’02]? A kernel-level API for implementing various probing structures? A user-level library that implements the MINT constructions (i.e., inference, labeling, and integration)? Used effectively to implement many MINT instantiationsAPIMANAGERDEFINITIONS LOGICSCHEDULERMONITORUserSpacePeriScopeKernelSpaceIP LayerApplication CallbacksAPI Calls† A Probing Engine for the Recovery of Internet Subgraphs(Available from WING web pages at http://www.cs.bu.edu/groups/wing)Azer Bestavros Practical and Efficient Construction of Network Caricatures 53Probing Structures? Example probing structures:? [pp]: Packet-pair probing [CC96] [P96] [P97].? [pp]: Bayesian probing [HBB00].? [pq]: Tailgated-pair probing [LB00].? [pm(pq)r-1pm]: Cartouche Probing [HBB01].? Probing structures differ in:? The number of packets.? The size of each probe packets.? The destination of each probe packet.? The inference function.Azer Bestavros Practical and Efficient Construction of Network Caricatures 54PeriScope Design Rationale? Ensure kernel code modularity and restrict changes to the networking stack.? Minimize user/kernel boundary crossings.? Provide enough primitives to enable the definition of arbitrary probing structures and techniques.? Provide a structured and well-defined interface for applications.10Azer Bestavros Practical and Efficient Construction of Network Caricatures 55Loss Topology: Internet testsIntegration over time of inferred loss topologies Loss topology inferred most of the time by PeriScopeLogical topology of hand-picked set of receiversUsing BP loss oracle implemented in PeriScopeAzer Bestavros Practical and Efficient Construction of Network Caricatures 56Cartouche Probing: Internet Tests? Implemented in PeriScope? Tested against pchar on path prefixes of known link speedspchar results  on January 30, 2002BB of links at BU and in Internet2 published by AbileneAzer Bestavros Practical and Efficient Construction of Network Caricatures 57Cartouche Probing: Internet Tests? Shared BB Experiments? Example: Measure BB shared between the BU?GTech and BU?UBC pathsAzer Bestavros Practical and Efficient Construction of Network Caricatures 58Cartouche Probing: Internet Testsr=2 r=3r=4 r=5Shared BB for GTech and UBC Effect of cross trafficCorrect estimateAzer Bestavros Practical and Efficient Construction of Network Caricatures 59Cartouche Probing: Internet Tests? Path Prefix BB Experiments? Example: Measure BB of first 6 hops from BU to all destinationsAzer Bestavros Practical and Efficient Construction of Network Caricatures 60Cartouche Probing: Internet TestsGeorgia TechUniversity of British ColumbiaEcole NormaleSuperieureHirosakiUniversityb1,5 over a 10Mbps LAN (top) and over 56Kbps modem (bottom)using Cartouches of size r=4 11Azer Bestavros Practical and Efficient Construction of Network Caricatures 61Cartouche Probing: Internet Tests95 percentile confidence interval estimates with r=4 Estimates of BB of arbitrary path segmentsAzer Bestavros Practical and Efficient Construction of Network Caricatures 62Summary? MINT is an effective framework for the representation of the sharing structure between a set of endpoints.? We embodied MINT in a Linux API (PeriScope), which we for inference and labeling of loss, BB and delay topologies.Azer Bestavros Practical and Efficient Construction of Network Caricatures 63Food for thought? How do we “better” validate all this?? Active versus passive?? Other metrics? Other properties?? MINT “triangulation”? MINT “beacons”?? MINT-enabled applications?? MINT for non Internet networks?? Caricatures for non-internet networks…http://www.cs.bu.edu/groups/wing

Practical and efficient construction of network caricatures

https://open.bu.edu/bitstream/2144/26101/1/MINT.pdf

Practical and efficient construction of network caricatures

Abstract

Similar works

Full text

Available Versions

Boston University Institutional Repository (OpenBU)