Gaining insight into AS-level outages through analysis of internet background radiation

Abstract—Internet Background Radiation (IBR) is unsolicited network traffic mostly generated by malicious software, e.g., worms, scans. In previous work, we extracted a signal from IBR traffic arriving at a large (/8) segment of unassigned IPv4 address space to identify large-scale disruptions of connectivity at an Autonomous System (AS) granularity, and used our technique to study episodes of government censorship and natural disasters [1]. Here we explore other IBR-derived metrics that may provide insights into the causes of macroscopic connectivity disruptions. We propose metrics indicating packet loss (e.g., due to link congestion) along a path from a specific AS to our observation point. We use three case studies to illustrate how our metrics can help identify packet loss characteristics of an outage. These metrics could be used in the diagnostic component of a semi-automated system for detecting and characterizing large-scale outages. I

Leveraging Internet Background Radiation for Opportunistic Network Analysis

In this dissertation, we evaluate the potential of unsolicited Internet traffic, called Internet Background Radiation (IBR), to provide insights into address space usage and network conditions. IBR is primarily collected through darknets, which are blocks of IP addresses dedicated to collecting unsolicited traffic resulting from scans, backscatter, misconfigurations, and bugs. We expect these pervasively sourced components to yield visibility into networks that are hard to measure (e.g., hosts behind firewalls or not appearing in logs) with traditional active and passive techniques. Using the largest collections of IBR available to academic researchers, we test this hypothesis by: (1) identifying the phenomena that induce many hosts to send IBR, (2) characterizing the factors that influence our visibility, including aspects of the traffic itself and measurement infrastructure, and (3) extracting insights from 11 diverse case studies, after excluding obvious cases of sender inauthenticity. Through IBR, we observe traffic from nearly every country, most ASes with routable prefixes, and millions of /24 blocks. Misconfigurations and bugs, often involving P2P networks, result in the widest coverage in terms of visible networks, though scanning traffic is applicable for in-depth and repeated analysis due to its large volume. We find, notwithstanding the extraordinary popularity of some IP addresses, similar observations using IBR collected in different darknets, and a predictable degradation using smaller darknets. Although the mix of IBR components evolves, our observations are consistent over time. Our case studies highlight the versatility of IBR and help establish guidelines for when researchers should consider using unsolicited traffic for opportunistic network analysis. Based on our experience, IBR may assist in: corroborating inferences made through other datasets (e.g., DHCP lease durations) supplementing current state-of-the art techniques (e.g., IPv4 address space utilization), exposing weaknesses in other datasets (e.g., missing router interfaces), identifying abused resources (e.g., open resolvers), testing Internet tools by acting as a diverse traffic sample (e.g., uptime heuristics), and reducing the number of required active probes (e.g., path change inferences). In nearly every case study, IBR improves our analysis of an Internet-wide behavior. We expect future studies to reap similar benefits by including IBR

Leveraging Internet Background Radiation for Opportunistic Network Analysis

In this dissertation, we evaluate the potential of unsolicited Internet traffic, called Internet Background Radiation (IBR), to provide insights into address space usage and network conditions. IBR is primarily collected through darknets, which are blocks of IP addresses dedicated to collecting unsolicited traffic resulting from scans, backscatter, misconfigurations, and bugs. We expect these pervasively sourced components to yield visibility into networks that are hard to measure (e.g., hosts behind firewalls or not appearing in logs) with traditional active and passive techniques. Using the largest collections of IBR available to academic researchers, we test this hypothesis by: (1) identifying the phenomena that induce many hosts to send IBR, (2) characterizing the factors that influence our visibility, including aspects of the traffic itself and measurement infrastructure, and (3) extracting insights from 11 diverse case studies, after excluding obvious cases of sender inauthenticity.Through IBR, we observe traffic from nearly every country, most ASes with routable prefixes, and millions of /24 blocks. Misconfigurations and bugs, often involving P2P networks, result in the widest coverage in terms of visible networks, though scanning traffic is applicable for in-depth and repeated analysis due to its large volume. We find, notwithstanding the extraordinary popularity of some IP addresses, similar observations using IBR collected in different darknets, and a predictable degradation using smaller darknets. Although the mix of IBR components evolves, our observations are consistent over time.Our case studies highlight the versatility of IBR and help establish guidelines for when researchers should consider using unsolicited traffic for opportunistic network analysis. Based on our experience, IBR may assist in: corroborating inferences made through other datasets (e.g., DHCP lease durations) supplementing current state-of-the art techniques (e.g., IPv4 address space utilization), exposing weaknesses in other datasets (e.g., missing router interfaces), identifying abused resources (e.g., open resolvers), testing Internet tools by acting as a diverse traffic sample (e.g., uptime heuristics), and reducing the number of required active probes (e.g., path change inferences). In nearly every case study, IBR improves our analysis of an Internet-wide behavior. We expect future studies to reap similar benefits by including IBR

Do You Know Where Your Cloud Files Are?

Clients of storage-as-a-service systems such as Amazon’s S3 want to be sure that the files they have entrusted to the cloud are available now and will be available in the future. Using protocols from previous work on proofs of retrievability and on provable data possession, clients can verify that their files are available now. But these protocols do not guarantee that the files are replicated onto multiple drives or multiple datacenters. Such tests are crucial if cloud storage is to provide resilience to natural disasters and power outages as well as improving the network latency to different parts of the world. In this paper, we study the problem of verifying that a cloud storage provider replicates the data in diverse geolocations. We provide a theoretical framework for verifyin

The impact of router outages on the AS-level internet

We propose and evaluate a new metric for understanding the dependence of the AS-level Internet on individual routers. Whereas prior work uses large volumes of reachability probes to infer outages, we design an efficient active probing technique that directly and unambiguously reveals router restarts. We use our technique to survey 149,560 routers across the Internet for 2.5 years. 59,175 of the surveyed routers (40%) experience at least one reboot, and we quantify the resulting impact of each router outage on global IPv4 and IPv6 BGP reachability. Our technique complements existing data and control plane outage analysis methods by providing a causal link from BGP reachability failures to the responsible router(s) and multi-homing configurations. While we found the Internet core to be largely robust, we identified specific routers that were single points of failure for the prefixes they advertised. In total, 2,385 routers -- 4.0% of the routers that restarted over the course of 2.5 years of probing -- were single points of failure for 3,396 IPv6 prefixes announced by 1,708 ASes. We inferred 59% of these routers were the customer-edge border router. 2,374 (70%) of the withdrawn prefixes were not covered by a less specific prefix, so 1,726 routers (2.9%) of those that restarted were single points of failure for at least one network. However, a covering route did not imply reachability during a router outage, as no previously-responsive address in a withdrawn more specific prefix responded during a one-week sample. We validate our reboot and single point of failure inference techniques with four networks, finding no false positive or false negative reboots, but find some false negatives in our single point of failure inferences

Competitive Analysis of Online Traffic Grooming in WDM Rings

This paper addresses the problem of traffic grooming in wavelength-division multiplexing (WDM) rings where connection requests arrive online. Each request specifies a pair of nodes that wish to communicate and also the desired bandwidth of this connection. If the request is to be satisfied, it must be allocated to one or more wavelengths with sufficient remaining capacity. We consider three distinct profit models specifying the profit associated with satisfying a connection request. We give results on offline and online algorithms for each of the three profit models. We use the paradigm of competitive analysis to theoretically analyze the quality of our online algorithms. Finally, experimental results are given to provide insight into the performance of these algorithms in practice