Beyond Counting: New Perspectives on the Active IPv4 Address Space

In this study, we report on techniques and analyses that enable us to capture Internet-wide activity at individual IP address-level granularity by relying on server logs of a large commercial content delivery network (CDN) that serves close to 3 trillion HTTP requests on a daily basis. Across the whole of 2015, these logs recorded client activity involving 1.2 billion unique IPv4 addresses, the highest ever measured, in agreement with recent estimates. Monthly client IPv4 address counts showed constant growth for years prior, but since 2014, the IPv4 count has stagnated while IPv6 counts have grown. Thus, it seems we have entered an era marked by increased complexity, one in which the sole enumeration of active IPv4 addresses is of little use to characterize recent growth of the Internet as a whole. With this observation in mind, we consider new points of view in the study of global IPv4 address activity. Our analysis shows significant churn in active IPv4 addresses: the set of active IPv4 addresses varies by as much as 25% over the course of a year. Second, by looking across the active addresses in a prefix, we are able to identify and attribute activity patterns to network restructurings, user behaviors, and, in particular, various address assignment practices. Third, by combining spatio-temporal measures of address utilization with measures of traffic volume, and sampling-based estimates of relative host counts, we present novel perspectives on worldwide IPv4 address activity, including empirical observation of under-utilization in some areas, and complete utilization, or exhaustion, in others.Comment: in Proceedings of ACM IMC 201

Impact of the COVID-19 pandemic on the Internet latency: A large-scale study

The COVID-19 pandemic dramatically changed the way of living of billions of people in a very short time frame. In this paper, we evaluate the impact on the Internet latency caused by the increased amount of human activities that are carried out on-line. The study focuses on Italy, which experienced significant restrictions imposed by local authorities, but results about Spain, France, Germany, Sweden, and the whole of Europe are also included. The analysis of a large set of measurements shows that the impact on the network can be significant, especially in terms of increased variability of latency. In Italy we observed that the standard deviation of the average additional delay – the additional time with respect to the minimum delay of the paths in the region – during lockdown is ∼3−4 times as much as the value before the pandemic. Similarly, in Italy, packet loss is ∼2−3 times as much as before the pandemic. The impact is not negligible also for the other countries and for the whole of Europe, but with different levels and distinct patterns

Whether weather causes contention: assessing the ongoing resilience opportunity of telecommuting

The Covid-19 pandemic resulted in an unprecedented overnight explosion in telecommuting. It has highlighted a new dependence on digital infrastructures and raised new questions regarding the resilience of internet connectivity as an alternative to travel. Pre-pandemic, we considered how telecommuting could offer an opportunity for resilience when travel was disrupted by weather extremes. We analysed five years’ of recorded broadband speed variation across England and Wales in order to quantify the changing demand for internet access during the working day under adverse weather conditions. Slower broadband speeds, also known as contention, are an indication of increased demand. Thus, during the working day, contention is an indication that external factors like weather can influence the choice to telecommute instead of travel. A multilevel regression model is estimated to investigate the relationship between contention during the working day and weather, whilst controlling for background spatial and demographic differences in internet services. Emergent patterns suggest that even before the pandemic, online connectivity was in greater demand when travel was disrupted or at risk of disruption. Our research provides insights into the roles that both the supply of and the demand for transport and digital technologies might play in increasing resilience and maintaining productivity during severe weather and other disruptions as experience of both types of working has become so widespread

Improving Resilience of Communication in Information Dissemination for Time-Critical Applications

Severe weather impacts life and in this dire condition, people rely on communication, to organize relief and stay in touch with their loved ones. In such situations, cellular network infrastructure\footnote{We refer to cellular network infrastructure as infrastructure for the entirety of this document} might be affected due to power outage, link failures, etc. This urges us to look at Ad-hoc mode of communication, to offload major traffic partially or fully from the infrastructure, depending on the status of it. We look into threefold approach, ranging from the case where the infrastructure is completely unavailable, to where it has been replaced by make shift low capacity mobile cellular base station. First, we look into communication without infrastructure and timely, dissemination of weather alerts specific to geographical areas. We look into the specific case of floods as they affect significant number of people. Due to the nature of the problem we can utilize the properties of Information Centric Networking (ICN) in this context, namely: i) Flexibility and high failure resistance: Any node in the network that has the information can satisfy the query ii) Robust: Only sensor and car need to communicate iii) Fine grained geo-location specific information dissemination. We analyze how message forwarding using ICN on top of Ad hoc network, approach compares to the one based on infrastructure, that is less resilient in the case of disaster. In addition, we compare the performance of different message forwarding strategies in VANETs (Vehicular Adhoc Networks) using ICN. Our results show that ICN strategy outperforms the infrastructure-based approach as it is 100 times faster for 63\% of total messages delivered. Then we look into the case where we have the cellular network infrastructure, but it is being pressured due to rapid increase in volume of network traffic (as seen during a major event) or it has been replaced by low capacity mobile tower. In this case we look at offloading as much traffic as possible from the infrastructure to device-to-device communication. However, the host-oriented model of the TCP/IP-based Internet poses challenges to this communication pattern. A scheme that uses an ICN model to fetch content from nearby peers, increases the resiliency of the network in cases of outages and disasters. We collected content popularity statistics from social media to create a content request pattern and evaluate our approach through the simulation of realistic urban scenarios. Additionally, we analyze the scenario of large crowds in sports venues. Our simulation results show that we can offload traffic from the backhaul network by up to 51.7\%, suggesting an advantageous path to support the surge in traffic while keeping complexity and cost for the network operator at manageable levels. Finally, we look at adaptive bit-rate streaming (ABR) streaming, which has contributed significantly to the reduction of video playout stalling, mainly in highly variable bandwidth conditions. ABR clients continue to suffer from the variation of bit rate qualities over the duration of a streaming session. Similar to stalling, these variations in bit rate quality have a negative impact on the users’ Quality of Experience (QoE). We use a trace from a large-scale CDN to show that such quality changes occur in a significant amount of streaming sessions and investigate an ABR video segment retransmission approach to reduce the number of such quality changes. As the new HTTP/2 standard is becoming increasingly popular, we also see an increase in the usage of HTTP/2 as an alternative protocol for the transmission of web traffic including video streaming. Using various network conditions, we conduct a systematic comparison of existing transport layer approaches for HTTP/2 that is best suited for ABR segment retransmissions. Since it is well known that both protocols provide a series of improvements over HTTP/1.1, we perform experiments both in controlled environments and over transcontinental links in the Internet and find that these benefits also “trickle up” into the application layer when it comes to ABR video streaming where HTTP/2 retransmissions can significantly improve the average quality bitrate while simultaneously minimizing bit rate variations over the duration of a streaming session. Taking inspiration from the first two approaches, we take into account the resiliency of a multi-path approach and further look at a multi-path and multi-stream approach to ABR streaming and demonstrate that losses on one path have very little impact on the other from the same multi-path connection and this increases throughput and resiliency of communication

Understanding Home Networks with Lightweight Privacy-Preserving Passive Measurement

Homes are involved in a significant fraction of Internet traffic. However, meaningful and comprehensive information on the structure and use of home networks is still hard to obtain. The two main challenges in collecting such information are the lack of measurement infrastructure in the home network environment and individuals’ concerns about information privacy. To tackle these challenges, the dissertation introduces Home Network Flow Logger (HNFL) to bring lightweight privacy-preserving passive measurement to home networks. The core of HNFL is a Linux kernel module that runs on resource-constrained commodity home routers to collect network traffic data from raw packets. Unlike prior passive measurement tools, HNFL is shown to work without harming either data accuracy or router performance. This dissertation also includes a months-long field study to collect passive measurement data from home network gateways where network traffic is not mixed by NAT (Network Address Translation) in a non-intrusive way. The comprehensive data collected from over fifty households are analyzed to learn the characteristics of home networks such as number and distribution of connected devices, traffic distribution among internal devices, network availability, downlink/uplink bandwidth, data usage patterns, and application traffic distribution

Inferring the presence of reverse proxies through timing analysis

This thesis presents a method for inferring the presence of a reverse proxy server using packet timing analysis from the vantage point of a client system. This method can determine whether Internet users are receiving web content from the actual source or from some potentially spoofed proxy device; leading to better risk assessment and understanding of the cyber terrain. By using only the measurement and comparison of three-way handshake and content request/delivery packet round trip times, we identify an accurate classifier that detects the presence of a reverse proxy server with over 98% accuracy. This is an improvement over other inference methods because all measurements can be done from an external client machine. A secondary yet significant contribution is the robust data set that was produced as a result of this research. We have collected a set of over 6 million data points from a known set of 30 globally dispersed machines, which was instrumental in our research efforts and will be used for further studies and exploration.http://archive.org/details/inferringpresenc1094545803Outstanding ThesisOutstanding ThesisMajor, United States ArmyApproved for public release; distribution is unlimited

Observing and Improving the Reliability of Internet Last-mile Links

People rely on having persistent Internet connectivity from their homes and mobile devices. However, unlike links in the core of the Internet, the links that connect people's homes and mobile devices, known as "last-mile" links, are not redundant. As a result, the reliability of any given link is of paramount concern: when last-mile links fail, people can be completely disconnected from the Internet. In addition to lacking redundancy, Internet last-mile links are vulnerable to failure. Such links can fail because the cables and equipment that make up last-mile links are exposed to the elements; for example, weather can cause tree limbs to fall on overhead cables, and flooding can destroy underground equipment. They can also fail, eventually, because cellular last-mile links can drain a smartphone's battery if an application tries to communicate when signal strength is weak. In this dissertation, I defend the following thesis: By building on existing infrastructure, it is possible to (1) observe the reliability of Internet last-mile links across different weather conditions and link types; (2) improve the energy efficiency of cellular Internet last-mile links; and (3) provide an incrementally deployable, energy-efficient Internet last-mile downlink that is highly resilient to weather-related failures. I defend this thesis by designing, implementing, and evaluating systems

Analyzing Internet reliability remotely with probing-based techniques

Internet reliability for home users is increasingly important as a variety of services that we use migrate to the Internet. Yet, we lack authoritative measures of residential Internet reliability. Measuring reliability requires the detection of Internet outage events experienced by home users. But residential Internet outages are rare events. Further, they can affect relatively few users. Thus, detecting residential Internet outages requires broad and longitudinal measurements of individual users' Internet connections. However, such measurements of Internet reliability are challenging to obtain accurately and at scale. Probing-based remote outage detection techniques can scale but their accuracy is questionable. These techniques detect Internet outages across time as well as across the IPv4 address space by sending active probes, such as pings and traceroutes, to users' IP addresses and use probe responses to infer Internet connectivity. However, they can infer false outages since their foundational assumption can sometimes be invalid: that the lack of response to an active probe is indicative of failure. In this dissertation, I show how to use probing-based techniques to measure residential Internet reliability by defending the following thesis: It is possible to remotely and accurately detect substantial outages experienced by any device with a stable public IP address that typically responds to active probes and use these outages to compare reliability across ISPs, media-types, geographical areas, and weather conditions. In the first part of the dissertation, I address the inaccuracy of probing-based techniques' detected outages and show how to use probe responses to correctly detect outages. I illustrate two scenarios where the lack of response to an active probe is not indicative of failure. In the first scenario, responses are delayed beyond the prober's timeout, leading these techniques to infer packet-loss instead of delay. In the second scenario, these techniques can falsely infer packet-loss when the address they are probing gets dynamically reassigned. I examine how often delayed responses and dynamic reassignment occur across ISPs to quantify the inaccuracy of these techniques. I show how outages can be inferred correctly even in networks with dynamic reassignment using complementary datasets that can reveal whether an address was dynamically reassigned before, during, and after a detected outage for that address. In the second part of the dissertation, I motivate why the detection of individual addresses' outages is necessary for analyzing residential reliability. An individual address typically represents one residential customer; therefore, detecting outages for individual addresses can allow capturing even small outages. Prior probing-based techniques focus upon the detection of edge network outages affecting a substantial set of addresses belonging to a BGP prefix or to a /24 address block. Here, I quantitatively demonstrate the extent to which prior techniques can miss residential outages. I show that even individual address outages occur rarely in most networks. When multiple simultaneous outages of related individual addresses occur, there is likely a common underlying cause. With this insight, I develop and evaluate an approach to find outage events that are statistically unlikely to have occurred independently. I show that the majority of such events do not affect entire /24 address blocks or BGP prefixes, and are therefore not likely to be detected by existing techniques which look for outages at these granularities. In the final part of the dissertation, I show how to use individual addresses' outages detected by probing-based techniques to assess Internet reliability across media-types, geographical areas, and weather conditions. Individual outages are not direct measures of reliability: they can occur independently because users disable equipment or can be observed falsely due to dynamic address renumbering. I use the insight that the statistical change in outage rate in different challenging environments (e.g., thunderstorm) can quantitatively expose actual outage “inflation”. I show how to study the effect of challenging environments upon the reliability of a group of addresses by analyzing the inflation in outage rate for that group during its presence. This dissertation's contributions will help achieve comprehensive measurements of Internet reliability that can be used to identify vulnerable networks and their challenges, inform which enhancements can help networks improve reliability, and evaluate the efficacy of deployed enhancements over time