BigBen: Telemetry Processing for Internet-wide Event Monitoring

This paper describes BigBen, a network telemetry processing system designed to enable accurate and timely reporting of Internet events (e.g., outages, attacks and configuration changes). BigBen is distinct from other Internet-wide event detection systems in its use of passive measurements of Network Time Protocol (NTP) traffic. We describe the architecture of BigBen, which includes (i) a distributed NTP traffic collection component, (ii) an Extract Transform Load (ETL) component, (iii) an event identification component, and (iv) a visualization and reporting component. We also describe a cloud-based implementation of BigBen developed to process large NTP data sets and provide daily event reporting. We demonstrate BigBen on a 15.5TB corpus of NTP data. We show that our implementation is efficient and could support hourly event reporting. We show that BigBen identifies a wide range of Internet events characterized by their location, scope and duration. We compare the events detected by BigBen vs. events detected by a large active probe-based detection system. We find only modest overlap and show how BigBen provides details on events that are not available from active measurements. Finally, we report on the perspective that BigBen provides on Internet events that were reported by third parties. In each case, BigBen confirms the event and provides details that were not available in prior reports, highlighting the utility of the passive, NTP-based approach.Comment: 12 page

Poster Abstract: Towards Active Measurements of Edge Network Outages ⋆

End-to-end reachability is a fundamental service of the Internet. We study network outages caused by natural disasters [2, 5], and political upheavals [8]. We propose a new approach to outage detection using active probing. Like prior outage detection methods [3, 4], our method uses ICMP echo request