Big Data and Analysis of Data Transfers for International Research Networks Using NetSage

Modern science is increasingly data-driven and collaborative in nature. Many scientific disciplines, including genomics, high-energy physics, astronomy, and atmospheric science, produce petabytes of data that must be shared with collaborators all over the world. The National Science Foundation-supported International Research Network Connection (IRNC) links have been essential to enabling this collaboration, but as data sharing has increased, so has the amount of information being collected to understand network performance. New capabilities to measure and analyze the performance of international wide-area networks are essential to ensure end-users are able to take full advantage of such infrastructure for their big data applications. NetSage is a project to develop a unified, open, privacy-aware network measurement, and visualization service to address the needs of monitoring today's high-speed international research networks. NetSage collects data on both backbone links and exchange points, which can be as much as 1Tb per month. This puts a significant strain on hardware, not only in terms storage needs to hold multi-year historical data, but also in terms of processor and memory needs to analyze the data to understand network behaviors. This paper addresses the basic NetSage architecture, its current data collection and archiving approach, and details the constraints of dealing with this big data problem of handling vast amounts of monitoring data, while providing useful, extensible visualization to end users

Alternate marking-based network telemetry for industrial WSNs

For continuous, persistent and problem-free operation of Industrial Wireless Sensor Networks (IWSN), it is critical to have visibility and awareness into what is happening on the network at any one time. Especially, for the use cases with strong needs for deterministic and real-time network services with latency and reliability guarantees, it is vital to monitor network devices continuously to guarantee their functioning, detect and isolate relevant problems and verify if all system requirements are being met simultaneously. In this context, this article investigates a light-weight telemetry solution for IWSNs, which enables the collection of accurate and continuous flowbased telemetry information, while adding no overhead on the monitored packets. The proposed monitoring solution adopts the recent Alternate Marking Performance Monitoring (AMPM) concept and mainly targets measuring end-to-end and hopby-hop reliability and delay performance in critical application flows. Besides, the technical capabilities and characteristics of the proposed solution are evaluated via a real-life implementation and practical experiments, validating its suitability for IWSNs

An Analysis on End-To-End Inference Methods based On Packet Probing in Network

Current Internet is a massive, distributed network which continues to grow in size as globalization takes major role in everyone’s life like e-commerce, social networking and related activities grow. The heterogeneous and largely unregulated structure of the Internet renders tasks such as optimized service provision, rate limiting certain classes of applications (e.g. peer-to-peer), provide bandwidth guarantee for certain applications, avoiding shared congestion in flows are increasingly challenging tasks. The problem is complicated by the fact that one cannot rely on the cooperation of individual servers and routers to aid in the collection of network traffic measurements vital for these tasks. Hence we go for network monitoring and inference methods based on packet probing in the network. This paper presents an analysis of different inference methods for network characteristics to deal with shared congestion, packet forwarding priority, network tomography and evaluates each methodology based on packet loss rate and delay variance