Gigascope: a stream database for network applications

We have developed Gigascope, a stream database for network applications including traffic analysis, intrusion detection, router configuration analysis, network research, network monitoring, and and performance monitoring and debugging. Gigascope is undergoing installation at many sites within the AT&T network, including at OC48 routers, for detailed monitoring. In this paper we describe our motivation for and constraints in developing Gigascope, the Gigascope architecture and query language, and performance issues. We conclude with a discussion of stream database research problems we have found in our application. 1

Abstract

Managing a large scale network requires a network monitoring infrastructure. However, network monitoring is a difficult application. In response to shortcomings in the readily available tools, we have developed Gigascope, a stream database system specialized for network monitoring. In this article, we discuss some of the constraints we faced when developing the Gigascope architecture, Gigascope applications, and how Gigascope is used.

Enhanced Streaming Services in a Content Distribution Network

Prism’s content naming, management, discovery, and redirection mechanisms support high-quality streaming media services in an IP-based content distribution network. With the emergence of broadband access networks and powerful personal computer systems, the demand for network-delivered full-motion streaming video is growing. While the traditional Web service model can be applied to IP-based streaming content, the user experience of quality does not compare favorably with cable, satellite, or broadcast television. On the other hand, current television broadcasting technologies limit user choice

CDN Brokering

Content distribution networks (CDNs) increase the capacity of individual Web sites and attempt to deliver content from caches that are located "closer" to end-users than the origin servers that provide the content. CDN brokering provides CDNs a way to interoperate by allowing one CDN to intelligently redirect clients dynamically to other CDNs. This paper describes the goals, architecture, and performance of a CDN brokerage system. Our system has been deployed on the Internet on a provisional basis, and our architectural ideas have helped advance the evolution of Internet standards for interoperating CDNs

Early Experiences on the Journey Towards Self-* Storage

Self-* systems are self-organizing, self-configuring, self-healing, self-tuning and, in general, selfmanaging. Ursa Minor is a large-scale storage infrastructure being designed and deployed at Carnegie Mellon University, with the goal of taking steps towards the self-* ideal. This paper discusses our early experiences with one specific aspect of storage management: performance tuning and projection. Ursa Minor uses self-monitoring and rudimentary system modeling to support analysis of how system changes would affect performance, exposing simple What...if query interfaces to administrators and tuning agents. We find that most performance predictions are sufficiently accurate (within 10-20%) and that the associated performance overhead is less than 6%. Such embedded support for What...if queries simplifies tuning automation and reduces the administrator expertise needed to make acquisition decisions

Ursa Minor: versatile cluster-based storage

No single encoding scheme or fault model is optimal for all data. A versatile storage system allows them to be matched to access patterns, reliability requirements, and cost goals on a per-data item basis. Ursa Minor is a cluster-based storage system that allows data-specific selection of, and on-line changes to, encoding schemes and fault models. Thus, different data types can share a scalable storage infrastructure and still enjoy specialized choices, rather than suffering from "one size fits all." Experiments with Ursa Minor show performance benefits of 2--3 when using specialized choices as opposed to a single, more general, configuration. Experiments also show that a single cluster supporting multiple workloads simultaneously is much more efficient when the choices are specialized for each distribution rather than forced to use a "one size fits all" configuration. When using the specialized distributions, aggregate cluster throughput nearly doubled