World Wide Web DOI 10.1007/s11280-007-0027-8 Prefetching in Content Distribution Networks via Web Communities Identification and Outsourcing

Abstract

Abstract Content distribution networks (CDNs) improve scalability and reliability, by replicating content to the “edge ” of the Internet. Apart from the pure networking issues of the CDNs relevant to the establishment of the infrastructure, some very crucial data management issues must be resolved to exploit the full potential of CDNs to reduce the “last mile ” latencies. A very important issue is the selection of the content to be prefetched to the CDN servers. All the approaches developed so far, assume the existence of adequate content popularity statistics to drive the prefetch decisions. Such information though, is not always available, or it is extremely volatile, turning such methods problematic. To address this issue, we develop selfadaptive techniques to select the outsourced content in a CDN infrastructure, which requires no apriori knowledge of request statistics. We identify clusters of “correlated ” Web pages in a site, called Web site communities, and make thes