Techniques of data prefetching, replication, and consistency in the Internet

Internet has become a major infrastructure for information sharing in our daily life, and indispensable to critical and large applications in industry, government, business, and education. Internet bandwidth (or the network speed to transfer data) has been dramatically increased, however, the latency time (or the delay to physically access data) has been reduced in a much slower pace. The rich bandwidth and lagging latency can be effectively coped with in Internet systems by three data management techniques: caching, replication, and prefetching. The focus of this dissertation is to address the latency problem in Internet by utilizing the rich bandwidth and large storage capacity for efficiently prefetching data to significantly improve the Web content caching performance, by proposing and implementing scalable data consistency maintenance methods to handle Internet Web address caching in distributed name systems (DNS), and to handle massive data replications in peer-to-peer systems. While the DNS service is critical in Internet, peer-to-peer data sharing is being accepted as an important activity in Internet.;We have made three contributions in developing prefetching techniques. First, we have proposed an efficient data structure for maintaining Web access information, called popularity-based Prediction by Partial Matching (PB-PPM), where data are placed and replaced guided by popularity information of Web accesses, thus only important and useful information is stored. PB-PPM greatly reduces the required storage space, and improves the prediction accuracy. Second, a major weakness in existing Web servers is that prefetching activities are scheduled independently of dynamically changing server workloads. Without a proper control and coordination between the two kinds of activities, prefetching can negatively affect the Web services and degrade the Web access performance. to address this problem, we have developed a queuing model to characterize the interactions. Guided by the model, we have designed a coordination scheme that dynamically adjusts the prefetching aggressiveness in Web Servers. This scheme not only prevents the Web servers from being overloaded, but it can also minimize the average server response time. Finally, we have proposed a scheme that effectively coordinates the sharing of access information for both proxy and Web servers. With the support of this scheme, the accuracy of prefetching decisions is significantly improved.;Regarding data consistency support for Internet caching and data replications, we have conducted three significant studies. First, we have developed a consistency support technique to maintain the data consistency among the replicas in structured P2P networks. Based on Pastry, an existing and popular P2P system, we have implemented this scheme, and show that it can effectively maintain consistency while prevent hot-spot and node-failure problems. Second, we have designed and implemented a DNS cache update protocol, called DNScup, to provide strong consistency for domain/IP mappings. Finally, we have developed a dynamic lease scheme to timely update the replicas in Internet

Evaluation, Analysis and adaptation of web prefetching techniques in current web

Abstract This dissertation is focused on the study of the prefetching technique applied to the World Wide Web. This technique lies in processing (e.g., downloading) a Web request before the user actually makes it. By doing so, the waiting time perceived by the user can be reduced, which is the main goal of the Web prefetching techniques. The study of the state of the art about Web prefetching showed the heterogeneity that exists in its performance evaluation. This heterogeneity is mainly focused on four issues: i) there was no open framework to simulate and evaluate the already proposed prefetching techniques; ii) no uniform selection of the performance indexes to be maximized, or even their definition; iii) no comparative studies of prediction algorithms taking into account the costs and benefits of web prefetching at the same time; and iv) the evaluation of techniques under very different or few significant workloads. During the research work, we have contributed to homogenizing the evaluation of prefetching performance by developing an open simulation framework that reproduces in detail all the aspects that impact on prefetching performance. In addition, prefetching performance metrics have been analyzed in order to clarify their definition and detect the most meaningful from the user's point of view. We also proposed an evaluation methodology to consider the cost and the benefit of prefetching at the same time. Finally, the importance of using current workloads to evaluate prefetching techniques has been highlighted; otherwise wrong conclusions could be achieved. The potential benefits of each web prefetching architecture were analyzed, finding that collaborative predictors could reduce almost all the latency perceived by users. The first step to develop a collaborative predictor is to make predictions at the server, so this thesis is focused on an architecture with a server-located predictor. The environment conditions that can be found in the web are alsDoménech I De Soria, J. (2007). Evaluation, Analysis and adaptation of web prefetching techniques in current web [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/1841Palanci

Building Internet caching systems for streaming media delivery

The proxy has been widely and successfully used to cache the static Web objects fetched by a client so that the subsequent clients requesting the same Web objects can be served directly from the proxy instead of other sources faraway, thus reducing the server\u27s load, the network traffic and the client response time. However, with the dramatic increase of streaming media objects emerging on the Internet, the existing proxy cannot efficiently deliver them due to their large sizes and client real time requirements.;In this dissertation, we design, implement, and evaluate cost-effective and high performance proxy-based Internet caching systems for streaming media delivery. Addressing the conflicting performance objectives for streaming media delivery, we first propose an efficient segment-based streaming media proxy system model. This model has guided us to design a practical streaming proxy, called Hyper-Proxy, aiming at delivering the streaming media data to clients with minimum playback jitter and a small startup latency, while achieving high caching performance. Second, we have implemented Hyper-Proxy by leveraging the existing Internet infrastructure. Hyper-Proxy enables the streaming service on the common Web servers. The evaluation of Hyper-Proxy on the global Internet environment and the local network environment shows it can provide satisfying streaming performance to clients while maintaining a good cache performance. Finally, to further improve the streaming delivery efficiency, we propose a group of the Shared Running Buffers (SRB) based proxy caching techniques to effectively utilize proxy\u27s memory. SRB algorithms can significantly reduce the media server/proxy\u27s load and network traffic and relieve the bottlenecks of the disk bandwidth and the network bandwidth.;The contributions of this dissertation are threefold: (1) we have studied several critical performance trade-offs and provided insights into Internet media content caching and delivery. Our understanding further leads us to establish an effective streaming system optimization model; (2) we have designed and evaluated several efficient algorithms to support Internet streaming content delivery, including segment caching, segment prefetching, and memory locality exploitation for streaming; (3) having addressed several system challenges, we have successfully implemented a real streaming proxy system and deployed it in a large industrial enterprise

Supervision de contenus multimédia : adaptation de contenu, politiques optimales de préchargement et coordination causale de flux

La qualité des systèmes d'informations distribués dépend de la pertinence du contenu mis à disposition, de la réactivité du service ainsi que de la cohérence des informations présentées. Nos travaux visent à améliorer ces trois critères de performance et passent par la prise en compte des caractéristiques de l'utilisateur, des ressources disponibles ou plus généralement du contexte d'exécution. Par conséquent, cette thèse comporte trois volets. Le premier volet se place dans le cadre de l'adaptation de systèmes d’information déployés dans des contextes dynamiques et stochastiques. Nous présentons une approche où des agents d’adaptation appliquent des politiques de décision séquentielle dans l'incertain. Nous modélisons ces agents par des Processus Décisionnels de Markov (PDM) selon que le contexte soit observable ou seulement partiellement observable (PDM Partiellement Observables). Dans le cas d’un service mobile de consultation de films, nous montrons en particulier qu’une politique d'adaptation de ce service à des ressources limitées peut être nuancée selon l'intérêt de l'utilisateur, estimé grâce à l’évaluation des signaux de retour implicite. Dans le deuxième volet, nous nous intéressons à l'optimisation de la réactivité d'un système qui propose des contenus hypermédia. Nous nous appuyons sur des techniques de préchargement pour réduire les latences. Comme précédemment, un PDM modélise les habitudes des utilisateurs et les ressources disponibles. La force de ce modèle réside dans sa capacité à fournir des politiques optimales de préchargement. Les premières politiques que nous obtenons sont simples. Nous enrichissons alors le modèle pour dériver des politiques de préchargement plus complexes et plus agressives et montrons leurs performances par simulation. Afin de personnaliser nos stratégies optimales nous proposons finalement un modèle PDMPO dont les politiques s'adaptent aux profils des utilisateurs. Le troisième volet se place dans le contexte des applications multimédia interactives distribuées et concerne le contrôle de la cohérence des flux multimédia répartis. Dans un tel contexte, plusieurs mécanismes de synchronisation sont nécessaires et plusieurs ordres logiques (fifo, causal, total) s'avèrent utiles. Nous proposons une boîte à outils capable de gérer plusieurs protocoles d’ordre partiel et d'assurer une délivrance correcte de chaque message, en respectant tous les ordres qui lui ont été imposés. Nous décrivons ensuite l’intégration des tolérances humaines vis-à-vis des courtes incohérences causales dans notre boîte à outils. Nos simulations montrent que de meilleures performances sont obtenues par cette méthode comparativement à d’autres approches, comme la causalité classique ou la Δ-causalité. ABSTRACT : Distributed systems information quality depends on service responsiveness, data consistency and its relevance according to user interests. The thesis aims to improve these three performance criteria by taking into account user characteristics, available ressources or more generally execution context. Naturally, the document is organized in three main parts. The first part discusses adaptation policies for information systems that are subject to dynamic and stochastic contexts. In our approach adaptation agents apply sequential decisional policies under uncertainty. We focus on the modeling of such decisional processes depending on whether the context is fully or partially observable. We use Markov Decision Processes (MDP) and Partially Observable MDP (POMDP) for modeling a movie browsing service in a mobile environment. Our model derives adaptation policies for this service that take into account the limited (and observable) resources. These policies are further refined according to the (partially observable) users’ interest level estimated from implicit feedback. Our theoretical models are validated through numerous simulations. The second part deals with hypermedia content delivery aiming to reduce navigation latencies by means of prefetching. As previously, we build upon an MDP model able to derive optimal prefetching policies integrating both user behaviour and ressource availability. First, we extend this model and propose more complex and aggressive policies. Second, the extended model is enriched by taking into account user's profile and therefore provides finer prefetching policies. It is worth noting that this model issues personnalized policies without explicily manipulating user profiles. The proposed extensions and the associated policies are validated through comparison with the original model and some heuristic approches. Finally, the third part considers multimedia applications in distributed contexts. In these contexts, highly interactive collaborative applications need to offer each user a consistent view of the interactions represented by the streams exchanged between dispersed groups of users. At the coordination level, strong ordering protocols for capturing and delivering streams' interactions (e.g. CAUSAL, TOTAL order) may be too expensive due to the variability of network conditions. We build upon previous work on expressing streams causality and propose a flexible coordination middleware for integrating different delivery modes (e.g. FIFO, CAUSAL, TOTAL) into a single channel (with respect to each of these protocols). Moreover, the proposed abstract channel can handle the mix of any partial or total order protocols. Integrating perceptual tolerance in our middleware, provides us with a coordination toolkit that performs better than Δ-causality, usually considered the best solutio

Partial prefetch for faster surfing in composite hypermedia

In this paper we present a prefetch technique, which incorporates a scheme similar to data streaming to minimize the response-lag. Unlike previous all or none techniques, we propose partial prefetch where the size of the lead segment is computed optimally so that only a minimum but sufficient amount of data is prefetched and buffered. The remaining segment is fetched if and only when the media is traversed. Thus, it delivers content without any increase in perceived response delay, and at the same time drastically minimizes unnecessary pre-load. The paper presents the scheme in the context of surfing in composite multimedia documents. It presents the technique and optimization scheme used for stream segmentation backed by analytical model and statistical simulation. We report remarkable increase in the responsiveness of web systems by a factor of 2-15 based on the specific situation