High-performance cluster computing, algorithms, implementations and performance evaluation for computation-intensive applications to promote complex scientific research on turbulent flows

Large-scale high-performance computing is a very rapidly growing field of research that plays a vital role in the advance of science, engineering, and modern industrial technology. Increasing sophistication in research has led to a need for bigger and faster computers or computer clusters, and high-performance computer systems are themselves stimulating the redevelopment of the methods of computation. Computing is fast becoming the most frequently used technique to explore new questions. We have developed high-performance computer simulation modeling software system on turbulent flows. Five papers are selected to present here from dozens of papers published in our efforts on complex software system development and knowledge discovery through computer simulations. The first paper describes the end-to-end computer simulation system development and simulation results that help understand the nature of complex shelterbelt turbulent flows. The second paper deals specifically with high-performance algorithm design and implementation in a cluster of computers. The third paper discusses the twelve design processes of parallel algorithms and software system as well as theoretical performance modeling and characterization of cluster computing. The fourth paper is about the computing framework of drag and pressure coefficients. The fifth paper is about simulated evapotranspiration and energy partition of inhomogeneous ecosystems. We discuss the end-to-end computer simulation system software development, distributed parallel computing performance modeling and system performance characterization. We design and compare several parallel implementations of our computer simulation system and show that the performance depends on algorithm design, communication channel pattern, and coding strategies that significantly impact load balancing, speedup, and computing efficiency. For a given cluster communication characteristics and a given problem complexity, there exists an optimal number of nodes. With this computer simulation system, we resolved many historically controversial issues and a lot of important problems

Distributed Dynamic Replica Placement and Request Redirection in Content Delivery Networks

Chiara Petrioli, Giancarlo Bongiovanni, Francesco Lo Prest

Aplicação de técnicas de inteligência artificial no desenvolvimento de agentes para gerência de redes

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica

Analysis of Task Assignment Policies in Scalable Distributed Web-server Systems

A distributed multi-server Web site can provide the scalability necessary to keep up with growing client demand at popular sites. Load balancing of these distributed Web-server systems, consisting of multiple Web servers for document retrieval and a Domain name server (DNS) for address resolution, opens interesting new problems. In this paper, we investigate the effects of using a more active DNS which, as an atypical centralized scheduler, applies some scheduling strategy in routing the requests to the most suitable Web server. Unlike traditional parallel/distributed systems in which a centralized scheduler has full control of the system, the DNS controls only a very small fraction of the requests reaching the multi-server Web site. This peculiarity, especially in the presence of highly skewed load, makes it very difficult to achieve acceptable load balancing and avoid overloading some Web server. This paper adapts traditional scheduling algorithms to the DNS, proposes new policies, a..

Analysis of task assignment policies in scalable distributed Web server systems

A distributed multiserver Web site can provide the scalability necessary to keep up with growing client demand at popular sites. Load balancing of these distributed Web-server systems, consisting of multiple, homogeneous Web servers for document retrieval and a Domain Name Server (DNS) for address resolution, opens interesting new problems. In this paper, we investigate the effects of using a more active DNS which, as an atypical centralized scheduler, applies some scheduling strategy in routing the requests to the most suitable Web server. Unlike traditional parallel/distributed systems in which a centralized scheduler has full control of the system, the DNS controls only a very small fraction of the requests reaching the multiserver Web site. This peculiarity, especially in the presence of highly skewed load, makes it very difficult to achieve acceptable load balancing and avoid overloading some Web servers. This paper adapts traditional scheduling algorithms to the DNS, proposes new policies, and examines their impact under different scenarios. Extensive simulation results show the advantage of strategies that make scheduling decisions on the basis of the domain that originates the client requests and limited server state information (e.g., whether a server is overloaded or not). An initially unexpected result is that using detailed server information, especially based on history, does not seem useful in predicting the future load and can often lead to degraded performance

Otimizando servidores web de alta demanda

Orientador : Paulo Licio de GeusDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Com o explosivo crescimento da Web, sua popularização e sua grande importância como meio de comunicação global, cada vez mais a atenção se volta para questões de desempenho. Pelo fato de ser um serviço relativamente recente, são poucos os estudos que abordam de modo geral todos os fatores que influenciam no seu desempenho e como tais fatores estão inter-relacionados. Assim, o objetivo deste trabalho é abordar o problema de desempenho na Web como um todo, levantando e analisando todos os componentes relacionados, fornecendo embasamento teórico para o entendimento desses tópicos e apresentando suas possíveis soluções, de forma a cobrir esta lacuna na bibliografia existente. Em especial, se concentra em tópicos de software na porção servidora de sistemas de grande demanda. Como resultado, o leitor encontrará um texto único na otimização servidores de grande porte, que é um trabalho de referência e ao mesmo tempo um guia de otimizaçãoAbstract: With the Web explosive growth, its popularization and its great importance as a way of global communication, more and more attention is devoted to its performance issues. Due to the Web being a relatively new service, very few works approach the issue of performance affecting factors in a broad sense, as well as their inter-relationship. The goal of this work is to fill that gap by enumerating and analyzing every component that affects Web performance. This is done by covering theoretical issues for a clear understanding of all topics involved and by presenting possible solutions and suggestions. The main focus of this work is on the server portion of high performance systems. As a result, the reader will find a unique text on optimizing performance of very large Web sites, that is both a reference work and a tuning guideMestradoMestre em Ciência da Computaçã

Effective task assignment strategies for distributed systems under highly variable workloads

Heavy-tailed workload distributions are commonly experienced in many areas of distributed computing. Such workloads are highly variable, where a small number of very large tasks make up a large proportion of the workload, making the load very hard to distribute effectively. Traditional task assignment policies are ineffective under these conditions as they were formulated based on the assumption of an exponentially distributed workload. Size-based task assignment policies have been proposed to handle heavy-tailed workloads, but their applications are limited by their static nature and assumption of prior knowledge of a task's service requirement. This thesis analyses existing approaches to load distribution under heavy-tailed workloads, and presents a new generalised task assignment policy that significantly improves performance for many distributed applications, by intelligently addressing the negative effects on performance that highly variable workloads cause. Many problems associated with the modelling and optimisations of systems under highly variable workloads were then addressed by a novel technique that approximated these workloads with simpler mathematical representations, without losing any of their pertinent original properties. Finally, we obtain advance queuing metrics (such as the variance of key measurements like waiting time and slowdown that are difficult to obtain analytically) through rigorous simulation