Geospatial Data Indexing Analysis and Visualization via Web Services with Autonomic Resource Management

With the exponential growth of the usage of web-based map services, the web GIS application has become more and more popular. Spatial data index, search, analysis, visualization and the resource management of such services are becoming increasingly important to deliver user-desired Quality of Service. First, spatial indexing is typically time-consuming and is not available to end-users. To address this, we introduce TerraFly sksOpen, an open-sourced an Online Indexing and Querying System for Big Geospatial Data. Integrated with the TerraFly Geospatial database [1-9], sksOpen is an efficient indexing and query engine for processing Top-k Spatial Boolean Queries. Further, we provide ergonomic visualization of query results on interactive maps to facilitate the user’s data analysis. Second, due to the highly complex and dynamic nature of GIS systems, it is quite challenging for the end users to quickly understand and analyze the spatial data, and to efficiently share their own data and analysis results with others. Built on the TerraFly Geo spatial database, TerraFly GeoCloud is an extra layer running upon the TerraFly map and can efficiently support many different visualization functions and spatial data analysis models. Furthermore, users can create unique URLs to visualize and share the analysis results. TerraFly GeoCloud also enables the MapQL technology to customize map visualization using SQL-like statements [10]. Third, map systems often serve dynamic web workloads and involve multiple CPU and I/O intensive tiers, which make it challenging to meet the response time targets of map requests while using the resources efficiently. Virtualization facilitates the deployment of web map services and improves their resource utilization through encapsulation and consolidation. Autonomic resource management allows resources to be automatically provisioned to a map service and its internal tiers on demand. v-TerraFly are techniques to predict the demand of map workloads online and optimize resource allocations, considering both response time and data freshness as the QoS target. The proposed v-TerraFly system is prototyped on TerraFly, a production web map service, and evaluated using real TerraFly workloads. The results show that v-TerraFly can accurately predict the workload demands: 18.91% more accurate; and efficiently allocate resources to meet the QoS target: improves the QoS by 26.19% and saves resource usages by 20.83% compared to traditional peak load-based resource allocation

Caracterização computacional para alocação distribuída para uma configuração com interface natural de usuário

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Ciência da Computação, Florianópolis, 2015.Em um sistema distribuído heterogêneo, como grades computacionais, a escolha do sistema computacional para processar uma tarefa é realizada por meio de heurísticas adotadas igualmente para todos os sistemas. Os métodos atuais para avaliação da carga computacional, em grades heterogêneas, não levam em consideração características qualitativas que afetam o desempenho. Sistemas computacionais aparentemente idênticos, com as mesmas características quantitativas (tal como a quantidade de núcleos de processamento e de memória), podem apresentar desempenhos desiguais. O método proposto consiste em uma política de informação ao balanceamento de carga e tem como objetivo mensurar a carga dos sistemas computacionais por meio da avaliação de seus recursos quantitativos, tanto os imutáveis (como a quantidade de núcleos de processamento) quanto os mutáveis (como o percentual de memória livre), e qualitativos, inerentes à arquitetura do sistema computacional. A comparação da carga computacional entre os sistemas permite que o balanceamento de carga seja realizado mesmo em sistemas distribuídos heterogêneos para que seja possível a escolha do sistema computacional no qual executar uma tarefa da forma mais eficiente. Esta pesquisa utiliza a ferramenta CVFlow, uma Interface Natural de Usuário destinada ao balanceamento de carga, para avaliar o método proposto. O experimento consiste no escalonamento de um conjunto de tarefas e na comparação do método proposto com o estado da arte presente na literatura. O método proposto fornece um conjunto de melhorias que distribuem a carga de forma mais homogênea entre os sistemas computacionais, evitando, assim, sobrecarregar um sistema específico, além de oferecer um desempenho superior na execução do conjunto de tarefas.Abstract : In a distributed heterogeneous system, such as grids, the choice of a computer system to process a task is performed by means of heuristics adopted equally for all systems. Current methods for assessing the computing load, on heterogeneous grids, do not take into account qualitative characteristics that affect performance. Computer systems apparently identical, with the same quantitative traits (such as the number of processing cores and memory), may provide different performance. The proposed method consists of an information policy to load balancing. It aims to measure the load of a computer systems through the assessment of their quantitative and qualitative features. Quantitative, both immutable (as the number of cores) and mutable (as the percentage of free memory). And the qualitative, inherent to the computer system architecture. Comparison of computational load between systems allows load balancing to be performed even in heterogeneous distributed systems, to be able to choose the computer system on which to perform a task more efficiently. This research uses the CVFlow tool, a Natural User Interface intended for load balancing, to evaluate the proposed method. The experiment consists of the scheduling of a set of tasks and the comparison of the proposed method with the state of the art. The proposed method provides a set of improvements that distribute the load more evenly among computer systems, avoid overloading a particular system, and provides a better performance on the execution of the set of tasks

Fuzzy Modeling and Control Based Virtual Machine Resource Management

Virtual machines (VMs) are powerful platforms for building agile datacenters and emerging cloud systems. However, resource management for a VM-based system is still a challenging task. First, the complexity of application workloads as well as the interference among competing workloads makes it difficult to understand their VMs’ resource demands for meeting their Quality of Service (QoS) targets; Second, the dynamics in the applications and system makes it also difficult to maintain the desired QoS target while the environment changes; Third, the transparency of virtualization presents a hurdle for guest-layer application and host-layer VM scheduler to cooperate and improve application QoS and system efficiency. This dissertation proposes to address the above challenges through fuzzy modeling and control theory based VM resource management. First, a fuzzy-logic-based nonlinear modeling approach is proposed to accurately capture a VM’s complex demands of multiple types of resources automatically online based on the observed workload and resource usages. Second, to enable fast adaption for resource management, the fuzzy modeling approach is integrated with a predictive-control-based controller to form a new Fuzzy Modeling Predictive Control (FMPC) approach which can quickly track the applications’ QoS targets and optimize the resource allocations under dynamic changes in the system. Finally, to address the limitations of black-box-based resource management solutions, a cross-layer optimization approach is proposed to enable cooperation between a VM’s host and guest layers and further improve the application QoS and resource usage efficiency. The above proposed approaches are prototyped and evaluated on a Xen-based virtualized system and evaluated with representative benchmarks including TPC-H, RUBiS, and TerraFly. The results demonstrate that the fuzzy-modeling-based approach improves the accuracy in resource prediction by up to 31.4% compared to conventional regression approaches. The FMPC approach substantially outperforms the traditional linear-model-based predictive control approach in meeting application QoS targets for an oversubscribed system. It is able to manage dynamic VM resource allocations and migrations for over 100 concurrent VMs across multiple hosts with good efficiency. Finally, the cross-layer optimization approach further improves the performance of a virtualized application by up to 40% when the resources are contended by dynamic workloads