448 research outputs found
Self-adaptation via concurrent multi-action evaluation for unknown context
Context-aware computing has been attracting growing attention in recent years. Generally, there are several ways for a context-aware system to select a course of action for a particular change of context. One way is for the system developers to encompass all possible context changes in the domain knowledge. Other methods include system inferences and adaptive learning whereby the system executes one action and evaluates the outcome and self-adapts/self-learns based on that. However, in situations where a system encounters unknown contexts, the iterative approach would become unfeasible when the size of the action space increases. Providing efficient solutions to this problem has been the main goal of this research project.
Based on the developed abstract model, the designed methodology replaces the single action implementation and evaluation by multiple actions implemented and evaluated concurrently. This parallel evaluation of actions speeds up significantly the evolution time taken to select the best action suited to unknown context compared to the iterative approach.
The designed and implemented framework efficiently carries out concurrent multi-action evaluation when an unknown context is encountered and finds the best course of action. Two concrete implementations of the framework were carried out demonstrating the usability and adaptability of the framework across multiple domains.
The first implementation was in the domain of database performance tuning. The concrete implementation of the framework demonstrated the ability of concurrent multi-action evaluation technique to performance tune a database when performance is regressed for an unknown reason.
The second implementation demonstrated the ability of the framework to correctly determine the threshold price to be used in a name-your-own-price channel when an unknown context is encountered.
In conclusion the research introduced a new paradigm of a self-adaptation technique for context-aware application. Among the existing body of work, the concurrent multi-action evaluation is classified under the abstract concept of experiment-based self-adaptation techniques
Survey of storage systems for high-performance computing
In current supercomputers, storage is typically provided by parallel distributed file systems for hot data and tape archives for cold data. These file systems are often compatible with local file systems due to their use of the POSIX interface and semantics, which eases development and debugging because applications can easily run both on workstations and supercomputers. There is a wide variety of file systems to choose from, each tuned for different use cases and implementing different optimizations. However, the overall application performance is often held back by I/O bottlenecks due to insufficient performance of file systems or I/O libraries for highly parallel workloads. Performance problems are dealt with using novel storage hardware technologies as well as alternative I/O semantics and interfaces. These approaches have to be integrated into the storage stack seamlessly to make them convenient to use. Upcoming storage systems abandon the traditional POSIX interface and semantics in favor of alternative concepts such as object and key-value storage; moreover, they heavily rely on technologies such as NVM and burst buffers to improve performance. Additional tiers of storage hardware will increase the importance of hierarchical storage management. Many of these changes will be disruptive and require application developers to rethink their approaches to data management and I/O. A thorough understanding of today's storage infrastructures, including their strengths and weaknesses, is crucially important for designing and implementing scalable storage systems suitable for demands of exascale computing
DRackSim: Simulator for Rack-scale Memory Disaggregation
Memory disaggregation has emerged as an alternative to traditional server
architecture in data centers. This paper introduces DRackSim, a simulation
infrastructure to model rack-scale hardware disaggregated memory. DRackSim
models multiple compute nodes, memory pools, and a rack-scale interconnect
similar to GenZ. An application-level simulation approach simulates an x86
out-of-order multi-core processor with a multi-level cache hierarchy at compute
nodes. A queue-based simulation is used to model a remote memory controller and
rack-level interconnect, which allows both cache-based and page-based access to
remote memory. DRackSim models a central memory manager to manage address space
at the memory pools. We integrate community-accepted DRAMSim2 to perform memory
simulation at local and remote memory using multiple DRAMSim2 instances. An
incremental approach is followed to validate the core and cache subsystem of
DRackSim with that of Gem5. We measure the performance of various HPC workloads
and show the performance impact for different nodes/pools configuration
Scalable Storage for Digital Libraries
I propose a storage system optimised for digital libraries. Its key features are its heterogeneous scalability; its integration and exploitation of rich semantic metadata associated with digital objects; its use of a name space; and its aggressive performance optimisation in the digital library domain
An Analytical Model-based Capacity Planning Approach for Building CSD-based Storage Systems
The data movement in large-scale computing facilities (from compute nodes to
data nodes) is categorized as one of the major contributors to high cost and
energy utilization. To tackle it, in-storage processing (ISP) within storage
devices, such as Solid-State Drives (SSDs), has been explored actively. The
introduction of computational storage drives (CSDs) enabled ISP within the same
form factor as regular SSDs and made it easy to replace SSDs within traditional
compute nodes. With CSDs, host systems can offload various operations such as
search, filter, and count. However, commercialized CSDs have different hardware
resources and performance characteristics. Thus, it requires careful
consideration of hardware, performance, and workload characteristics for
building a CSD-based storage system within a compute node. Therefore, storage
architects are hesitant to build a storage system based on CSDs as there are no
tools to determine the benefits of CSD-based compute nodes to meet the
performance requirements compared to traditional nodes based on SSDs. In this
work, we proposed an analytical model-based storage capacity planner called
CSDPlan for system architects to build performance-effective CSD-based compute
nodes. Our model takes into account the performance characteristics of the host
system, targeted workloads, and hardware and performance characteristics of
CSDs to be deployed and provides optimal configuration based on the number of
CSDs for a compute node. Furthermore, CSDPlan estimates and reduces the total
cost of ownership (TCO) for building a CSD-based compute node. To evaluate the
efficacy of CSDPlan, we selected two commercially available CSDs and 4
representative big data analysis workloads
A differentiated proposal of three dimension i/o performance characterization model focusing on storage environments
The I/O bottleneck remains a central issue in high-performance environments. Cloud
computing, high-performance computing (HPC) and big data environments share many underneath difficulties to deliver data at a desirable time rate requested by high-performance
applications. This increases the possibility of creating bottlenecks throughout the application feeding process by bottom hardware devices located in the storage system layer.
In the last years, many researchers have been proposed solutions to improve the I/O
architecture considering different approaches. Some of them take advantage of hardware
devices while others focus on a sophisticated software approach. However, due to the
complexity of dealing with high-performance environments, creating solutions to improve
I/O performance in both software and hardware is challenging and gives researchers many
opportunities. Classifying these improvements in different dimensions allows researchers
to understand how these improvements have been built over the years and how it progresses. In addition, it also allows future efforts to be directed to research topics that
have developed at a lower rate, balancing the general development process. This research
present a three-dimension characterization model for classifying research works on I/O
performance improvements for large scale storage computing facilities. This classification
model can also be used as a guideline framework to summarize researches providing an
overview of the actual scenario. We also used the proposed model to perform a systematic
literature mapping that covered ten years of research on I/O performance improvements
in storage environments. This study classified hundreds of distinct researches identifying
which were the hardware, software, and storage systems that received more attention over
the years, which were the most researches proposals elements and where these elements
were evaluated. In order to justify the importance of this model and the development
of solutions that targets I/O performance improvements, we evaluated a subset of these
improvements using a a real and complete experimentation environment, the Grid5000.
Analysis over different scenarios using a synthetic I/O benchmark demonstrates how the
throughput and latency parameters behaves when performing different I/O operations
using distinct storage technologies and approaches.O gargalo de E/S continua sendo um problema central em ambientes de alto desempenho. Os ambientes de computação em nuvem, computação de alto desempenho (HPC) e big data compartilham muitas dificuldades para fornecer dados em uma taxa de tempo desejável solicitada por aplicações de alto desempenho. Isso aumenta a possibilidade de criar gargalos em todo o processo de alimentação de aplicativos pelos dispositivos de hardware inferiores localizados na camada do sistema de armazenamento. Nos últimos anos, muitos pesquisadores propuseram soluções para melhorar a arquitetura de E/S considerando diferentes abordagens. Alguns deles aproveitam os dispositivos de hardware, enquanto outros se concentram em uma abordagem sofisticada de software. No entanto, devido à complexidade de lidar com ambientes de alto desempenho, criar soluções para melhorar o desempenho de E/S em software e hardware é um desafio e oferece aos pesquisadores muitas oportunidades. A classificação dessas melhorias em diferentes dimensões permite que os pesquisadores entendam como essas melhorias foram construídas ao longo dos anos e como elas progridem. Além disso, também permite que futuros esforços sejam direcionados para tópicos de pesquisa que se desenvolveram em menor proporção, equilibrando o processo geral de desenvolvimento. Esta pesquisa apresenta um modelo de caracterização tridimensional para classificar trabalhos de pesquisa sobre melhorias de desempenho de E/S para instalações de computação de armazenamento em larga escala. Esse modelo de classificação também pode ser usado como uma estrutura de diretrizes para resumir as pesquisas, fornecendo uma visão geral do cenário real. Também usamos o modelo proposto para realizar um mapeamento sistemático da literatura que abrangeu dez anos de pesquisa sobre melhorias no desempenho de E/S em ambientes de armazenamento. Este estudo classificou centenas de pesquisas distintas, identificando quais eram os dispositivos de hardware, software e sistemas de armazenamento que receberam mais atenção ao longo dos anos, quais foram os elementos de proposta mais pesquisados e onde esses elementos foram avaliados. Para justificar a importância desse modelo e o desenvolvimento de soluções que visam melhorias no desempenho de E/S, avaliamos um subconjunto dessas melhorias usando um ambiente de experimentação real e completo, o Grid5000. Análises em cenários diferentes usando um benchmark de E/S sintética demonstra como os parâmetros de vazão e latência se comportam ao executar diferentes operações de E/S usando tecnologias e abordagens distintas de armazenamento
Computing at massive scale: Scalability and dependability challenges
Large-scale Cloud systems and big data analytics frameworks are now widely used for practical services and applications. However, with the increase of data volume, together with the heterogeneity of workloads and resources, and the dynamic nature of massive user requests, the uncertainties and complexity of resource management and service provisioning increase dramatically, often resulting in poor resource utilization, vulnerable system dependability, and user-perceived performance degradations. In this paper we report our latest understanding of the current and future challenges in this particular area, and discuss both existing and potential solutions to the problems, especially those concerned with system efficiency, scalability and dependability. We first introduce a data-driven analysis methodology for characterizing the resource and workload patterns and tracing performance bottlenecks in a massive-scale distributed computing environment. We then examine and analyze several fundamental challenges and the solutions we are developing to tackle them, including for example incremental but decentralized resource scheduling, incremental messaging communication, rapid system failover, and request handling parallelism. We integrate these solutions with our data analysis methodology in order to establish an engineering approach that facilitates the optimization, tuning and verification of massive-scale distributed systems. We aim to develop and offer innovative methods and mechanisms for future computing platforms that will provide strong support for new big data and IoE (Internet of Everything) applications
- …