Using Global Behavior Modeling to Improve QoS in Cloud Data Storage Services

The cloud computing model aims to make large-scale data-intensive computing affordable even for users with limited financial resources, that cannot invest into expensive infrastructures necesssary to run them. In this context, MapReduce is emerging as a highly scalable programming paradigm that enables high-throughput data-intensive processing as a cloud service. Its performance is highly dependent on the underlying storage service, responsible to efficiently support massively parallel data accesses by guaranteeing a high throughput under heavy access concurrency. In this context, quality of service plays a crucial role: the storage service needs to sustain a stable throughput for each individual accesss, in addition to achieving a high aggregated throughput under concurrency. In this paper we propose a technique to address this problem using component monitoring, application-side feedback and behavior pattern analysis to automatically infer useful knowledge about the causes of poor quality of service and provide an easy way to reason in about potential improvements. We apply our proposal to Blob Seer, a representative data storage service specifically designed to achieve high aggregated throughputs and show through extensive experimentation substantial improvements in the stability of individual data read accesses under MapReduce workloads

Intermediate QoS Prototype for the EDGI Infrastructure

This document provides the first deliverable of EDGI JRA2. It is produced by the INRIA team, the SZTAKI team, the LAL/IN2P3 team and the University of Coimbra team. This document aims at describing achievements and results of JRA2 tasks "Advanced QoS Scheduler and Oracle" and "Support In Science Gateway". Hybrid Distributed Computing Infrastructures (DCIs) allow users to combine Grids, Desktop Grids, Clouds, etc. to obtain for their users large computing capabilities. The EDGI infrastructure belongs to this kind of DCIs. The document presents the SpeQuloS framework to provide quality of service (QoS) for application executed on the EDGI infrastructure. It also introduces EDGI QoS portal, an user-friendly and integrated access to QoS features for users of EDGI infrastructure. In this document, we first introduce new results from JRA2.1 task, which collected and analyzed batch execution on Desktop Grid. Then, we present the advanced Cloud Scheduling and Oracle strategies designed inside the SpeQuloS framework (task JRA2.2). We demonstrate efficiency of these strategies using performance evaluation carried out with simulations. Next, we introduce Credit System architecture and QoS user portal as part of the JRA2 Support In Science Gateway (task JRA2.3). Finally, we conclude and provide references to JRA2 production.Ce document fournit le premier livrable pour la tâche JRA2 du projet européen European Desktop Grid Initiative (FP7 EDGI). Il est produit par les équipes de l'INRIA, de SZTAKI, du LAL/IN2P3 et de l'Université de Coimbra. Ce document vise à décrire les réalisations et les résultats qui concernent la qualité de service pour l'infrastructure de grilles de PCs européenne EDGI

Predicting Network Failures with AI Techniques

Network failure is the unintentional interruption of internet services, resulting in widespread client frustration. It is especially true for time-sensitive services in the healthcare industry, smart grid control, and mobility control, among others. In addition, the COVID-19 pandemic has compelled many businesses to operate remotely, making uninterrupted internet access essential. Moreover, Internet Service Providers (ISPs) lose millions of dollars annually due to network failure, which has a negative impact on their businesses. Currently, redundant network equipment is used as a restoration technique to resolve this issue of network failure. This technique requires a strategy for failure identification and prediction to run faultlessly and without delay. However, we lack a suitable generic network failure identification and prediction system due to the unavailability of publicly accessible failure data. This study simulates network traffic to gather failure data based on a general network failure guideline. Furthermore, various state-of-the-art Machine Learning and Deep Learning methods were applied to the generated data. Notably, our proposed Deep Learning model for failure identification provides accuracy, precision, recall, and F1 scores in the range of 97% to 99% for three different demonstration networks. Additionally, our proposed Long Short Term Memory model gives low root mean square error rates of 0.9751 for failure prediction

Flexible distributed computing with volunteered resources

PhDNowadays, computational grids have evolved to a stage where they can comprise many volunteered resources owned by different individual users and/or institutions, such as desktop grids and volunteered computing grids. This brings benefits for large-scale computing, as more resources are available to exploit. On the other hand, the inherent characteristics of the volunteered resources bring some challenges for efficiently exploiting them. For example, jobs may not be able to be executed by some resources, as the computing resources can be heterogeneous. Furthermore, the resources can be volatile as the resource owners usually have the right to decide when and how to donate the idle Central Processing Unit (CPU) cycles of their computers. Therefore, in order to utilise volunteered resources efficiently, this research investigated solutions from different aspects. Firstly, this research proposes a new computational Grid architecture based on Java and Java application migration technologies to provide fundamental support for coping with these challenges. This proposed architecture supports heterogeneous resources, ensuring local activities are not affected by Grid jobs and enabling resources to carry out live and automatic Java application migration. Secondly, this research work proposes some job-scheduling and migration algorithms based on resource availability prediction and/or artificial intelligence techniques. To examine the proposed algorithms, this work includes a series of experiments in both synthetic and practical scenarios and compares the performance of the proposed algorithms with existing ones across a variety of scenarios. According to the critical assessment, each algorithm has its own distinct advantages and performs well when certain conditions are met. In addition, this research analyses the characteristics of resources in terms of the availability pattern of practical volunteer-based grids. The analysis shows that each environment has its own characteristics and each volunteered resource’s availability tends to possess weak correlations across different days and times-of-day.British Telco

安全で高性能なボランティアコンピューティング基盤に関する研究

Tohoku University小林 広明課

Multi-State Grid Resource Availability Characterization

Abstract—The functional heterogeneity of non-dedicated computational grids will increase with the inclusion of resources from desktop grids, P2P systems, and even mobile grids. Machine failure characteristics, as well as individual and organizational policies for resource usage by the grid, will increasingly vary even more than they already do. Since grid applications also vary as to how well they tolerate the failure of the host on which they run, grid schedulers must begin to predict and consider how resources will transition between availability modes. Toward this goal, this paper introduces five availability states, and characterizes a Condor pool trace that uncovers when, how, and why its resources reside in, and transition between, these states. This characterization suggests resource categories that schedulers can use to make better mapping decisions. Simulations that characterize how a variety of jobs would run on the traced resources demonstrate this approach’s potential for performance improvement. A simple predictor based on the previous day's behavior indicates that the states and categories are somewhat predictable, thereby supporting the potential usefulness of multi-state grid resource availability characterization. 1 I