AGOCS – Accurate Google Cloud Simulator Framework

This paper presents the Accurate Google Cloud Simulator (AGOCS) – a novel high-fidelity Cloud workload simulator based on parsing real workload traces, which can be conveniently used on a desktop machine for day-to-day research. Our simulation is based on real-world workload traces from a Google Cluster with 12.5K nodes, over a period of a calendar month. The framework is able to reveal very precise and detailed parameters of the executed jobs, tasks and nodes as well as to provide actual resource usage statistics. The system has been implemented in Scala language with focus on parallel execution and an easy-to-extend design concept. The paper presents the detailed structural framework for AGOCS and discusses our main design decisions, whilst also suggesting alternative and possibly performance enhancing future approaches. The framework is available via the Open Source GitHub repository

A Deep Dive into the Google Cluster Workload Traces: Analyzing the Application Failure Characteristics and User Behaviors

Large-scale cloud data centers have gained popularity due to their high availability, rapid elasticity, scalability, and low cost. However, current data centers continue to have high failure rates due to the lack of proper resource utilization and early failure detection. To maximize resource efficiency and reduce failure rates in large-scale cloud data centers, it is crucial to understand the workload and failure characteristics. In this paper, we perform a deep analysis of the 2019 Google Cluster Trace Dataset, which contains 2.4TiB of workload traces from eight different clusters around the world. We explore the characteristics of failed and killed jobs in Google's production cloud and attempt to correlate them with key attributes such as resource usage, job priority, scheduling class, job duration, and the number of task resubmissions. Our analysis reveals several important characteristics of failed jobs that contribute to job failure and hence, could be used for developing an early failure prediction system. Also, we present a novel usage analysis to identify heterogeneity in jobs and tasks submitted by users. We are able to identify specific users who control more than half of all collection events on a single cluster. We contend that these characteristics could be useful in developing an early job failure prediction system that could be utilized for dynamic rescheduling of the job scheduler and thus improving resource utilization in large-scale cloud data centers while reducing failure rates

Algorithms for advance bandwidth reservation in media production networks

Media production generally requires many geographically distributed actors (e.g., production houses, broadcasters, advertisers) to exchange huge amounts of raw video and audio data. Traditional distribution techniques, such as dedicated point-to-point optical links, are highly inefficient in terms of installation time and cost. To improve efficiency, shared media production networks that connect all involved actors over a large geographical area, are currently being deployed. The traffic in such networks is often predictable, as the timing and bandwidth requirements of data transfers are generally known hours or even days in advance. As such, the use of advance bandwidth reservation (AR) can greatly increase resource utilization and cost efficiency. In this paper, we propose an Integer Linear Programming formulation of the bandwidth scheduling problem, which takes into account the specific characteristics of media production networks, is presented. Two novel optimization algorithms based on this model are thoroughly evaluated and compared by means of in-depth simulation results

The lifecycle of geotagged data

The world is a big place. At any given instant something is happening somewhere, but even when nothing in particular is going on people still find ways to generate data, such as posting on s

Cloud Computing Trace Characterization and Synthetic Workload Generation

This thesis researches cloud computing workload characteristics and synthetic workload generation. A heuristic presented in the work guides the process of workload trace characterization and synthetic workload generation. Analysis of a cloud trace provides insight into client request behaviors and statistical parameters. A versatile workload generation tool creates client connections, controls request rates, defines number of jobs, produces tasks within each job, and manages task durations. The test system consists of multiple clients creating workloads and a server receiving request, all contained within a virtual machine environment. Statistical analysis verifies the synthetic workload experimental results are consistent with real workload behaviors and characteristics

Framework for Analyzing Customer Involvement in Product-service Systems

Abstract In manufacturing, product-service systems (PSS) that create value by coupling a physical product and a service have been attracting attention. In PSS, it is important for providers to enhance the value-in-use that is perceived by customers in utilizing a product and/or service. Customers play a key role in realizing such value and therefore, are regarded as co-producers in the value-creation process. Although customer involvement plays an essential role in realizing value, previous research has revealed its risks. Therefore, PSS providers are required to adopt a suitable strategy for involving customers. However, current studies do not necessarily offer much guidance on determining such strategies. To solve this problem, this paper proposes a framework that analyzes the benefits and risks of customer involvement in PSS development. This framework aims to identify factors that influence benefits and risks from the viewpoints of characteristics of a PSS and its customer involvement. The effectiveness of the proposed framework is validated through a case study