2,829 research outputs found
A Platform for Automating Chaos Experiments
The Netflix video streaming system is composed of many interacting services.
In such a large system, failures in individual services are not uncommon. This
paper describes the Chaos Automation Platform, a system for running failure
injection experiments on the production system to verify that failures in
non-critical services do not result in system outages.Comment: Conference publicatio
Khaos: Dynamically Optimizing Checkpointing for Dependable Distributed Stream Processing
Distributed Stream Processing systems are becoming an increasingly essential
part of Big Data processing platforms as users grow ever more reliant on their
ability to provide fast access to new results. As such, making timely decisions
based on these results is dependent on a system's ability to tolerate failure.
Typically, these systems achieve fault tolerance and the ability to recover
automatically from partial failures by implementing checkpoint and rollback
recovery. However, owing to the statistical probability of partial failures
occurring in these distributed environments and the variability of workloads
upon which jobs are expected to operate, static configurations will often not
meet Quality of Service constraints with low overhead.
In this paper we present Khaos, a new approach which utilizes the parallel
processing capabilities of virtual cloud automation technologies for the
automatic runtime optimization of fault tolerance configurations in Distributed
Stream Processing jobs. Our approach employs three subsequent phases which
borrows from the principles of Chaos Engineering: establish the steady-state
processing conditions, conduct experiments to better understand how the system
performs under failure, and use this knowledge to continuously minimize Quality
of Service violations. We implemented Khaos prototypically together with Apache
Flink and demonstrate its usefulness experimentally
!CHAOS: A cloud of controls
The paper is aimed to present the !CHAOS open source project
aimed to develop a prototype of a national private Cloud Computing infrastructure, devoted to accelerator control systems and large experiments of High Energy Physics (HEP). The !CHAOS project has been financed by MIUR (Italian Ministry of Research and Education) and aims to develop a new concept of control system and data acquisition framework by providing, with a high level of abstraction, all the services needed for controlling and managing a large scientific, or non-scientific, infrastructure. A beta version of the !CHAOS infrastructure will be released at the end of December 2015 and will run on private Cloud infrastructures based on OpenStack
- …