Integrating Scale Out and Fault Tolerance in Stream Processing using Operator State Management

As users of big data applications expect fresh results, we witness a new breed of stream processing systems (SPS) that are designed to scale to large numbers of cloud-hosted machines. Such systems face new challenges: (i) to benefit from the pay-as-you-go model of cloud computing, they must scale out on demand, acquiring additional virtual machines (VMs) and parallelising operators when the workload increases; (ii) failures are common with deployments on hundreds of VMs - systems must be fault-tolerant with fast recovery times, yet low per-machine overheads. An open question is how to achieve these two goals when stream queries include stateful operators, which must be scaled out and recovered without affecting query results. Our key idea is to expose internal operator state explicitly to the SPS through a set of state management primitives. Based on them, we describe an integrated approach for dynamic scale out and recovery of stateful operators. Externalised operator state is checkpointed periodically by the SPS and backed up to upstream VMs. The SPS identifies individual operator bottlenecks and automatically scales them out by allocating new VMs and partitioning the check-pointed state. At any point, failed operators are recovered by restoring checkpointed state on a new VM and replaying unprocessed tuples. We evaluate this approach with the Linear Road Benchmark on the Amazon EC2 cloud platform and show that it can scale automatically to a load factor of L=350 with 50 VMs, while recovering quickly from failures. Copyright © 2013 ACM

ROBOSIM, a simulator for robotic systems

ROBOSIM, a simulator for robotic systems, was developed by NASA to aid in the rapid prototyping of automation. ROBOSIM has allowed the development of improved robotic systems concepts for both earth-based and proposed on-orbit applications while significantly reducing development costs. In a cooperative effort with an area university, ROBOSIM was further developed for use in the classroom as a safe and cost-effective way of allowing students to study robotic systems. Students have used ROBOSIM to study existing robotic systems and systems which they have designed in the classroom. Since an advanced simulator/trainer of this type is beneficial not only to NASA projects and programs but industry and academia as well, NASA is in the process of developing this technology for wider public use. An update on the simulators's new application areas, the improvements made to the simulator's design, and current efforts to ensure the timely transfer of this technology are presented

Excited-state quantum phase transitions in a two-fluid Lipkin model

Background: Composed systems have became of great interest in the framework of the ground state quantum phase transitions (QPTs) and many of their properties have been studied in detail. However, in these systems the study of the so called excited-state quantum phase transitions (ESQPTs) have not received so much attention. Purpose: A quantum analysis of the ESQPTs in the two-fluid Lipkin model is presented in this work. The study is performed through the Hamiltonian diagonalization for selected values of the control parameters in order to cover the most interesting regions of the system phase diagram. [Method:] A Hamiltonian that resembles the consistent-Q Hamiltonian of the interacting boson model (IBM) is diagonalized for selected values of the parameters and properties such as the density of states, the Peres lattices, the nearest-neighbor spacing distribution, and the participation ratio are analyzed. Results: An overview of the spectrum of the two-fluid Lipkin model for selected positions in the phase diagram has been obtained. The location of the excited-state quantum phase transition can be easily singled out with the Peres lattice, with the nearest-neighbor spacing distribution, with Poincar\'e sections or with the participation ratio. Conclusions: This study completes the analysis of QPTs for the two-fluid Lipkin model, extending the previous study to excited states. The ESQPT signatures in composed systems behave in the same way as in single ones, although the evidences of their presence can be sometimes blurred. The Peres lattice turns out to be a convenient tool to look into the position of the ESQPT and to define the concept of phase in the excited states realm