Retro: Targeted Resource Management in Multi-tenant Distributed Systems

Abstract In distributed systems shared by multiple tenants, effective resource management is an important pre-requisite to providing quality of service guarantees. Many systems deployed today lack performance isolation and experience contention, slowdown, and even outages caused by aggressive workloads or by improperly throttled maintenance tasks such as data replication. In this work we present Retro, a resource management framework for shared distributed systems. Retro monitors per-tenant resource usage both within and across distributed systems, and exposes this information to centralized resource management policies through a high-level API. A policy can shape the resources consumed by a tenant using Retro's control points, which enforce sharing and ratelimiting decisions. We demonstrate Retro through three policies providing bottleneck resource fairness, dominant resource fairness, and latency guarantees to high-priority tenants, and evaluate the system across five distributed systems: HBase, Yarn, MapReduce, HDFS, and Zookeeper. Our evaluation shows that Retro has low overhead, and achieves the policies' goals, accurately detecting contended resources, throttling tenants responsible for slowdown and overload, and fairly distributing the remaining cluster capacity

Effect of the crystallographic c-axis orientation on the tribological properties of the few-layer PtSe2

Two-dimensional (2D) transition metal dichalcogenides are potential candidates for ultrathin solid-state lubricants in low-dimensional systems owing to their flatness, high in-plane mechanical strength, and low shear interlayer strength. Yet, the effects of surface topography and surface chemistry on the tribological properties of 2D layers are still unclear. In this work, we performed a comparative investigation of nanoscale tribological properties of ultra-thin highly-ordered PtSe2 layers deposited on the sapphire substrates with the in-plane and out-of-plane crystallographic orientation of the PtSe2 c-axis flakes, and epitaxial PtSe2 layers. PtSe2 c-axis orientation was found to has an impact on the nanotribological, morphological and electrical properties of PtSe2, in particular the change in the alignment of the PtSe2 flakes from vertical (VA) to horizontal (HA) led to the lowering of the coefficient of friction from 0.21 to 0.16. This observation was accompanied by an increase in the root-mean-square surface roughness from 1.0 to 1.7 nm for the HA and VA films, respectively. The epitaxial films showed lower friction caused by lowering adhesion when compared to other investigated films, whereas the friction coefficient was similar to films with HA flakes. The observed trends in nanoscale friction is attributed to a different distribution of PtSe2 structure

Automating Datacenter Operations Using Machine Learning

Today's Internet datacenters run many complex and large-scale Web applications that are very difficult to manage. The main challenges are understanding user workloads and application performance, and quickly identifying and resolving performance problems. Statistical Machine Learning (SML) provides a methodology for quickly processing the large quantities of monitoring data generated by these applications, finding repeating patterns in their behavior, and building accurate models of their performance. This dissertation argues that SML is a useful tool for simplifying and automating datacenter operations and demonstrates application of SML to three important problems in this area: characterization and synthesis of workload spikes, dynamic resource allocation in stateful systems, and quick and accurate identification of recurring performance problems

Distributed resource management across process boundaries

Multi-tenant distributed systems composed of small services, such as Service-oriented Architectures (SOAs) and Micro-services, raise new challenges in attaining high performance and efficient resource utilization. In these systems, a request execution spans tens to thousands of processes, and the execution paths and resource demands on different services are generally not known when a request first enters the system. In this paper, we highlight the fundamental challenges of regulating load and scheduling in SOAs while meeting end-to-end performance objectives on metrics of concern to both tenants and operators. We design Wisp, a framework for building SOAs that transparently adapts rate limiters and request schedulers system-wide according to operator policies to satisfy end-to-end goals while responding to changing system conditions. In evaluations against production as well as synthetic workloads, Wisp successfully enforces a range of end-to-end performance objectives, such as reducing average latencies, meeting deadlines, providing fairness and isolation, and avoiding system overload

Distributed regression: an efficient framework for modeling sensor network data

We present distributed regression, an efficient and general framework for in-network modeling of sensor data. In this framework, the nodes of the sensor network collaborate to optimally fit a global function to each of their local measurements. The algorithm is based upon kernel linear regression, where the model takes the form of a weighted sum of local basis functions; this provides an expressive yet tractable class of models for sensor network data. Rather than transmitting data to one another or outside the network, nodes communicate constraints on the model parameters, drastically reducing the communication required. After the algorithm is run, each node can answer queries for its local region, or the nodes can efficiently transmit the parameters of the model to a user outside the network. We present an evaluation of the algorithm based upon data from a 48-node sensor network deployment at the Intel Research- Berkeley Lab, demonstrating that our distributed algorithm converges to the optimal solution at a fast rate and is very robust to packet losses