1 research outputs found
Automating Cluster Management with Weave
Modern cluster management systems like Kubernetes and Openstack grapple with
hard combinatorial optimization problems: load balancing, placement,
scheduling, and configuration. Currently, developers tackle these problems by
designing custom application-specific algorithms---an approach that is proving
unsustainable, as ad-hoc solutions both perform poorly and introduce
overwhelming complexity to the system, making it challenging to add important
new features.
We propose a radically different architecture, where programmers drive
cluster management tasks declaratively, using SQL queries over cluster state
stored in a relational database. These queries capture in a natural way both
constraints on the cluster configuration as well as optimization objectives.
When a cluster reconfiguration is required at runtime, our tool, called Weave,
synthesizes an encoding of these queries into an optimization model, which it
solves using an off-the-shelf solver.
We demonstrate Weave's efficacy by powering three production-grade systems
with it: a Kubernetes scheduler, a virtual machine management solution, and a
distributed transactional datastore. Using Weave, we expressed complex cluster
management policies in under 20 lines of SQL, easily added new features to
these existing systems, and significantly improved placement quality and
convergence times