1 research outputs found
Achieving High Throughput and Elasticity in a Larger-than-Memory Store
Millions of sensors, mobile applications and machines are now generating
billions of events. This has led to specialized many-core key-value stores
(KVSs) that can ingest and index these events at high rates (100
Mops/s/machine). These systems are efficient if events are generated on the
same machine, but, to be practical and cost-effective they must ingest events
over the network and scale across cloud resources elastically.
We present Shadowfax, a new distributed key-value store that transparently
spans DRAM, SSDs, and cloud blob storage while serving 130 Mops/s/VM over
commodity Azure VMs using conventional Linux TCP. Beyond high single-VM
performance, Shadowfax uses a unique approach to distributed reconfiguration
that avoids any server-side key ownership checks or cross-core coordination
both during normal operation and migration. Hence, Shadowfax can shift load in
17 s to improve cluster throughput by 10% with little disruption. Compared to
the state-of-the-art, it has 8x better throughput (than Seastar+memcached) and
scales out 6x faster