Software Shared Memory Support on Clusters of Symmetric MultiProcessors Using Remote-Write Networks

Abstract

Low-latency, remote-write-access networks have recently become commodity items. These networks can connect clusters of symmetric multiprocessors (SMPs) to form very cost-effective, large scale parallel systems. Software-based distributed shared memory (SDSM) is a natural choice for the underlying platform. However, to exploit the platform's full potential, sharing across SMPs must be managed without compromising the efficiency of sharing within an SMP. Cashmere-2L is a "two-level" SDSM protocol that delivers the platform's potential through novel software techniques that leverage, without compromising, the efficiency of the hardware coherence. The protocol implements a moderately lazy release consistency model with page directories, home-nodes, and multiple concurrent writers. By avoiding global meta-data locks and TLB shootdown, Cashmere2L is able to maintain a high level of asynchrony. The prototype Cashmere-2L system currently runs on an 8-node, 32-processor DEC AlphaServer cluster ..