Using Remote Access Histories for Thread Scheduling in Distributed Shared Memory Systems

We developed the remote access histories mechanism (RAHM), a technique that can be used to dynamically, adaptively, and transparently improve the locality of memory references in distributed shared memory systems (DSMs). The RAHM helps the run-time system make correct decisions concerning the redistribution of threads to hosts -- operations that typically involve high overhead. The RAHM is independent of the memory consistency model, and comes into play only when the memory model fails to achieve locality of memory reference. The RAHM piggyback on existing communication and adds only a negligible overhead. The rahm is designed to work in dynamically changing systems, such as non-dedicated distributed environments. The rahm is fully implemented in user-space as part of the millipede system, a Virtual Parallel Machine (VPM) designed for non-dedicated NT/PC clusters. The performance evaluation shows a potential for dramatic improvement in the efficiency of shared-memory parallel computation..