. This paper explores the foundations and philosophy that have made PVM both effective and widespread: a simple system abstraction and API, transparent heterogeneity, and dynamic system configuration. From a high-performance programming point of view, we examine the features that make PVM useful and those that make hardware-level performance difficult to achieve. The key conclusion from this analysis is that PVM, MPI, and similar paradigms suffer from a monolithic approach to the distributed computing problem. The approaches simply cannot meet the large range of service requirements for advanced distributed computing environments. The notion of a Generalized PlugIn Machine (GPM) is introduced that allows programs to exert better control over their operating environment. This environment has the potential to provide mechanisms for better performance, richer system dynamics, and fault-tolerance. Pluggable components, such as messaging substrates, dynamically-attached debugging agents, or..