Multiprocessors have permitted astounding increases in computational performance, but many cannot meet the intense I/O requirements of some scienti c applications. An important component of any solution to this I/O bottleneck is a parallel le system that can provide high-bandwidth access to tremendous amounts of data in parallel to hundreds or thousands of processors. Most successful systems are based on a solid understanding of the expected workload, but thus far there have been no comprehensive workload characterizations of multiprocessor le systems. This paper presents the results of a three week tracing study in which all le-related activity on a massively parallel computer was recorded. Our instrumentation di ers from previous e orts in that it collects information about every I/O request and about the mix of jobs running in a production environment. We also present the results of a trace-driven caching simulation and recommendations for designers of multiprocessor le systems.