Reducing the SPEC2006 Benchmark Suite for SimulationBased Abstract Computer Architecture Research

Abstract

Present day   computer   architects   use   advanced microarchitecture simulators   to   test   the   performance   of processor designs. The simulator workloads are generally benchmarks,  which are representative of specific types of real world   applications. Because   microarchitecture implementations increase in complexity and  the simulation workloads are   required   to   represent   complicated applications,  the simulation time has greatly  increased. To solve the problem,  researchers are looking into ways to reduce the   amount   of   time   benchmarks   run,   while maintaining the   same   workload   characterization   of   the longer benchmarks. MinneSPEC   is   a   representative reduction of SPEC2000,  with the reduced input sets found using SimpleScalar profiling tools [1]. With the release of SPEC CPU2006,  new benchmarks have been added to the SPEC benchmarking suite which will be used to evaluate performance in   tomorrow's   microprocessors. These benchmarks are considerably larger than SPEC2000 and using SimpleScalar to profile their workloads would take a large amount of time and effort. This paper suggests a different reduction   technique   which   gathers   profiling information using   processor   performance   counters accessed using PAPI. Since workloads are running on a native system instead of a simulator,  profiling information can be gathered in a much shorter amount of time. This allows for fine grained tuning of reduced input sets so more representative reduced benchmarks can be found in a much shorter amount of time. Using this technique,  we were able to reduce five SPEC2006 benchmarks to under 1