Standardizing the performance assessment of reconfigurable processor architectures

Comparing and evaluating the performance of reconfigurable processors is a difficult task. For this research field to progress in a more meaningful and scientific fashion, there needs to be a method of measuring progress between different reconfigurable architectures as well as with respect to traditional computing technologies. This paper presents the Reconfigurable Architecture TEsting Suite, or RATES, which defines a standard for describing and using benchmarks for reconfigurable architectures. RATES is a set of functional benchmarks, is totally independent from the architecture and language, and usable on any processing platform be it general purpose or reconfigurable. It requires standard algorithms to allow comparisons amongst architectures but allows custom algorithms to highlight specific features. Although creating a standard set of benchmarks seems to be an obvious solution, there are many issues to be addressed if this is to be realized. The first is that there is no standard language for algorithmic abstraction that can be compiled to all reconfigurable processors. This is partially due to designers choosing different computational models for their processors, and thus adopting whatever language model enables a reasonable mapping to their architecture. RATES addresses this problem by abstracting the benchmark problem definition from the source code. A second problem arises from the fact that there aren’t any standard benchmarks for these processors. Thus researchers may choose different algorithms when trying to implement the same benchmark to ameliorate performance. However, if a researcher does not use the same algorithm to implement the same benchmark there is some question as to the usefulness of the results for interproject comparisons. Problems also arise from considering what types of tests should be included in a reconfigurable benchmark suite. Most projects use numerous kernels to test their reconfig