Towards Transparent Parallelization of Connectionist Systems

Abstract

Much work has been done in the area of parallel simulation of connectionist systems. However, usually parallel implementation issues for artificial neural networks have been discussed in general terms, but the actual parallel programs implement specific network models and are written in programming languages like C or C++. This paper deals with the transparent parallelization of neural networks. The goal is to automatically derive parallel code for MIMD and SPMD architectures from abstract descriptions of networks. In this, unit parallelism and training set parallelism are discussed. First, an outline of the abstract neural network description language CONNECT is given. The language combines procedural, functional, and object--oriented paradigms and allows for readable and, at the same time, complete definitions of connectionist systems. Currently, C++ code can be generated from CONNECT specifications. The code generation process is explained, and it is shown how unit parallelism can b..