This paper describes a new approach for evolving recurrent  neural networks using Genetic Programming. A system has been developed  to train weightless neural networks using construction rules. The  network construction rules are evolved by the Genetic Programming system  which build the solution neural networks. The use of rules allows  networks to be constructed modularly. Experimentation with decomposable  Boolean functions has revealed that the performance of the system  is superior to a non-modular version of the system