publisher: Elsevier articletitle: A modular architecture for transparent computation in recurrent neural networks journaltitle: Neural Networks articlelink: http://dx.doi.org/10.1016/j.neunet.2016.09.001 content_type: article copyright: © 2016 Elsevier Ltd. All rights reserved

beim Graben, P

Carmantini, GS

Desroches, M

Rodrigues, S

Neural Networks

arXiv

Computation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simulation of a range of automata. We then show that the Gödelization of versatile shifts defines nonlinear dynamical automata, dynamical systems evolving on a vectorial space. Finally, we present a mapping between nonlinear dynamical automata and recurrent artificial neural networks. The mapping defines an architecture characterized by its granular modularity, where data, symbolic operations and their control are not only distinguishable in activation space, but also spatially localizable in the network itself, while maintaining a distributed encoding of symbolic representations. The resulting networks simulate automata in real-time and are programmed directly, in the absence of network training. To discuss the unique characteristics of the architecture and their consequences, we present two examples: (i) the design of a Central Pattern Generator from a finite-state locomotive controller, and (ii) the creation of a network simulating a system of interactive automata that supports the parsing of garden-path sentences as investigated in psycholinguistics experiments

PEARL (Univ. of Plymouth)

University of PlymouthPEARL https://pearl.plymouth.ac.uk01 University of Plymouth Research Outputs University of Plymouth Research Outputs2017-01-01A modular architecture for transparentcomputation in recurrent neuralnetworksCarmantini, GShttp://hdl.handle.net/10026.1/671010.1016/j.neunet.2016.09.001Neural NetworksElsevier BVAll content in PEARL is protected by copyright law. Author manuscripts are made available in accordance withpublisher policies. Please cite only the published version using the details provided on the item record ordocument. In the absence of an open licence (e.g. Creative Commons), permissions for further reuse of contentshould be sought from the publisher or author.Version 1.1 | 17.02.2016  UNIVERSITY OF PLYMOUTH  DEPOSIT LICENCE AGREEMENT FOR RESEARCH     PEARL supports the long-term preservation of research outputs, and open access to them. To undertake this role, the administrators of PEARL need permission from authors to store, copy and migrate the material in order to ensure that it can be preserved and accessed  in the future.  This Deposit License Agreement is designed to give PEARL’s administrators the right to do this, and to confirm that the author has the right to submit the material to the repository. The Licence Agreement is non-exclusive, and the depositor does not give away any of their rights to the repository.  Depositor’s declaration You (the Author) agree to deposit an electronic copy of your research (the Work) in the University of Plymouth (the University) research repository (PEARL). You agree that: 1. By accepting and submitting this license, the Author grants the University the non-exclusive rights to reproduce, translate, migrate (as defined below), duplicate for back up processes and/or distribute the Work, including the abstract, worldwide in print and electronic format and in any medium, including but not limited to audio or video.  2. The Author retains all rights in the Work in its present version or future versions. The research repository administrators or their agents may, without changing content, digitise and migrate the Work to any medium or format for the purpose of future preservation and accessibility. The University may keep more than one copy of this submission for purposes of security, back-up and preservation. 3. The Work deposited in PEARL will be accessible to individuals and institutions via the internet subject to the terms and conditions granted in this agreement to the University and to the user of the research. Through the medium of the internet, files will also be available to automated agents, and may be searched and copied by text mining and plagiarism detection software. 4. You are the Author and owner of the copyright in the Work and/or you have the authority of the author and owner of the copyright in the Work to make this agreement and grant the University a licence to make available the Work in digitised format through PEARL for the purposes of non-commercial research, private study, criticism, review and news reporting, illustration for teaching, and/or other educational purposes in electronic or print form  Version 1.1 | 17.02.2016  5. If your research does include any substantial subsidiary material for which you do not hold copyright, you have sought and obtained permission to make this third party material available to the public in digital format via a stand-alone device or a communications network and that this permission encompasses the rights that you have granted to the University. Any such third-party owned material should be clearly identified and acknowledged within the text or content of the submission   6. You have exercised reasonable care to ensure that the Work is original, and does not to the best of your knowledge break any UK law, infringe any third party's copyright or other Intellectual Property Right, or contain any confidential material.   7. If the Work is based upon research that has been sponsored or supported by an agency or organisation other than the University, or is covered by any publisher’s agreement, you represent that you have fulfilled any right of review or other obligations required by such contract or agreement. The University shall retain the right to remove the Work for professional or administrative reasons, or if it is found to violate the legal rights of any person. The University will notify the depositor and the item will be removed from public view until the position is clarified. 8. The University does not have any obligation to take legal action on behalf of yourself, or other rights holders, in the event of infringement of intellectual property rights, breach of contract or of any other right, in the work. PEARL’s rights and responsibilities: 1. PEARL will clearly identify your name(s) as the author(s) or owner(s) of the submission, and will not make any alteration, other than as allowed by this license, to your submission. 2. PEARL undertakes not to distribute or translate this work for any commercial gain. The licence is required to allow deposit into the Plymouth Electronic Archive and Research Library and its successors. This licence may be passed to third party institutions (such as the British Library) if national or international policy advocates or requires a single repository of materials. Under no other condition will the University transfer the paper to others. 3. PEARL will ensure that the relevant metadata harvesting tools have access to the citation (including abstract) of the work to allow greater information discovery.  

English

A modular architecture for transparent computation in recurrent neural networks

Giovanni S. Carmantini

Peter beim Graben

Mathieu Desroches

Serafim Rodrigues

Crossref

International audienceComputation is classically studied in terms of automata, formal languages and algorithms; yet, the relation between neural dynamics and symbolic representations and operations is still unclear in traditional eliminative connectionism. Therefore, we suggest a unique perspective on this central issue, to which we would like to refer as to transparent connectionism, by proposing accounts of how symbolic computation can be implemented in neural substrates. In this study we first introduce a new model of dynamics on a symbolic space, the versatile shift, showing that it supports the real-time simulation of a range of automata. We then show that the Gödelization of versatile shifts defines nonlinear dynamical automata, dynamical systems evolving on a vectorial space. Finally, we present a mapping between nonlinear dynamical automata and recurrent artificial neural networks. The mapping defines an architecture characterized by its granular modularity, where data, symbolic operations and their control are not only distinguishable in activation space, but also spatially localizable in the network itself, while maintaining a distributed encoding of symbolic representations. The resulting networks simulate automata in real-time and are programmed directly, in absence of network training. To discuss the unique characteristics of the architecture and their consequences, we present two examples: i) the design of a Central Pattern Generator from a finite-state locomotive controller, and ii) the creation of a network simulating a system of interactive automata that supports the parsing of garden-path sentences as investigated in psycholinguistics experiments

Carmantini, Giovanni

Beim Graben, Peter

Desroches, Mathieu

Rodrigues, Serafim

HAL: Hyper Article en Ligne

INRIA a CCSD electronic archive server

https://pearl.plymouth.ac.uk/cgi/viewcontent.cgi?filename=0&article=2864&context=secam-research&type=additional

A modular architecture for transparent computation in recurrent neural networks

Abstract

Similar works

Full text

Available Versions

PEARL (Univ. of Plymouth)

Crossref

HAL: Hyper Article en Ligne

INRIA a CCSD electronic archive server