A polynomial training algorithm for calculating perceptrons of optimal stability

Recomi (REpeated COrrelation Matrix Inversion) is a polynomially fast algorithm for searching optimally stable solutions of the perceptron learning problem. For random unbiased and biased patterns it is shown that the algorithm is able to find optimal solutions, if any exist, in at worst O(N^4) floating point operations. Even beyond the critical storage capacity alpha_c the algorithm is able to find locally stable solutions (with negative stability) at the same speed. There are no divergent time scales in the learning process. A full proof of convergence cannot yet be given, only major constituents of a proof are shown.Comment: 11 pages, Latex, 4 EPS figure

Neural networks for non-linear adaptive filtering

Neural networks are shown to be a class of non-linear adaptive filters, which can be trained permanently with a possibly infinite number of timeordered examples ; this is an altogether différent framework from the usual, non-adaptive training of neural networks . A family of new gradientbased algorithms is proposed.Nous introduisons une famille d'algorithmes adaptatifs permettant l'utilisation de réseaux de neurones comme filtres adaptatifs non linéaires, systèmes susceptibles de subir un apprentissage permanent à partir d'un nombre éventuellement infini d'exemples présentés dans un ordre déterminé. Ces algorithmes, fondés sur des techniques d'évaluation du gradient d'une fonction de coût, s'inscrivent dans un cadre différent de celui de l'apprentissage classique des réseaux de neurones, qui est habituellement non adaptati

Derivation of Hebb's rule

On the basis of the general form for the energy needed to adapt the connection strengths of a network in which learning takes place, a local learning rule is found for the changes of the weights. This biologically realizable learning rule turns out to comply with Hebb's neuro-physiological postulate, but is not of the form of any of the learning rules proposed in the literature. It is shown that, if a finite set of the same patterns is presented over and over again to the network, the weights of the synapses converge to finite values. Furthermore, it is proved that the final values found in this biologically realizable limit are the same as those found via a mathematical approach to the problem of finding the weights of a partially connected neural network that can store a collection of patterns. The mathematical solution is obtained via a modified version of the so-called method of the pseudo-inverse, and has the inverse of a reduced correlation matrix, rather than the usual correlation matrix, as its basic ingredient. Thus, a biological network might realize the final results of the mathematician by the energetically economic rule for the adaption of the synapses found in this article.Comment: 29 pages, LaTeX, 3 figure

The cerebellar transcriptome during postnatal development of the Ts1Cje mouse, a segmental trisomy model for Down syndrome

The central nervous system of persons with Down syndrome presents cytoarchitectural abnormalities that likely result from gene-dosage effects affecting the expression of key developmental genes. To test this hypothesis, we have investigated the transcriptome of the cerebellum of the Ts1Cje mouse model of Down syndrome during postnatal development using microarrays and quantitative PCR (qPCR). Genes present in three copies were consistently overexpressed, with a mean ratio relative to euploid of 1.52 as determined by qPCR. Out of 63 three-copy genes tested, only five, nine and seven genes had ratios >2 or <1.2 at postnatal days 0 (P0), P15 and P30, respectively. This gene-dosage effect was associated with a dysregulation of the expression of some two-copy genes. Out of 8258 genes examined, the Ts1Cje/euploid ratios differed significantly from 1.0 for 406 (80 and 154 with ratios above 1.5 and below 0.7, respectively), 333 (11 above 1.5 and 55 below 0.7) and 246 genes (59 above 1.5 and 69 below 0.7) at P0, P15 and P30, respectively. Among the two-copy genes differentially expressed in the trisomic cerebellum, six homeobox genes, two belonging to the Notch pathway, were severely repressed. Overall, at P0, transcripts involved in cell differentiation and development were over-represented among the dysregulated genes, suggesting that cell differentiation and migration might be more altered than cell proliferation. Finally, global gene profiling revealed that transcription in Ts1Cje mice is more affected by the developmental changes than by the trisomic state, and that there is no apparent detectable delay in the postnatal development of the cerebellum of Ts1Cje mic

Phylostratigraphic tracking of cancer genes suggests a link to the emergence of multicellularity in metazoa

Background: Phylostratigraphy is a method used to correlate the evolutionary origin of founder genes (that is, functional founder protein domains) of gene families with particular macroevolutionary transitions. It is based on a model of genome evolution that suggests that the origin of complex phenotypic innovations will be accompanied by the emergence of such founder genes, the descendants of which can still be traced in extant organisms. The origin of multicellularity can be considered to be a macroevolutionary transition, for which new gene functions would have been required. Cancer should be tightly connected to multicellular life since it can be viewed as a malfunction of interaction between cells in a multicellular organism. A phylostratigraphic tracking of the origin of cancer genes should, therefore, also provide insights into the origin of multicellularity. Results: We find two strong peaks of the emergence of cancer related protein domains, one at the time of the origin of the first cell and the other around the time of the evolution of the multicellular metazoan organisms. These peaks correlate with two major classes of cancer genes, the 'caretakers', which are involved in general functions that support genome stability and the 'gatekeepers', which are involved in cellular signalling and growth processes. Interestingly, this phylogenetic succession mirrors the ontogenetic succession of tumour progression, where mutations in caretakers are thought to precede mutations in gatekeepers. Conclusions: A link between multicellularity and formation of cancer has often been predicted. However, this has not so far been explicitly tested. Although we find that a significant number of protein domains involved in cancer predate the origin of multicellularity, the second peak of cancer protein domain emergence is, indeed, connected to a phylogenetic level where multicellular animals have emerged. The fact that we can find a strong and consistent signal for this second peak in the phylostratigraphic map implies that a complex multi-level selection process has driven the transition to multicellularity

Specification And Implementation Of A Digital Hopfield-Type Associative Memory With On-Chip Training

This paper addresses the definition of the requirements for the design of a neural network associative memory, with on-chip training, in standard digital CMOS technology. We investigate various learning rules which are integrable in silicon, and we study the associative memory properties of the resulting networks. We also investigate the relationships between the architecture of the circuit and the learning rule, in order to minimize the extra circuitry required for the implementation of training. We describe a 64neuron associative memory with on-chip training, which has been manufactured, and we outline its future extensions. Beyond the application to the specific circuit described in the paper, the general methodology for determining the accuracy requirements can be applied to other circuits and to other auto-associative memory architectures. 1. INTRODUCTION The present paper describes the specification and the silicon integration of a Hopfield neural network designed (i) to operate ..