Techniques of replica symmetry breaking and the storage problem of the McCulloch-Pitts neuron

In this article the framework for Parisi's spontaneous replica symmetry breaking is reviewed, and subsequently applied to the example of the statistical mechanical description of the storage properties of a McCulloch-Pitts neuron. The technical details are reviewed extensively, with regard to the wide range of systems where the method may be applied. Parisi's partial differential equation and related differential equations are discussed, and a Green function technique introduced for the calculation of replica averages, the key to determining the averages of physical quantities. The ensuing graph rules involve only tree graphs, as appropriate for a mean-field-like model. The lowest order Ward-Takahashi identity is recovered analytically and is shown to lead to the Goldstone modes in continuous replica symmetry breaking phases. The need for a replica symmetry breaking theory in the storage problem of the neuron has arisen due to the thermodynamical instability of formerly given solutions. Variational forms for the neuron's free energy are derived in terms of the order parameter function x(q), for different prior distribution of synapses. Analytically in the high temperature limit and numerically in generic cases various phases are identified, among them one similar to the Parisi phase in the Sherrington-Kirkpatrick model. Extensive quantities like the error per pattern change slightly with respect to the known unstable solutions, but there is a significant difference in the distribution of non-extensive quantities like the synaptic overlaps and the pattern storage stability parameter. A simulation result is also reviewed and compared to the prediction of the theory.Comment: 103 Latex pages (with REVTeX 3.0), including 15 figures (ps, epsi, eepic), accepted for Physics Report

Techniques of replica symmetry breaking and the storage problem of the McCulloch-Pitts neuron

In this article the framework for Parisi's spontaneous replica symmetry breaking is reviewed, and subsequently applied to the example of the statistical mechanical description of the storage properties of a McCulloch-Pitts neuron. The technical details are reviewed extensively, with regard to the wide range of systems where the method may be applied. Parisi's partial differential equation and related differential equations are discussed, and a Green function technique introduced for the calculation of replica averages, the key to determining the averages of physical quantities. The ensuing graph rules involve only tree graphs, as appropriate for a mean-field-like model. The lowest order Ward-Takahashi identity is recovered analytically and is shown to lead to the Goldstone modes in continuous replica symmetry breaking phases. The need for a replica symmetry breaking theory in the storage problem of the neuron has arisen due to the thermodynamical instability of formerly given solutions. Variational forms for the neuron's free energy are derived in terms of the order parameter function x(q), for different prior distribution of synapses. Analytically in the high temperature limit and numerically in generic cases various phases are identified, among them one similar to the Parisi phase in the Sherrington-Kirkpatrick model. Extensive quantities like the error per pattern change slightly with respect to the known unstable solutions, but there is a significant difference in the distribution of non-extensive quantities like the synaptic overlaps and the pattern storage stability parameter. A simulation result is also reviewed and compared to the prediction of the theory.Comment: 103 Latex pages (with REVTeX 3.0), including 15 figures (ps, epsi, eepic), accepted for Physics Report

Neural network based architectures for aerospace applications

A brief history of the field of neural networks research is given and some simple concepts are described. In addition, some neural network based avionics research and development programs are reviewed. The need for the United States Air Force and NASA to assume a leadership role in supporting this technology is stressed

Identification of genetic drivers of colorectal cancer via bioinformatics and machine learning

Machine learning methods have been widely used in a range of areas within genetics and genomics, it is maybe one of the most useful tools for the interpretation of large genomic data sets and has been used to annotate and analyse a wide variety of genomic sequence elements due to its ability to analyze and learn how to extract data insights from large heterogeneous data sets. In this work, we mainly focus on identifying gene markers that are associated with an increased risk of colorectal cancer (CRC) one of the most common cancers worldwide, showing the highest mortality. In this research, we look into feature selection methods based on variant relevancy toward the development of hereditary diseases. With this approach, we aim to find rel- evant frequently occurring variants and also rare variant occurrences, this way we will identify potentially valuable disease biomarkers. We analysed 8339 different variants and determined 765 to be relevant to CRC. We will also use feature clustering methods for the identification of co-occurrence between certain genetic variants, this will allow us to identify genetic links and non-co-occurring variants that are both rare and associated with an increased risk of development of CRC. Using this method we can determine differ- ent co-occurring variant groups with an additional one being composed of independent variants. We expect the identification of these gene markers to allow for better clinical manage- ment of the patients, namely due to the identification of genetic predispositions to CRC that will allow for a better risk assessment of patients and change the type of exams to be performed and their frequency, which will have a strong impact not only on their clinical screening but also on that of their family members, this can allow for early identification of tumours or even benign lesions, therefore contributing to CRC prevention. We believe that this study will contribute to the overall understanding of CRC causes and will further advance the study of its prevention. We also expect to give insights on how to identify the biological mechanisms underlying gene variant occurrences for not only CRC but also other hereditary cancer syndromes.Métodos de aprendizagem automática têm sido amplamente utilizados em diversas áreas dentro da genética e genômica. A aprendizagem automática é talvez uma das ferramentas mais úteis para a interpretação de grandes conjuntos de dados genômicos e tem sido usado para anotar e analisar uma ampla variedade de elementos de sequências genô- micas. A sua capacidade para analisar e aprender a extraindo informação de grandes conjuntos de dados heterogéneos. Vamos nos concentrar principalmente na identificação de marcadores genéticos que estão associados a um risco aumentado de cancro colo-retal (CCR), um dos cancros mais comuns em todo o mundo, apresentando uma das maiores mortalidades. Neste estudo, analisamos os métodos de feature selection com base na relevância da variante genética para o desenvolvimento de CCR. Com estes métodos, pretendemos en- contrar variantes relevantes que ocorrem com frequência e também variantes raras, desta forma identificaremos biomarcadores potencialmente valiosos. Analisamos 8339 varian- tes diferentes e determinamos que 765 são relevantes para o desenvolvimento de CCR. Também usaremos métodos de clustering de variantes genéticas para a identificação de correlação entre certas variantes genéticas, o que nos permitirá identificar ligações genéti- cas e ocorrências de variantes independentes que estão associadas a um risco aumentado de desenvolvimento de CCR. Usando esse método, determinamos que há 4 diferentes gru- pos de variantes relevantes, sendo um adicional composto por variantes independentes. Esperamos que a identificação destes marcadores genéticos permita uma melhor ges- tão clínica dos doentes, nomeadamente devido à identificação de predisposições genéticas para CCR que permitirão uma melhor avaliação do risco dos doentes e alterar o tipo de exames a serem realizados e a sua frequência, que terá forte impacto não só na sua triagem clínica, mas também na dos seus familiares, isto pode permitir a identificação precoce de tumores ou mesmo lesões benignas, contribuindo assim para a prevenção de CCR. Acreditamos que este estudo contribuirá para a compreensão geral das causas CCR e avançará o estudo da sua prevenção. Também esperamos fornecer métodos de como identificar os mecanismos biológicos subjacentes às ocorrências de variantes genéticas não apenas para CCR, mas também para outras síndromes de câncer hereditário

Heuristic pattern correction scheme using adaptively trained generalized regression neural networks

In many pattern classification problems, an intelligent neural system is required which can learn the newly encountered but misclassified patterns incrementally, while keeping a good classification performance over the past patterns stored in the network. In the paper, an heuristic pattern correction scheme is proposed using adaptively trained generalized regression neural networks (GRNNs). The scheme is based upon both network growing and dual-stage shrinking mechanisms. In the network growing phase, a subset of the misclassified patterns in each incoming data set is iteratively added into the network until all the patterns in the incoming data set are classified correctly. Then, the redundancy in the growing phase is removed in the dual-stage network shrinking. Both long- and short-term memory models are considered in the network shrinking, which are motivated from biological study of the brain. The learning capability of the proposed scheme is investigated through extensive simulation studie