Chess Endgames and Neural Networks

The existence of endgame databases challenges us to extract higher-grade information and knowledge from their basic data content. Chess players, for example, would like simple and usable endgame theories if such holy grail exists: endgame experts would like to provide such insights and be inspired by computers to do so. Here, we investigate the use of artificial neural networks (NNs) to mine these databases and we report on a first use of NNs on KPK. The results encourage us to suggest further work on chess applications of neural networks and other data-mining techniques

Cell division and migration in a 'genotype' for neural networks

Much research has been dedicated recently to applying genetic algorithms to populations of neural networks. However, while in real organisms the inherited genotype maps in complex ways into the resulting phenotype, in most of this research the development process that creates the individual phenotype is ignored. In this paper we present a model of neural development which includes cell division and cell migration in addition to axonal growth and branching. This reflects, in a very simplified way, what happens in the ontogeny of real organisms. The development process of our artificial organisms shows successive phases of functional differentiation and specialization. In addition, we find that mutations that affect different phases of development have very different evolutionary consequences. A single change in the early stages of cell division/migration can have huge effects on the phenotype while changes in later stages have usually a less drammatic impact. Sometimes changes that affect the first developental stages may be retained producing sudden changes in evolutionary history

Precis of neuroconstructivism: how the brain constructs cognition

Neuroconstructivism: How the Brain Constructs Cognition proposes a unifying framework for the study of cognitive development that brings together (1) constructivism (which views development as the progressive elaboration of increasingly complex structures), (2) cognitive neuroscience (which aims to understand the neural mechanisms underlying behavior), and (3) computational modeling (which proposes formal and explicit specifications of information processing). The guiding principle of our approach is context dependence, within and (in contrast to Marr [1982]) between levels of organization. We propose that three mechanisms guide the emergence of representations: competition, cooperation, and chronotopy; which themselves allow for two central processes: proactivity and progressive specialization. We suggest that the main outcome of development is partial representations, distributed across distinct functional circuits. This framework is derived by examining development at the level of single neurons, brain systems, and whole organisms. We use the terms encellment, embrainment, and embodiment to describe the higher-level contextual influences that act at each of these levels of organization. To illustrate these mechanisms in operation we provide case studies in early visual perception, infant habituation, phonological development, and object representations in infancy. Three further case studies are concerned with interactions between levels of explanation: social development, atypical development and within that, developmental dyslexia. We conclude that cognitive development arises from a dynamic, contextual change in embodied neural structures leading to partial representations across multiple brain regions and timescales, in response to proactively specified physical and social environment

Artificial neural networks for diagnosis and survival prediction in colon cancer

ANNs are nonlinear regression computational devices that have been used for over 45 years in classification and survival prediction in several biomedical systems, including colon cancer. Described in this article is the theory behind the three-layer free forward artificial neural networks with backpropagation error, which is widely used in biomedical fields, and a methodological approach to its application for cancer research, as exemplified by colon cancer. Review of the literature shows that applications of these networks have improved the accuracy of colon cancer classification and survival prediction when compared to other statistical or clinicopathological methods. Accuracy, however, must be exercised when designing, using and publishing biomedical results employing machine-learning devices such as ANNs in worldwide literature in order to enhance confidence in the quality and reliability of reported data

Deep Learning Techniques for Music Generation -- A Survey

This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content. We propose a methodology based on five dimensions for our analysis: Objective - What musical content is to be generated? Examples are: melody, polyphony, accompaniment or counterpoint. - For what destination and for what use? To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file). Representation - What are the concepts to be manipulated? Examples are: waveform, spectrogram, note, chord, meter and beat. - What format is to be used? Examples are: MIDI, piano roll or text. - How will the representation be encoded? Examples are: scalar, one-hot or many-hot. Architecture - What type(s) of deep neural network is (are) to be used? Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks. Challenge - What are the limitations and open challenges? Examples are: variability, interactivity and creativity. Strategy - How do we model and control the process of generation? Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation. For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described and are used to exemplify the various choices of objective, representation, architecture, challenge and strategy. The last section includes some discussion and some prospects.Comment: 209 pages. This paper is a simplified version of the book: J.-P. Briot, G. Hadjeres and F.-D. Pachet, Deep Learning Techniques for Music Generation, Computational Synthesis and Creative Systems, Springer, 201

A Review of Formal Methods applied to Machine Learning

We review state-of-the-art formal methods applied to the emerging field of the verification of machine learning systems. Formal methods can provide rigorous correctness guarantees on hardware and software systems. Thanks to the availability of mature tools, their use is well established in the industry, and in particular to check safety-critical applications as they undergo a stringent certification process. As machine learning is becoming more popular, machine-learned components are now considered for inclusion in critical systems. This raises the question of their safety and their verification. Yet, established formal methods are limited to classic, i.e. non machine-learned software. Applying formal methods to verify systems that include machine learning has only been considered recently and poses novel challenges in soundness, precision, and scalability. We first recall established formal methods and their current use in an exemplar safety-critical field, avionic software, with a focus on abstract interpretation based techniques as they provide a high level of scalability. This provides a golden standard and sets high expectations for machine learning verification. We then provide a comprehensive and detailed review of the formal methods developed so far for machine learning, highlighting their strengths and limitations. The large majority of them verify trained neural networks and employ either SMT, optimization, or abstract interpretation techniques. We also discuss methods for support vector machines and decision tree ensembles, as well as methods targeting training and data preparation, which are critical but often neglected aspects of machine learning. Finally, we offer perspectives for future research directions towards the formal verification of machine learning systems

Modification of an asynchronous dexterous hand movement decoder for hardware implementation

One of the goals of modern prosthetics research is to provide natural, neurologically driven control of a prosthetic device, preferably in a portable format. Previously, an algorithm for asynchronously decoding individuated finger and wrist movements from recordings of neural activity in the primary motor cortex was developed by Aggarwal et al. and implemented in software. The first objective of this work was to determine what effect simplifying Aggarwal\u27s algorithm by using linear Artificial neural networks instead of nonlinear ones would have on movement detection and classification accuracy. The simplified algorithm developed in this work was demonstrated to achieve movement detection and classification accuracies of 99.7%, 89.9%, and 95.3% for an individuated movement decoding task across three subjects using neural recordings from 80 neurons. In comparison, the original algorithm demonstrated accuracies of 96.2%, 90.5%, and 99.8% for the same task and subjects using neural recordings from 40 neurons. Additionally, the simplified algorithm was demonstrated to have a detection and classification accuracy of 80.5% for a combined movement task, whereas the original algorithm achieved accuracy of 92.5%. Even though a greater input space size was required for the linear decoder, the computational intensity was reduced with a mean accuracy loss in the individuated movement task of only 0.53%. However, the 12% loss of accuracy observed in the combined movement task is considered unacceptable and suggests that the simplified algorithm is not appropriate for this task. The second objective of this work was to create a digital hardware implementation of the simplified linear artificial neural network version of Aggarwal\u27s algorithm. A scalable, fully parallel architecture was designed for implementation on a Xilinx Virtex-4 FX60 FPGA. This implementation could be realized with an input dimension of up to 60 neurons on this FPGA, although computations were performed on the order of 104 times faster than was necessary for realtime operation, indicating that there is an opportunity to reduce hardware size in exchange for computational speed. This work is an important exploration into the eventual goal of incorporating a hardware movement decoder in a prosthetic device and demonstrates that a hardware implementation is feasible using currently available technology