An efficient automated parameter tuning framework for spiking neural networks

As the desire for biologically realistic spiking neural networks (SNNs) increases, tuning the enormous number of open parameters in these models becomes a difficult challenge. SNNs have been used to successfully model complex neural circuits that explore various neural phenomena such as neural plasticity, vision systems, auditory systems, neural oscillations, and many other important topics of neural function. Additionally, SNNs are particularly well-adapted to run on neuromorphic hardware that will support biological brain-scale architectures. Although the inclusion of realistic plasticity equations, neural dynamics, and recurrent topologies has increased the descriptive power of SNNs, it has also made the task of tuning these biologically realistic SNNs difficult. To meet this challenge, we present an automated parameter tuning framework capable of tuning SNNs quickly and efficiently using evolutionary algorithms (EA) and inexpensive, readily accessible graphics processing units (GPUs). A sample SNN with 4104 neurons was tuned to give V1 simple cell-like tuning curve responses and produce self-organizing receptive fields (SORFs) when presented with a random sequence of counterphase sinusoidal grating stimuli. A performance analysis comparing the GPU-accelerated implementation to a single-threaded central processing unit (CPU) implementation was carried out and showed a speedup of 65× of the GPU implementation over the CPU implementation, or 0.35 h per generation for GPU vs. 23.5 h per generation for CPU. Additionally, the parameter value solutions found in the tuned SNN were studied and found to be stable and repeatable. The automated parameter tuning framework presented here will be of use to both the computational neuroscience and neuromorphic engineering communities, making the process of constructing and tuning large-scale SNNs much quicker and easier

Preventing premature convergence and proving the optimality in evolutionary algorithms

http://ea2013.inria.fr//proceedings.pdfInternational audienceEvolutionary Algorithms (EA) usually carry out an efficient exploration of the search-space, but get often trapped in local minima and do not prove the optimality of the solution. Interval-based techniques, on the other hand, yield a numerical proof of optimality of the solution. However, they may fail to converge within a reasonable time due to their inability to quickly compute a good approximation of the global minimum and their exponential complexity. The contribution of this paper is a hybrid algorithm called Charibde in which a particular EA, Differential Evolution, cooperates with a Branch and Bound algorithm endowed with interval propagation techniques. It prevents premature convergence toward local optima and outperforms both deterministic and stochastic existing approaches. We demonstrate its efficiency on a benchmark of highly multimodal problems, for which we provide previously unknown global minima and certification of optimality

Estudio preliminar de estrategias híbridas de cómputo CPU-GPU para acelerar algoritmos evolutivos

Debido a su gran capacidad para encontrar buenas soluciones en tiempos de ejecución razonables las metaheurísticas son ampliamente utilizadas para la resolución de problemas de optimización. Dentro de las metaheurísticas se destacan los Algoritmos Genéticos (GAs). Sin embargo, resolver problemas con instancias de gran tamaño puede ser difícil incluso para este tipo de estrategias. Por esta razón, la paralelización de metaheurísticas es una alternativa interesante para disminuir los tiempos de ejecución de estos algoritmos. En los últimos años, las GPUs han sufrido una evolución explosiva. Originalmente eran dispositivos diseñados para un único propósito específico, el procesamiento gráfico, pero en pocos años se transformaron en verdaderos multiprocesadores de memoria compartida. En base a esto, las GPUs se presentan como una plataforma poderosa para implementar algoritmos paralelos. En este reporte, presentamos un estudio preliminar de paralelización de un algoritmo genético simple incluyendo estrategias híbridas de cómputo CPU-GPU. La propuesta presentada se basa en el esquema de paralelismo de GAs Maestro-Esclavo. Se presentan los resultados obtenidos utilizando una GPU de bajo rango (NVidia 9800 GTX+), alcanzando valores de speedup de 9x

Wizualizacja zjawisk topnienia i sublimacji

Niniejsza monografia dotyczy wizualizacji zjawisk topnienia i sublimacji, które są przejściem fazowym z ciała stałego odpowiednio do cieczy i gazu. Modelem granicy miedzy dwoma fazami jest powierzchnia międzyfazowa, dlatego topnienie i sublimacja mogą być rozpatrywane jako przesuwanie powierzchni międzyfazowej z towarzysząca mu wymiana ciepła. Wizualizacja omawianych zjawisk wymaga omówienia różnych jej aspektów – od sposobu reprezentacji danych graficznych, przez algorytmy przetwarzania tych danych i ich optymalizacje, problemy renderingu czasu rzeczywistego, po metody weryfikacji jej wyników. Wymienione kwestie zostały zebrane w niniejszej ksiażce

Principled design of evolutionary learning sytems for large scale data mining

Currently, the data mining and machine learning fields are facing new challenges because of the amount of information that is collected and needs processing. Many sophisticated learning approaches cannot simply cope with large and complex domains, because of the unmanageable execution times or the loss of prediction and generality capacities that occurs when the domains become more complex. Therefore, to cope with the volumes of information of the current realworld problems there is a need to push forward the boundaries of sophisticated data mining techniques. This thesis is focused on improving the efficiency of Evolutionary Learning systems in large scale domains. Specifically the objective of this thesis is improving the efficiency of the Bioinformatic Hierarchical Evolutionary Learning (BioHEL) system, a system designed with the purpose of handling large domains. This is a classifier system that uses an Iterative Rule Learning approach to generate a set of rules one by one using consecutive Genetic Algorithms. This system have shown to be very competitive so far in large and complex domains. In particular, BioHEL has obtained very important results when solving protein structure prediction problems and has won related merits, such as being placed among the best algorithms for this purpose at the Critical Assessment of Techniques for Protein Structure Prediction (CASP) in 2008 and 2010, and winning the bronze medal at the HUMIES Awards for Human-competitive results in 2007. However, there is still a need to analyse this system in a principled way to determine how the current mechanisms work together to solve larger domains and determine the aspects of the system that can be improved towards this aim. To fulfil the objective of this thesis, the work is divided in two parts. In the first part of the thesis exhaustive experimentation was carried out to determine ways in which the system could be improved. From this exhaustive analysis three main weaknesses are pointed out: a) the problem-dependancy of parameters in BioHEL's fitness function, which results in having a system difficult to set up and which requires an extensive preliminary experimentation to determine the adequate values for these parameters; b) the execution time of the learning process, which at the moment does not use any parallelisation techniques and depends on the size of the training sets; and c) the lack of global supervision over the generated solutions which comes from the usage of the Iterative Rule Learning paradigm and produces larger rule sets in which there is no guarantee of minimality or maximal generality. The second part of the thesis is focused on tackling each one of the weaknesses abovementioned to have a system capable of handling larger domains. First a heuristic approach to set parameters within BioHEL's fitness function is developed. Second a new parallel evaluation process that runs on General Purpose Graphic Processing Units was developed. Finally, post-processing operators to tackle the generality and cardinality of the generated solutions are proposed. By means of these enhancements we managed to improve the BioHEL system to reduce both the learning and the preliminary experimentation time, increase the generality of the final solutions and make the system more accessible for end-users. Moreover, as the techniques discussed in this thesis can be easily extended to other Evolutionary Learning systems we consider them important additions to the research in this field towards tackling large scale domains