Democratic population decisions result in robust policy-gradient learning: A parametric study with GPU simulations

High performance computing on the Graphics Processing Unit (GPU) is an emerging field driven by the promise of high computational power at a low cost. However, GPU programming is a non-trivial task and moreover architectural limitations raise the question of whether investing effort in this direction may be worthwhile. In this work, we use GPU programming to simulate a two-layer network of Integrate-and-Fire neurons with varying degrees of recurrent connectivity and investigate its ability to learn a simplified navigation task using a policy-gradient learning rule stemming from Reinforcement Learning. The purpose of this paper is twofold. First, we want to support the use of GPUs in the field of Computational Neuroscience. Second, using GPU computing power, we investigate the conditions under which the said architecture and learning rule demonstrate best performance. Our work indicates that networks featuring strong Mexican-Hat-shaped recurrent connections in the top layer, where decision making is governed by the formation of a stable activity bump in the neural population (a "non-democratic" mechanism), achieve mediocre learning results at best. In absence of recurrent connections, where all neurons "vote" independently ("democratic") for a decision via population vector readout, the task is generally learned better and more robustly. Our study would have been extremely difficult on a desktop computer without the use of GPU programming. We present the routines developed for this purpose and show that a speed improvement of 5x up to 42x is provided versus optimised Python code. The higher speed is achieved when we exploit the parallelism of the GPU in the search of learning parameters. This suggests that efficient GPU programming can significantly reduce the time needed for simulating networks of spiking neurons, particularly when multiple parameter configurations are investigated. © 2011 Richmond et al

HIV-1 Polymerase Inhibition by Nucleoside Analogs: Cellular- and Kinetic Parameters of Efficacy, Susceptibility and Resistance Selection

Nucleoside analogs (NAs) are used to treat numerous viral infections and cancer. They compete with endogenous nucleotides (dNTP/NTP) for incorporation into nascent DNA/RNA and inhibit replication by preventing subsequent primer extension. To date, an integrated mathematical model that could allow the analysis of their mechanism of action, of the various resistance mechanisms, and their effect on viral fitness is still lacking. We present the first mechanistic mathematical model of polymerase inhibition by NAs that takes into account the reversibility of polymerase inhibition. Analytical solutions for the model point out the cellular- and kinetic aspects of inhibition. Our model correctly predicts for HIV-1 that resistance against nucleoside analog reverse transcriptase inhibitors (NRTIs) can be conferred by decreasing their incorporation rate, increasing their excision rate, or decreasing their affinity for the polymerase enzyme. For all analyzed NRTIs and their combinations, model-predicted macroscopic parameters (efficacy, fitness and toxicity) were consistent with observations. NRTI efficacy was found to greatly vary between distinct target cells. Surprisingly, target cells with low dNTP/NTP levels may not confer hyper-susceptibility to inhibition, whereas cells with high dNTP/NTP contents are likely to confer natural resistance. Our model also allows quantification of the selective advantage of mutations by integrating their effects on viral fitness and drug susceptibility. For zidovudine triphosphate (AZT-TP), we predict that this selective advantage, as well as the minimal concentration required to select thymidine-associated mutations (TAMs) are highly cell-dependent. The developed model allows studying various resistance mechanisms, inherent fitness effects, selection forces and epistasis based on microscopic kinetic data. It can readily be embedded in extended models of the complete HIV-1 reverse transcription process, or analogous processes in other viruses and help to guide drug development and improve our understanding of the mechanisms of resistance development during treatment

Thermodynamics and relativistic kinetic theory for q-generalized Bose-Einstein and Fermi-Dirac systems

The thermodynamics and covariant kinetic theory have been elaborately investigated in a non-extensive environment considering the non-extensive generalization of Bose-Einstein (BE) and Fermi-Dirac (FD) statistics. Starting with Tsallis' entropy formula, the fundamental principles of thermostatistics have been established for a grand canonical system having q-generalized BE/FD degrees of freedom. The many particle kinetic theory has been set up in terms of the relativistic transport equation with q-generalized Uehling-Uhlenbeck collision term. The conservation laws have been realized in terms of appropriate moments of the transport equation. The thermodynamic quantities have been obtained in weak non-extensive environment for a massive pion-nucleon and a massless quark-gluon system with non-zero baryon chemical potential. In order to get an estimate of the impact of non-extensivity on the system dynamics, the q-modified Debye mass and hence the q-modified effective coupling have been estimated for a quark-gluon system.by Sukanya Mitr