Mathematical properties of neuronal TD-rules and differential Hebbian learning: a comparison

A confusingly wide variety of temporally asymmetric learning rules exists related to reinforcement learning and/or to spike-timing dependent plasticity, many of which look exceedingly similar, while displaying strongly different behavior. These rules often find their use in control tasks, for example in robotics and for this rigorous convergence and numerical stability is required. The goal of this article is to review these rules and compare them to provide a better overview over their different properties. Two main classes will be discussed: temporal difference (TD) rules and correlation based (differential hebbian) rules and some transition cases. In general we will focus on neuronal implementations with changeable synaptic weights and a time-continuous representation of activity. In a machine learning (non-neuronal) context, for TD-learning a solid mathematical theory has existed since several years. This can partly be transfered to a neuronal framework, too. On the other hand, only now a more complete theory has also emerged for differential Hebb rules. In general rules differ by their convergence conditions and their numerical stability, which can lead to very undesirable behavior, when wanting to apply them. For TD, convergence can be enforced with a certain output condition assuring that the δ-error drops on average to zero (output control). Correlation based rules, on the other hand, converge when one input drops to zero (input control). Temporally asymmetric learning rules treat situations where incoming stimuli follow each other in time. Thus, it is necessary to remember the first stimulus to be able to relate it to the later occurring second one. To this end different types of so-called eligibility traces are being used by these two different types of rules. This aspect leads again to different properties of TD and differential Hebbian learning as discussed here. Thus, this paper, while also presenting several novel mathematical results, is mainly meant to provide a road map through the different neuronally emulated temporal asymmetrical learning rules and their behavior to provide some guidance for possible applications

Clinicopathological Profile and Surgical Treatment of Abdominal Tuberculosis: A Single Centre Experience in Northwestern Tanzania.

Abdominal tuberculosis continues to be a major public health problem worldwide and poses diagnostic and therapeutic challenges to general surgeons practicing in resource-limited countries. This study was conducted to describe the clinicopathological profile and outcome of surgical treatment of abdominal tuberculosis in our setting and compare with what is described in literature. A prospective descriptive study of patients who presented with abdominal tuberculosis was conducted at Bugando Medical Centre (BMC) in northwestern Tanzania from January 2006 to February 2012. Ethical approval to conduct the study was obtained from relevant authorities. Statistical data analysis was performed using SPSS version 17.0. Out of 256 patients enrolled in the study, males outnumbered females. The median age was 28 years (range = 16-68 years). The majority of patients (77.3%) had primary abdominal tuberculosis. A total of 127 (49.6%) patients presented with intestinal obstruction, 106 (41.4%) with peritonitis, 17 (6.6%) with abdominal masses and 6 (2.3%) patients with multiple fistulae in ano. Forty-eight (18.8%) patients were HIV positive. A total of 212 (82.8%) patients underwent surgical treatment for abdominal tuberculosis. Bands /adhesions (58.5%) were the most common operative findings. Ileo-caecal region was the most common bowel involved in 122 (57.5%) patients. Release of adhesions and bands was the most frequent surgical procedure performed in 58.5% of cases. Complication and mortality rates were 29.7% and 18.8% respectively. The overall median length of hospital stay was 32 days and was significantly longer in patients with complications (p < 0.001). Advanced age (age ≥ 65 years), co-morbid illness, late presentation, HIV positivity and CD4+ count < 200 cells/μl were statistically significantly associated with mortality (p < 0.0001). The follow up of patients were generally poor as only 37.5% of patients were available for follow up at twelve months after discharge. Abdominal tuberculosis constitutes a major public health problem in our environment and presents a diagnostic challenge requiring a high index of clinical suspicion. Early diagnosis, early anti-tuberculous therapy and surgical treatment of the associated complications are essential for survival

Enzymatic Shaving of the Tegument Surface of Live Schistosomes for Proteomic Analysis: A Rational Approach to Select Vaccine Candidates

Adult schistosome parasites can reside in the host bloodstream for decades surrounded by components of the immune system. It was originally proposed that their survival depended on the secretion of an inert bilayer, the membranocalyx, to protect the underlying plasma membrane from attack. We have investigated whether any proteins were exposed on the surface of live worms using incubation with selected hydrolases, in combination with mass spectrometry to identify released proteins. We show that a small number of parasite proteins are accessible to the enzymes and so could represent constituents of the membranocalyx. We also identified several proteins acquired by the parasite on contact with host cells. In addition, components of the cytolytic complement pathway were detected, but these appeared not to harm the worm, indicating that some of its own surface proteins could inhibit the lytic pathway. We suggest that, collectively, the ‘superficial’ parasite proteins may provide good candidates for a schistosome vaccine

An Imperfect Dopaminergic Error Signal Can Drive Temporal-Difference Learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards

A taxonomy for community-based care programs focused on HIV/AIDS prevention, treatment, and care in resource-poor settings

Community-based care (CBC) can increase access to key services for people affected by HIV/AIDS through the mobilization of community interests and resources and their integration with formal health structures. Yet, the lack of a systematic framework for analysis of CBC focused on HIV/AIDS impedes our ability to understand and study CBC programs. We sought to develop taxonomy of CBC programs focused on HIV/AIDS in resource-limited settings in an effort to understand their key characteristics, uncover any gaps in programming, and highlight the potential roles they play. Our review aimed to systematically identify key CBC programs focused on HIV/AIDS in resource-limited settings. We used both bibliographic database searches (Medline, CINAHL, and EMBASE) for peer-reviewed literature and internet-based searches for gray literature. Our search terms were &#x2018;HIV&#x2019; or &#x2018;AIDS&#x2019; and &#x2018;community-based care&#x2019; or &#x2018;CBC&#x2019;. Two co-authors developed a descriptive taxonomy through an iterative, inductive process using the retrieved program information. We identified 21 CBC programs useful for developing taxonomy. Extensive variation was observed within each of the nine categories identified: region, vision, characteristics of target populations, program scope, program operations, funding models, human resources, sustainability, and monitoring and evaluation strategies. While additional research may still be needed to identify the conditions that lead to overall program success, our findings can help to inform our understanding of the various aspects of CBC programs and inform potential logic models for CBC programming in the context of HIV/AIDS in resource-limited settings. Importantly, the findings of the present study can be used to develop sustainable HIV/AIDS-service delivery programs in regions with health resource shortages

Temporal-Difference Reinforcement Learning with Distributed Representations

Temporal-difference (TD) algorithms have been proposed as models of reinforcement learning (RL). We examine two issues of distributed representation in these TD algorithms: distributed representations of belief and distributed discounting factors. Distributed representation of belief allows the believed state of the world to distribute across sets of equivalent states. Distributed exponential discounting factors produce hyperbolic discounting in the behavior of the agent itself. We examine these issues in the context of a TD RL model in which state-belief is distributed over a set of exponentially-discounting “micro-Agents”, each of which has a separate discounting factor (γ). Each µAgent maintains an independent hypothesis about the state of the world, and a separate value-estimate of taking actions within that hypothesized state. The overall agent thus instantiates a flexible representation of an evolving world-state. As with other TD models, the value-error (δ) signal within the model matches dopamine signals recorded from animals in standard conditioning reward-paradigms. The distributed representation of belief provides an explanation for the decrease in dopamine at the conditioned stimulus seen in overtrained animals, for the differences between trace and delay conditioning, and for transient bursts of dopamine seen at movement initiation. Because each µAgent also includes its own exponential discounting factor, the overall agent shows hyperbolic discounting, consistent with behavioral experiments

The "Maggie" filament: Physical properties of a giant atomic cloud

The atomic phase of the interstellar medium plays a key role in the formation process of molecular clouds. Due to the line-of-sight confusion in the Galactic plane that is associated with its ubiquity, atomic hydrogen emission has been challenging to study. Employing the high-angular resolution data from the THOR survey, we identify one of the largest, coherent, mostly atomic HI filaments in the Milky Way at the line-of-sight velocities around -54 km/s. The giant atomic filament "Maggie", with a total length of 1.2 kpc, is not detected in most other tracers, and does not show signs of active star formation. At a kinematic distance of 17 kpc, Maggie is situated below (by 500 pc) but parallel to the Galactic HI disk and is trailing the predicted location of the Outer Arm by 5-10 km/s in longitude-velocity space. The centroid velocity exhibits a smooth gradient of less than $\pm$3 km/s /10 pc and a coherent structure to within $\pm$6 km/s. The line widths of 10 km/s along the spine of the filament are dominated by non-thermal effects. After correcting for optical depth effects, the mass of Maggie's dense spine is estimated to be $7.2\times10^5\,M_{\odot}$. The mean number density of the filament is 4$\rm\,cm^{-3}$, which is best explained by the filament being a mix of cold and warm neutral gas. In contrast to molecular filaments, the turbulent Mach number and velocity structure function suggest that Maggie is driven by transonic to moderately supersonic velocities that are likely associated with the Galactic potential rather than being subject to the effects of self-gravity or stellar feedback. The column density PDF displays a log-normal shape around a mean of $N_{\rm HI} = 4.8\times 10^{20}\rm\,cm^{-2}$, thus reflecting the absence of dominating effects of gravitational contraction

Glutaredoxin-1 Overexpression Enhances Neovascularization and Diminishes Ventricular Remodeling in Chronic Myocardial Infarction

Oxidative stress plays a critical role in the pathophysiology of cardiac failure, including the modulation of neovascularization following myocardial infarction (MI). Redox molecules thioredoxin (Trx) and glutaredoxin (Grx) superfamilies actively maintain intracellular thiol-redox homeostasis by scavenging reactive oxygen species. Among these two superfamilies, the pro-angiogenic function of Trx-1 has been reported in chronic MI model whereas similar role of Grx-1 remains uncertain. The present study attempts to establish the role of Grx-1 in neovascularization and ventricular remodeling following MI. Wild-type (WT) and Grx-1 transgenic (Grx-1Tg/+) mice were randomized into wild-type sham (WTS), Grx-1Tg/+ Sham (Grx-1Tg/+S), WTMI, Grx-1Tg/+MI. MI was induced by permanent occlusion of the LAD coronary artery. Sham groups underwent identical time-matched surgical procedures without LAD ligation. Significant increase in arteriolar density was observed 7 days (d) after surgical intervention in the Grx-1Tg/+MI group as compared to the WTMI animals. Further, improvement in myocardial functional parameters 30 d after MI was observed including decreased LVIDs, LVIDd, increased ejection fraction and, fractional shortening was also observed in the Grx-1Tg/+MI group as compared to the WTMI animals. Moreover, attenuation of oxidative stress and apoptotic cardiomyocytes was observed in the Grx-1Tg/+MI group as compared to the WTMI animals. Increased expression of p-Akt, VEGF, Ang-1, Bcl-2, survivin and DNA binding activity of NF-κB were observed in the Grx-1Tg/+MI group when compared to WTMI animals as revealed by Western blot analysis and Gel-shift analysis, respectively. These results are the first to demonstrate that Grx-1 induces angiogenesis and diminishes ventricular remodeling apparently through neovascularization mediated by Akt, VEGF, Ang-1 and NF-κB as well as Bcl-2 and survivin-mediated anti-apoptotic pathway in the infarcted myocardium

STDP Allows Fast Rate-Modulated Coding with Poisson-Like Spike Trains

Spike timing-dependent plasticity (STDP) has been shown to enable single neurons to detect repeatedly presented spatiotemporal spike patterns. This holds even when such patterns are embedded in equally dense random spiking activity, that is, in the absence of external reference times such as a stimulus onset. Here we demonstrate, both analytically and numerically, that STDP can also learn repeating rate-modulated patterns, which have received more experimental evidence, for example, through post-stimulus time histograms (PSTHs). Each input spike train is generated from a rate function using a stochastic sampling mechanism, chosen to be an inhomogeneous Poisson process here. Learning is feasible provided significant covarying rate modulations occur within the typical timescale of STDP (∼10–20 ms) for sufficiently many inputs (∼100 among 1000 in our simulations), a condition that is met by many experimental PSTHs. Repeated pattern presentations induce spike-time correlations that are captured by STDP. Despite imprecise input spike times and even variable spike counts, a single trained neuron robustly detects the pattern just a few milliseconds after its presentation. Therefore, temporal imprecision and Poisson-like firing variability are not an obstacle to fast temporal coding. STDP provides an appealing mechanism to learn such rate patterns, which, beyond sensory processing, may also be involved in many cognitive tasks