2,592 research outputs found

    Mathematical properties of neuronal TD-rules and differential Hebbian learning: a comparison

    Get PDF
    A confusingly wide variety of temporally asymmetric learning rules exists related to reinforcement learning and/or to spike-timing dependent plasticity, many of which look exceedingly similar, while displaying strongly different behavior. These rules often find their use in control tasks, for example in robotics and for this rigorous convergence and numerical stability is required. The goal of this article is to review these rules and compare them to provide a better overview over their different properties. Two main classes will be discussed: temporal difference (TD) rules and correlation based (differential hebbian) rules and some transition cases. In general we will focus on neuronal implementations with changeable synaptic weights and a time-continuous representation of activity. In a machine learning (non-neuronal) context, for TD-learning a solid mathematical theory has existed since several years. This can partly be transfered to a neuronal framework, too. On the other hand, only now a more complete theory has also emerged for differential Hebb rules. In general rules differ by their convergence conditions and their numerical stability, which can lead to very undesirable behavior, when wanting to apply them. For TD, convergence can be enforced with a certain output condition assuring that the ÎŽ-error drops on average to zero (output control). Correlation based rules, on the other hand, converge when one input drops to zero (input control). Temporally asymmetric learning rules treat situations where incoming stimuli follow each other in time. Thus, it is necessary to remember the first stimulus to be able to relate it to the later occurring second one. To this end different types of so-called eligibility traces are being used by these two different types of rules. This aspect leads again to different properties of TD and differential Hebbian learning as discussed here. Thus, this paper, while also presenting several novel mathematical results, is mainly meant to provide a road map through the different neuronally emulated temporal asymmetrical learning rules and their behavior to provide some guidance for possible applications

    Alkaline air: changing perspectives on nitrogen and air pollution in an ammonia-rich world

    Get PDF
    Ammonia and ammonium have received less attention than other forms of air pollution, with limited progress in controlling emissions at UK, European and global scales. By contrast, these compounds have been of significant past interest to science and society, the recollection of which can inform future strategies. Sal ammoniac (nĆ«shādir, nao sha) is found to have been extremely valuable in long-distance trade (ca AD 600–1150) from Egypt and China, where 6–8 kg N could purchase a human life, while air pollution associated with nĆ«shādir collection was attributed to this nitrogen form. Ammonia was one of the keys to alchemy—seen as an early experimental mesocosm to understand the world—and later became of interest as ‘alkaline air’ within the eighteenth century development of pneumatic chemistry. The same economic, chemical and environmental properties are found to make ammonia and ammonium of huge relevance today. Successful control of acidifying SO2 and NOx emissions leaves atmospheric NH3 in excess in many areas, contributing to particulate matter (PM2.5) formation, while leading to a new significance of alkaline air, with adverse impacts on natural ecosystems. Investigations of epiphytic lichens and bog ecosystems show how the alkalinity effect of NH3 may explain its having three to five times the adverse effect of ammonium and nitrate, respectively. It is concluded that future air pollution policy should no longer neglect ammonia. Progress is likely to be mobilized by emphasizing the lost economic value of global N emissions ($200 billion yr−1), as part of developing the circular economy for sustainable nitrogen management

    Single Gene Deletions of Orexin, Leptin, Neuropeptide Y, and Ghrelin Do Not Appreciably Alter Food Anticipatory Activity in Mice

    Get PDF
    Timing activity to match resource availability is a widely conserved ability in nature. Scheduled feeding of a limited amount of food induces increased activity prior to feeding time in animals as diverse as fish and rodents. Typically, food anticipatory activity (FAA) involves temporally restricting unlimited food access (RF) to several hours in the middle of the light cycle, which is a time of day when rodents are not normally active. We compared this model to calorie restriction (CR), giving the mice 60% of their normal daily calorie intake at the same time each day. Measurement of body temperature and home cage behaviors suggests that the RF and CR models are very similar but CR has the advantage of a clearly defined food intake and more stable mean body temperature. Using the CR model, we then attempted to verify the published result that orexin deletion diminishes food anticipatory activity (FAA) but observed little to no diminution in the response to CR and, surprisingly, that orexin KO mice are refractory to body weight loss on a CR diet. Next we tested the orexigenic neuropeptide Y (NPY) and ghrelin and the anorexigenic hormone, leptin, using mouse mutants. NPY deletion did not alter the behavior or physiological response to CR. Leptin deletion impaired FAA in terms of some activity measures, such as walking and rearing, but did not substantially diminish hanging behavior preceding feeding time, suggesting that leptin knockout mice do anticipate daily meal time but do not manifest the full spectrum of activities that typify FAA. Ghrelin knockout mice do not have impaired FAA on a CR diet. Collectively, these results suggest that the individual hormones and neuropepetides tested do not regulate FAA by acting individually but this does not rule out the possibility of their concerted action in mediating FAA

    Comparison of breast and bowel cancer screening uptake patterns in a common cohort of South Asian women in England

    Get PDF
    Background: Inequalities in uptake of cancer screening by ethnic minority populations are well documented in a number of international studies. However, most studies to date have explored screening uptake for a single cancer only. This paper compares breast and bowel cancer screening uptake for a cohort of South Asian women invited to undertake both, and similarly investigates these women's breast cancer screening behaviour over a period of fifteen years. Methods: Screening data for rounds 1, 2 and 5 (1989-2004) of the NHS breast cancer screening programme and for round 1 of the NHS bowel screening pilot (2000-2002) were obtained for women aged 50-69 resident in the English bowel screening pilot site, Coventry and Warwickshire, who had been invited to undertake breast and bowel cancer screening in the period 2000-2002. Breast and bowel cancer screening uptake levels were calculated and compared using the chi-squared test. Results: 72,566 women were invited to breast and bowel cancer screening after exclusions. Of these, 3,539 were South Asian and 69,027 non-Asian; 18,730 had been invited to mammography over the previous fifteen years (rounds 1 to 5). South Asian women were significantly less likely to undertake both breast and bowel cancer screening; 29.9% (n = 1,057) compared to 59.4% (n = 40,969) for non-Asians (p < 0.001). Women in both groups who consistently chose to undertake breast cancer screening in rounds 1, 2 and 5 were more likely to complete round 1 bowel cancer screening. However, the likelihood of completion of bowel cancer screening was still significantly lower for South Asians; 49.5% vs. 82.3% for non-Asians, p < 0.001. South Asian women who undertook breast cancer screening in only one round were no more likely to complete bowel cancer screening than those who decided against breast cancer screening in all three rounds. In contrast, similar women in the non-Asian population had an increased likelihood of completing the new bowel cancer screening test. The likelihood of continued uptake of mammography after undertaking screening in round 1 differed between South Asian religio-linguistic groups. Noticeably, women in the Muslim population were less likely to continue to participate in mammography than those in other South Asian groups. Conclusions: Culturally appropriate targeted interventions are required to reduce observed disparities in cancer screening uptakes

    Unintended Consequences of Incentive Provision for Behaviour Change and Maintenance around Childbirth

    Get PDF
    Financial (positive or negative) and non-financial incentives or rewards are increasingly used in attempts to influence health behaviours. While unintended consequences of incentive provision are discussed in the literature, evidence syntheses did not identify any primary research with the aim of investigating unintended consequences of incentive interventions for lifestyle behaviour change. Our objective was to investigate perceived positive and negative unintended consequences of incentive provision for a shortlist of seven promising incentive strategies for smoking cessation in pregnancy and breastfeeding. A multi-disciplinary, mixed-methods approach included involving two service-user mother and baby groups from disadvantaged areas with experience of the target behaviours as study co-investigators. Systematic reviews informed the shortlist of incentive strategies. Qualitative semi-structured interviews and a web-based survey of health professionals asked open questions on positive and negative consequences of incentives. The participants from three UK regions were a diverse sample with and without direct experience of incentive interventions: 88 pregnant women/recent mothers/partners/family members; 53 service providers; 24 experts/decision makers and interactive discussions with 63 conference attendees. Maternity and early years health professionals (n = 497) including doctors, midwives, health visitors, public health and related staff participated in the survey. Qualitative analysis identified ethical, political, cultural, social and psychological implications of incentive delivery at population and individual levels. Four key themes emerged: how incentives can address or create inequalities; enhance or diminish intrinsic motivation and wellbeing; have a positive or negative effect on relationships with others within personal networks or health providers; and can impact on health systems and resources by raising awareness and directing service delivery, but may be detrimental to other health care areas. Financial incentives are controversial and generated emotive and oppositional responses. The planning, design and delivery of future incentive interventions should evaluate unexpected consequences to inform the evidence for effectiveness, cost-effectiveness and future implementation

    Learning and innovative elements of strategy adoption rules expand cooperative network topologies

    Get PDF
    Cooperation plays a key role in the evolution of complex systems. However, the level of cooperation extensively varies with the topology of agent networks in the widely used models of repeated games. Here we show that cooperation remains rather stable by applying the reinforcement learning strategy adoption rule, Q-learning on a variety of random, regular, small-word, scale-free and modular network models in repeated, multi-agent Prisoners Dilemma and Hawk-Dove games. Furthermore, we found that using the above model systems other long-term learning strategy adoption rules also promote cooperation, while introducing a low level of noise (as a model of innovation) to the strategy adoption rules makes the level of cooperation less dependent on the actual network topology. Our results demonstrate that long-term learning and random elements in the strategy adoption rules, when acting together, extend the range of network topologies enabling the development of cooperation at a wider range of costs and temptations. These results suggest that a balanced duo of learning and innovation may help to preserve cooperation during the re-organization of real-world networks, and may play a prominent role in the evolution of self-organizing, complex systems.Comment: 14 pages, 3 Figures + a Supplementary Material with 25 pages, 3 Tables, 12 Figures and 116 reference
    • 

    corecore