Sample-Efficient Model-Free Reinforcement Learning with Off-Policy Critics

Value-based reinforcement-learning algorithms provide state-of-the-art results in model-free discrete-action settings, and tend to outperform actor-critic algorithms. We argue that actor-critic algorithms are limited by their need for an on-policy critic. We propose Bootstrapped Dual Policy Iteration (BDPI), a novel model-free reinforcement-learning algorithm for continuous states and discrete actions, with an actor and several off-policy critics. Off-policy critics are compatible with experience replay, ensuring high sample-efficiency, without the need for off-policy corrections. The actor, by slowly imitating the average greedy policy of the critics, leads to high-quality and state-specific exploration, which we compare to Thompson sampling. Because the actor and critics are fully decoupled, BDPI is remarkably stable, and unusually robust to its hyper-parameters. BDPI is significantly more sample-efficient than Bootstrapped DQN, PPO, and ACKTR, on discrete, continuous and pixel-based tasks. Source code: https://github.com/vub-ai-lab/bdpi.Comment: Accepted at the European Conference on Machine Learning 2019 (ECML

The formation of a functional retinal pigment epithelium occurs on porous polytetrafluoroethylene substrates independently of the surface chemistry

Subretinal transplantation of functioning retinal pigment epithelial (RPE) cells may have the potential to preserve or restore vision in patients affected by blinding diseases such as age-related macular degeneration (AMD). One of the critical steps in achieving this is the ability to grow a functioning retinal pigment epithelium, which may need a substrate on which to grow and to aid transplantation. Tailoring the physical and chemical properties of the substrate should help the engineered tissue to function in the long term. The purpose of the study was to determine whether a functioning monolayer of RPE cells could be produced on expanded polytetrafluoroethylene substrates modified by either an ammonia plasma treatment or an n-Heptylamine coating, and whether the difference in surface chemistries altered the extracellular matrix the cells produced. Primary human RPE cells were able to form a functional, cobblestone monolayer on both substrates, but the formation of an extracellular matrix to exhibit a network structure took months, whereas on non-porous substrates with the same surface chemistry, a similar appearance was observed after a few weeks. This study suggests that the surface chemistry of these materials may not be the most critical factor in the development of growth of a functional monolayer of RPE cells as long as the cells can attach and proliferate on the surface. This has important implications in the design of strategies to optimise the clinical outcomes of subretinal transplant procedures

First Steps towards Underdominant Genetic Transformation of Insect Populations

The idea of introducing genetic modifications into wild populations of insects to stop them from spreading diseases is more than 40 years old. Synthetic disease refractory genes have been successfully generated for mosquito vectors of dengue fever and human malaria. Equally important is the development of population transformation systems to drive and maintain disease refractory genes at high frequency in populations. We demonstrate an underdominant population transformation system in Drosophila melanogaster that has the property of being both spatially self-limiting and reversible to the original genetic state. Both population transformation and its reversal can be largely achieved within as few as 5 generations. The described genetic construct {Ud} is composed of two genes; (1) a UAS-RpL14.dsRNA targeting RNAi to a haploinsufficient gene RpL14 and (2) an RNAi insensitive RpL14 rescue. In this proof-of-principle system the UAS-RpL14.dsRNA knock-down gene is placed under the control of an Actin5c-GAL4 driver located on a different chromosome to the {Ud} insert. This configuration would not be effective in wild populations without incorporating the Actin5c-GAL4 driver as part of the {Ud} construct (or replacing the UAS promoter with an appropriate direct promoter). It is however anticipated that the approach that underlies this underdominant system could potentially be applied to a number of species. Figure

Safe prescribing training provision for junior doctors: is this optimal?

Background The aim of this study was to determine the training provisions in practical safe prescribing for foundation doctors in NHS hospitals located in the South Thames region. Methods A web-based questionnaire was distributed by e-mail to all 1762 foundation doctors in the South Thames Foundation School (STFS) region. In addition, a separate questionnaire was distributed to prescribing training Leads at 15 NHS Hospital Trusts. Quantitative data were analysed using descriptive statistics and thematic analysis was performed on qualitative data. Results Trainers: 10 Prescribing Leads (67 %) responded. Of the 9 NHS Trusts that offered safe prescribing training in their induction programme, 5 included a practical prescribing session. By the end of the foundation year, 6 NHS Trusts had provided at least one dedicated practical prescribing session for F1s compared with 2 NHS Trusts for F2s. Trainees: A total of 124 foundation trainees (7.2 %) responded (69 F1s and 55 F2s). 87 % of F1s received dedicated training in safe prescribing at their Trust induction (n = 60) in comparison to 49 % of F2s (n = 27). 80 % of F1s (n = 55) had a practical prescribing session during induction versus 27 % of F2s (n = 15). The difference was significant, X2 (1, N = 124) = 34.23, p <0.0001. Emerging themes from qualitative data included, recognition of medical education as a continuum, importance of working relationships with pharmacists and neglect of F2s. Conclusions There appears to be a lack of emphasis on the training of F2 doctors in practical safe prescribing compared with F1 doctors. There should be standardisation of safe prescribing training provisions, particularly in the induction period and for F2 doctors

Neural correlates of sexual cue reactivity in individuals with and without compulsive sexual behaviours

Although compulsive sexual behaviour (CSB) has been conceptualized as a "behavioural" addiction and common or overlapping neural circuits may govern the processing of natural and drug rewards, little is known regarding the responses to sexually explicit materials in individuals with and without CSB. Here, the processing of cues of varying sexual content was assessed in individuals with and without CSB, focusing on neural regions identified in prior studies of drug-cue reactivity. 19 CSB subjects and 19 healthy volunteers were assessed using functional MRI comparing sexually explicit videos with non-sexual exciting videos. Ratings of sexual desire and liking were obtained. Relative to healthy volunteers, CSB subjects had greater desire but similar liking scores in response to the sexually explicit videos. Exposure to sexually explicit cues in CSB compared to non-CSB subjects was associated with activation of the dorsal anterior cingulate, ventral striatum and amygdala. Functional connectivity of the dorsal anterior cingulate-ventral striatum-amygdala network was associated with subjective sexual desire (but not liking) to a greater degree in CSB relative to non-CSB subjects. The dissociation between desire or wanting and liking is consistent with theories of incentive motivation underlying CSB as in drug addictions. Neural differences in the processing of sexual-cue reactivity were identified in CSB subjects in regions previously implicated in drug-cue reactivity studies. The greater engagement of corticostriatal limbic circuitry in CSB following exposure to sexual cues suggests neural mechanisms underlying CSB and potential biological targets for interventions

Two treatments, one disease: childhood malaria management in Tanga, Tanzania

<p>Abstract</p> <p>Background</p> <p>In the Tanga District of coastal Tanzania, malaria is one of the primary causes of mortality for children under the age of five. While some children are treated with malaria medications in biomedical facilities, as the World Health Organization recommends, others receive home-care or treatment from traditional healers. Recognition of malaria is difficult because symptoms can range from fever with uncomplicated malaria to convulsions with severe malaria. This study explores why caregivers in the Tanga District of Tanzania pursue particular courses of action to deal with malaria in their children.</p> <p>Methods</p> <p>Qualitative data were collected through interviews with three samples: female caregivers of children under five (N = 61), medical practitioners (N = 28), and traditional healers (N = 18) in urban, peri-urban, and rural areas. The female caregiver sample is intentionally stratified to reflect the greater population of the Tanga District in level of education, marital status, gender of household head, religion, and tribal group affiliation. Qualitative data were counted, coded and analysed using NVivo7 software.</p> <p>Results</p> <p>Results indicate that a variety of factors influence treatment choice, including socio-cultural beliefs about malaria symptoms, associations with spiritual affliction requiring traditional healing, knowledge of malaria, and fear of certain anti-malaria treatment procedures. Most notably, some caregivers identified convulsions as a spiritual condition, unrelated to malaria. While nearly all caregivers reported attending biomedical facilities to treat children with fever (N = 60/61), many caregivers stated that convulsions are best treated by traditional healers (N = 26/61). Qualitative interviews with medical practitioners and traditional healers confirmed this belief.</p> <p>Conclusion</p> <p>Results offer insight into current trends in malaria management and have implications in healthcare policy, educational campaigns, and the importance of integrating traditional and biomedical approaches.</p

β-catenin negatively regulates expression of the prostaglandin transporter PGT in the normal intestinal epithelium and colorectal tumour cells: A role in the chemopreventive efficacy of aspirin

Background: Levels of the pro-tumorigenic prostaglandin PGE 2 are increased in colorectal cancer, previously attributed to increased synthesis through COX-2 upregulation and, more recently, to decreased catabolism. The functionally linked genes 15-prostaglandin dehydrogenase (15-PGDH) and the prostaglandin transporter PGT co-operate in prostaglandin degradation and are downregulated in colorectal cancer. We previously reported repression of 15-PGDH expression by the Wnt/β-catenin pathway, commonly deregulated during early colorectal neoplasia. Here we asked whether β-catenin also regulates PGT expression. Methods: The effect of β-catenin deletion in vivo was addressed by PGT immunostaining of β-catenin/lox-villin-cre-ERT2 mouse tissue. The effect of siRNA-mediated β-catenin knockdown and dnTCF4 induction in vitro was addressed by semi-quantitative and quantitative real-time RT-PCR and immunoblotting. Results: This study shows for the first time that deletion of β-catenin in murine intestinal epithelium in vivo upregulates PGT protein, especially in the crypt epithelium. Furthermore, β-catenin knockdown in vitro increases PGT expression in both colorectal adenoma-and carcinoma-derived cell lines, as does dnTCF4 induction in LS174T cells.Conclusions:These data suggest that β-catenin employs a two-pronged approach to inhibiting prostaglandin turnover during colorectal neoplasia by repressing PGT expression in addition to 15-PGDH. Furthermore, our data highlight a potential mechanism that may contribute to the non-selective NSAID aspirins chemopreventive efficacy. © 2012 Cancer Research UK All rights reserved

Molecular evidence of Rickettsia felis infection in dogs from northern territory, Australia

The prevalence of spotted fever group rickettsial infection in dogs from a remote indigenous community in the Northern Territory (NT) was determined using molecular tools. Blood samples collected from 130 dogs in the community of Maningrida were subjected to a spotted fever group (SFG)-specific PCR targeting the ompB gene followed by a Rickettsia felis-specific PCR targeting the gltA gene of R. felis. Rickettsia felis ompB and gltA genes were amplified from the blood of 3 dogs. This study is the first report of R. felis infection in indigenous community dogs in NT

Sustainability and Long Term-Tenure: Lion Trophy Hunting in Tanzania

It is argued that trophy hunting of large, charismatic mammal species can have considerable conservation benefits but only if undertaken sustainably. Social-ecological theory suggests such sustainability only results from developing governance systems that balance financial and biological requirements. Here we use lion (Panthera leo) trophy hunting data from Tanzania to investigate how resource ownership patterns influence hunting revenue and offtake levels. Tanzania contains up to half of the global population of free-ranging lions and is also the main location for lion trophy hunting in Africa. However, there are concerns that current hunting levels are unsustainable. The lion hunting industry in Tanzania is run by the private sector, although the government leases each hunting block to companies, enforces hunting regulation, and allocates them a species-specific annual quota per block. The length of these leases varies and theories surrounding property rights and tenure suggest hunting levels would be less sustainable in blocks experiencing a high turnover of short-term leases. We explored this issue using lion data collected from 1996 to 2008 in the Selous Game Reserve (SGR), the most important trophy hunting destination in Tanzania. We found that blocks in SGR with the highest lion hunting offtake were also those that experienced the steepest declines in trophy offtake. In addition, we found this high hunting offtake and the resultant offtake decline tended to be in blocks under short-term tenure. In contrast, lion hunting levels in blocks under long-term tenure matched more closely the recommended sustainable offtake of 0.92 lions per 1000 km2. However, annual financial returns were higher from blocks under short-term tenure, providing $133 per km2 of government revenue as compared to$62 per km2 from long-term tenure blocks. Our results provide evidence for the importance of property rights in conservation, and support calls for an overhaul of the system in Tanzania by developing competitive market-based approaches for block allocation based on long-term tenure of ten years

Child malaria treatment decisions by mothers of children less than five years of age attending an outpatient clinic in south-west Nigeria: an application of the PEN-3 cultural model

<p>Abstract</p> <p>Background</p> <p>Using the PEN-3 cultural model, this study sought to understand mothers treatment decisions about their child febrile illness by examining positive health beliefs and practices held by mothers, examine existential (unique) practices that are indigenous to mothers and have no harmful health consequences, and explore negative beliefs and practices that limit recommended responses to febrile illness in children.</p> <p>Methods</p> <p>This qualitative study was conducted in the paediatric section of an outpatient clinic in south-west Nigeria. A total of 123 mothers with children less than five years of age with febrile illness diagnosed as malaria by physicians were individually interviewed on their treatment-seeking practices prior to visiting the clinic and their reasons for attendance at the clinic.</p> <p>Results</p> <p>For some mothers interviewed, effective treatment from the clinic for their child's febrile illness, coupled with physician's approach with malaria diagnosis and treatment practices was important in generating positive maternal treatment-seeking responses to child febrile illness. In addition, beliefs related to a child teething highlighted existential decisions with treatment-seeking for child febrile illness in this setting. Finally, the belief that febrile illness is not all that severe despite noticeable signs and symptoms was a concerning negative perception shared by some mothers in this study.</p> <p>Conclusion</p> <p>The findings highlight the need to consider not only the responses that may serve as barriers to effective treatment, but also an acknowledgment of the positive and existential responses that are equally critical in influencing mothers' management of malaria in their children.</p