Impatient Bandits: Optimizing Recommendations for the Long-Term Without Delay

Recommender systems are a ubiquitous feature of online platforms. Increasingly, they are explicitly tasked with increasing users' long-term satisfaction. In this context, we study a content exploration task, which we formalize as a multi-armed bandit problem with delayed rewards. We observe that there is an apparent trade-off in choosing the learning signal: Waiting for the full reward to become available might take several weeks, hurting the rate at which learning happens, whereas measuring short-term proxy rewards reflects the actual long-term goal only imperfectly. We address this challenge in two steps. First, we develop a predictive model of delayed rewards that incorporates all information obtained to date. Full observations as well as partial (short or medium-term) outcomes are combined through a Bayesian filter to obtain a probabilistic belief. Second, we devise a bandit algorithm that takes advantage of this new predictive model. The algorithm quickly learns to identify content aligned with long-term success by carefully balancing exploration and exploitation. We apply our approach to a podcast recommendation problem, where we seek to identify shows that users engage with repeatedly over two months. We empirically validate that our approach results in substantially better performance compared to approaches that either optimize for short-term proxies, or wait for the long-term outcome to be fully realized.Comment: Presented at the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '23

Somato-dendritic vasopressin and oxytocin secretion in endocrine and autonomic regulation

Somato‐dendritic secretion was first demonstrated over 30 years ago. However, although its existence has become widely accepted, the function of somato‐dendritic secretion is still not completely understood. Hypothalamic magnocellular neurosecretory cells were among the first neuronal phenotypes in which somato‐dendritic secretion was demonstrated and are among the neurones for which the functions of somato‐dendritic secretion are best characterised. These neurones secrete the neuropeptides, vasopressin and oxytocin, in an orthograde manner from their axons in the posterior pituitary gland into the blood circulation to regulate body fluid balance and reproductive physiology. Retrograde somato‐dendritic secretion of vasopressin and oxytocin modulates the activity of the neurones from which they are secreted, as well as the activity of neighbouring populations of neurones, to provide intra‐ and inter‐population signals that coordinate the endocrine and autonomic responses for the control of peripheral physiology. Somato‐dendritic vasopressin and oxytocin have also been proposed to act as hormone‐like signals in the brain. There is some evidence that somato‐dendritic secretion from magnocellular neurosecretory cells modulates the activity of neurones beyond their local environment where there are no vasopressin‐ or oxytocin‐containing axons but, to date, there is no conclusive evidence for, or against, hormone‐like signalling throughout the brain, although it is difficult to imagine that the levels of vasopressin found throughout the brain could be underpinned by release from relatively sparse axon terminal fields. The generation of data to resolve this issue remains a priority for the field.http://wileyonlinelibrary.com/journal/jne2021-04-17hj2020Immunolog

HAX-1 overexpression, splicing and cellular localization in tumors

<p>Abstract</p> <p>Background</p> <p>HAX-1 has been described as a protein potentially involved in carcinogenesis and especially metastasis. Its involvement in regulation of apoptosis and cell migration along with some data indicating its overexpression in cancer cell lines and tumors suggests that HAX-1 may play a role in neoplastic transformation. Here we present the first systematic analysis of HAX-1 expression in several solid tumors.</p> <p>Methods</p> <p>Using quantitative RT-PCR, we have determined the mRNA levels of <it>HAX1 </it>splice variant I in several solid tumors. We have also analyzed by semiquantitative and quantitative RT-PCR the expression of five <it>HAX-1 </it>splice variants in breast cancer samples and in normal tissue from the same individuals. Quantitative PCR was also employed to analyze the effect of estrogen on <it>HAX1 </it>expression in breast cancer cell line. Immunohistochemical analysis of HAX-1 was performed on normal and breast cancer samples.</p> <p>Results</p> <p>The results reveal statistically important <it>HAX1 </it>up-regulation in breast cancer, lung cancer and melanoma, along with some minor variations in the splicing pattern. HAX-1 up-regulation in breast cancer samples was confirmed by immunohistochemical analysis, which also revealed an intriguing HAX-1 localization in the nuclei of the tumor cells, associated with strong ER status.</p> <p>Conclusion</p> <p>HAX-1 elevated levels in cancer tissues point to its involvement in neoplastic transformation, especially in breast cancer. The connection between HAX-1 nuclear location and ER status in breast cancer samples remains to be clarified.</p

Effects of prenatal food and micronutrient supplementation on child growth from birth to 54 months of age: a randomized trial in Bangladesh

<p>Abstract</p> <p>Background</p> <p>There is a lack of information on the optimal timing of food supplementation to malnourished pregnant women and possible combined effects of food and multiple micronutrient supplementations (MMS) on their offspring's growth. We evaluated the effects of prenatal food and micronutrient interventions on postnatal child growth. The hypothesis was that prenatal MMS and early invitation to food supplementation would increase physical growth in the offspring during 0-54 months and a combination of these interventions would further improve these outcomes.</p> <p>Methods</p> <p>In the large, randomized MINIMat trial (Maternal and Infant Nutrition Interventions in Matlab), Bangladesh, 4436 pregnant women were enrolled between November 2001 and October 2003 and their children were followed until March 2009. Participants were randomized into six groups comprising 30 mg Fe and 400 μg folic acid (Fe30F), 60 mg Fe and 400 μg folic acid (Fe60F) or MMS combined with either an early (immediately after identification of pregnancy) or a later usual (at the time of their choosing, i.e., usual care in this community) program invitation to food supplementation. The anthropometry of 3267 children was followed from birth to 54 months, and 2735 children were available for analysis at 54 months.</p> <p>Results</p> <p>There were no differences in characteristics of mothers and households among the different intervention groups. The average birth weight was 2694 g and birth length was 47.7 cm, with no difference among intervention groups. Early invitation to food supplementation (in comparison with usual invitation) reduced the proportion of stunting from early infancy up to 54 months for boys (p = 0.01), but not for girls (p = 0.31). MMS resulted in more stunting than standard Fe60F (p = 0.02). There was no interaction between the food and micronutrient supplementation on the growth outcome.</p> <p>Conclusions</p> <p>Early food supplementation in pregnancy reduced the occurrence of stunting during 0-54 months in boys, but not in girls, and prenatal MMS increased the proportion of stunting in boys. These effects on postnatal growth suggest programming effects in early fetal life.</p> <p>Trial registration number</p> <p>ISRCTN: <a href="http://www.controlled-trials.com/ISRCTN16581394">ISRCTN16581394</a></p

Tests of multiple molecular markers for the identification of Great Spotted and Syrian Woodpeckers and their hybrids

Great Spotted and Syrian Woodpeckers (Dendrocopos major and D. syriacus) are known to hybridize in nature; however, the extent of this phenomenon is not known due to difficulties in hybrid detection based on plumage analyses. Here, we tested five markers (one mitochondrial and four nuclear) and a set of six microsatellite loci for the identification of these two Woodpeckers and their hybrids. Sequencing of DNA from 26 individuals of both Woodpeckers from different parts of their ranges: one allopatric (D. major; Norway) and two sympatric (Poland and Bulgaria) showed that both species can be clearly separated based on all sequence markers. The highest number of fixed nucleotide sites were found in the mtDNA control region and intron 5 of the transforming growth factor. Analyses of microsatellite data distinguished the two species, but all loci showed a large number of common alleles and their utility in identifying hybrids is therefore doubtful. According to the DNA sequence analyses, 2 out of 18 specimens within the sympatric range in Poland were identified as possible hybrids, most probably paternal backcrosses. Moreover, both hybrids are from synantropic populations (settled in cities), whereas none of the D. major sampled in forests and in its allopatric range (Norway) showed signs of an intermixed genotype. Further research on hybridization and introgression in woodpeckers is undoubtedly needed and could be useful for understanding ecological and ethological interactions among these species, particularly for D. syriacus, which is relatively rare in Europe

Cardiopoietic cell therapy for advanced ischemic heart failure: results at 39 weeks of the prospective, randomized, double blind, sham-controlled CHART-1 clinical trial

Cardiopoietic cells, produced through cardiogenic conditioning of patients' mesenchymal stem cells, have shown preliminary efficacy. The Congestive Heart Failure Cardiopoietic Regenerative Therapy (CHART-1) trial aimed to validate cardiopoiesis-based biotherapy in a larger heart failure cohort

Discrimination of amino acids by electronic tongue

Obecnie potrzebne są metody szybkiego, dokładnego, a przede wszystkim taniego oznaczania aminokwasów w próbkach biologicznych. Do takich technik należy potencjometryczny elektroniczny język, który pozwala na rozróżnianie próbek ciekłych. W pracy przedstawiono zastosowanie potencjometrycznego elektronicznego języka do klasyfikowania 5 aminokwasów. Sprawdzono możliwość rozpoznania poszczególnych analitów w roztworach o różnych pH, a następnie wykorzystując metodę częściowych najmniejszych kwadratów (PLS) rozróżniono poszczególne aminokwasy.In the last few years medical discoveries showed that presence of amino acids can be associated with many diseases. Usually, these compounds are determined using HPLC method (high performance liquid chromatography), however, due to its noxiousness, new methods are investigated. Application of electronic tongue system can be an alternative method for fast, accurate, affordable determination of amino acids in biological samples. This device is used for analysis and classification of complex liquid samples. Its main advantage is the ability to distinguish between samples containing various components on-line, without prior sample preparation and modification. In this paper there are presented preliminary studies on the recognition of 5 amino acids (glutamic acid, alanine, serine, arginine, aspartic acid) with use of a sensor array composed of ion-selective electrodes of various selectivity. Chemical images of the samples were obtained by measuring the electrode signals in solutions of amino acids for various pH ranging from 2.5 to 11.5. The possibility of identification of individual analytes in solutions of different pH was examined with use of the Partial Least Squares Analysis

Fourier policy gradients

We propose a new way of deriving policy gradient updates for reinforcement learning. Our technique, based on Fourier analysis, recasts integrals that arise with expected policy gradients as convolutions and turns them into multiplications. The obtained analytical solutions allow us to capture the low variance benefits of EPG in a broad range of settings. For the critic, we treat trigonometric and radial basis functions, two function families with the universal approximation property. The choice of policy can be almost arbitrary, including mixtures or hybrid continuous-discrete probability distributions. Moreover, we derive a general family of sample-based estimators for stochastic policy gradients, which unifies existing results on sample-based approximation. We believe that this technique has the potential to shape the next generation of policy gradient approaches, powered by analytical results

Fourier policy gradients

We propose a new way of deriving policy gradient updates for reinforcement learning. Our technique, based on Fourier analysis, recasts integrals that arise with expected policy gradients as convolutions and turns them into multiplications. The obtained analytical solutions allow us to capture the low variance benefits of EPG in a broad range of settings. For the critic, we treat trigonometric and radial basis functions, two function families with the universal approximation property. The choice of policy can be almost arbitrary, including mixtures or hybrid continuous-discrete probability distributions. Moreover, we derive a general family of sample-based estimators for stochastic policy gradients, which unifies existing results on sample-based approximation. We believe that this technique has the potential to shape the next generation of policy gradient approaches, powered by analytical results