Reinforcement learning for personalized dialogue management

Language systems have been of great interest to the research community and have recently reached the mass market through various assistant platforms on the web. Reinforcement Learning methods that optimize dialogue policies have seen successes in past years and have recently been extended into methods that personalize the dialogue, e.g. take the personal context of users into account. These works, however, are limited to personalization to a single user with whom they require multiple interactions and do not generalize the usage of context across users. This work introduces a problem where a generalized usage of context is relevant and proposes two Reinforcement Learning (RL)-based approaches to this problem. The first approach uses a single learner and extends the traditional POMDP formulation of dialogue state with features that describe the user context. The second approach segments users by context and then employs a learner per context. We compare these approaches in a benchmark of existing non-RL and RL-based methods in three established and one novel application domain of financial product recommendation. We compare the influence of context and training experiences on performance and find that learning approaches generally outperform a handcrafted gold standard

Prevalence of Trachoma in Unity State, South Sudan: Results from a Large-Scale Population-Based Survey and Potential Implications for Further Surveys

Large parts of South Sudan are thought to be trachoma endemic but baseline data, required to initiate interventions, are few. District-by-district surveys, currently recommended by the World Health Organization (WHO), are often not financially or logistically viable. We therefore adapted existing WHO guidelines and combined eight counties (equivalent to districts) of Unity State into one survey area, randomly sampling 40 villages using a population-based survey design. This decision was based on a trachoma risk map and a trachoma rapid assessment, both identifying the state as likely to be highly endemic. The survey confirmed trachoma as being hyperendemic throughout Unity State, meaning that large-scale intervention should be initiated now. Simulation studies were conducted to determine the likely outcome if fewer (n = 20) or more (n = 60) villages had been sampled, confirming that precision decreased or increased, respectively. Importantly, simulation results also showed that all three sample sizes would have led to the same conclusion, namely the need for large-scale intervention. This finding suggests that district-by-district surveys may not be required for areas where trachoma is suspected to be highly prevalent but that are lacking baseline data; instead districts may be combined into a larger survey area

Contribution of Intrinsic Reactivity of the HIV-1 Envelope Glycoproteins to CD4-Independent Infection and Global Inhibitor Sensitivity

Human immunodeficiency virus (HIV-1) enters cells following sequential activation of the high-potential-energy viral envelope glycoprotein trimer by target cell CD4 and coreceptor. HIV-1 variants differ in their requirements for CD4; viruses that can infect coreceptor-expressing cells that lack CD4 have been generated in the laboratory. These CD4-independent HIV-1 variants are sensitive to neutralization by multiple antibodies that recognize different envelope glycoprotein epitopes. The mechanisms underlying CD4 independence, global sensitivity to neutralization and the association between them are still unclear. By studying HIV-1 variants that differ in requirements for CD4, we investigated the contribution of CD4 binding to virus entry. CD4 engagement exposes the coreceptor-binding site and increases the “intrinsic reactivity” of the envelope glycoproteins; intrinsic reactivity describes the propensity of the envelope glycoproteins to negotiate transitions to lower-energy states upon stimulation. Coreceptor-binding site exposure and increased intrinsic reactivity promote formation/exposure of the HR1 coiled coil on the gp41 transmembrane glycoprotein and allow virus entry upon coreceptor binding. Intrinsic reactivity also dictates the global sensitivity of HIV-1 to perturbations such as exposure to cold and the binding of antibodies and small molecules. Accordingly, CD4 independence of HIV-1 was accompanied by increased susceptibility to inactivation by these factors. We investigated the role of intrinsic reactivity in determining the sensitivity of primary HIV-1 isolates to inhibition. Relative to the more common neutralization-resistant (“Tier 2-like”) viruses, globally sensitive (“Tier 1”) viruses exhibited increased intrinsic reactivity, i.e., were inactivated more efficiently by cold exposure or by a given level of antibody binding to the envelope glycoprotein trimer. Virus sensitivity to neutralization was dictated both by the efficiency of inhibitor/antibody binding to the envelope glycoprotein trimer and by envelope glycoprotein reactivity to the inhibitor/antibody binding event. Quantitative differences in intrinsic reactivity contribute to HIV-1 strain variability in global susceptibility to neutralization and explain the long-observed relationship between increased inhibitor sensitivity and decreased entry requirements for target cell CD4