Using informative behavior to increase engagement while learning from human reward

In this work, we address a relatively unexplored aspect of designing agents that learn from human reward. We investigate how an agent’s non-task behavior can affect a human trainer’s training and agent learning. We use the TAMER framework, which facilitates the training of agents by human-generated reward signals, i.e., judgements of the quality of the agent’s actions, as the foundation for our investigation. Then, starting from the premise that the interaction between the agent and the trainer should be bi-directional, we propose two new training interfaces to increase a human trainer’s active involvement in the training process and thereby improve the agent’s task performance. One provides information on the agent’s uncertainty which is a metric calculated as data coverage, the other on its performance. Our results from a 51-subject user study show that these interfaces can induce the trainers to train longer and give more feedback. The agent’s performance, however, increases only in response to the addition of performance-oriented information, not by sharing uncertainty levels. These results suggest that the organizational maxim about human behavior, “you get what you measure”—i.e., sharing metrics with people causes them to focus on optimizing those metrics while de-emphasizing other objectives—also applies to the training of agents. Using principle component analysis, we show how trainers in the two conditions train agents differently. In addition, by simulating the influence of the agent’s uncertainty–informative behavior on a human’s training behavior, we show that trainers could be distracted by the agent sharing its uncertainty levels about its actions, giving poor feedback for the sake of reducing the agent’s uncertainty without improving the agent’s performance

Planck intermediate results. XLI. A map of lensing-induced B-modes

The secondary cosmic microwave background (CMB) $B$-modes stem from the post-decoupling distortion of the polarization $E$-modes due to the gravitational lensing effect of large-scale structures. These lensing-induced $B$-modes constitute both a valuable probe of the dark matter distribution and an important contaminant for the extraction of the primary CMB $B$-modes from inflation. Planck provides accurate nearly all-sky measurements of both the polarization $E$-modes and the integrated mass distribution via the reconstruction of the CMB lensing potential. By combining these two data products, we have produced an all-sky template map of the lensing-induced $B$-modes using a real-space algorithm that minimizes the impact of sky masks. The cross-correlation of this template with an observed (primordial and secondary) $B$-mode map can be used to measure the lensing $B$-mode power spectrum at multipoles up to $2000$. In particular, when cross-correlating with the $B$-mode contribution directly derived from the Planck polarization maps, we obtain lensing-induced $B$-mode power spectrum measurement at a significance level of $12\,\sigma$, which agrees with the theoretical expectation derived from the Planck best-fit $\Lambda$CDM model. This unique nearly all-sky secondary $B$-mode template, which includes the lensing-induced information from intermediate to small ($10\lesssim \ell\lesssim 1000$) angular scales, is delivered as part of the Planck 2015 public data release. It will be particularly useful for experiments searching for primordial $B$-modes, such as BICEP2/Keck Array or LiteBIRD, since it will enable an estimate to be made of the lensing-induced contribution to the measured total CMB $B$-modes.Comment: 20 pages, 12 figures; Accepted for publication in A&A; The B-mode map is part of the PR2-2015 Cosmology Products; available as Lensing Products in the Planck Legacy Archive http://pla.esac.esa.int/pla/#cosmology; and described in the 'Explanatory Supplement' https://wiki.cosmos.esa.int/planckpla2015/index.php/Specially_processed_maps#2015_Lensing-induced_B-mode_ma

Who Sets the Agenda? Analyzing Key Actors and Dynamics of Economic Diversification in Kazakhstan Throughout 2011–2016

This contribution attempts to answer the key question: Who sets the agenda for economic diversification in the context of Kazakhstan? This question remains critical in current scholarly debates. Although Kazakhstan, a young post-Soviet developing nation, has received fair scholarly attention with regard to the agenda setting stage of the policy cycle, the existing literature has yet failed to (1) empirically establish who actually sets the agenda for a certain policy issue and (2) employ the Internet research methods. This paper seeks to fill these gaps. The literature review of Kazakh-specific agenda setting publications suggests that among the major actors, the government tends to exert predominant influence, though other actors may also play a role, for example, media and academia. This research is driven by Internet penetration rate data and focuses on the period from January 2011 until December 2016. The findings lead to two key conclusions. First, think tanks seem to set the government agenda for economic diversification policy in Kazakhstan. Second, the government, while exhibiting the larger agenda setting magnitude vis-à-vis the other actors, shapes the subsequent debates as measured by the number of relevant references in media, think tanks, and academic publications. This research seeks to contribute to existing agenda setting theories in the Internet era by defining the most important actor(s), specifically in the Kazakh context based on longitudinal dynamics in attention

Mechanisms of TSC-mediated Control of Synapse Assembly and Axon Guidance

Tuberous sclerosis complex is a dominant genetic disorder produced by mutations in either of two tumor suppressor genes, TSC1 and TSC2; it is characterized by hamartomatous tumors, and is associated with severe neurological and behavioral disturbances. Mutations in TSC1 or TSC2 deregulate a conserved growth control pathway that includes Ras homolog enriched in brain (Rheb) and Target of Rapamycin (TOR). To understand the function of this pathway in neural development, we have examined the contributions of multiple components of this pathway in both neuromuscular junction assembly and photoreceptor axon guidance in Drosophila. Expression of Rheb in the motoneuron, but not the muscle of the larval neuromuscular junction produced synaptic overgrowth and enhanced synaptic function, while reductions in Rheb function compromised synapse development. Synapse growth produced by Rheb is insensitive to rapamycin, an inhibitor of Tor complex 1, and requires wishful thinking, a bone morphogenetic protein receptor critical for functional synapse expansion. In the visual system, loss of Tsc1 in the developing retina disrupted axon guidance independently of cellular growth. Inhibiting Tor complex 1 with rapamycin or eliminating the Tor complex 1 effector, S6 kinase (S6k), did not rescue axon guidance abnormalities of Tsc1 mosaics, while reductions in Tor function suppressed those phenotypes. These findings show that Tsc-mediated control of axon guidance and synapse assembly occurs via growth-independent signaling mechanisms, and suggest that Tor complex 2, a regulator of actin organization, is critical in these aspects of neuronal development

Women's Preferences for Treatment of Perinatal Depression and Anxiety : A Discrete Choice Experiment

Perinatal depression and anxiety (PNDA) are an international healthcare priority, associated with significant short- and long-term problems for women, their children and families. Effective treatment is available but uptake is suboptimal: some women go untreated whilst others choose treatments without strong evidence of efficacy. Better understanding of women's preferences for treatment is needed to facilitate uptake of effective treatment. To address this issue, a discrete choice experiment (DCE) was administered to 217 pregnant or postnatal women in Australia, who were recruited through an online research company and had similar sociodemographic characteristics to Australian data for perinatal women. The DCE investigated preferences regarding cost, treatment type, availability of childcare, modality and efficacy. Data were analysed using logit-based models accounting for preference and scale heterogeneity. Predicted probability analysis was used to explore relative attribute importance and policy change scenarios, including how these differed by women's sociodemographic characteristics. Cost and treatment type had the greatest impact on choice, such that a policy of subsidising effective treatments was predicted to double their uptake compared with the base case. There were differences in predicted uptake associated with certain sociodemographic characteristics: for example, women with higher educational attainment were more likely to choose effective treatment. The findings suggest policy directions for decision makers whose goal is to reduce the burden of PNDA on women, their children and families

To Fear is to Gain? The Role of Fear Recognition in Risky Decision Making in TBI Patients and Healthy Controls

Fear is an important emotional reaction that guides decision making in situations of ambiguity or uncertainty. Both recognition of facial expressions of fear and decision making ability can be impaired after traumatic brain injury (TBI), in particular when the frontal lobe is damaged. So far, it has not been investigated how recognition of fear influences risk behavior in healthy subjects and TBI patients. The ability to recognize fear is thought to be related to the ability to experience fear and to use it as a warning signal to guide decision making. We hypothesized that a better ability to recognize fear would be related to a better regulation of risk behavior, with healthy controls outperforming TBI patients. To investigate this, 59 healthy subjects and 49 TBI patients were assessed with a test for emotion recognition (Facial Expression of Emotion: Stimuli and Tests) and a gambling task (Iowa Gambling Task (IGT)). The results showed that, regardless of post traumatic amnesia duration or the presence of frontal lesions, patients were more impaired than healthy controls on both fear recognition and decision making. In both groups, a significant relationship was found between better fear recognition, the development of an advantageous strategy across the IGT and less risk behavior in the last blocks of the IGT. Educational level moderated this relationship in the final block of the IGT. This study has important clinical implications, indicating that impaired decision making and risk behavior after TBI can be preceded by deficits in the processing of fear

Chlorophylls, ligands and assembly of light-harvesting complexes in chloroplasts

Chlorophyll (Chl) b serves an essential function in accumulation of light-harvesting complexes (LHCs) in plants. In this article, this role of Chl b is explored by considering the properties of Chls and the ligands with which they interact in the complexes. The overall properties of the Chls, not only their spectral features, are altered as consequences of chemical modifications on the periphery of the molecules. Important modifications are introduction of oxygen atoms at specific locations and reduction or desaturation of sidechains. These modifications influence formation of coordination bonds by which the central Mg atom, the Lewis acid, of Chl molecules interacts with amino acid sidechains, as the Lewis base, in proteins. Chl a is a versatile Lewis acid and interacts principally with imidazole groups but also with sidechain amides and water. The 7-formyl group on Chl b withdraws electron density toward the periphery of the molecule and consequently the positive Mg is less shielded by the molecular electron cloud than in Chl a. Chl b thus tends to form electrostatic bonds with Lewis bases with a fixed dipole, such as water and, in particular, peptide backbone carbonyl groups. The coordination bonds are enhanced by H-bonds between the protein and the 7-formyl group. These additional strong interactions with Chl b are necessary to achieve assembly of stable LHCs

Planck early results II : The thermal performance of Planck

Peer reviewe