An Efficient Bandit Algorithm for Realtime Multivariate Optimization

Optimization is commonly employed to determine the content of web pages, such as to maximize conversions on landing pages or click-through rates on search engine result pages. Often the layout of these pages can be decoupled into several separate decisions. For example, the composition of a landing page may involve deciding which image to show, which wording to use, what color background to display, etc. Such optimization is a combinatorial problem over an exponentially large decision space. Randomized experiments do not scale well to this setting, and therefore, in practice, one is typically limited to optimizing a single aspect of a web page at a time. This represents a missed opportunity in both the speed of experimentation and the exploitation of possible interactions between layout decisions. Here we focus on multivariate optimization of interactive web pages. We formulate an approach where the possible interactions between different components of the page are modeled explicitly. We apply bandit methodology to explore the layout space efficiently and use hill-climbing to select optimal content in realtime. Our algorithm also extends to contextualization and personalization of layout selection. Simulation results show the suitability of our approach to large decision spaces with strong interactions between content. We further apply our algorithm to optimize a message that promotes adoption of an Amazon service. After only a single week of online optimization, we saw a 21% conversion increase compared to the median layout. Our technique is currently being deployed to optimize content across several locations at Amazon.com.Comment: KDD'17 Audience Appreciation Awar

A Zero Attention Model for Personalized Product Search

Product search is one of the most popular methods for people to discover and purchase products on e-commerce websites. Because personal preferences often have an important influence on the purchase decision of each customer, it is intuitive that personalization should be beneficial for product search engines. While synthetic experiments from previous studies show that purchase histories are useful for identifying the individual intent of each product search session, the effect of personalization on product search in practice, however, remains mostly unknown. In this paper, we formulate the problem of personalized product search and conduct large-scale experiments with search logs sampled from a commercial e-commerce search engine. Results from our preliminary analysis show that the potential of personalization depends on query characteristics, interactions between queries, and user purchase histories. Based on these observations, we propose a Zero Attention Model for product search that automatically determines when and how to personalize a user-query pair via a novel attention mechanism. Empirical results on commercial product search logs show that the proposed model not only significantly outperforms state-of-the-art personalized product retrieval models, but also provides important information on the potential of personalization in each product search session

Corticosterone Acts in the Nucleus Accumbens to Enhance Dopamine Signaling and Potentiate Reinstatement of Cocaine Seeking

Stressful life events are important contributors to relapse in recovering cocaine addicts, but the mechanisms by which they influence motivational systems are poorly understood. Studies suggest that stress may “set the stage” for relapse by increasing the sensitivity of brain reward circuits to drug-associated stimuli. We examined the effects of stress and corticosterone on behavioral and neurochemical responses of rats to a cocaine prime after cocaine self-administration and extinction. Exposure of rats to acute electric footshock stress did not by itself reinstate drug-seeking behavior but potentiated reinstatement in response to a subthreshold dose of cocaine. This effect of stress was not observed in adrenalectomized animals, and was reproduced in nonstressed animals by administration of corticosterone at a dose that reproduced stress-induced plasma levels. Pretreatment with the glucocorticoid receptor antagonist RU38486 did not block the corticosterone effect. Corticosterone potentiated cocaine-induced increases in extracellular dopamine in the nucleus accumbens (NAc), and pharmacological blockade of NAc dopamine receptors blocked corticosterone-induced potentiation of reinstatement. Intra-accumbens administration of corticosterone reproduced the behavioral effects of stress and systemic corticosterone. Corticosterone treatment acutely decreased NAc dopamine clearance measured by fast-scan cyclic voltammetry, suggesting that inhibition of uptake2-mediated dopamine clearance may underlie corticosterone effects. Consistent with this hypothesis, intra-accumbens administration of the uptake2 inhibitor normetanephrine potentiated cocaine-induced reinstatement. Expression of organic cation transporter 3, a corticosterone-sensitive uptake2 transporter, was detected on NAc neurons. These findings reveal a novel mechanism by which stress hormones can rapidly regulate dopamine signaling and contribute to the impact of stress on drug intake

A fluorescent nanosensor paint detects dopamine release at axonal varicosities with high spatiotemporal resolution

The neurotransmitter dopamine (DA) controls multiple behaviors and is perturbed in several major brain diseases. DA is released from large populations of specialized structures called axon varicosities. Determining the DA release mechanisms at such varicosities is essential for a detailed understanding of DA biology and pathobiology but has been limited by the low spatial resolution of DA detection methods. We used a near-infrared fluorescent DA nanosensor paint, adsorbed nanosensors detecting release of dopamine (AndromeDA), to detect DA secretion from cultured murine dopaminergic neurons with high spatial and temporal resolution. We found that AndromeDA detects discrete DA release events and extracellular DA diffusion and observed that DA release varies across varicosities. To systematically detect DA release hotspots, we developed a machine learning–based analysis tool. AndromeDA permitted the simultaneous visualization of DA release for up to 100 dopaminergic varicosities, showing that DA release hotspots are heterogeneous and occur at only ∼17% of all varicosities, indicating that many varicosities are functionally silent. Using AndromeDA, we determined that DA release requires Munc13-type vesicle priming proteins, validating the utility of AndromeDA as a tool to study the molecular and cellular mechanism of DA secretion

Texture Coding in the Rat Whisker System: Slip-Stick Versus Differential Resonance

Rats discriminate surface textures using their whiskers (vibrissae), but how whiskers extract texture information, and how this information is encoded by the brain, are not known. In the resonance model, whisker motion across different textures excites mechanical resonance in distinct subsets of whiskers, due to variation across whiskers in resonance frequency, which varies with whisker length. Texture information is therefore encoded by the spatial pattern of activated whiskers. In the competing kinetic signature model, different textures excite resonance equally across whiskers, and instead, texture is encoded by characteristic, nonuniform temporal patterns of whisker motion. We tested these models by measuring whisker motion in awake, behaving rats whisking in air and onto sandpaper surfaces. Resonant motion was prominent during whisking in air, with fundamental frequencies ranging from approximately 35 Hz for the long Delta whisker to approximately 110 Hz for the shorter D3 whisker. Resonant vibrations also occurred while whisking against textures, but the amplitude of resonance within single whiskers was independent of texture, contradicting the resonance model. Rather, whiskers resonated transiently during discrete, high-velocity, and high-acceleration slip-stick events, which occurred prominently during whisking on surfaces. The rate and magnitude of slip-stick events varied systematically with texture. These results suggest that texture is encoded not by differential resonant motion across whiskers, but by the magnitude and temporal pattern of slip-stick motion. These findings predict a temporal code for texture in neural spike trains

From QASC to QASCIP: successful Australian translational scale-up and spread of a proven intervention in acute stroke using a prospective pre-test/post-test study design

Objectives: To embed an evidence-based intervention to manage FEver, hyperglycaemia (Sugar) and Swallowing (the FeSS protocols) in stroke, previously demonstrated in the Quality in Acute Stroke Care (QASC) trial to decrease 90-day death and dependency, into all stroke services in New South Wales (NSW), Australia’s most populous state. Design: Pre-test/post-test prospective study. Setting: 36 NSW stroke services. Methods: Our clinical translational initiative, the QASC Implementation Project (QASCIP), targeted stroke services to embed 3 nurse-led clinical protocols (the FeSS protocols) into routine practice. Clinical champions attended a 1-day multidisciplinary training workshop and received standardised educational resources and ongoing support. Using the National Stroke Foundation audit collection tool and processes, patient data from retrospective medical record self-reported audits for 40 consecutive patients with stroke per site pre-QASCIP (1 July 2012 to 31 December 2012) were compared with prospective self-reported data from 40 consecutive patients with stroke per site post-QASCIP (1 November 2013 to 28 February 2014). Inter-rater reliability was substantial for 10 of 12 variables. Primary outcome measures: Proportion of patients receiving care according to the FeSS protocols pre-QASCIP to post-QASCIP. Results: All 36 (100%) NSW stroke services participated, nominating 100 site champions who attended our educational workshops. The time from start of intervention to completion of post-QASCIP data collection was 8 months. All (n=36, 100%) sites provided medical record audit data for 2144 patients (n=1062 pre-QASCIP; n=1082 post-QASCIP). Pre-QASCIP to post-QASCIP, proportions of patients receiving the 3 targeted clinical behaviours increased significantly: management of fever (pre: 69%; post: 78%; p=0.003), hyperglycaemia (pre: 23%; post: 34%; p=0.0085) and swallowing (pre: 42%; post: 51%; p=0.033). Conclusions: We obtained unprecedented statewide scale-up and spread to all NSW stroke services of a nurse-led intervention previously proven to improve long-term patient outcomes. As clinical leaders search for strategies to improve quality of care, our initiative is replicable and feasible in other acute care settings