R-UCB: a Contextual Bandit Algorithm for Risk-Aware Recommender Systems

Mobile Context-Aware Recommender Systems can be naturally modelled as an exploration/exploitation trade-off (exr/exp) problem, where the system has to choose between maximizing its expected rewards dealing with its current knowledge (exploitation) and learning more about the unknown user's preferences to improve its knowledge (exploration). This problem has been addressed by the reinforcement learning community but they do not consider the risk level of the current user's situation, where it may be dangerous to recommend items the user may not desire in her current situation if the risk level is high. We introduce in this paper an algorithm named R-UCB that considers the risk level of the user's situation to adaptively balance between exr and exp. The detailed analysis of the experimental results reveals several important discoveries in the exr/exp behaviour

Pathways to Coastal Resiliency: the Adaptive Gradients Framework

Current and future climate-related coastal impacts such as catastrophic and repetitive flooding, hurricane intensity, and sea level rise necessitate a new approach to developing and managing coastal infrastructure. Traditional “hard” or “grey” engineering solutions are proving both expensive and inflexible in the face of a rapidly changing coastal environment. Hybrid solutions that incorporate natural, nature-based, structural, and non-structural features may better achieve a broad set of goals such as ecological enhancement, long-term adaptation, and social benefits, but broad consideration and uptake of these approaches has been slow. One barrier to the widespread implementation of hybrid solutions is the lack of a relatively quick but holistic evaluation framework that places these broader environmental and societal goals on equal footing with the more traditional goal of exposure reduction. To respond to this need, the Adaptive Gradients Framework was developed and pilot-tested as a qualitative, flexible, and collaborative process guide for organizations to understand, evaluate, and potentially select more diverse kinds of infrastructural responses. These responses would ideally include natural, nature-based, and regulatory/cultural approaches, as well as hybrid designs combining multiple approaches. It enables rapid expert review of project designs based on eight metrics called “gradients”, which include exposure reduction, cost efficiency, institutional capacity, ecological enhancement, adaptation over time, greenhouse gas reduction, participatory process, and social benefits. The framework was conceptualized and developed in three phases: relevant factors and barriers were collected from practitioners and experts by survey; these factors were ranked by importance and used to develop the initial framework; several case studies were iteratively evaluated using this technique; and the framework was finalized for implementation. The article presents the framework and a pilot test of its application, along with resources that would enable wider application of the framework by practitioners and theorists

A Gradient in Education Due to Health? Evidence from the Study of Health Behavior in School-Aged Children

Research exploring the relationship between education and health suggests that people with higher levels of schooling report better health. To emphasize health as a determinant of educational achievement, this article establishes a gradient in education by health among Canadian students. Using data from the 2006 Health Behaviour in School-aged Children (HBSC) study, the relationship between self-rated health and achievement is examined for 8,626 students from 131 schools. The variation of the gradient in education by health within and between schools suggests that increases in self-rated health are associated with increased achievement for students. Moreover, the within-school regression accounted for 2.7 % of the variation in achievement due to health, whereas the between-school regression slope accounted for 19.8% of the variation in achievement due to health. Inequalities in achievement associated with health were more pronounced between schools than within schools. Policy implications as they relate to the findings are discussed.La recherche portant sur le rapport entre l’éducation et la santé donne à penser que les gens les plus instruits se disent en meilleure santé. Afin de mettre en relief la santé comme facteur déterminant dans le niveau d’instruction, cet article développe une échelle liant le niveau de scolarité et la santé chez les élèves canadiens. Puisant dans des données de l’enquête sur les comportements liés à la santé chez les enfants d’âge scolaire (2006), nous examinons le rapport entre la santé et la scolarité telles que décrites par 8 626 élèves provenant de 131 écoles. La variation notée dans le rapport scolarité/santé à l’intérieur des écoles et entre elles donne à penser que plus l’état de santé déclaré est positif, plus le rendement est élevé chez les élèves. De plus, la régression au sein des écoles représente 2,7% de la variation dans le rendement attribuable à la santé, alors que la régression entre les écoles représente 19,8 % de la variation dans le rendement attribuable à la santé. Les inégalités dans le rendement associé à la santé étaient plus prononcées entre les écoles qu’au sein des écoles. Nous discutons des incidences sur la politique qui découlent de ces résultats

Improving Search through A3C Reinforcement Learning based Conversational Agent

We develop a reinforcement learning based search assistant which can assist users through a set of actions and sequence of interactions to enable them realize their intent. Our approach caters to subjective search where the user is seeking digital assets such as images which is fundamentally different from the tasks which have objective and limited search modalities. Labeled conversational data is generally not available in such search tasks and training the agent through human interactions can be time consuming. We propose a stochastic virtual user which impersonates a real user and can be used to sample user behavior efficiently to train the agent which accelerates the bootstrapping of the agent. We develop A3C algorithm based context preserving architecture which enables the agent to provide contextual assistance to the user. We compare the A3C agent with Q-learning and evaluate its performance on average rewards and state values it obtains with the virtual user in validation episodes. Our experiments show that the agent learns to achieve higher rewards and better states.Comment: 17 pages, 7 figure