Detecting Adversarial Directions in Deep Reinforcement Learning to Make Robust Decisions

Learning in MDPs with highly complex state representations is currently possible due to multiple advancements in reinforcement learning algorithm design. However, this incline in complexity, and furthermore the increase in the dimensions of the observation came at the cost of volatility that can be taken advantage of via adversarial attacks (i.e. moving along worst-case directions in the observation space). To solve this policy instability problem we propose a novel method to detect the presence of these non-robust directions via local quadratic approximation of the deep neural policy loss. Our method provides a theoretical basis for the fundamental cut-off between safe observations and adversarial observations. Furthermore, our technique is computationally efficient, and does not depend on the methods used to produce the worst-case directions. We conduct extensive experiments in the Arcade Learning Environment with several different adversarial attack techniques. Most significantly, we demonstrate the effectiveness of our approach even in the setting where non-robust directions are explicitly optimized to circumvent our proposed method.Comment: Published in ICML 202

Futures Studies in the Interactive Society

This book consists of papers which were prepared within the framework of the research project (No. T 048539) entitled Futures Studies in the Interactive Society (project leader: Éva Hideg) and funded by the Hungarian Scientific Research Fund (OTKA) between 2005 and 2009. Some discuss the theoretical and methodological questions of futures studies and foresight; others present new approaches to or procedures of certain questions which are very important and topical from the perspective of forecast and foresight practice. Each study was conducted in pursuit of improvement in futures fields

Trust, regulatory processes and NICE decision-making: Appraising cost-effectiveness models through appraising people and systems.

This article presents an ethnographic study of regulatory decision-making regarding the cost-effectiveness of expensive medicines at the National Institute for Health and Care Excellence (NICE) in England. We explored trust as one important mechanism by which problems of complexity and uncertainty were resolved. Existing studies note the salience of trust for regulatory decisions, by which the appraisal of people becomes a proxy for appraising technologies themselves. Although such (dis)trust in manufacturers was one important influence, we describe a more intricate web of (dis)trust relations also involving various expert advisors, fellow committee members and committee Chairs. Within these complex chains of relations, we found examples of both more blind-acquiescent and more critical-investigative forms of trust as well as, at times, pronounced distrust. Difficulties in overcoming uncertainty through other means obliged trust in some contexts, although not in others. (Dis)trust was constructed through inferences involving abstract systems alongside actors’ oral and written presentations-of-self. Systemic features and ‘forced options’ to trust indicate potential insidious processes of regulatory capture

What are the impacts and cost-effectiveness of strategies to improve performance of untrained and under-trained teachers in the classroom in developing countries?

What are the impacts and cost effectiveness of strategies to improve performance of untrained and under-trained teachers in the classroom in developing countries

Organizational Differences in Managerial Compensation and Financial Performance

The present study has two general purposes. First, based on the compensation strategy literature, we examine the extent to which organizations facing similar conditions make different managerial compensation decisions regarding base pay, bonus pay, and eligibility for long-term incentives. Second, working from expectancy and agency theory perspectives, we explore the consequences of these decisions for subsequent firm performance as measured by return on assets. Using longitudinal data on approximately 16,000 top and middle level managers and 200 organizations, significant between-organization differences in compensation decisions are found. The smallest organization effects are on the level of base pay. The largest organization effects are on bonus levels and eligibility for long-term incentives. In other words, our results suggest that organizations tend to distinguish themselves through decisions about pay contingency or variability rather than through decisions about the level of base pay. To study consequences, residualized measures (adjusted for employee and job factors) of organization pay level and pay mix are used. Pay level is not associated with organization financial performance. On the other hand, greater contingency of pay in the form of bonuses and long-term incentives is associated with better financial performance

Theories of behaviour change synthesised into a set of theoretical groupings: Introducing a thematic series on the Theoretical Domains Framework

Behaviour change is key to increasing the uptake of evidence into healthcare practice. Designing behaviour-change interventions first requires problem analysis, ideally informed by theory. Yet the large number of partly overlapping theories of behaviour makes it difficult to select the most appropriate theory. The need for an overarching theoretical framework of behaviour change was addressed in research in which 128 explanatory constructs from 33 theories of behaviour were identified and grouped. The resulting Theoretical Domains Framework (TDF) appears to be a helpful basis for investigating implementation problems. Research groups in several countries have conducted TDF-based studies. It seems timely to bring together the experience of these teams in a thematic series to demonstrate further applications and to report key developments. This overview article describes the TDF, provides a brief critique of the framework, and introduces this thematic series. In a brief review to assess the extent of TDF-based research, we identified 133 papers that cite the framework. Of these, 17 used the TDF as the basis for empirical studies to explore health professionals’ behaviour. The identified papers provide evidence of the impact of the TDF on implementation research. Two major strengths of the framework are its theoretical coverage and its capacity to elicit beliefs that could signify key mediators of behaviour change. The TDF provides a useful conceptual basis for assessing implementation problems, designing interventions to enhance healthcare practice, and understanding behaviour-change processes. We discuss limitations and research challenges and introduce papers in this series

Multimodal Hierarchical Dirichlet Process-based Active Perception

In this paper, we propose an active perception method for recognizing object categories based on the multimodal hierarchical Dirichlet process (MHDP). The MHDP enables a robot to form object categories using multimodal information, e.g., visual, auditory, and haptic information, which can be observed by performing actions on an object. However, performing many actions on a target object requires a long time. In a real-time scenario, i.e., when the time is limited, the robot has to determine the set of actions that is most effective for recognizing a target object. We propose an MHDP-based active perception method that uses the information gain (IG) maximization criterion and lazy greedy algorithm. We show that the IG maximization criterion is optimal in the sense that the criterion is equivalent to a minimization of the expected Kullback--Leibler divergence between a final recognition state and the recognition state after the next set of actions. However, a straightforward calculation of IG is practically impossible. Therefore, we derive an efficient Monte Carlo approximation method for IG by making use of a property of the MHDP. We also show that the IG has submodular and non-decreasing properties as a set function because of the structure of the graphical model of the MHDP. Therefore, the IG maximization problem is reduced to a submodular maximization problem. This means that greedy and lazy greedy algorithms are effective and have a theoretical justification for their performance. We conducted an experiment using an upper-torso humanoid robot and a second one using synthetic data. The experimental results show that the method enables the robot to select a set of actions that allow it to recognize target objects quickly and accurately. The results support our theoretical outcomes.Comment: submitte

Using Experiments to Foster Innovation and Improve the Effectiveness of Energy Efficiency Programs

This paper argues that the establishment of a process designed to manage innovation must be developed in California to foster the creation of needed program improvements and develop new and more effective energy efficiency delivery programs. This paper discusses several key institutional problems that must be overcome to achieve significant progress

E-HRM: Innovation or irritation. An explorative empirical study in five large companies on web-based HRM

Technological optimistic voices assume that, from a technical perspective, the IT possibilities for HRM are endless: in principal all HR processes can be supported by IT. E-HRM is the relatively new term for this IT supported HRM, especially through the use of web technology. This paper aims at demystifying e-HRM by answering the following questions: what actually is e-HRM?, what are the goals of starting with e-HRM?, what types can be distinguished? and what are the outcomes of e-HRM? Based upon the literature, an e-HRM research model is developed and, guided by this model, five organizations have been studied that have already been on the "e-HR road" for a number of years. We conclude that the goals of e-HRM are mainly to improve HR's administrative efficiency/to achieve cost reduction. Next to this goals, international companies seem to use the introduction of e-HRM to standardize/harmonize HR policies and processes. Further, there is a 'gap' between e-HRM in a technical sense and e-HRM in a practical sense in the five companies involved in our study. Finally, e-HRM hardly helped to improve employee competences, but resulted in cost reduction and a reduction of the administrative burden

Resolution, Recovery and Survival: The Evolution of Payment Disputes in Post-Socialist Europe

What determines the mechanism chosen to resolve a commercial dispute? To what degree does the aggrieved recover damages? And does the relationship survive in the aftermath? The answers to these questions affect expectations as to the costs of transacting and, thereby, the development of markets. But they have received almost no attention in the economic literature on the post-socialist transition. This article exploits a rich survey of small and medium-sized manufacturing enterprises in three post-socialist countries to explain behavioral responses to an inter-firm payment dispute. Particular attention is given to how the evolution of disputes is sensitive to both the geographic distance between trade partners and membership in a business association.http://deepblue.lib.umich.edu/bitstream/2027.42/40147/3/wp761.pd