Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Maps

With advances in reinforcement learning (RL), agents are now being developed in high-stakes application domains such as healthcare and transportation. Explaining the behavior of these agents is challenging, as the environments in which they act have large state spaces, and their decision-making can be affected by delayed rewards, making it difficult to analyze their behavior. To address this problem, several approaches have been developed. Some approaches attempt to convey the $\textit{global}$ behavior of the agent, describing the actions it takes in different states. Other approaches devised $\textit{local}$ explanations which provide information regarding the agent's decision-making in a particular state. In this paper, we combine global and local explanation methods, and evaluate their joint and separate contributions, providing (to the best of our knowledge) the first user study of combined local and global explanations for RL agents. Specifically, we augment strategy summaries that extract important trajectories of states from simulations of the agent with saliency maps which show what information the agent attends to. Our results show that the choice of what states to include in the summary (global information) strongly affects people's understanding of agents: participants shown summaries that included important states significantly outperformed participants who were presented with agent behavior in a randomly set of chosen world-states. We find mixed results with respect to augmenting demonstrations with saliency maps (local information), as the addition of saliency maps did not significantly improve performance in most cases. However, we do find some evidence that saliency maps can help users better understand what information the agent relies on in its decision making, suggesting avenues for future work that can further improve explanations of RL agents

Looking for a psychology for the inner rational agent

Research in psychology and behavioural economics shows that individuals’ choices often depend on ‘irrelevant’ contextual factors. This presents problems for normative economics, which has traditionally used preference-satisfaction as its criterion. A common response is to claim that individuals have context-independent latent preferences which are ‘distorted’ by psychological factors, and that latent preferences should be respected. This response implicitly uses a model of human action in which each human being has an ‘inner rational agent’. I argue that this model is psychologically ungrounded. Although references to latent preferences appear in psychologically-based explanations of context-dependent choice, latent preferences serve no explanatory purpose

Progress in Behavioral Game Theory

Is game theory meant to describe actual choices by people and institutions or not? It is remarkable how much game theory has been done while largely ignoring this question. The seminal book by von Neumann and Morgenstern, The Theory of Games and Economic Behavior, was clearly about how rational players would play against others they knew were rational. In more recent work, game theorists are not always explicit about what they aim to describe or advise. At one extreme, highly mathematical analyses have proposed rationality requirements that people and firms are probably not smart enough to satisfy in everyday decisions. At the other extreme, adaptive and evolutionary approaches use very simple models-mostly developed to describe nonhuman animals-in which players may not realize they are playing a game at all. When game theory does aim to describe behavior, it often proceeds with a disturbingly low ratio of careful observation to theorizing

The Institutionalization of Institutional Theory

[Excerpt] Our primary aims in this effort are twofold: to clarify the independent theoretical contributions of institutional theory to analyses of organizations, and to develop this theoretical perspective further in order to enhance its use in empirical research. There is also a more general, more ambitious objective here, and that is to build a bridge between two distinct models of social actor that underlie most organizational analyses, which we refer to as a rational actor model and an institutional model. The former is premised on the assumption that individuals are constantly engaged in calculations of the costs and benefits of different action choices, and that behavior reflects such utility-maximizing calculations. In the latter model, by contrast, \u27oversocialized\u27 individuals are assumed to accept and follow social norms unquestioningly, without any real reflection or behavioral resistance based on their own particular, personal interests. We suggest that these two general models should be treated not as oppositional but rather as representing two ends of a continuum of decision-making processes and behaviors. Thus, a key problem for theory and research is to specify the conditions under which behavior is more likely to resemble one end of this continuum or the other. In short, what is needed are theories of when rationality is likely to be more or less bounded. A developed conception of institutionalization processes provides a useful point of departure for exploring this issue

Knowing where you are walking: the benefits and hazards of using theoretical roadmaps and research to guide community consultation practice

This paper progresses the 2006 Trans-Tasman Conference theme by considering 'do you know where you are walking?' and more importantly 'do you know how to get there?'. The community psychologist’s aspirations for the 'common good' anticipate outcomes such as social justice, sense of community and empowerment, and prescribe congruent values-based praxis to accomplish it. Such concepts and visions compel us to act for social change; they also enlighten the challenging decisions and processes we encounter in our personal every day walk of life. This journey can be complex as the 'doing' of community psychology often involves an emotive personal as well as professional commitment, sometimes associated with a particular political agenda. This paper considers how mapping the psychological landscape of a community, along with its history and geography, can help guide the practitioner through difficult terrain. Some of the author’s experiences are considered from theoretical perspectives including sense of community, community consultation processes, social ecology, community readiness and capacity building. These narratives illustrate how such theoretical maps can help the community psychologist to avoid walking in circles, down dead-end streets and up one-way streets while 'walking the walk'

Practice-based Qualitative Research: Participant Experiences of Walk-in Counselling and Traditional Counselling

Walk-in single session counselling is becoming a more widely used model for delivering mental health services across Ontario. This paper reports findings from the qualitative phase of a mixed method study, exploring the experiences of those attending walk-in counselling (WIC) model compared to the traditional service delivery model employing a wait list. We used a comparative case study design for the qualitative phase. Findings reveal that participant outcomes of the walk-in counselling model is influenced by accessibility, how a participant makes sense of the service, and the degree to which a participant is motivated and able to engage in counselling. WIC supports the mental health system by reducing wait lists associated with traditional service delivery models, and meeting the needs many people identify for immediate consultation. Other participants still perceive themselves as requiring ongoing counselling over time and involving in-depth exploration. This research supports health systems providing access to both models