Simple threshold rules solve explore/exploit trade‐offs in a resource accumulation search task

How, and how well, do people switch between exploration and exploitation to search for and accumulate resources? We study the decision processes underlying such exploration/exploitation trade‐offs using a novel card selection task that captures the common situation of searching among multiple resources (e.g., jobs) that can be exploited without depleting. With experience, participants learn to switch appropriately between exploration and exploitation and approach optimal performance. We model participants' behavior on this task with random, threshold, and sampling strategies, and find that a linear decreasing threshold rule best fits participants' results. Further evidence that participants use decreasing threshold‐based strategies comes from reaction time differences between exploration and exploitation; however, participants themselves report non‐decreasing thresholds. Decreasing threshold strategies that “front‐load” exploration and switch quickly to exploitation are particularly effective in resource accumulation tasks, in contrast to optimal stopping problems like the Secretary Problem requiring longer exploration

Active Sensing as Bayes-Optimal Sequential Decision Making

Sensory inference under conditions of uncertainty is a major problem in both machine learning and computational neuroscience. An important but poorly understood aspect of sensory processing is the role of active sensing. Here, we present a Bayes-optimal inference and control framework for active sensing, C-DAC (Context-Dependent Active Controller). Unlike previously proposed algorithms that optimize abstract statistical objectives such as information maximization (Infomax) [Butko & Movellan, 2010] or one-step look-ahead accuracy [Najemnik & Geisler, 2005], our active sensing model directly minimizes a combination of behavioral costs, such as temporal delay, response error, and effort. We simulate these algorithms on a simple visual search task to illustrate scenarios in which context-sensitivity is particularly beneficial and optimization with respect to generic statistical objectives particularly inadequate. Motivated by the geometric properties of the C-DAC policy, we present both parametric and non-parametric approximations, which retain context-sensitivity while significantly reducing computational complexity. These approximations enable us to investigate the more complex problem involving peripheral vision, and we notice that the difference between C-DAC and statistical policies becomes even more evident in this scenario.Comment: Scheduled to appear in UAI 201

Re-visions of rationality?

Empirical evidence suggests proponents of the ‘adaptive toolbox’ framework of human judgment need to rethink their vision of rationality

Adaptive behavior in optimal sequential search

Sequential decision making-making a decision where available options are encountered successively-is a hallmark of everyday life. Such decisions require deciding to accept or reject an alternative without knowing potential future options. Prior work focused on understanding choice behavior by developing decision models that capture human choices in such tasks. We investigated people's adaptive behavior in changing environments in light of their cognitive strategies. We present two studies in which we modified (a) outcome variance and (b) the time horizon and provide empirical evidence that people adapt to both context manipulations. Furthermore, we apply a recently developed threshold model of optimal stopping to our data to disentangle different cognitive processes involved in optimal stopping behavior. The results from Study 1 show that participants adaptively scaled the values of the sampling distribution to its variance, suggesting that the value of an option is perceived in relative rather than absolute terms. The results from Study 2 suggest that increasing the time horizon decreases the initial acceptance level, but less strongly than would be optimal. Furthermore, for longer sequences, participants more weakly adjusted this acceptance threshold over time than for shorter sequences. Further correlations between individual estimates in each condition indicate that individual differences between the participants' thresholds remain fairly stable between the conditions, pointing toward an additive effect of our manipulations

A Cognitive Modeling Analysis of Risk in Sequential Choice Tasks

There exists a variety of instruments that assess risk propensity, or an individual's intrinsic tendency to be risk seeking. This thesis looks at four widely-studied cognitive tasks (the optimal stopping problem, the Balloon Analogue Risk Task, bandit problems, and a preferential choice gambling task) and three commonly used risk questionnaires (Risk Propensity Scale, Risk Taking Index, and Domain-Specific Risk-Taking Scale). Although these decision-making tasks and risk questionnaires have been studied extensively in isolation, there has been less research comparing measures of risk propensity across them. The motivation for examining the relationships between the tasks is that if an individual has a fundamental propensity to take risks, then this trait should be reflected in various questionnaires and cognitive tasks in which behavior is sensitive to risk. Within-subjects data was collected through Amazon Mechanical Turk from 56 participants. As measures of risk from the decision-making tasks, four cognitive models are implemented in which there are psychological variables that can be interpreted as risk propensity. Modeling results, based on Bayesian inferences about parameters and their correlations, show that people's risk behavior is consistent within tasks, but there is less evidence that the way people manage risk in each domain generalizes across tasks and questionnaires

Stochastic models of evidence accumulation in changing environments

Organisms and ecological groups accumulate evidence to make decisions. Classic experiments and theoretical studies have explored this process when the correct choice is fixed during each trial. However, we live in a constantly changing world. What effect does such impermanence have on classical results about decision making? To address this question we use sequential analysis to derive a tractable model of evidence accumulation when the correct option changes in time. Our analysis shows that ideal observers discount prior evidence at a rate determined by the volatility of the environment, and the dynamics of evidence accumulation is governed by the information gained over an average environmental epoch. A plausible neural implementation of an optimal observer in a changing environment shows that, in contrast to previous models, neural populations representing alternate choices are coupled through excitation. Our work builds a bridge between statistical decision making in volatile environments and stochastic nonlinear dynamics.Comment: 26 pages, 7 figure

Predictive intelligence to the edge through approximate collaborative context reasoning

We focus on Internet of Things (IoT) environments where a network of sensing and computing devices are responsible to locally process contextual data, reason and collaboratively infer the appearance of a specific phenomenon (event). Pushing processing and knowledge inference to the edge of the IoT network allows the complexity of the event reasoning process to be distributed into many manageable pieces and to be physically located at the source of the contextual information. This enables a huge amount of rich data streams to be processed in real time that would be prohibitively complex and costly to deliver on a traditional centralized Cloud system. We propose a lightweight, energy-efficient, distributed, adaptive, multiple-context perspective event reasoning model under uncertainty on each IoT device (sensor/actuator). Each device senses and processes context data and infers events based on different local context perspectives: (i) expert knowledge on event representation, (ii) outliers inference, and (iii) deviation from locally predicted context. Such novel approximate reasoning paradigm is achieved through a contextualized, collaborative belief-driven clustering process, where clusters of devices are formed according to their belief on the presence of events. Our distributed and federated intelligence model efficiently identifies any localized abnormality on the contextual data in light of event reasoning through aggregating local degrees of belief, updates, and adjusts its knowledge to contextual data outliers and novelty detection. We provide comprehensive experimental and comparison assessment of our model over real contextual data with other localized and centralized event detection models and show the benefits stemmed from its adoption by achieving up to three orders of magnitude less energy consumption and high quality of inference