Optimism as a Prior Belief about the Probability of Future Reward

Optimists hold positive a priori beliefs about the future. In Bayesian statistical theory, a priori beliefs can be overcome by experience. However, optimistic beliefs can at times appear surprisingly resistant to evidence, suggesting that optimism might also influence how new information is selected and learned. Here, we use a novel Pavlovian conditioning task, embedded in a normative framework, to directly assess how trait optimism, as classically measured using self-report questionnaires, influences choices between visual targets, by learning about their association with reward progresses. We find that trait optimism relates to an a priori belief about the likelihood of rewards, but not losses, in our task. Critically, this positive belief behaves like a probabilistic prior, i.e. its influence reduces with increasing experience. Contrary to findings in the literature related to unrealistic optimism and self-beliefs, it does not appear to influence the iterative learning process directly

Near-Optimal BRL using Optimistic Local Transitions

Model-based Bayesian Reinforcement Learning (BRL) allows a found formalization of the problem of acting optimally while facing an unknown environment, i.e., avoiding the exploration-exploitation dilemma. However, algorithms explicitly addressing BRL suffer from such a combinatorial explosion that a large body of work relies on heuristic algorithms. This paper introduces BOLT, a simple and (almost) deterministic heuristic algorithm for BRL which is optimistic about the transition function. We analyze BOLT's sample complexity, and show that under certain parameters, the algorithm is near-optimal in the Bayesian sense with high probability. Then, experimental results highlight the key differences of this method compared to previous work.Comment: ICML201

Better Optimism By Bayes: Adaptive Planning with Rich Models

The computational costs of inference and planning have confined Bayesian model-based reinforcement learning to one of two dismal fates: powerful Bayes-adaptive planning but only for simplistic models, or powerful, Bayesian non-parametric models but using simple, myopic planning strategies such as Thompson sampling. We ask whether it is feasible and truly beneficial to combine rich probabilistic models with a closer approximation to fully Bayesian planning. First, we use a collection of counterexamples to show formal problems with the over-optimism inherent in Thompson sampling. Then we leverage state-of-the-art techniques in efficient Bayes-adaptive planning and non-parametric Bayesian methods to perform qualitatively better than both existing conventional algorithms and Thompson sampling on two contextual bandit-like problems.Comment: 11 pages, 11 figure

Bayesian Reinforcement Learning via Deep, Sparse Sampling

We address the problem of Bayesian reinforcement learning using efficient model-based online planning. We propose an optimism-free Bayes-adaptive algorithm to induce deeper and sparser exploration with a theoretical bound on its performance relative to the Bayes optimal policy, with a lower computational complexity. The main novelty is the use of a candidate policy generator, to generate long-term options in the planning tree (over beliefs), which allows us to create much sparser and deeper trees. Experimental results on different environments show that in comparison to the state-of-the-art, our algorithm is both computationally more efficient, and obtains significantly higher reward in discrete environments.Comment: Published in AISTATS 202

Experts, Conflicts of Interest, and the Controversial Role of Reputation

This paper studies the impact of reputation on the reporting strategy of experts that face conflicts of interest. The framework we propose applies to different settings involv- ing decision makers that rely on experts for making informed decisions, such as financial analysts and goverment agencies. We show that reputation has a non-monotonic effect on the degree of information revelation. In general, truthful revelation is more likely to occur when there is more uncertainty on an expert's ability. Furthermore, above a certain threshold, an increase in reputation always makes truthful revelation more difficult to achieve. Our results shed light on the relationship between the institutional features of the reporting environment and informational efficiency.

Is there a pessimistic bias in individual beliefs ? Evidence from survey data.

The aim of this paper is to determine whether individuals exhibit a behavioral bias towards pessimism in their beliefs, in a lottery or more generally in an investment opportunities framework. For this purpose, we design a field survey on a sample of 1,540 individuals aiming at deriving a measure of pessimism from answers to hypothetical scenarios. In the context of our experiment, we observe that individuals are on average pessimistic. We analyze how pessimism is distributed among individuals, in particular in link with gender, age and income. We also analyze how our notion of pessimism is related to more general notions of pessimism already introduced in psychology. We finally estimate the possible impact of this pessimistic bias on the financial markets equilibrium risk premium.pessimism; lottery; judged probability;