A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. Bayesian optimization employs the Bayesian technique of setting a prior over the objective function and combining it with evidence to get a posterior function. This permits a utility-based selection of the next observation to make on the objective function, which must take into account both exploration (sampling from areas of high uncertainty) and exploitation (sampling areas likely to offer improvement over the current best observation). We also present two detailed extensions of Bayesian optimization, with experiments---active user modelling with preferences, and hierarchical reinforcement learning---and a discussion of the pros and cons of Bayesian optimization based on our experiences

Scenario selection optimization in system engineering projects under uncertainty: a multi-objective ant colony method based on a learning mechanism

This paper presents a multi-objective Ant Colony Optimization (MOACO) algorithm based on a learning mechanism (named MOACO-L) for the optimization of project scenario selection under uncertainty in a system engineering (SE) process. The objectives to minimize are the total cost of the project, its total duration and the global risk. Risk is considered as an uncertainty about task costs and task durations in the project graph. The learning mechanism aims to improve the MOACO algorithm for the selection of optimal project scenarios in aSE project by considering the uncertainties on the project objectives. The MOACO-L algorithm is then developed by taking into account ants’ past experiences. The learning mechanism allows a better exploration of the search space and an improvement of the MOACO algorithm performance. To validate our approach, some experimental results are presented

Transcranial Direct Corrent stimulation (tDCS) of the anterior prefrontal cortex (aPFC) modulates reinforcement learning and decision-making under uncertainty: A doubleblind crossover study

Reinforcement learning refers to the ability to acquire information from the outcomes of prior choices (i.e. positive and negative) in order to make predictions on the effect of future decision and adapt the behaviour basing on past experiences. The anterior prefrontal cortex (aPFC) is considered to play a key role in the representation of event value, reinforcement learning and decision-making. However, a causal evidence of the involvement of this area in these processes has not been provided yet. The aim of the study was to test the role of the orbitofrontal cortex in feedback processing, reinforcement learning and decision-making under uncertainly. Eighteen healthy individuals underwent three sessions of tDCS over the prefrontal pole (anodal, cathodal, sham) during a probabilistic learning (PL) task. In the PL task, participants were invited to learn the covert probabilistic stimulusoutcome association from positive and negative feedbacks in order to choose the best option. Afterwards, a probabilistic selection (PS) task was delivered to assess decisions based on the stimulus-reward associations acquired in the PL task. During cathodal tDCS, accuracy in the PL task was reduced and participants were less prone to maintain their choice after positive feedback or to change it after a negative one (i.e., winstay and lose-shift behavior). In addition, anodal tDCS affected the subsequent PS task by reducing the ability to choose the best alternative during hard probabilistic decisions. In conclusion, the present study suggests a causal role of aPFC in feedback trial-by-trial behavioral adaptation and decision-making under uncertainty

Bootstrapping trust evaluations through stereotypes

Publisher PD

What drives contract design in strategic alliances? Taking stock and how to proceed

We collect and assess prior empirical evidence on contract design in alliances that has been published since Parkhe’s (1993) seminal study on inter-firm contracts. We elaborate on the effects of transaction-related factors, experience gained from prior relationships, and deliberate learning efforts on contracts. Our paper offers three contributions. First, we systematically review the existing literature on alliance contracts and summarize our findings. Second, while prior research has traditionally focused on contractual complexity, we place the content of contracts center stage and identify three contractual functions. While existing studies on contractual functions predominantly refer to safeguarding as a response to appropriation concerns, we also consider coordination and contingency adaptability as outcomes of adaptation concerns. Third, we disentangle the differential influences of previous experiences on distinct contractual functions and show that experience gained from prior relationships has different effects on safeguarding and contingency adaptability than on coordination. Overall, we add to the systematization of the current debate on alliance contract design and trace promising avenues for future research on the impact of transaction- and experience-related factors on the complexity and content of alliance contracts