No-Regret Online Prediction with Strategic Experts

We study a generalization of the online binary prediction with expert advice framework where at each round, the learner is allowed to pick $m\geq 1$ experts from a pool of $K$ experts and the overall utility is a modular or submodular function of the chosen experts. We focus on the setting in which experts act strategically and aim to maximize their influence on the algorithm's predictions by potentially misreporting their beliefs about the events. Among others, this setting finds applications in forecasting competitions where the learner seeks not only to make predictions by aggregating different forecasters but also to rank them according to their relative performance. Our goal is to design algorithms that satisfy the following two requirements: 1) $\textit{Incentive-compatible}$: Incentivize the experts to report their beliefs truthfully, and 2) $\textit{No-regret}$: Achieve sublinear regret with respect to the true beliefs of the best fixed set of $m$ experts in hindsight. Prior works have studied this framework when $m=1$ and provided incentive-compatible no-regret algorithms for the problem. We first show that a simple reduction of our problem to the $m=1$ setting is neither efficient nor effective. Then, we provide algorithms that utilize the specific structure of the utility functions to achieve the two desired goals

Alignment Problems With Current Forecasting Platforms

We present alignment problems in current forecasting platforms, such as Good Judgment Open, CSET-Foretell or Metaculus. We classify those problems as either reward specification problems or principal-agent problems, and we propose solutions. For instance, the scoring rule used by Good Judgment Open is not proper, and Metaculus tournaments disincentivize sharing information and incentivize distorting one's true probabilities to maximize the chances of placing in the top few positions which earn a monetary reward. We also point out some partial similarities between the problem of aligning forecasters and the problem of aligning artificial intelligence systems.Comment: 39 pages, 13 figure

Classification of Empirical Work on Sales Promotion: A Synthesis for Managerial Decision Making

Sales Promotion activities have gained strategic focus as markets are getting complex and competitive. Key managerial concerns in this area are budget allocation across elements of promotions as well as trade vis. consumer promotion, how to design individual sales promotion techniques and a calendar in face of competitive promotions, how to manage them and evaluate the short-term and long-term impact of the same. The objective of this paper is to present, through Meta-analysis, an overview of recent contributions appearing in scholastic journals relevant to the field of Sales Promotion, to classify them into different classificatory framework, report key findings, highlight the managerial implications and raise issues. The database used is the EBSCO host available on VSLLAN (Library)- Indian Institute of Management Ahmedabad). The selection procedure consisted of peer-reviewed scholarly contributions for recent five year period. Out of more than 700 articles 64 article were selected which were analyzed for classifying them into • Perspective addressed: Manufacturer, retailer or consumer. • Market [country where the research was undertaken] • Type of promotion activity addressed - coupon, contest, price cut etc. • Management function addressed: planning, implementation, control [evaluation] • It was found that majority of the articles addressed manufacturers perspectives ; almost all studies were done in developed countries ; coupon as a consumer promotion tool was widely researched; and more than half of the articles were addressing planning related issues. Finally attempt has been made to synthesize managerial implications of the studies under broad topic areas for guidelines for managers.

Learning to forecast: The probabilistic time series forecasting challenge

We report on a course project in which students submit weekly probabilistic forecasts of two weather variables and one financial variable. This real-time format allows students to engage in practical forecasting, which requires a diverse set of skills in data science and applied statistics. We describe the context and aims of the course, and discuss design parameters like the selection of target variables, the forecast submission process, the evaluation of forecast performance, and the feedback provided to students. Furthermore, we describe empirical properties of students' probabilistic forecasts, as well as some lessons learned on our part

Elicitation of expectations using Colonel Blotto

We develop a mechanism based on the Colonel Blotto game to elicit (subjective) expectations in a group-based manner. In this game, two players allocate resources over possible future events. A fixed prize is awarded based on the amounts the players allocate to the realized event. We consider two payoff variations: under the proportional-prize rule, the award is split proportionally to the resources that players allocate to the realized event; under the winner-takes-all rule, the full award is given to the player who allocate the most resources to the realized event. When probabilities by which events realize are common knowledge to the players, both games are Bayesian–Nash incentive compatible in the sense that (expected) equilibrium allocations perfectly reflect the true realization probabilities. By means of a laboratory experiment, we find that in a setting where realization probabilities are common knowledge the game with the proportional-prize rule (Prop) elicits better distributions compared to both the winner-takes-all variation (Win) and a benchmark mechanism based on an individual-based proper scoring rule (Ind). Without common knowledge of realization probabilities Prop is at least as good as Ind, showing that it is possible to use a game to elicit expectations in a similar fashion to using a proper scoring rule

Prosocial Option Increases Women’s Entry Into Competition

We provide evidence that women enter competitions at the same rate as men when the incentive for winning includes the option to share part of the rewards with the losers (i.e., when the incentive system is socially oriented). Using an experiment (with N = 238 subjects from three laboratories), we find that about 16% more men than women choose to compete in the standard tournament; this gender gap is eliminated in the socially oriented incentive treatment. While men’s choice to compete remains unchanged, at around 52% in both conditions, women increase their entry rate from 35% in the standard tournament to 60% when the incentive includes a socially oriented option

On Eliciting Beliefs in Strategic Games

Several recent studies in experimental economics have tried to measure beliefs of subjects engaged in strategic games with other subjects. Using data from one such study we conduct an experiment where our experienced subjects observe early rounds of strategy choices from that study and are given monetary incentives to report forecasts of choices in later rounds. We elicit beliefs using three different scoring rules: linear, logarithmic, and quadratic. We compare forecasts across the scoring rules and compare the forecasts of our trained observers to forecasts of the actual players in the original experiment. We find significant differences across scoring rules. The improper linear scoring rule produces forecasts closer to 0 and 1 than the proper rules, and these forecasts are poorly calibrated. The two proper scoring rules induce significantly different distributions of forecasts. We find that forecasts by observers under both proper scoring rules are significantly different from the forecasts of the actual players, in terms of accuracy, calibration, and the distribution of forecasts. We also find evidence for belief convergence among the observers