Forecasting Economic Indices - Design, Performance, and Learning in Prediction Markets

Macroeconomic forecasts are used extensively in industry and government even though the historical accuracy and reliability is disputed. This thesis develops and studies a prediction market designed to forecast macro-economic indicators in Germany. The market forecasts performed well in comparison to the \u27Bloomberg\u27-survey forecasts. Distinguishing between trading behavior and performance the thesis provides insights into the interplay between interface, information and decision-making

Identifying Experts in Virtual Forecasting Communities

Macroeconomic forecasts are used extensively in industry and government even though the historical accuracy and reliability is questionable. Over the last couple of years prediction markets as a community forecasting method have gained interest in the scientific world and in industry. An arising question is how to detect valuable user input and identify experts in such online communities. Detecting such input would possibly enable us to improve the information aggregation mechanism and the forecast performance of such systems. We design a prediction market for economic derivatives that aggregates macro-economic information. Using market-based measures we find that user input can be evaluated ad-hoc. Further analysis shows that aggregated measures outperform established methods -such as reputation- in identifying forecasting experts. Moreover, using data from a two year field-experiment we find that expertise is stable for longer time horizons

Crowd Labor Markets as Platform for IS Research: First Evidence from Electronic Markets

Crowd labor markets such as Amazon Mechanical Turk (MTurk) have emerged as popular platforms where researchers can inexpensively run web-based experiments. Recent work even suggests that MTurk can be used to run large-scale field experiments such as prediction markets in which participants interact synchronously in real-time. Besides technical issues, several methodological questions arise and lead to the question of how results from MTurk and laboratory experiments compare. In this work we provide first insights into running market experiments on MTurk and compare the key property of markets, information efficiency, to a laboratory setting. The results are mixed at best. On MTurk, information aggregation took place less frequently than in the lab. Our results suggest that MTurk participants cannot handle as much complexity as laboratory participants in time-pressured, synchronized experiments

Beware of Performance Indicators - How Visual Cues Increase the Disposition Effect

Online trading interfaces are important instruments for retail investors. For sound reasons, regulators obligate online brokers to inform customers about certain trade related risks. Research has shown that different behavioral biases can decrease traders’ performance and hence lead to pecuniary losses. The disposition to hold losing stocks too long and sell winning stocks too early (‘disposition effect’) is such a deviation from rational behavior. The disposition effect is analyzed for the prediction market ‘Kurspiloten’ which predicts selected stock prices and counts nearly 2000 active traders and more than 200,000 orders. We show that the disposition effect can be aggravated by visual feedback on a trader’s performance via colored trend direction arrows and percentages. However, we find no evidence that such an interface modification leads to higher activity. Furthermore, we can not confirm that creating awareness of the disposition effect with textual information is suited to decreasing its strength

Participation, Feedback & Incentives in a Competitive Forecasting Community

Macro-economic forecasts are used extensively in industry and government even though the historical accuracy and reliability is questionable. Over the last couple of years prediction markets as a community forecasting method have gained interest. An arising question is how to design incentive schemes and feedback mechanisms to motivate participants to contribute to such an information exchange. We design a prediction market for economic derivatives that aggregates macro-economic information. We show that the level of participation is mainly driven by a weekly newsletter which acts as a reminder. In public goods projects participation feedback has been found to increase participants\u27 contributions. We find that the induced competitiveness of market environments seem to superpose classical feedback mechanisms. We show that forecast errors fall over the prediction horizon. The market generated forecasts compare well to the Bloomberg-survey forecasts, the industry standard. Additionally we can predict community forecast error by using an implicit market measure

FEEDBACK AND PERFORMANCE IN CROWD WORK: A REAL EFFORT EXPERIMENT

Online labor markets gain momentum: Frequently, requsters post micro-tasks and workers choose which tasks to complete for a payment. In virtual, short-lived, and commonly one-shot labor relations, one challenge is to properly incentivize worker effort and quality of work. We present a real effort experiment on a crowd work platform studying the effect of feedback on worker performance. Rank order tournaments might or might not disclose a worker´s current competitive position. One might expect that feedback on the competitive position spurs competition and, in effect, effort and performance. On the contrary, we find evidence that in rank order tournaments, performance feedback tends to have a negative impact on workers´ performance. This effect is mediated by task completion. Furthermore when playing against strong competitors, feedback makes workers more likely to quit the task altogether and, thus, show lower performance. When the competitors are weak, workers tend to complete the task but with reduced effort. Thus, providing performance feedback might not be advisable in crowd labor markets

Incorporating Emotional Information in Decision Systems

Abstract – The media equation [22] states that users react to systems as they would to another person, while continuously emitting social signals. Today’s users expect systems to be empathetic and understand these social signals. Decision systems are a specific sub-branch, facing the need to incorporate affective information, to facilitate users to maximize their cognitive resources. To this end, we attempt to incorporate affective information in the form of physiology to learn users ’ decision behavior. In a controlled experiment, we record participants ’ decisions and measure physiological signals elicited from subjects. To predict the binary decision to buy or sell, three algorithms, multi-layer perceptron, radial basis function, and decision trees, are compared, and they yield recognition rates of 76%, 73 % and 77.2 % respectively. Taking these results, we propose that a decision tree with feature-level fusion, factors in affective information in this controlled context best. These results however have to be extrapolated to decision contexts that elicit emotions more strongly. Keywords—Multimodal Systems, Emotion, User Behavior

ANALYSIS OF THE DISPOSITION EFFECT: ASYMMETRY AND PREDICTION ACCURACY

The disposition effect describes investors’ common tendency of selling a winning investment too soon and holding on to losing investments too long. We analyze the disposition effect in a prediction market for economic indices. We show that the effect for individual traders as well as on an aggregated level. Furthermore we find a significant asymmetry of the disposition effect. The effect can almost exclusively be attributed to the percentage of gains realized (PGR). Additionally we link the aggregated disposition effect and market efficiency. A common hypothesis of the behavioral finance literature is that if participants make systematically biased decisions, market efficiency will suffer. Our setup is well-suited to studying the behavioral aspects of decision making because, in contrast to financial markets (i) the value of shares in our market is ultimately known and (ii) we can measure the participants’ behavioral biases (i.e the disposition effect). Against intuition we find no correlation between the disposition effect and prediction accuracy - a proxy for market efficiency