Predicting overprecision in range estimation

Theme for 2016: Recognizing and representing eventsOverprecision (overconfidence in interval estimation) is a bias with clear implications for economic outcomes in industries reliant on forecasting possible ranges for future prices and unknown states of nature - such as mineral and petroleum exploration. Prior research has shown the ranges people provide are too narrow given the knowledge they have – that is, they underestimate uncertainty and are overconfident in their knowledge. The underlying causes of this bias are, however, still unclear and individual differences research has shed little light on traits predictive of susceptibility. Taking this as a starting point, this paper directly contrasts the Naïve Sampling Model and Informativeness-Accuracy Tradeoff accounts of overprecision – seeing which better predicts performance in an interval estimation task. This was achieved by identifying traits associated with these theories – Short Term Memory and Need for Cognitive Closure, respectively. Analyses indicate that NFCC but not STM predicts interval width and thus, potentially, impacts overprecision.Matthew Kaesler, Matthew B. Welsh, Carolyn Semmle

What is your level of overconfidence? A strictly incentive compatible measurement of absolute and relative overconfidence.

This study contributes to the ongoing discussion on the appropriate measurement of overconfidence, in particular, its strictly incentive compatible measurement in experiments. Despite a number of significant advances in recent research, several important issues remain to be solved. These relate to the strictness of incentive compatibility, the identification of well-calibrated participants, the trichotomous classification into over- or underconfident and well-calibrated participants, and the generalization to measuring beliefs about the performance relative to other people. This paper develops a measurement of overconfidence that is improved regarding all four of these issues. We theoretically prove that our method is strictly incentive compatible and robust to risk attitudes within the framework of Cumulative Prospect Theory. Furthermore, our method allows the measurement of various levels of overconfidence and the direct comparison of absolute and relative confidence. We tested our method, and the results meet our expectations, replicate recent results, and show that a population can be simultaneously overconfident, well-calibrated, and underconfident. In our specific case, we find that more than ninety-five percent of the population believe to be better than twenty-five percent; about fifty percent believe to be better than fifty percent; and only seven percent believe to be better than seventy-five percent.Belief elicitation, Overconfidence, Better than average, Incentive compatibility

Overconfidence in Phishing Email Detection

This study examines overconfidence in phishing email detection. Researchers believe that overconfidence (i.e., where one’s judgmental confidence exceeds one’s actual performance in decision making) can lead to one’s adopting risky behavior in uncertain situations. This study focuses on what leads to overconfidence in phishing detection. We performed a survey experiment with 600 subjects to collect empirical data for the study. In the experiment, each subject judged a set of randomly selected phishing emails and authentic business emails. Specifically, we examined two metrics of overconfidence (i.e., overprecision and overestimation). Results show that cognitive effort decreased overconfidence, while variability in attention allocation, dispositional optimism, and familiarity with the business entities in the emails all increased overconfidence in phishing email detection. The effect of perceived self-efficacy of detecting phishing emails on overconfidence was marginal. In addition, all confidence beliefs poorly predicted detection accuracy and poorly explained its variance, which highlights the issue of relying on them to guide one’s behavior in detecting phishing. We discuss mechanisms to reduce overconfidence

What makes you over- or underconfident? Self-estimation in general knowledge test and its relation to personality traits and narcissism

The first aim of this study was to gain information on how people self-estimate their performance in general knowledge questionnaire during a personnel assessment day. That is, do personnel assessment participants demonstrate overconfidence, underconfidence, or are they self-aware of their performance? Secondly, this thesis aimed at finding which personality traits may predispose individuals to over- or underconfidence. This study was conducted in a real-life assessment center. Data was collected in 2019 by a Finnish HR-consulting company Psycon Corp. Sample collection occurred from March to April. Data collection was a part of a broader personnel selection procedure which aimed selection of candidates for expert and leader positions. Candidates participating personnel selection at Psycon were given a possibility to participate in the general knowledge questionnaire measuring over- and underconfidence specifically developed for this study. The final sample (n = 155) consisted of participants who filled in both the general knowledge questionnaire and personality inventories, including extraversion and neuroticism related sub-traits and narcissism. The studied personality traits were Narcissism, neuroticism related sub-trait Worrying & Self-criticism, and two extraversion related sub-traits of Need for Sociability & Company and Need to Lead & Decide. Firstly, the candidates participating personnel assessment day estimated their performance accurately and, on average, did not demonstrate over- or underconfidence. Secondly, both Need for Sociability & Company and Narcissism associated with overconfidence when simple correlations and separate regression models were studied. However, it is notable that only neuroticism was related to self-estimation of performance in the final model, including all the personality variables. In conclusion, individuals scoring high in neuroticism typically underestimated their performance. Furthermore, individuals high in extraversion subscale or narcissism tend to be overconfident, but the associations were modest and, in this study, insignificant when tested in regression analysis containing multiple variables. Therefore, neuroticism demands attention when studying the "risk-factors" of underconfidence. When considering the opposite pole, overconfidence, extraversion, and narcissism are potential contributors, but according to this study, their predictive ability is at best modest. Including self-estimation of performance, in other words, assessing biases towards under- or overconfidence, provides information of the potential thinking style a candidate views him-/her-self, how he/she embraces risks and what type of information he/she includes into evaluations before making decisions. Additionally, studying candidate’s over- and underconfidence may reveal aspects of optimism, self-view, self-efficacy, and self-awareness to mention a few possible explanations for under-/overconfidence. Methodologically this procedure brings incremental validity over and beyond traditional self-reports hence increases the predictive power of assessment battery. Assessing under- or overconfidence provides, therefore valuable information in the recruiting context. For example, neuroticism, coupled with the realistic self-estimation ability, may be a better fit to a position requiring high decision-making skills than neuroticism combined with underconfidence. Vice versa, individual high in narcissism (or extraversion) and with a realistic self-estimation ability may perform better in a position requiring risk management or high-quality decision making than an individual scoring high in narcissism (or extraversion) and overconfidence. Even though further studying is required to fully understand which factors lie under biased confidence and how these biases impact on work-related behavior assessment of over-/underconfidence provides interesting and additional information beyond self-reporting into personnel assessment battery

Climate scientists’ wide prediction intervals may be more likely but are perceived to be less certain

The use of interval forecasts allows climate scientists to issue predictions with high levels of certainty even for areas fraught with uncertainty, since wide intervals are objectively more likely to capture the truth than narrow intervals. However, wide intervals are also less informative about what the outcome will be than narrow intervals, implying a lack of knowledge or subjective uncertainty in the forecaster. In six experiments, we investigate how lay people perceive the (un)certainty associated with wide and narrow interval forecasts, and find that the preference for accuracy (seeing wide intervals as “objectively” certain) vs. informativeness (seeing wide intervals as indicating “subjective” uncertainty) is influenced by contextual cues (e.g., question formulation). Most importantly, we find that people more commonly and intuitively associate wide intervals with uncertainty than with certainty. Our research thus challenges the wisdom of using wide intervals to construct statements of high certainty in climate change reports

Exploring Financial Literacy and Overconfident Investor Behavior

This study examined the factors predicting overconfidence in U.S. investors and the relationship between overconfidence and seeking financial advice. This study adopted a quantitative research method using the 2018 NFCS Investor Survey data to explore the relationship between financial literacy and investor behavior in the U.S. stock market. Theories in financial literacy and overconfident behavior are combined to identify factors that predict overconfident behavior in U.S. investors. A logistic regression model was utilized to understand the relationship between financial literacy, demographics, and overconfident investor behavior. The results show a positive relationship between overconfident behavior and portfolio value, seeking financial advice, and conducting research activity. Results also showed that overconfidence is higher in male investors, younger investors, and investors with lower incomes. These findings are useful to individuals and corporations across several applications. Individuals can increase self-awareness regarding their own behaviors to identify certain biases, such as overconfidence, to help them avoid making large financial mistakes. Financial advisors can utilize these findings to become more aware of their clients that are likely to demonstrate overconfident behavior and help them mitigate these risks. Government entities can incorporate financial literacy programs that will establish baseline financial literacy competency in primary and secondary education programs

Three Experimental Accounting Studies

Diese Dissertation umfasst drei Studien. Die erste Studie untersucht Overprecision („Überpräzision“). Ich untersuche, wie Menschen Spannweitenschätzungen vornehmen. Teilnehmende müssen die Größe der Spannweite (Präzision) mit der Wahrscheinlichkeit, dass sie den wahren Wert einschließt (Richtigkeit) balancieren. Die Ergebnisse zeigen, dass Menschen inhärente individuelle Vorlieben für Präzision zu haben scheinen. Gleichzeitig werden vorhersagbar zusätzliche Informationen genutzt, um die Richtigkeit der Schätzungen zu erhöhen. Dafür wird entweder Präzision geopfert, oder die Spannweitenschätzung insgesamt verschoben. Die Richtigkeit der Schätzung wird jedoch nicht maximiert, sondern ein Teil für höhere Präzision aufgegeben. Die zweite Studie untersucht, wie sich die Übersetzung von Finanzberichterstattung auf die Wahrnehmung einer Firma als attraktives Investment auswirkt. Sie beleuchtet drei verschiedene Kanäle: Lesbarkeit, Stimmung, und Präzision der Veröffentlichung. In einem Umfrageexperiment lesen Kleinanleger deutsche und englische Prognoseberichte deutscher Firmen. Die Ergebnisse zeigen, dass die deutschen Berichte als besser lesbar wahrgenommen werden. Im Gegensatz zu vorheriger Literatur ist die Lesbarkeit nicht mit einer höheren Investmentattraktivität korreliert. Allein die Stimmung des Textes zeigt eine Korrelation mit höherer Attraktivität. Die dritte Studie untersucht, wie das Angebot von formativen Onlinetests die Leistungen von Studierenden in der Klausur beeinflusst. Sie untersucht zudem, ob die Leistung sich unterscheidet, je nachdem ob die Studierenden zeitlich begrenzten Zugang zu den Tests haben oder ob sie jederzeit auf die Tests zugreifen können. Ein Experiment, welches es ermöglicht den kausalen Intention-to-treat-Effekt zu bestimmen, zeigt, dass die formativen Onlinetests die Studienleistung erhöhen können, allerdings nur für Studierende, welche sich nicht freiwillig für die Tests gemeldet hatten und in der kontinuierlichen Lerngruppe waren.This dissertation comprises three papers. The first study examines overprecision. I examine how people provide range estimates, a challenging task that requires people to balance the width of the range (i.e., its precision) with the probability of the range covering the true value (i.e., accuracy). I find that people appear to have inherent individual preferences for a certain level of precision. At the same time, they appear to predictably incorporate additional information in order to increase accuracy by either sacrificing precision or shifting their ranges altogether. Still, they do not seem to maximise accuracy, but are willing to expend some of it to provide more precise estimates. The second study examines how the translation of financial disclosures changes investors’ perceptions of firms as an attractive investment. It examines three possible channels: readability, tone, and precision of the underlying disclosure. In a survey experiment, retail investors read forecast reports of German firms, provided in German and English. The findings indicate that the German versions are easier to read. Contrary to prior literature, the easier readability does not translate into higher investment attractiveness. Solely tone appears to be correlated with investment attractiveness. The third study analyses how offering formative online assessments influences student performance in the final exam. It further examines whether students perform differently depending on whether they have time-restricted access to the assessments, or whether they can access the assessments at any time. An experiment which allows for the identification of the causal intention-to-treat effect shows that offering formative online assessments can enhance student performance, but only for students who do not opt for taking the test voluntarily and who are in a continuous learning environment

Overconfidence is universal? Elicitation of genuine overconfidence (EGO) procedure reveals systematic differences across domain, task knowledge, and incentives in four populations

Overconfidence is sometimes assumed to be a human universal, but there remains a dearth of data systematically measuring overconfidence across populations and contexts. Moreover, cross-cultural experiments often fail to distinguish between placement and precision and worse still, often compare population-mean placement estimates rather than individual performance subtracted from placement. Here we introduce a procedure for concurrently capturing both placement and precision at an individual level based on individual performance: The Elicitation of Genuine Overconfidence (EGO) procedure. We conducted experiments using the EGO procedure, manipulating domain, task knowledge, and incentives across four populations—Japanese, Hong Kong Chinese, Euro Canadians, and East Asian Canadians. We find that previous measures of population-level overconfidence may have been misleading; rather than universal, overconfidence is highly context dependent. Our results reveal cross-cultural differences in sensitivity to incentives and differences in overconfidence strategies, with underconfidence, accuracy, and overconfidence. Comparing sexes, we find inconsistent results for overplacement, but that males are consistently more confident in their placement. These findings have implications for our understanding of the adaptive value of overconfidence and its role in explaining population-level and individual-level differences in economic and psychological behavior