Essays on Natural Language Processing and Central Banking.

Humans generally interact, communicate, and form social structures using natural language. Due to the high dimensionality of language, much of the wealth of information from these interactions has been barred from the economic profession. However, recent technological advancements lead to increasing use of text as an underlying datasource in economic and financial applications. This trend has been further accelerated by Nobel laureate Robert J. Shiller's presidential address to the American Economic Association "Narrative Economics", in which he argues for more elaboration on narratives - stories that affect individual decisions and collective actions - by the economic scientific community. Addressing this gap in the literature, research has been published utilizing textual information to quantify latent variables such as uncertainty, forecasting macroeconomic variables in real time, and asset price predictions. In conjunction with the rise of natural language processing applications, there has been a shift in perspective on monetary policy with regards to central bank transparency and communication. Transitioning from the presumption that monetary policy is limited to interest rate actions, communication has advanced to become a key tool in the central banker's toolbox. Ever since, words are used to anchor expectations and self-enforce the central banks' desired equilibrium path. As a result, research on monetary policy has been relentless in the pursuit of adopting novel techniques as well as incorporating new unstructured data sources such as news-articles, press conference statements, and speeches. This string of literature is regularly complemented by an extension of the traditional empirical toolbox, borrowing novel techniques from the field of machine learning. The here presented cumulative dissertation consists of four essays that touch on all these fields, namely text as data, monetary policy, and machine learning. My primary focus is on the European Central Bank (ECB), but the methodology and ideas can be extended to other central banks as well. Throughout this thesis, textual information is incorporated from different data sources, analyzed using different techniques in order to approximate different latent variables. As a result, text is employed as a dependent variable at times and as an independent variable at other times. Specifically, the first essay leverages the relative frequency of terms used in ECB press statements as anecdotal evidence for the diversity of the central banks' communication with regard to their topics, whereas the second essay counts positive and negative terms in speeches to approximate the latent variable of central bank loss. The third essay examines the impact of linguistic complexity on financial market participants by conducting a readability test on the ECB's introduction statements, and the final essay dives into computational linguistics to develop a novel central bank-specific language model for better quantifying monetary policy communication. The following is a brief summary of the four essays included in this thesis. My first essay analyzes rule-based monetary policy in the euro area before and after the financial crisis. Jonas Gross and I argue that the environment in which policymakers operate is far more complex than traditional model-based analysis of policy rules permits. We complement this view with evidence from ECB press conferences, demonstrating that the central bank discusses a wide range of topics beyond the traditional Taylor-rule variables. Since each variable has the potential to be relevant in understanding the central bank's reaction function, we combine a literature review with natural language processing to identify a set of potential determinants. The traditional approach of selecting a single interest rate response function is then contrasted by applying a Bayesian model averaging approach to these determinants. We account for model uncertainty by including a large number of determinants and estimating a total of 33.000 different model combinations. Our results suggest that in contrast to the ongoing criticism, the ECB primarily reacts to inflation in its interest rate decision. In fact, our analysis finds that inflation is a significant variable in almost all of the examined model combinations. Furthermore, we find that the ECB reacts to changes in economic activity determinants such as unemployment and production as well. These economic activity indicators were a priority for the ECB prior to the financial crisis but have since declined in relevance, suggesting that inflation is the sole driver of monetary policy decisions in the post-crisis period. Finally, we assess our findings with textual evidence from the ECB press conferences, where, in accordance with the previous results, we find the same shift. My second essay focuses on the ECB's objective itself, quantifying the central bank's satisfaction with current economic conditions through textual analysis. By maximizing an implied objective function, the ECB is assumed to pursue inflation targeting with a subordinate focus on supporting the general economic policy of the European Union. I compute the central bank's sentiment using the ECB's public communication by counting the number of positive and negative words in speeches, allowing me to quantify the objective. Assuming a typical functional form for the objective allows me to estimate the optimal levels with respect to inflation and economic activity, i.e. the bliss points in which the central banks communication is the most positive. Using a dictionary approach to estimate the sentiment index yields several interesting results. The most surprising is, unquestionable, a concave inflation objective with an implied inflation target beyond the banks' mandate and best described as 'above, but close to 2%'. Deviations from this bliss point appear to lower the satisfaction, and hence the optimistic language in speeches. With respect to the subordinate objective, I find a convex objective towards output growth and a linear objective towards the unemployment rate. Furthermore, my results suggest that deviations from the primary objective, the inflation rate, appear to have no greater effect on the speeches' language than deviations from either of the subordinate objectives. In fact, in contrast to inflation, both output and unemployment are consistently significant variables. Finally, contrary to findings in the United States, financial market conditions have no significant influence on the ECB's sentiment. In the third essay, Bernd Hayo, Kai Henseler, Marc Steffen Rapp, and I investigate the impact of central bank communication on financial markets. We are particularly interested in the communication's complexity and how it affects financial market trading. To examine this relationship empirically, we employ high-frequency data from European stock index futures during the introductory statement of the ECB's press conferences. A readability test on the introductory statement during the press conference determines the statements' linguistic complexity. In conjunction with the central banks' unique communication design, we are able to separate the effect of verbal complexity on trading during the introductory statements and the subsequent Q&A session. Our sample contains announcements of novel UMPM, enabling us to investigate whether the content of the introductory statements interacts with the reaction of traders to its linguistic complexity. We find that the Q&A sessions are - in terms of linguistic complexity - less complex and thus more comprehensible. When UMPM are announced, contemporaneous trading volumes are negatively correlated with complexity, resulting in a temporal shift of trading towards the less complex Q&A session. This shift is first indication that financial markets respond to linguistic complexity in a context-specific manner. This line of reasoning is strengthened further by the observation that events containing UMPM are less similar in terms of wording to previous statements. As a result, we believe that financial market traders are underreacting to novel complex information in introductory statements regarding UMPM. The subsequent discussion and clarification of the cognitively costly content during the Q&A session mitigates this effect, shifting trading from the introductory statement phase to the Q&A phase of the ECB's press conference. The final essay concerns the quantification of central bank communication, i.e. it explores how text in monetary policy can be effectively summarised and analysed. Martin Baumgärtner and I propose a novel language model, build on machine learning, as a tool to quantify central bankers qualitative information. The necessity and feasibility of measuring central bank communication in this manner stems from two major developments in the fields of monetary policy and machine learning over the last two decades. On the one hand, central bankers' communication, as well as its analysis, has increased substantially. This progress necessitates some form of quantification of the qualitative components, a research topic dominated by dictionary approaches. On the other hand, advances at the intersection of linguistics and computer science enabled the use of machine learning to train language models capable of adequately capturing the languages multidimensionality and context-dependence. The resulting models are regularly open source. However, the technical jargon of central bankers renders them generally unsuitable for use in the field. This essay aims to apply computational linguistics research to monetary policy by developing a language model exclusively trained on central bank communication. To accomplish this, we gather a large and diverse text corpus, which we use to compare a number of state-of-the-art machine learning algorithms. Choosing the most promising, we develop a central bank specific language model. Several applications are presented to showcase the broad applicability of our language model. First, we propose a novel technique for comparing central banks, affirming that similarity is driven by mutual objectives. Next, we construct a time-series index that reflects the ECB's willingness to act as a lender of last resort. The index suggests that communication similar to Mario Draghi's 'whatever it takes' speech can calm financial markets during times of high uncertainty. The third application emphasizes the presence of prejudices even in central bankers' technical language. We demonstrate how social patterns, such as occupational gender distribution, are reflected in their communication. The final application is a forecasting exercise that suggests that speeches may be more accurate predictors than previous research suggests

Biological Hypercomputation and Degrees of Freedom

This chapter presents the idea of biological hypercomputation (BH) and discusses how and why it entails degrees of freedom. Crossing a biological and computational point of view, the claim is made that living beings cannot be considered as machines in any sense of the word, the arguments are provided, and the consequence is drawn: the complexity of life is the very process by which living beings gain degrees of freedom. The leading thread for the analysis here is the relationship between matter, energy and information

The Simulator: Understanding Adaptive Sampling in the Moderate-Confidence Regime

We propose a novel technique for analyzing adaptive sampling called the {\em Simulator}. Our approach differs from the existing methods by considering not how much information could be gathered by any fixed sampling strategy, but how difficult it is to distinguish a good sampling strategy from a bad one given the limited amount of data collected up to any given time. This change of perspective allows us to match the strength of both Fano and change-of-measure techniques, without succumbing to the limitations of either method. For concreteness, we apply our techniques to a structured multi-arm bandit problem in the fixed-confidence pure exploration setting, where we show that the constraints on the means imply a substantial gap between the moderate-confidence sample complexity, and the asymptotic sample complexity as $\delta \to 0$ found in the literature. We also prove the first instance-based lower bounds for the top-k problem which incorporate the appropriate log-factors. Moreover, our lower bounds zero-in on the number of times each \emph{individual} arm needs to be pulled, uncovering new phenomena which are drowned out in the aggregate sample complexity. Our new analysis inspires a simple and near-optimal algorithm for the best-arm and top-k identification, the first {\em practical} algorithm of its kind for the latter problem which removes extraneous log factors, and outperforms the state-of-the-art in experiments

A Neural Approach to Discourse Relation Signal Detection

Previous data-driven work investigating the types and distributions of discourse relation signals, including discourse markers such as 'however' or phrases such as 'as a result' has focused on the relative frequencies of signal words within and outside text from each discourse relation. Such approaches do not allow us to quantify the signaling strength of individual instances of a signal on a scale (e.g. more or less discourse-relevant instances of 'and'), to assess the distribution of ambiguity for signals, or to identify words that hinder discourse relation identification in context ('anti-signals' or 'distractors'). In this paper we present a data-driven approach to signal detection using a distantly supervised neural network and develop a metric, Δs (or 'delta-softmax'), to quantify signaling strength. Ranging between -1 and 1 and relying on recent advances in contextualized words embeddings, the metric represents each word's positive or negative contribution to the identifiability of a relation in specific instances in context. Based on an English corpus annotated for discourse relations using Rhetorical Structure Theory and signal type annotations anchored to specific tokens, our analysis examines the reliability of the metric, the places where it overlaps with and differs from human judgments, and the implications for identifying features that neural models may need in order to perform better on automatic discourse relation classification

The Simplest Inflationary Potentials

Inflation is a highly favoured theory for the early Universe. It is compatible with current observations of the cosmic microwave background and large scale structure and is a driver in the quest to detect primordial gravitational waves. It is also, given the current quality of the data, highly under-determined with a large number of candidate implementations. We use a new method in symbolic regression to generate all possible simple scalar field potentials for one of two possible basis sets of operators. Treating these as single-field, slow-roll inflationary models we then score them with an information-theoretic metric ("minimum description length") that quantifies their efficiency in compressing the information in the Planck data. We explore two possible priors on the parameter space of potentials, one related to the functions' structural complexity and one that uses a Katz back-off language model to prefer functions that may be theoretically motivated. This enables us to identify the inflaton potentials that optimally balance simplicity with accuracy at explaining the Planck data, which may subsequently find theoretical motivation. Our exploratory study opens the door to extraction of fundamental physics directly from data, and may be augmented with more refined theoretical priors in the quest for a complete understanding of the early Universe.Comment: 13+4 pages, 4 figures; submitted to Physical Review

Dagstuhl News January - December 2011

"Dagstuhl News" is a publication edited especially for the members of the Foundation "Informatikzentrum Schloss Dagstuhl" to thank them for their support. The News give a summary of the scientific work being done in Dagstuhl. Each Dagstuhl Seminar is presented by a small abstract describing the contents and scientific highlights of the seminar as well as the perspectives or challenges of the research topic

Ibn Jaldūn en su léxico subjetivo. La constelación emocional de un intelectual en transición

This study is a linguistic analysis of the subjective lexicon used by the North African thinker ‘Abd al-Raḥmān Ibn Khaldūn (d. 1406) in the epilogue of his historical work Kitāb al-‘Ibar, known as Ta‘rīf, in which he writes a biography of himself. The paper outlines the usefulness of building an analytical schema of the linguistic phenomena in which the first person emerges, especially in self-referencing, actions, emotions, and evaluations, in order to fully grasp the complexity of Ibn Khaldūn’s self-expression. A hybrid quantitative and qualitative methodology helps to extract, organize, analyze, and interpret his use of language to express himself in relation to the narration of his life’s journey and his psychological evolution. The data show an intimate expression of a premodern Arabic subject, leading Ta‘rīf to be considered an evolutionary link to the modern autobiography, which takes on great significance in the process of cultural transition of the Islamic West towards European modernity.El presente estudio se centra en el análisis lingüístico del léxico subjetivo utilizado por el pensador norteafricano ‘Abd al-Raḥmān Ibn Jaldūn (m. 1406) en el epílogo de su obra histórica Kitāb al-‘Ibar, conocida como Ta‘rīf, donde se biografía a sí mismo. El objetivo es esbozar la utilidad de construir un esquema analítico de los fenómenos lingüísticos en los que emerge la primera persona, especialmente en la autorreferencia, las acciones, las emociones y las evaluaciones, para captar la complejidad de la autoexpresión de Ibn Jaldūn. Una metodología híbrida cuantitativa y cualitativa ayuda a extraer, organizar, analizar e interpretar su uso del lenguaje para expresarse en relación con la narración de su trayectoria vital y su evolución psicológica. Los datos muestran una expresión íntima de un sujeto árabe premoderno, lo que lleva a considerar el Ta‘rīf como un eslabón evolutivo de la autobiografía moderna, que adquiere una gran importancia dentro del proceso de transición cultural del Occidente islámico hacia la modernidad europea.CONICYT FONDECYT/POSTDOCTORADO/Project No 319019

The Computational Lens: from Quantum Physics to Neuroscience

Two transformative waves of computing have redefined the way we approach science. The first wave came with the birth of the digital computer, which enabled scientists to numerically simulate their models and analyze massive datasets. This technological breakthrough led to the emergence of many sub-disciplines bearing the prefix "computational" in their names. Currently, we are in the midst of the second wave, marked by the remarkable advancements in artificial intelligence. From predicting protein structures to classifying galaxies, the scope of its applications is vast, and there can only be more awaiting us on the horizon. While these two waves influence scientific methodology at the instrumental level, in this dissertation, I will present the computational lens in science, aiming at the conceptual level. Specifically, the central thesis posits that computation serves as a convenient and mechanistic language for understanding and analyzing information processing systems, offering the advantages of composability and modularity. This dissertation begins with an illustration of the blueprint of the computational lens, supported by a review of relevant previous work. Subsequently, I will present my own works in quantum physics and neuroscience as concrete examples. In the concluding chapter, I will contemplate the potential of applying the computational lens across various scientific fields, in a way that can provide significant domain insights, and discuss potential future directions.Comment: PhD thesis, Harvard University, Cambridge, Massachusetts, USA. 2023. Some chapters report joint wor