Self-Supervised Learning with an Information Maximization Criterion

Self-supervised learning allows AI systems to learn effective representations from large amounts of data using tasks that do not require costly labeling. Mode collapse, i.e., the model producing identical representations for all inputs, is a central problem to many self-supervised learning approaches, making self-supervised tasks, such as matching distorted variants of the inputs, ineffective. In this article, we argue that a straightforward application of information maximization among alternative latent representations of the same input naturally solves the collapse problem and achieves competitive empirical results. We propose a self-supervised learning method, CorInfoMax, that uses a second-order statistics-based mutual information measure that reflects the level of correlation among its arguments. Maximizing this correlative information measure between alternative representations of the same input serves two purposes: (1) it avoids the collapse problem by generating feature vectors with non-degenerate covariances; (2) it establishes relevance among alternative representations by increasing the linear dependence among them. An approximation of the proposed information maximization objective simplifies to a Euclidean distance-based objective function regularized by the log-determinant of the feature covariance matrix. The regularization term acts as a natural barrier against feature space degeneracy. Consequently, beyond avoiding complete output collapse to a single point, the proposed approach also prevents dimensional collapse by encouraging the spread of information across the whole feature space. Numerical experiments demonstrate that CorInfoMax achieves better or competitive performance results relative to the state-of-the-art SSL approaches

Semantics-based information extraction for detecting economic events

As today's financial markets are sensitive to breaking news on economic events, accurate and timely automatic identification of events in news items is crucial. Unstructured news items originating from many heterogeneous sources have to be mined in order to extract knowledge useful for guiding decision making processes. Hence, we propose the Semantics-Based Pipeline for Economic Event Detection (SPEED), focusing on extracting financial events from news articles and annotating these with meta-data at a speed that enables real-time use. In our implementation, we use some components of an existing framework as well as new components, e.g., a high-performance Ontology Gazetteer, a Word Group Look-Up component, a Word Sense Disambiguator, and components for detecting economic events. Through their interaction with a domain-specific ontology, our novel, semantically enabled components constitute a feedback loop which fosters future reuse of acquired knowledge in the event detection process

All downhill from the PhD? The typical impact trajectory of US academic careers

© 2020 The Authors. Published by MIT Press. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1162/qss_a_00072.Within academia, mature researchers tend to be more senior, but do they also tend to write higher impact articles? This article assesses long-term publishing (16+ years) United States (US) researchers, contrasting them with shorter-term publishing researchers (1, 6 or 10 years). A long-term US researcher is operationalised as having a first Scopus-indexed journal article in exactly 2001 and one in 2016-2019, with US main affiliations in their first and last articles. Researchers publishing in large teams (11+ authors) were excluded. The average field and year normalised citation impact of long- and shorter-term US researchers’ journal articles decreases over time relative to the national average, with especially large falls to the last articles published that may be at least partly due to a decline in self-citations. In many cases researchers start by publishing above US average citation impact research and end by publishing below US average citation impact research. Thus, research managers should not assume that senior researchers will usually write the highest impact papers

An Economic Analysis of Color-Blind Affirmative Action

This article offers an economic analysis of color-blind alternatives to conventional affirmative action policies in higher education, focusing on efficiency issues. When the distribution of applicants' traits is fixed (i.e., in the short-run) color blindness leads colleges to shift weight from academic traits that predict performance to social traits that proxy for race. Using data on matriculates at several selective colleges and universities, we estimate that the short-run efficiency cost of "blind" relative to "sighted" affirmative action is comparable to the cost colleges would incur were they to ignore standardized test scores when deciding on admissions. We then build a model of applicant competition with endogenous effort in order to study long-run incentive effects. We show that, compared to the sighted alternative, color-blind affirmative action is inefficient because it flattens the function mapping effort into a probability of admission in the model's equilibrium. "Implementing race-neutral programs will help educational institutions minimize litigation risks they currently face… . If we are persistent in implementing race-neutral approaches, the end result will be to fulfill the great words of Dr. Martin Luther King Jr., who dreamed of the day that all children will be judged by the content of their character and not the color of their skin."--US Department of Education. Race-Neutral Alternatives in Postsecondary Education: Innovative Approaches to Diversity , Washington, D.C.: U.S. Department of Education, Office of Civil Rights March 2003, pp. 7, 40. The Author 2007. Published by Oxford University Press on behalf of Yale University. All rights reserved. For permissions, please email: [email protected], Oxford University Press.

A Quick Tour of Word Sense Disambiguation, Induction and Related Approaches

Abstract. Word Sense Disambiguation (WSD) and Word Sense Induction (WSI) are two fundamental tasks in Natural Language Processing (NLP), i.e., those of, respectively, automatically assigning meaning to words in context from a predefined sense inventory and discovering senses from text for a given input word. The two tasks have generally been hard to perform with high accuracy. However, today innovations in approach to WSD and WSI are promising to open up many interesting new horizons in NLP and Information Retrieval applications. This paper is a quick tour on how to start doing research in this exciting field and suggests the hottest topics to focus on. Keywords: computational lexical semantics, Word Sense Disambiguation, Word Sense Induction, text understanding