Deep Weighted Averaging Classifiers

Recent advances in deep learning have achieved impressive gains in classification accuracy on a variety of types of data, including images and text. Despite these gains, however, concerns have been raised about the calibration, robustness, and interpretability of these models. In this paper we propose a simple way to modify any conventional deep architecture to automatically provide more transparent explanations for classification decisions, as well as an intuitive notion of the credibility of each prediction. Specifically, we draw on ideas from nonparametric kernel regression, and propose to predict labels based on a weighted sum of training instances, where the weights are determined by distance in a learned instance-embedding space. Working within the framework of conformal methods, we propose a new measure of nonconformity suggested by our model, and experimentally validate the accompanying theoretical expectations, demonstrating improved transparency, controlled error rates, and robustness to out-of-domain data, without compromising on accuracy or calibration.Comment: 13 pages, 8 figures, 5 tables, added DOI and updated to meet ACM formatting requirements, In Proceedings of FAT* (2019

Uncertainty in Natural Language Generation: From Theory to Applications

Recent advances of powerful Language Models have allowed Natural Language Generation (NLG) to emerge as an important technology that can not only perform traditional tasks like summarisation or translation, but also serve as a natural language interface to a variety of applications. As such, it is crucial that NLG systems are trustworthy and reliable, for example by indicating when they are likely to be wrong; and supporting multiple views, backgrounds and writing styles -- reflecting diverse human sub-populations. In this paper, we argue that a principled treatment of uncertainty can assist in creating systems and evaluation protocols better aligned with these goals. We first present the fundamental theory, frameworks and vocabulary required to represent uncertainty. We then characterise the main sources of uncertainty in NLG from a linguistic perspective, and propose a two-dimensional taxonomy that is more informative and faithful than the popular aleatoric/epistemic dichotomy. Finally, we move from theory to applications and highlight exciting research directions that exploit uncertainty to power decoding, controllable generation, self-assessment, selective answering, active learning and more

Deformation Quantization: Twenty Years After

We first review the historical developments, both in physics and in mathematics, that preceded (and in some sense provided the background of) deformation quantization. Then we describe the birth of the latter theory and its evolution in the past twenty years, insisting on the main conceptual developments and keeping here as much as possible on the physical side. For the physical part the accent is put on its relations to, and relevance for, "conventional" physics. For the mathematical part we concentrate on the questions of existence and equivalence, including most recent developments for general Poisson manifolds; we touch also noncommutative geometry and index theorems, and relations with group theory, including quantum groups. An extensive (though very incomplete) bibliography is appended and includes background mathematical literature.Comment: 39 pages; to be published with AIP Press in Proceedings of the 1998 Lodz conference "Particles, Fields and Gravitation". LaTeX (compatibility mode) with aipproc styl

Learning Neural Graph Representations in Non-Euclidean Geometries

The success of Deep Learning methods is heavily dependent on the choice of the data representation. For that reason, much of the actual effort goes into Representation Learning, which seeks to design preprocessing pipelines and data transformations that can support effective learning algorithms. The aim of Representation Learning is to facilitate the task of extracting useful information for classifiers and other predictor models. In this regard, graphs arise as a convenient data structure that serves as an intermediary representation in a wide range of problems. The predominant approach to work with graphs has been to embed them in an Euclidean space, due to the power and simplicity of this geometry. Nevertheless, data in many domains exhibit non-Euclidean features, making embeddings into Riemannian manifolds with a richer structure necessary. The choice of a metric space where to embed the data imposes a geometric inductive bias, with a direct impact on the performance of the models. This thesis is about learning neural graph representations in non-Euclidean geometries and showcasing their applicability in different downstream tasks. We introduce a toolkit formed by different graph metrics with the goal of characterizing the topology of the data. In that way, we can choose a suitable target embedding space aligned to the shape of the dataset. By virtue of the geometric inductive bias provided by the structure of the non-Euclidean manifolds, neural models can achieve higher performances with a reduced parameter footprint. As a first step, we study graphs with hierarchical structures. We develop different techniques to derive hierarchical graphs from large label inventories. Noticing the capacity of hyperbolic spaces to represent tree-like arrangements, we incorporate this information into an NLP model through hyperbolic graph embeddings and showcase the higher performance that they enable. Second, we tackle the question of how to learn hierarchical representations suited for different downstream tasks. We introduce a model that jointly learns task-specific graph embeddings from a label inventory and performs classification in hyperbolic space. The model achieves state-of-the-art results on very fine-grained labels, with a remarkable reduction of the parameter size. Next, we move to matrix manifolds to work on graphs with diverse structures and properties. We propose a general framework to implement the mathematical tools required to learn graph embeddings on symmetric spaces. These spaces are of particular interest given that they have a compound geometry that simultaneously contains Euclidean as well as hyperbolic subspaces, allowing them to automatically adapt to dissimilar features in the graph. We demonstrate a concrete implementation of the framework on Siegel spaces, showcasing their versatility on different tasks. Finally, we focus on multi-relational graphs. We devise the means to translate Euclidean and hyperbolic multi-relational graph embedding models into the space of symmetric positive definite (SPD) matrices. To do so we develop gyrocalculus in this geometry and integrate it with the aforementioned framework

Gauge Theories of Gravitation

During the last five decades, gravity, as one of the fundamental forces of nature, has been formulated as a gauge theory of the Weyl-Cartan-Yang-Mills type. The present text offers commentaries on the articles from the most prominent proponents of the theory. In the early 1960s, the gauge idea was successfully applied to the Poincar\'e group of spacetime symmetries and to the related conserved energy-momentum and angular momentum currents. The resulting theory, the Poincar\'e gauge theory, encompasses Einstein's general relativity as well as the teleparallel theory of gravity as subcases. The spacetime structure is enriched by Cartan's torsion, and the new theory can accommodate fermionic matter and its spin in a perfectly natural way. This guided tour starts from special relativity and leads, in its first part, to general relativity and its gauge type extensions \`a la Weyl and Cartan. Subsequent stopping points are the theories of Yang-Mills and Utiyama and, as a particular vantage point, the theory of Sciama and Kibble. Later, the Poincar\'e gauge theory and its generalizations are explored and special topics, such as its Hamiltonian formulation and exact solutions, are studied. This guide to the literature on classical gauge theories of gravity is intended to be a stimulating introduction to the subject.Comment: 169 pages, pdf file, v3: extended to a guide to the literature on classical gauge theories of gravit