Entity centric neural models for natural language processing

This thesis explores how to enhance natural language understanding by incorporating entity information into neural network models. It tackles three key questions:1. Leveraging entities for understanding tasks: This work introduces Entity-GCN, a model that performs multi-step reasoning on a graph where nodes represent entity mentions and edges represent relationships. This method achieved state-of-the-art results on a multi-document question-answering dataset.2. Identifying and disambiguating entities using large language models: This research proposes a novel system that retrieves entities by generating their names token-by-token, overcoming limitations of traditional methods and significantly reducing memory footprint. This approach is also extended to a multilingual setting and further optimized for efficiency.3. Interpreting and controlling entity knowledge within models: This thesis presents a post-hoc interpretation technique to analyze how decisions are made across layers in neural models, allowing for visualization and analysis of knowledge representation. Additionally, a method for editing factual knowledge about entities is proposed, enabling correction of model predictions without costly retraining

Entity centric neural models for natural language processing

This thesis explores how to enhance natural language understanding by incorporating entity information into neural network models. It tackles three key questions:1. Leveraging entities for understanding tasks: This work introduces Entity-GCN, a model that performs multi-step reasoning on a graph where nodes represent entity mentions and edges represent relationships. This method achieved state-of-the-art results on a multi-document question-answering dataset.2. Identifying and disambiguating entities using large language models: This research proposes a novel system that retrieves entities by generating their names token-by-token, overcoming limitations of traditional methods and significantly reducing memory footprint. This approach is also extended to a multilingual setting and further optimized for efficiency.3. Interpreting and controlling entity knowledge within models: This thesis presents a post-hoc interpretation technique to analyze how decisions are made across layers in neural models, allowing for visualization and analysis of knowledge representation. Additionally, a method for editing factual knowledge about entities is proposed, enabling correction of model predictions without costly retraining

Book Reviews

The Variational Auto-Encoder (VAE) is one of the most used unsupervised machine learning models. But although the default choice of a Gaussian distribution for both the prior and posterior represents a mathematically convenient distribution often leading to competitive results, we show that this parameterization fails to model data with a latent hyperspherical structure. To address this issue we propose using a von Mises-Fisher (vMF) distribution instead, leading to a hyperspherical latent space. Through a series of experiments we show how such a hyperspherical VAE, or $\mathcal{S}$-VAE, is more suitable for capturing data with a hyperspherical latent structure, while outperforming a normal, $\mathcal{N}$-VAE, in low dimensions on other data types.Comment: GitHub repository: http://github.com/nicola-decao/s-vae-tf, Blogpost: https://nicola-decao.github.io/s-va

Editing Factual Knowledge in Language Models

The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or become obsolete over time. We present KNOWLEDGEEDITOR, a method which can be used to edit this knowledge and, thus, fix 'bugs' or unexpected predictions without the need for expensive retraining or fine-tuning. Besides being computationally efficient, KNOWLEDGEEDITOR does not require any modifications in LM pre-training (e.g., the use of meta-learning). In our approach, we train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the trained hyper-network is then used to predict the weight update at test time. We show KNOWLEDGEEDITOR's efficacy with two popular architectures and knowledge-intensive tasks: i) a BERT model fine-tuned for fact-checking, and ii) a sequence-to-sequence BART model for question answering. With our method, changing a prediction on the specific wording of a query tends to result in a consistent change in predictions also for its paraphrases. We show that this can be further encouraged by exploiting (e.g., automatically-generated) paraphrases during training. Interestingly, our hyper-network can be regarded as a 'probe' revealing which components need to be changed to manipulate factual knowledge; our analysis shows that the updates tend to be concentrated on a small subset of components.</p

The quantum inflaton, primordial perturbations and CMB fluctuations

We compute the primordial scalar, vector and tensor metric perturbations arising from quantum field inflation. Quantum field inflation takes into account the nonperturbative quantum dynamics of the inflaton consistently coupled to the dynamics of the (classical) cosmological metric. For chaotic inflation, the quantum treatment avoids the unnatural requirements of an initial state with all the energy in the zero mode. For new inflation it allows a consistent treatment of the explosive particle production due to spinodal instabilities. Quantum field inflation (under conditions that are the quantum analog of slow roll) leads, upon evolution, to the formation of a condensate starting a regime of effective classical inflation. We compute the primordial perturbations taking the dominant quantum effects into account. The results for the scalar, vector and tensor primordial perturbations are expressed in terms of the classical inflation results. For a N-component field in a O(N) symmetric model, adiabatic fluctuations dominate while isocurvature or entropy fluctuations are negligible. The results agree with the current WMAP observations and predict corrections to the power spectrum in classical inflation.Such corrections are estimated to be of the order of m^2/[N H^2] where m is the inflaton mass and H the Hubble constant at horizon crossing. This turns to be about 4% for the cosmologically relevant scales. This quantum field treatment of inflation provides the foundations to the classical inflation and permits to compute quantum corrections to it.Comment: 23 pages, no figures. Improved version to appear in Phys. Rev.

Noise-robust method for image segmentation

Segmentation of noisy images is one of the most challenging problems in image analysis and any improvement of segmentation methods can highly influence the performance of many image processing applications. In automated image segmentation, the fuzzy c-means (FCM) clustering has been widely used because of its ability to model uncertainty within the data, applicability to multi-modal data and fairly robust behaviour. However, the standard FCM algorithm does not consider any information about the spatial linage context and is highly sensitive to noise and other imaging artefacts. Considering above mentioned problems, we developed a new FCM-based approach for the noise-robust fuzzy clustering and we present it in this paper. In this new iterative algorithm we incorporated both spatial and feature space information into the similarity measure and the membership function. We considered that spatial information depends on the relative location and features of the neighbouring pixels. The performance of the proposed algorithm is tested on synthetic image with different noise levels and real images. Experimental quantitative and qualitative segmentation results show that our method efficiently preserves the homogeneity of the regions and is more robust to noise than other FCM-based methods

QED Renormalization Given in A Mass-Dependent Subtraction and The Renormalization Group Approach

The QED renormalization is restudied by using a mass-dependent subtraction which is performed at a time-like renormalization point. The subtraction exactly respects necessary physical and mathematical requirements such as the gauge symmetry, the Lorentz- invariance and the mathematical convergence. Therefore, the renormalized results derived in the subtraction scheme are faithful and have no ambiguity. Especially, it is proved that the solution of the renormalization group equation satisfied by a renormalized wave function, propagator or vertex can be fixed by applying the renormalization boundary condition and, thus, an exact S-matrix element can be expressed in the form as written in the tree diagram approximation provided that the coupling constant and the fermion mass are replaced by their effective ones. In the one-loop approximation, the effective coupling constant and the effective fermion mass obtained by solving their renormalization group equations are given in rigorous and explicit expressions which are suitable in the whole range of distance and exhibit physically reasonable asymptotic behaviors.Comment: 29 pages, 4 figure

Dense loops, supersymmetry, and Goldstone phases in two dimensions

Loop models in two dimensions can be related to O(N) models. The low-temperature dense-loops phase of such a model, or of its reformulation using a supergroup as symmetry, can have a Goldstone broken-symmetry phase for N<2. We argue that this phase is generic for -2< N <2 when crossings of loops are allowed, and distinct from the model of non-crossing dense loops first studied by Nienhuis [Phys. Rev. Lett. 49, 1062 (1982)]. Our arguments are supported by our numerical results, and by a lattice model solved exactly by Martins et al. [Phys. Rev. Lett. 81, 504 (1998)].Comment: RevTeX, 5 pages, 3 postscript figure

Out of Equilibrium Non-perturbative Quantum Field Dynamics in Homogeneous External Fields

The quantum dynamics of the symmetry broken lambda (Phi^2)^2 scalar field theory in the presence of an homogeneous external field is investigated in the large N limit. We choose as initial state the ground state for a constant external field J .The sign of the external field is suddenly flipped from J to - J at a given time and the subsequent quantum dynamics calculated. Spinodal instabilities and parametric resonances produce large quantum fluctuations in the field components transverse to the external field. This allows the order parameter to turn around the maximum of the potential for intermediate times. Subsequently, the order parameter starts to oscillate near the global minimum for external field - J, entering a novel quasi-periodic regime.Comment: LaTex, 30 pages, 12 .ps figures, improved version to appear in Phys Rev

Bivalve Haemocyte Subpopulations: A Review

Bivalve molluscs stand out for their ecological success and their key role in the functioning of aquatic ecosystems, while also constituting a very valuable commercial resource. Both ecological success and production of bivalves depend on their effective immune defence function, in which haemocytes play a central role acting as both the undertaker of the cellular immunity and supplier of the humoral immunity. Bivalves have different types of haemocytes, which perform different functions. Hence, identification of cell subpopulations and their functional characterisation in immune responses is essential to fully understand the immune system in bivalves. Nowadays, there is not a unified nomenclature that applies to all bivalves. Characterisation of bivalve haemocyte subpopulations is often combined with 1) other multiple parameter assays to determine differences between cell types in immune-related physiological activities, such as phagocytosis, oxidative stress and apoptosis; and 2) immune response to different stressors such as pathogens, temperature, acidification and pollution. This review summarises the major and most recent findings in classification and functional characterisation of the main haemocyte types of bivalve molluscs.En prens