14,948 research outputs found
Entity centric neural models for natural language processing
This thesis explores how to enhance natural language understanding by incorporating entity information into neural network models. It tackles three key questions:1. Leveraging entities for understanding tasks: This work introduces Entity-GCN, a model that performs multi-step reasoning on a graph where nodes represent entity mentions and edges represent relationships. This method achieved state-of-the-art results on a multi-document question-answering dataset.2. Identifying and disambiguating entities using large language models: This research proposes a novel system that retrieves entities by generating their names token-by-token, overcoming limitations of traditional methods and significantly reducing memory footprint. This approach is also extended to a multilingual setting and further optimized for efficiency.3. Interpreting and controlling entity knowledge within models: This thesis presents a post-hoc interpretation technique to analyze how decisions are made across layers in neural models, allowing for visualization and analysis of knowledge representation. Additionally, a method for editing factual knowledge about entities is proposed, enabling correction of model predictions without costly retraining
Entity centric neural models for natural language processing
This thesis explores how to enhance natural language understanding by incorporating entity information into neural network models. It tackles three key questions:1. Leveraging entities for understanding tasks: This work introduces Entity-GCN, a model that performs multi-step reasoning on a graph where nodes represent entity mentions and edges represent relationships. This method achieved state-of-the-art results on a multi-document question-answering dataset.2. Identifying and disambiguating entities using large language models: This research proposes a novel system that retrieves entities by generating their names token-by-token, overcoming limitations of traditional methods and significantly reducing memory footprint. This approach is also extended to a multilingual setting and further optimized for efficiency.3. Interpreting and controlling entity knowledge within models: This thesis presents a post-hoc interpretation technique to analyze how decisions are made across layers in neural models, allowing for visualization and analysis of knowledge representation. Additionally, a method for editing factual knowledge about entities is proposed, enabling correction of model predictions without costly retraining
Book Reviews
The Variational Auto-Encoder (VAE) is one of the most used unsupervised
machine learning models. But although the default choice of a Gaussian
distribution for both the prior and posterior represents a mathematically
convenient distribution often leading to competitive results, we show that this
parameterization fails to model data with a latent hyperspherical structure. To
address this issue we propose using a von Mises-Fisher (vMF) distribution
instead, leading to a hyperspherical latent space. Through a series of
experiments we show how such a hyperspherical VAE, or -VAE, is
more suitable for capturing data with a hyperspherical latent structure, while
outperforming a normal, -VAE, in low dimensions on other data
types.Comment: GitHub repository: http://github.com/nicola-decao/s-vae-tf, Blogpost:
https://nicola-decao.github.io/s-va
Editing Factual Knowledge in Language Models
The factual knowledge acquired during pre-training and stored in the parameters of Language Models (LMs) can be useful in downstream tasks (e.g., question answering or textual inference). However, some facts can be incorrectly induced or become obsolete over time. We present KNOWLEDGEEDITOR, a method which can be used to edit this knowledge and, thus, fix 'bugs' or unexpected predictions without the need for expensive retraining or fine-tuning. Besides being computationally efficient, KNOWLEDGEEDITOR does not require any modifications in LM pre-training (e.g., the use of meta-learning). In our approach, we train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the trained hyper-network is then used to predict the weight update at test time. We show KNOWLEDGEEDITOR's efficacy with two popular architectures and knowledge-intensive tasks: i) a BERT model fine-tuned for fact-checking, and ii) a sequence-to-sequence BART model for question answering. With our method, changing a prediction on the specific wording of a query tends to result in a consistent change in predictions also for its paraphrases. We show that this can be further encouraged by exploiting (e.g., automatically-generated) paraphrases during training. Interestingly, our hyper-network can be regarded as a 'probe' revealing which components need to be changed to manipulate factual knowledge; our analysis shows that the updates tend to be concentrated on a small subset of components.</p
The quantum inflaton, primordial perturbations and CMB fluctuations
We compute the primordial scalar, vector and tensor metric perturbations
arising from quantum field inflation. Quantum field inflation takes into
account the nonperturbative quantum dynamics of the inflaton consistently
coupled to the dynamics of the (classical) cosmological metric. For chaotic
inflation, the quantum treatment avoids the unnatural requirements of an
initial state with all the energy in the zero mode. For new inflation it allows
a consistent treatment of the explosive particle production due to spinodal
instabilities. Quantum field inflation (under conditions that are the quantum
analog of slow roll) leads, upon evolution, to the formation of a condensate
starting a regime of effective classical inflation. We compute the primordial
perturbations taking the dominant quantum effects into account. The results for
the scalar, vector and tensor primordial perturbations are expressed in terms
of the classical inflation results. For a N-component field in a O(N) symmetric
model, adiabatic fluctuations dominate while isocurvature or entropy
fluctuations are negligible. The results agree with the current WMAP
observations and predict corrections to the power spectrum in classical
inflation.Such corrections are estimated to be of the order of m^2/[N H^2]
where m is the inflaton mass and H the Hubble constant at horizon crossing.
This turns to be about 4% for the cosmologically relevant scales. This quantum
field treatment of inflation provides the foundations to the classical
inflation and permits to compute quantum corrections to it.Comment: 23 pages, no figures. Improved version to appear in Phys. Rev.
Noise-robust method for image segmentation
Segmentation of noisy images is one of the most challenging problems in image analysis and any improvement of segmentation methods can highly influence the performance of many image processing applications. In automated image segmentation, the fuzzy c-means (FCM) clustering has been widely used because of its ability to model uncertainty within the data, applicability to multi-modal data and fairly robust behaviour. However, the standard FCM algorithm does not consider any information about the spatial linage context and is highly sensitive to noise and other imaging artefacts. Considering above mentioned problems, we developed a new FCM-based approach for the noise-robust fuzzy clustering and we present it in this paper. In this new iterative algorithm we incorporated both spatial and feature space information into the similarity measure and the membership function. We considered that spatial information depends on the relative location and features of the neighbouring pixels. The performance of the proposed algorithm is tested on synthetic image with different noise levels and real images. Experimental quantitative and qualitative segmentation results show that our method efficiently preserves the homogeneity of the regions and is more robust to noise than other FCM-based methods
QED Renormalization Given in A Mass-Dependent Subtraction and The Renormalization Group Approach
The QED renormalization is restudied by using a mass-dependent subtraction
which is performed at a time-like renormalization point. The subtraction
exactly respects necessary physical and mathematical requirements such as the
gauge symmetry, the Lorentz- invariance and the mathematical convergence.
Therefore, the renormalized results derived in the subtraction scheme are
faithful and have no ambiguity. Especially, it is proved that the solution of
the renormalization group equation satisfied by a renormalized wave function,
propagator or vertex can be fixed by applying the renormalization boundary
condition and, thus, an exact S-matrix element can be expressed in the form as
written in the tree diagram approximation provided that the coupling constant
and the fermion mass are replaced by their effective ones. In the one-loop
approximation, the effective coupling constant and the effective fermion mass
obtained by solving their renormalization group equations are given in rigorous
and explicit expressions which are suitable in the whole range of distance and
exhibit physically reasonable asymptotic behaviors.Comment: 29 pages, 4 figure
Dense loops, supersymmetry, and Goldstone phases in two dimensions
Loop models in two dimensions can be related to O(N) models. The
low-temperature dense-loops phase of such a model, or of its reformulation
using a supergroup as symmetry, can have a Goldstone broken-symmetry phase for
N<2. We argue that this phase is generic for -2< N <2 when crossings of loops
are allowed, and distinct from the model of non-crossing dense loops first
studied by Nienhuis [Phys. Rev. Lett. 49, 1062 (1982)]. Our arguments are
supported by our numerical results, and by a lattice model solved exactly by
Martins et al. [Phys. Rev. Lett. 81, 504 (1998)].Comment: RevTeX, 5 pages, 3 postscript figure
Out of Equilibrium Non-perturbative Quantum Field Dynamics in Homogeneous External Fields
The quantum dynamics of the symmetry broken lambda (Phi^2)^2 scalar field
theory in the presence of an homogeneous external field is investigated in the
large N limit. We choose as initial state the ground state for a constant
external field J .The sign of the external field is suddenly flipped from
J to - J at a given time and the subsequent quantum dynamics calculated.
Spinodal instabilities and parametric resonances produce large quantum
fluctuations in the field components transverse to the external field. This
allows the order parameter to turn around the maximum of the potential for
intermediate times. Subsequently, the order parameter starts to oscillate near
the global minimum for external field - J, entering a novel quasi-periodic
regime.Comment: LaTex, 30 pages, 12 .ps figures, improved version to appear in Phys
Rev
Bivalve Haemocyte Subpopulations: A Review
Bivalve molluscs stand out for their ecological success and their key role in the functioning of aquatic ecosystems, while also constituting a very valuable commercial resource. Both ecological success and production of bivalves depend on their effective immune defence function, in which haemocytes play a central role acting as both the undertaker of the cellular immunity and supplier of the humoral immunity. Bivalves have different types of haemocytes, which perform different functions. Hence, identification of cell subpopulations and their functional characterisation in immune responses is essential to fully understand the immune system in bivalves. Nowadays, there is not a unified nomenclature that applies to all bivalves. Characterisation of bivalve haemocyte subpopulations is often combined with 1) other multiple parameter assays to determine differences between cell types in immune-related physiological activities, such as phagocytosis, oxidative stress and apoptosis; and 2) immune response to different stressors such as pathogens, temperature, acidification and pollution. This review summarises the major and most recent findings in classification and functional characterisation of the main haemocyte types of bivalve molluscs.En prens
- …