444 research outputs found
A Unified Encoder-Decoder Framework with Entity Memory
Entities, as important carriers of real-world knowledge, play a key role in
many NLP tasks. We focus on incorporating entity knowledge into an
encoder-decoder framework for informative text generation. Existing approaches
tried to index, retrieve, and read external documents as evidence, but they
suffered from a large computational overhead. In this work, we propose an
encoder-decoder framework with an entity memory, namely EDMem. The entity
knowledge is stored in the memory as latent representations, and the memory is
pre-trained on Wikipedia along with encoder-decoder parameters. To precisely
generate entity names, we design three decoding methods to constrain entity
generation by linking entities in the memory. EDMem is a unified framework that
can be used on various entity-intensive question answering and generation
tasks. Extensive experimental results show that EDMem outperforms both
memory-based auto-encoder models and non-memory encoder-decoder models.Comment: Accepted by the 2022 Conference on Empirical Methods in Natural
Language Processing (EMNLP 2022
A FPC-ROOT Algorithm for 2D-DOA Estimation in Sparse Array
To improve the performance of two-dimensional direction-of-arrival (2D DOA) estimation in sparse array, this paper presents a Fixed Point Continuation Polynomial Roots (FPC-ROOT) algorithm. Firstly, a signal model for DOA estimation is established based on matrix completion and it can be proved that the proposed model meets Null Space Property (NSP). Secondly, left and right singular vectors of received signals matrix are achieved using the matrix completion algorithm. Finally, 2D DOA estimation can be acquired through solving the polynomial roots. The proposed algorithm can achieve high accuracy of 2D DOA estimation in sparse array, without solving autocorrelation matrix of received signals and scanning of two-dimensional spectral peak. Besides, it decreases the number of antennas and lowers computational complexity and meanwhile avoids the angle ambiguity problem. Computer simulations demonstrate that the proposed FPC-ROOT algorithm can obtain the 2D DOA estimation precisely in sparse array
Suboptimal subspace construction for log-determinant approximation
Variance reduction is a crucial idea for Monte Carlo simulation and the
stochastic Lanczos quadrature method is a dedicated method to approximate the
trace of a matrix function. Inspired by their advantages, we combine these two
techniques to approximate the log-determinant of large-scale symmetric positive
definite matrices. Key questions to be answered for such a method are how to
construct or choose an appropriate projection subspace and derive guaranteed
theoretical analysis. This paper applies some probabilistic approaches
including the projection-cost-preserving sketch and matrix concentration
inequalities to construct a suboptimal subspace. Furthermore, we provide some
insights on choosing design parameters in the underlying algorithm by deriving
corresponding approximation error and probabilistic error estimations.
Numerical experiments demonstrate our method's effectiveness and illustrate the
quality of the derived error bounds
Beyond Generic: Enhancing Image Captioning with Real-World Knowledge using Vision-Language Pre-Training Model
Current captioning approaches tend to generate correct but "generic"
descriptions that lack real-world knowledge, e.g., named entities and
contextual information. Considering that Vision-Language Pre-Training (VLP)
models master massive such knowledge from large-scale web-harvested data, it is
promising to utilize the generalizability of VLP models to incorporate
knowledge into image descriptions. However, using VLP models faces challenges:
zero-shot inference suffers from knowledge hallucination that leads to
low-quality descriptions, but the generic bias in downstream task fine-tuning
hinders the VLP model from expressing knowledge. To address these concerns, we
propose a simple yet effective method called Knowledge-guided Replay
(K-Replay), which enables the retention of pre-training knowledge during
fine-tuning. Our approach consists of two parts: (1) a knowledge prediction
task on automatically collected replay exemplars to continuously awaken the VLP
model's memory about knowledge, thus preventing the model from collapsing into
the generic pattern; (2) a knowledge distillation constraint to improve the
faithfulness of generated descriptions hence alleviating the knowledge
hallucination. To evaluate knowledge-enhanced descriptions, we construct a
novel captioning benchmark KnowCap, containing knowledge of landmarks, famous
brands, special foods and movie characters. Experimental results show that our
approach effectively incorporates knowledge into descriptions, outperforming
strong VLP baseline by 20.9 points (78.7->99.6) in CIDEr score and 20.5
percentage points (34.0%->54.5%) in knowledge recognition accuracy. Our code
and data is available at https://github.com/njucckevin/KnowCap.Comment: Accepted at ACM Multimedia (ACMMM) 202
Microgrid distribution system dynamic reactive power optimization based on improved particle swarm algorithms
Abstract Due to the low accuracy and convergence of existing particle swarm algorithm in the micro power dynamic reactive power optimization in distribution system, this paper proposes an improved particle swarm algorithm based on the state of the particle and inertia weight optimization. This algorithm first adjusts the status of the states of the particles. Then using Sigmoid mapping to optimize the search ability of the inertia weight in particle swarms algorithm. Finally, using the optimal learning strategies to improve the convergence of particle swarm optimization algorithm. Through simulation experiments, the proposed improving particle swarm algorithm based on particle state and inertia weight optimization owing better convergence than traditional particle swarm optimization. Only small error was obtained during dynamic reactive power optimization in micro power distribution system
- …