45 research outputs found
Improving Contextualized Topic Models with Negative Sampling
Topic modeling has emerged as a dominant method for exploring large document
collections. Recent approaches to topic modeling use large contextualized
language models and variational autoencoders. In this paper, we propose a
negative sampling mechanism for a contextualized topic model to improve the
quality of the generated topics. In particular, during model training, we
perturb the generated document-topic vector and use a triplet loss to encourage
the document reconstructed from the correct document-topic vector to be similar
to the input document and dissimilar to the document reconstructed from the
perturbed vector. Experiments for different topic counts on three publicly
available benchmark datasets show that in most cases, our approach leads to an
increase in topic coherence over that of the baselines. Our model also achieves
very high topic diversity.Comment: Accepted at 19th International Conference on Natural Language
Processing (ICON 2022
InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions
Most existing knowledge graphs suffer from incompleteness, which can be
alleviated by inferring missing links based on known facts. One popular way to
accomplish this is to generate low-dimensional embeddings of entities and
relations, and use these to make inferences. ConvE, a recently proposed
approach, applies convolutional filters on 2D reshapings of entity and relation
embeddings in order to capture rich interactions between their components.
However, the number of interactions that ConvE can capture is limited. In this
paper, we analyze how increasing the number of these interactions affects link
prediction performance, and utilize our observations to propose InteractE.
InteractE is based on three key ideas -- feature permutation, a novel feature
reshaping, and circular convolution. Through extensive experiments, we find
that InteractE outperforms state-of-the-art convolutional link prediction
baselines on FB15k-237. Further, InteractE achieves an MRR score that is 9%,
7.5%, and 23% better than ConvE on the FB15k-237, WN18RR and YAGO3-10 datasets
respectively. The results validate our central hypothesis -- that increasing
feature interaction is beneficial to link prediction performance. We make the
source code of InteractE available to encourage reproducible research.Comment: Accepted at AAAI 202
ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations
Graph Neural Networks (GNN) have been shown to work effectively for modeling
graph structured data to solve tasks such as node classification, link
prediction and graph classification. There has been some recent progress in
defining the notion of pooling in graphs whereby the model tries to generate a
graph level representation by downsampling and summarizing the information
present in the nodes. Existing pooling methods either fail to effectively
capture the graph substructure or do not easily scale to large graphs. In this
work, we propose ASAP (Adaptive Structure Aware Pooling), a sparse and
differentiable pooling method that addresses the limitations of previous graph
pooling architectures. ASAP utilizes a novel self-attention network along with
a modified GNN formulation to capture the importance of each node in a given
graph. It also learns a sparse soft cluster assignment for nodes at each layer
to effectively pool the subgraphs to form the pooled graph. Through extensive
experiments on multiple datasets and theoretical analysis, we motivate our
choice of the components used in ASAP. Our experimental results show that
combining existing GNN architectures with ASAP leads to state-of-the-art
results on multiple graph classification benchmarks. ASAP has an average
improvement of 4%, compared to current sparse hierarchical state-of-the-art
method.Comment: The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI
2020
Relationship among seam strength, weft-way fabric strength and stitch density of B. Twill jute bag
The influence of weft-way fabric strength and stitch density on the seam strength of B. Twill jute bag has been studied. Study reveals that a correlation exists among seam strength, weft-way fabric strength and stitch density of B. Twill jute bags sewn with herakle stitches. A regression equation for seam strength as a function of two factors namely weft-way fabric strength and stitch density has been established
SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes
We present SCULPT, a novel 3D generative model for clothed and textured 3D
meshes of humans. Specifically, we devise a deep neural network that learns to
represent the geometry and appearance distribution of clothed human bodies.
Training such a model is challenging, as datasets of textured 3D meshes for
humans are limited in size and accessibility. Our key observation is that there
exist medium-sized 3D scan datasets like CAPE, as well as large-scale 2D image
datasets of clothed humans and multiple appearances can be mapped to a single
geometry. To effectively learn from the two data modalities, we propose an
unpaired learning procedure for pose-dependent clothed and textured human
meshes. Specifically, we learn a pose-dependent geometry space from 3D scan
data. We represent this as per vertex displacements w.r.t. the SMPL model.
Next, we train a geometry conditioned texture generator in an unsupervised way
using the 2D image data. We use intermediate activations of the learned
geometry model to condition our texture generator. To alleviate entanglement
between pose and clothing type, and pose and clothing appearance, we condition
both the texture and geometry generators with attribute labels such as clothing
types for the geometry, and clothing colors for the texture generator. We
automatically generated these conditioning labels for the 2D images based on
the visual question answering model BLIP and CLIP. We validate our method on
the SCULPT dataset, and compare to state-of-the-art 3D generative models for
clothed human bodies. We will release the codebase for research purposes
Generation of Highlights from Research Papers Using Pointer-Generator Networks and SciBERT Embeddings
Nowadays many research articles are prefaced with research highlights to
summarize the main findings of the paper. Highlights not only help researchers
precisely and quickly identify the contributions of a paper, they also enhance
the discoverability of the article via search engines. We aim to automatically
construct research highlights given certain segments of the research paper. We
use a pointer-generator network with coverage mechanism and a contextual
embedding layer at the input that encodes the input tokens into SciBERT
embeddings. We test our model on a benchmark dataset, CSPubSum and also present
MixSub, a new multi-disciplinary corpus of papers for automatic research
highlight generation. For both CSPubSum and MixSub, we have observed that the
proposed model achieves the best performance compared to related variants and
other models proposed in the literature. On the CSPubSum data set, our model
achieves the best performance when the input is only the abstract of a paper as
opposed to other segments of the paper. It produces ROUGE-1, ROUGE-2 and
ROUGE-L F1-scores of 38.26, 14.26 and 35.51, respectively, METEOR F1-score of
32.62, and BERTScore F1 of 86.65 which outperform all other baselines. On the
new MixSub data set, where only the abstract is the input, our proposed model
(when trained on the whole training corpus without distinguishing between the
subject categories) achieves ROUGE-1, ROUGE-2 and ROUGE-L F1-scores of 31.78,
9.76 and 29.3, respectively, METEOR F1-score of 24.00, and BERTScore F1 of
85.25, outperforming other models.Comment: 18 pages, 7 figures, 7 table