202,292 research outputs found
IsoScore: Measuring the Uniformity of Embedding Space Utilization
The recent success of distributed word representations has led to an
increased interest in analyzing the properties of their spatial distribution.
Several studies have suggested that contextualized word embedding models do not
isotropically project tokens into vector space. However, current methods
designed to measure isotropy, such as average random cosine similarity and the
partition score, have not been thoroughly analyzed and are not appropriate for
measuring isotropy. We propose IsoScore: a novel tool that quantifies the
degree to which a point cloud uniformly utilizes the ambient vector space.
Using rigorously designed tests, we demonstrate that IsoScore is the only tool
available in the literature that accurately measures how uniformly distributed
variance is across dimensions in vector space. Additionally, we use IsoScore to
challenge a number of recent conclusions in the NLP literature that have been
derived using brittle metrics of isotropy. We caution future studies from using
existing tools to measure isotropy in contextualized embedding space as
resulting conclusions will be misleading or altogether inaccurate.Comment: ACL 2022 camera ready versio
Learning and Transferring IDs Representation in E-commerce
Many machine intelligence techniques are developed in E-commerce and one of
the most essential components is the representation of IDs, including user ID,
item ID, product ID, store ID, brand ID, category ID etc. The classical
encoding based methods (like one-hot encoding) are inefficient in that it
suffers sparsity problems due to its high dimension, and it cannot reflect the
relationships among IDs, either homogeneous or heterogeneous ones. In this
paper, we propose an embedding based framework to learn and transfer the
representation of IDs. As the implicit feedbacks of users, a tremendous amount
of item ID sequences can be easily collected from the interactive sessions. By
jointly using these informative sequences and the structural connections among
IDs, all types of IDs can be embedded into one low-dimensional semantic space.
Subsequently, the learned representations are utilized and transferred in four
scenarios: (i) measuring the similarity between items, (ii) transferring from
seen items to unseen items, (iii) transferring across different domains, (iv)
transferring across different tasks. We deploy and evaluate the proposed
approach in Hema App and the results validate its effectiveness.Comment: KDD'18, 9 page
The hidden subgroup problem and quantum computation using group representations
The hidden subgroup problem is the foundation of many quantum algorithms. An efficient solution is known for the problem over abelian groups, employed by both Simon's algorithm and Shor's factoring and discrete log algorithms. The nonabelian case, however, remains open; an efficient solution would give rise to an efficient quantum algorithm for graph isomorphism. We fully analyze a natural generalization of the algorithm for the abelian case to the nonabelian case and show that the algorithm determines the normal core of a hidden subgroup: in particular, normal subgroups can be determined. We show, however, that this immediate generalization of the abelian algorithm does not efficiently solve graph isomorphism
Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec
We present a novel approach to learn distributed representation of sentences from unlabeled data by modeling both content and context of a sentence. The content model learns sentence representation by predicting its words. On the other hand, the context model comprises a neighbor prediction component and a regularizer to model distributional and proximity hypotheses, respectively. We propose an online algorithm to train the model components jointly. We evaluate the models in a setup, where contextual information is available. The experimental results on tasks involving classification, clustering, and ranking of sentences show that our model outperforms the best existing models by a wide margin across multiple datasets
- …