2 research outputs found
Generative adversarial learning of Sinkhorn algorithm initializations
The Sinkhorn algorithm [Cut13] is the state-of-the-art to compute approximations of optimal transport distances between discrete probability distributions, making use of an entropically regularized formulation of the problem. The algorithm is guaranteed to converge, no matter its initialization. This lead to little attention being paid to initializing it, and simple starting vectors like the n-dimensional one-vector are common choices. We train a neural network to compute initializations for the algorithm, which significantly outperform standard initializations. The network predicts a potential of the optimal transport dual problem, where training is conducted in an adversarial fashion using a second, generating network. The network is universal in the sense that it is able to generalize to any pair of distributions of fixed dimension. Furthermore, we show that for certain applications the network can be used independently
Jina Embeddings: A Novel Set of High-Performance Sentence Embedding Models
Jina Embeddings constitutes a set of high-performance sentence embedding
models adept at translating various textual inputs into numerical
representations, thereby capturing the semantic essence of the text. While
these models are not exclusively designed for text generation, they excel in
applications such as dense retrieval and semantic textual similarity. This
paper details the development of Jina Embeddings, starting with the creation of
a high-quality pairwise and triplet dataset. It underlines the crucial role of
data cleaning in dataset preparation, gives in-depth insights into the model
training process, and concludes with a comprehensive performance evaluation
using the Massive Textual Embedding Benchmark (MTEB).Comment: 9 pages, 2 page appendix, EMNLP 2023 Industrial Trac