Approximating Word Ranking and Negative Sampling for Word Embedding

CBOW (Continuous Bag-Of-Words) is one of the most commonly used techniques to generate word embeddings in various NLP tasks. However, it fails to reach the optimal performance due to uniform involvements of positive words and a simple sampling distribution of negative words. To resolve these issues, we propose OptRank to optimize word ranking and approximate negative sampling for bettering word embedding. Specifically, we first formalize word embedding as a ranking problem. Then, we weigh the positive words by their ranks such that highly ranked words have more importance, and adopt a dynamic sampling strategy to select informative negative words. In addition, an approximation method is designed to efficiently compute word ranks. Empirical experiments show that OptRank consistently outperforms its counterparts on a benchmark dataset with different sampling scales, especially when the sampled subset is small. The code and datasets can be obtained from https://github.com/ouououououou/OptRank

Boundary integrated neural networks (BINNs) for acoustic radiation and scattering

This paper presents a novel approach called the boundary integrated neural networks (BINNs) for analyzing acoustic radiation and scattering. The method introduces fundamental solutions of the time-harmonic wave equation to encode the boundary integral equations (BIEs) within the neural networks, replacing the conventional use of the governing equation in physics-informed neural networks (PINNs). This approach offers several advantages. Firstly, the input data for the neural networks in the BINNs only require the coordinates of "boundary" collocation points, making it highly suitable for analyzing acoustic fields in unbounded domains. Secondly, the loss function of the BINNs is not a composite form, and has a fast convergence. Thirdly, the BINNs achieve comparable precision to the PINNs using fewer collocation points and hidden layers/neurons. Finally, the semi-analytic characteristic of the BIEs contributes to the higher precision of the BINNs. Numerical examples are presented to demonstrate the performance of the proposed method

VSE-ens: Visual-Semantic Embeddings with Efficient Negative Sampling

Jointing visual-semantic embeddings (VSE) have become a research hotpot for the task of image annotation, which suffers from the issue of semantic gap, i.e., the gap between images' visual features (low-level) and labels' semantic features (high-level). This issue will be even more challenging if visual features cannot be retrieved from images, that is, when images are only denoted by numerical IDs as given in some real datasets. The typical way of existing VSE methods is to perform a uniform sampling method for negative examples that violate the ranking order against positive examples, which requires a time-consuming search in the whole label space. In this paper, we propose a fast adaptive negative sampler that can work well in the settings of no figure pixels available. Our sampling strategy is to choose the negative examples that are most likely to meet the requirements of violation according to the latent factors of images. In this way, our approach can linearly scale up to large datasets. The experiments demonstrate that our approach converges 5.02x faster than the state-of-the-art approaches on OpenImages, 2.5x on IAPR-TCI2 and 2.06x on NUS-WIDE datasets, as well as better ranking accuracy across datasets.Comment: Published by The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18

VSE-ens: Visual-Semantic Embeddings with Efficient Negative Sampling

Jointing visual-semantic embeddings (VSE) have become a research hotpot for the task of image annotation, which suffers from the issue of semantic gap, i.e., the gap between images' visual features (low-level) and labels' semantic features (high-level). This issue will be even more challenging if visual features cannot be retrieved from images, that is, when images are only denoted by numerical IDs as given in some real datasets. The typical way of existing VSE methods is to perform a uniform sampling method for negative examples that violate the ranking order against positive examples, which requires a time-consuming search in the whole label space. In this paper, we propose a fast adaptive negative sampler that can work well in the settings of no figure pixels available. Our sampling strategy is to choose the negative examples that are most likely to meet the requirements of violation according to the latent factors of images. In this way, our approach can linearly scale up to large datasets. The experiments demonstrate that our approach converges 5.02x faster than the state-of-the-art approaches on OpenImages, 2.5x on IAPR-TCI2 and 2.06x on NUS-WIDE datasets, as well as better ranking accuracy across datasets.Comment: Published by The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18

Exploring Adapter-based Transfer Learning for Recommender Systems: Empirical Studies and Practical Insights

Adapters, a plug-in neural network module with some tunable parameters, have emerged as a parameter-efficient transfer learning technique for adapting pre-trained models to downstream tasks, especially for natural language processing (NLP) and computer vision (CV) fields. Meanwhile, learning recommendation models directly from raw item modality features -- e.g., texts of NLP and images of CV -- can enable effective and transferable recommender systems (called TransRec). In view of this, a natural question arises: can adapter-based learning techniques achieve parameter-efficient TransRec with good performance? To this end, we perform empirical studies to address several key sub-questions. First, we ask whether the adapter-based TransRec performs comparably to TransRec based on standard full-parameter fine-tuning? does it hold for recommendation with different item modalities, e.g., textual RS and visual RS. If yes, we benchmark these existing adapters, which have been shown to be effective in NLP and CV tasks, in the item recommendation settings. Third, we carefully study several key factors for the adapter-based TransRec in terms of where and how to insert these adapters? Finally, we look at the effects of adapter-based TransRec by either scaling up its source training data or scaling down its target training data. Our paper provides key insights and practical guidance on unified & transferable recommendation -- a less studied recommendation scenario. We promise to release all code & datasets for future research

TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedback

Learning large-scale pre-trained models on broad-ranging data and then transfer to a wide range of target tasks has become the de facto paradigm in many machine learning (ML) communities. Such big models are not only strong performers in practice but also offer a promising way to break out of the task-specific modeling restrictions, thereby enabling task-agnostic and unified ML systems. However, such a popular paradigm is mainly unexplored by the recommender systems (RS) community. A critical issue is that standard recommendation models are primarily built on categorical identity features. That is, the users and the interacted items are represented by their unique IDs, which are generally not shareable across different systems or platforms. To pursue the transferable recommendations, we propose studying pre-trained RS models in a novel scenario where a user's interaction feedback involves a mixture-of-modality (MoM) items, e.g., text and images. We then present TransRec, a very simple modification made on the popular ID-based RS framework. TransRec learns directly from the raw features of the MoM items in an end-to-end training manner and thus enables effective transfer learning under various scenarios without relying on overlapped users or items. We empirically study the transferring ability of TransRec across four different real-world recommendation settings. Besides, we look at its effects by scaling source and target data size. Our results suggest that learning neural recommendation models from MoM feedback provides a promising way to realize universal RS

Non-suicidal self-injury and suicidal ideation among adolescents: the chain-mediating role of rumination and decentering

ObjectiveTo explore the relationship between non-suicidal self-injury and suicidal ideation in adolescents and examine the roles of rumination and decentering in that relationship.MethodBy means of a questionnaire, 175 adolescent patients in a psychiatric hospital in Fujian Province were given the Functional Assessment of Self-Mutilation: Chinese Version, Positive and Negative Suicide Ideation, Ruminative Response Scale: Chinese Version, and Experiences Questionnaire: Decentering Scale.Results(1) Adolescent non-suicidal self-injury was significantly positively related to suicidal ideation and rumination and significantly negatively related to decentering. Suicidal ideation was significantly positively related to rumination and significantly negatively related to decentering. Rumination was significantly negatively related to decentering. (2) Rumination and decentering played a complete chain-mediating role between non-suicidal self-injury and suicidal ideation. Non-suicidal self-injury was found to indirectly affect suicidal ideation along three pathways: the independent mediating role of rumination (the mediating effect accounted for 40.166%), independent mediating role of decentering (the mediating effect accounted for 41.274%), and chain-mediating role of rumination and decentering (the mediating effect accounted for 14.958%).ConclusionAdolescent non-suicidal self-injury can indirectly affect suicidal ideation through rumination and decentering. In the future, mindfulness and other methods should be used to improve individuals’ levels of decentering and cultivate emotional regulation abilities, so as to reduce the incidence of non-suicidal self-injury and suicide in adolescents

NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation

Learning a recommender system model from an item's raw modality features (such as image, text, audio, etc.), called MoRec, has attracted growing interest recently. One key advantage of MoRec is that it can easily benefit from advances in other fields, such as natural language processing (NLP) and computer vision (CV). Moreover, it naturally supports transfer learning across different systems through modality features, known as transferable recommender systems, or TransRec. However, so far, TransRec has made little progress, compared to groundbreaking foundation models in the fields of NLP and CV. The lack of large-scale, high-quality recommendation datasets poses a major obstacle. To this end, we introduce NineRec, a TransRec dataset suite that includes a large-scale source domain recommendation dataset and nine diverse target domain recommendation datasets. Each item in NineRec is represented by a text description and a high-resolution cover image. With NineRec, we can implement TransRec models in an end-to-end training manner instead of using pre-extracted invariant features. We conduct a benchmark study and empirical analysis of TransRec using NineRec, and our findings provide several valuable insights. To support further research, we make our code, datasets, benchmarks, and leaderboards publicly available at https://github.com/westlake-repl/NineRec

Extended Wiener-Khinchin theorem for quantum spectral analysis

The classical Wiener-Khinchin theorem (WKT), which can extract spectral information by classical interferometers through Fourier transform, is a fundamental theorem used in many disciplines. However, there is still need for a quantum version of WKT, which could connect correlated biphoton spectral information by quantum interferometers. Here, we extend the classical WKT to its quantum counterpart, i.e., extended WKT (e-WKT), which is based on two-photon quantum interferometry. According to the e-WKT, the difference-frequency distribution of the biphoton wavefunctions can be extracted by applying a Fourier transform on the time-domain Hong-Ou-Mandel interference (HOMI) patterns, while the sum-frequency distribution can be extracted by applying a Fourier transform on the time-domain NOON state interference (NOONI) patterns. We also experimentally verified the WKT and e-WKT in a Mach-Zehnder interference (MZI), a HOMI and a NOONI. This theorem can be directly applied to quantum spectroscopy, where the spectral correlation information of biphotons can be obtained from time-domain quantum interferences by Fourier transform. This may open a new pathway for the study of light-matter interaction at the single photon level.Comment: 13 pages, 5 figure