Unsupervised Learning of Style-sensitive Word Vectors

This paper presents the first study aimed at capturing stylistic similarity between words in an unsupervised manner. We propose extending the continuous bag of words (CBOW) model (Mikolov et al., 2013) to learn style-sensitive word vectors using a wider context window under the assumption that the style of all the words in an utterance is consistent. In addition, we introduce a novel task to predict lexical stylistic similarity and to create a benchmark dataset for this task. Our experiment with this dataset supports our assumption and demonstrates that the proposed extensions contribute to the acquisition of style-sensitive word embeddings.Comment: 7 pages, Accepted at The 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018

半島マレーシアのエンダウ・ロンピン国立公園の観光利用地域における野生哺乳類の日周活動パターンへの人為影響

東京都立大

Finding a Shortest Non-zero Path in Group-Labeled Graphs via Permanent Computation

A group-labeled graph is a directed graph with each arc labeled by a group element, and the label of a path is defined as the sum of the labels of the traversed arcs. In this paper, we propose a polynomial time randomized algorithm for the problem of finding a shortest s-t path with a non-zero label in a given group-labeled graph (which we call the Shortest Non-Zero Path Problem). This problem generalizes the problem of finding a shortest path with an odd number of edges, which is known to be solvable in polynomial time by using matching algorithms. Our algorithm for the Shortest Non-Zero Path Problem is based on the ideas of Björklund and Husfeldt (Proceedings of the 41st international colloquium on automata, languages and programming, part I. LNCS 8572, pp 211–222, 2014). We reduce the problem to the computation of the permanent of a polynomial matrix modulo two. Furthermore, by devising an algorithm for computing the permanent of a polynomial matrix modulo 2r for any fixed integer r, we extend our result to the problem of packing internally-disjoint s-t paths

Theoretical Estimation of the Acoustic Energy Generation and Absorption Caused by Jet Oscillation

We investigate the energy transfer between the fluid field and acoustic field caused by a jet driven by an acoustic particle velocity field across it, which is the key to understanding the aerodynamic sound generation of flue instruments, such as the recorder, flute, and organ pipe. Howe’s energy corollary allows us to estimate the energy transfer between these two fields. For simplicity, we consider the situation such that a free jet is driven by a uniform acoustic particle velocity field across it. We improve the semi-empirical model of the oscillating jet, i.e., exponentially growing jet model, which has been studied in the field of musical acoustics, and introduce a polynomially growing jet model so as to apply Howe’s formula to it. It is found that the relative phase between the acoustic oscillation and jet oscillation, which changes with the distance from the flue exit, determines the quantity of the energy transfer between the two fields. The acoustic energy is mainly generated in the downstream area, but it is consumed in the upstream area near the flue exit in driving the jet. This theoretical examination well explains the numerical calculation of Howe’s formula for the two-dimensional flue instrument model in our previous work [Fluid Dyn. Res. 46, 061411 (2014) ] as well as the experimental result of Yoshikawa et al. [ J. Sound Vib. 331, 2558 (2012) ]

Transformer Language Models Handle Word Frequency in Prediction Head

Prediction head is a crucial component of Transformer language models. Despite its direct impact on prediction, this component has often been overlooked in analyzing Transformers. In this study, we investigate the inner workings of the prediction head, specifically focusing on bias parameters. Our experiments with BERT and GPT-2 models reveal that the biases in their word prediction heads play a significant role in the models' ability to reflect word frequency in a corpus, aligning with the logit adjustment method commonly used in long-tailed learning. We also quantify the effect of controlling the biases in practical auto-regressive text generation scenarios; under a particular setting, more diverse text can be generated without compromising text quality.Comment: 11 pages, 12 figures, accepted to ACL 2023 Findings (short paper

Contrastive Learning-based Sentence Encoders Implicitly Weight Informative Words

The performance of sentence encoders can be significantly improved through the simple practice of fine-tuning using contrastive loss. A natural question arises: what characteristics do models acquire during contrastive learning? This paper theoretically and experimentally shows that contrastive-based sentence encoders implicitly weight words based on information-theoretic quantities; that is, more informative words receive greater weight, while others receive less. The theory states that, in the lower bound of the optimal value of the contrastive learning objective, the norm of word embedding reflects the information gain associated with the distribution of surrounding words. We also conduct comprehensive experiments using various models, multiple datasets, two methods to measure the implicit weighting of models (Integrated Gradients and SHAP), and two information-theoretic quantities (information gain and self-information). The results provide empirical evidence that contrastive fine-tuning emphasizes informative words.Comment: 16 pages, 6 figures, accepted to EMNLP 2023 Findings (short paper

Microstructure Evolution of Carbon Steel by Hot Equal Channel Angular Extrusion

AbstractAn equal channel angular extrusion (ECAE) process equipment which enable a repetitive hot ECAE process without ejecting workpiece with route A and C are developed. This equipment has T-shape 3 actuator axis in horizontal plane and is capable of simulating the formation of fine grained steels in the transformation route. Each actuator (mechanical servo press unit) can be controlled by both position and load with programed motion. The outline of the developed ECAE equipment and the results of preliminary application of the ECAE equipment at an elevated temperature at various pressing speeds ranging from 2 to 32mm/s for a Nb alloyed steel are present. 2 passes via route C at ram speed 16mm/s are also conducted. The ferrite grain size of about 2μm steel is obtained throughout the workpiece at ram speed of 32mm/s, preheated temperature 960oC