Unveiling Multilinguality in Transformer Models: Exploring Language Specificity in Feed-Forward Networks

Recent research suggests that the feed-forward module within Transformers can be viewed as a collection of key-value memories, where the keys learn to capture specific patterns from the input based on the training examples. The values then combine the output from the 'memories' of the keys to generate predictions about the next token. This leads to an incremental process of prediction that gradually converges towards the final token choice near the output layers. This interesting perspective raises questions about how multilingual models might leverage this mechanism. Specifically, for autoregressive models trained on two or more languages, do all neurons (across layers) respond equally to all languages? No! Our hypothesis centers around the notion that during pretraining, certain model parameters learn strong language-specific features, while others learn more language-agnostic (shared across languages) features. To validate this, we conduct experiments utilizing parallel corpora of two languages that the model was initially pretrained on. Our findings reveal that the layers closest to the network's input or output tend to exhibit more language-specific behaviour compared to the layers in the middle

EMMT Release (version 1.0)

We present Eyetracked Multi-Modal Translation (EMMT), a dataset containing monocular eye movement recordings, audio data and 4-electrode wearable electroencephalogram (EEG) data of 43 participants while engaged in sight translation task supported by an image

Stroop Effect in Multi-Modal Sight Translation

This study investigates the human translation process from English to Czech in a multi-modal scenario (images) using reaction times. We make a distinction between ambiguous and unambiguous sentences where in the former, more information would be needed in order to make a proper translation (e.g. gender of the subject). Simultaneously, we also provide visual aid to help in disambiguation, which is necessary for the ambiguous sentences. We confirm that ambiguous sentences take longer to translate and the provision of disambiguating visual aid slows the translation process. When provided with an unrelated visual aid, humans are able to recognize and spend less time on it but still significantly more than in other conditions. These findings are a clear manifestation of the Stroop effect (longer processing times for incongruent combinations)

EMMT: A simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios

We present the Eyetracked Multi-Modal Translation (EMMT) corpus, a dataset containing monocular eye movement recordings, audio and 4-electrode electroencephalogram (EEG) data of 43 participants. The objective was to collect cognitive signals as responses of participants engaged in a number of language intensive tasks involving different text-image stimuli settings when translating from English to Czech. Each participant was exposed to 32 text-image stimuli pairs and asked to (1) read the English sentence, (2) translate it into Czech, (3) consult the image, (4) translate again, either updating or repeating the previous translation. The text stimuli consisted of 200 unique sentences with 616 unique words coupled with 200 unique images as the visual stimuli. The recordings were collected over a two week period and all the participants included in the study were Czech natives with strong English skills. Due to the nature of the tasks involved in the study and the relatively large number of participants involved, the corpus is well suited for research in Translation Process Studies, Cognitive Sciences among other disciplines.Comment: Submitted to Nature Scientific Dat

EMMT (Eyetracked Multi-Modal Translation)

Eyetracked Multi-Modal Translation (EMMT) is a simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios. It contains monocular eye movement recordings, audio data and 4-electrode wearable electroencephalogram (EEG) data of 43 participants while engaged in sight translation supported by an image. The details about the experiment and the dataset can be found in the README file

What do the eyes really see? An eye-tracking account of language processing

This experimental study aims to investigate the translation process from English to Czech in a multimodal scenario by using an eye tracker. We investigate specific aspects of translating ambiguous and unambiguous sentences, and simultaneously, we focus on the possible impact of visual information on the translation process. Thus, we show how mechanisms of visual search, as well as the presence and attention mechanisms involved in such translation processes, can be explored based on various eye-movement data, i.e., cognitive mechanisms involved in reading original sentences and producing the corresponding translation are studied using a plethora of eye-tracking-specific metrics. Among other things, the paper demonstrates how the Stroop effect is visible in the experimental setup

Photoredox/Nickel Dual Catalysis for C(sp2)-C(sp3) Cross Electrophile Coupling Reaction Using Phenols and Alcohols as the Latent Electrophile Sources

Introducing alkyl groups, particularly through innovative C(sp2)-C(sp3) bond-forming methods utilizing abundant feedstocks, holds promise for expanding chemical diversity, especially in drug discovery programs. Herein, we employed biomass-derived abundant phenols and alkyl alcohols as the coupling partners for the C(sp2)-C(sp3) cross-electrophile coupling reactions. Our innovation involved activating both the coupling partner via the same activator and designing a catalytic system that activates stronger bonds while ensuring coss-selectivity. The visible-light photoredox/nickel dual catalytic systems accommodate large substrate scope tolerating diverse functional groups. Besides, both the activation and cross-coupling reaction could be performed in one pot, and the reaction could be scaled up. Preliminary mechanistic studies involving luminescence quenching, cyclic voltammetry, radical quenching, and radical clock studies elucidated the proposed reaction mechanism

Comparison of analgesic effect of intra-articular administration of levobupivacaine and clonidine versus ropivacaine and clonidine in day care knee arthroscopic surgery under spinal anesthesia

Introduction: Intra-articular (IA) local anesthetics are often used for the management and prevention of pain after arthroscopic knee surgery. Clonidine prolongs the duration of local anesthetics. In this study, analgesic effect of intra-articular administration of levobupivacaine and clonidine was compared with ropivacaine and clonidine in knee joint arthroscopic surgery under spinal anesthesia. Method: 88 patients, aged between 15 to 55 years, ASA I and II undergoing knee arthroscopy under spinal anesthesia were assigned into two equal groups (n = 44) in a randomized double blind protocol. Patients in Group L received 10 ml of 0.50% levobupivacaine and 1 mcg/kg clonidine and Group R received 10 ml of 0.75% ropivacaine and 1 mcg/kg of clonidine through intra-articular route at the end of the procedure. In the post-operative period, pain intensity was assessed by VAS (Visual Analogue Scale) Score recorded at 1 st , 5 th , 8 th , 12 th , 18 th post-operative hours. Duration of analgesia, total rescue analgesic dose in first 18 hours and any side effects were also recorded. Result: Group L experienced significantly longer duration of effective postoperative analgesia and lesser rescue analgesic compared to group R. Group R had higher mean VAS score at 5 th and 12 th post-operative hours (P < 0.05). No side effects were observed among the groups. Conclusion: Intra-articular administration of levobupivacaine and clonidine give better post-operative pain relief by increasing duration of analgesia, and decreasing need of rescue analgesic compared to intra-articular ropivacaine and clonidine