Digging Errors in NMT: Evaluating and Understanding Model Errors from Partial Hypothesis Space

Solid evaluation of neural machine translation (NMT) is key to its understanding and improvement. Current evaluation of an NMT system is usually built upon a heuristic decoding algorithm (e.g., beam search) and an evaluation metric assessing similarity between the translation and golden reference. However, this system-level evaluation framework is limited by evaluating only one best hypothesis and search errors brought by heuristic decoding algorithms. To better understand NMT models, we propose a novel evaluation protocol, which defines model errors with model's ranking capability over hypothesis space. To tackle the problem of exponentially large space, we propose two approximation methods, top region evaluation along with an exact top-$k$ decoding algorithm, which finds top-ranked hypotheses in the whole hypothesis space, and Monte Carlo sampling evaluation, which simulates hypothesis space from a broader perspective. To quantify errors, we define our NMT model errors by measuring distance between the hypothesis array ranked by the model and the ideally ranked hypothesis array. After confirming the strong correlation with human judgment, we apply our evaluation to various NMT benchmarks and model architectures. We show that the state-of-the-art Transformer models face serious ranking issues and only perform at the random chance level in the top region. We further analyze model errors on architectures with different depths and widths, as well as different data-augmentation techniques, showing how these factors affect model errors. Finally, we connect model errors with the search algorithms and provide interesting findings of beam search inductive bias and correlation with Minimum Bayes Risk (MBR) decoding.Comment: To be appeared as a main conference paper at EMNLP 202

Integration of transcriptomics and metabolomics reveals the responses of the maternal circulation and maternal-fetal interface to LPS-induced preterm birth in mice

BackgroundTerm birth (TB) and preterm birth (PTB) are characterized by uterine contractions, rupture of the chorioamniotic membrane, decidual activation, and other physiological and pathological changes. In this study, we hypothesize that inflammation can cause changes in mRNA expression and metabolic stability in the placenta, decidua, chorioamniotic membrane, uterus and peripheral blood, ultimately leading to PTB.MethodsTo comprehensively assess the effects of inflammation on mRNA expression and metabolite production in different tissues of pregnancy, we used a mouse PTB model by intraperitoneally injecting lipopolysaccharide (LPS) and integrated transcriptomics and metabolomics studies.ResultsOur analysis identified 152 common differentially expressed genes (DEGs) and 8 common differentially expressed metabolites (DEMs) in the placenta, decidua, chorioamniotic membrane, uterus, and peripheral blood, or placenta and uterus after LPS injection, respectively. Our bioinformatics analysis revealed significant enrichment of the NOD-like receptor signaling pathway (mmu04621), TNF signaling pathway (mmu04668), IL-17 signaling pathway (mmu04657), and NF-kappa B signaling pathway in the transcriptomics of different tissues, and Hormone synthesis, Lysosome, NOD-like receptor signaling pathway, and Protein digest and absorption pathway in metabolomics. Moreover, we found that several upstream regulators and master regulators, including STAT1, STAT3, and NFKB1, were altered after exposure to inflammation in the different tissues. Interaction network analysis of transcriptomics and metabolomics DEGs and DEMs also revealed functional changes in mice intraperitoneally injected with LPS.ConclusionsOverall, our study identified significant and biologically relevant alterations in the placenta, decidua, chorioamniotic membrane, uterus, peripheral blood transcriptome and the placenta and uterus metabolome in mice exposed to LPS. Thus, a comprehensive analysis of different pregnancy tissues in mice intraperitoneally injected with LPS by combining transcriptomics and metabolomics may help to systematically understand the local and systemic changes associated with PTB caused by inflammation

Decreased Information Replacement of Working Memory After Sleep Deprivation: Evidence From an Event-Related Potential Study

Working memory (WM) components are altered after total sleep deprivation (TSD), both with respect to information replacement and result judgment. However, the electrophysiological mechanisms of WM alterations following sleep restriction remain largely unknown. To identify such mechanisms, event-related potentials were recorded during the n-back WM task, before and after 36 h sleep deprivation. Thirty-one young volunteers participated in this study and performed a two-back WM task with simultaneous electroencephalography (EEG) recording before and after TSD and after 8 h time in bed for recovery (TIBR). Repeated measures analysis of variance revealed that, compared to resting wakefulness, sleep deprivation induced a decrease in the P200 amplitude and induced longer reaction times. ERP-component scalp topographies results indicated that such decrease primarily occurred in the frontal cortex. The N200 and P300 amplitudes also decreased after TSD. Our results suggest that decreased information replacement of WM occurs after 36 h of TSD and that 8 h TIBR after a long period of TSD leads to partial restoration of WM functions. The present findings represent the EEG profile of WM during mental fatigue

Joint Alternate Small Convolution and Feature Reuse for Hyperspectral Image Classification

A hyperspectral image (HSI) contains fine and rich spectral information and spatial information of ground objects, which has great potential in applications. It is also widely used in precision agriculture, marine monitoring, military reconnaissance and many other fields. In recent years, a convolutional neural network (CNN) has been successfully used in HSI classification and has provided it with outstanding capacity for improving classification effects. To get rid of the bondage of strong correlation among bands for HSI classification, an effective CNN architecture is proposed for HSI classification in this work. The proposed CNN architecture has several distinct advantages. First, each 1D spectral vector that corresponds to a pixel in an HSI is transformed into a 2D spectral feature matrix, thereby emphasizing the difference among samples. In addition, this architecture can not only weaken the influence of strong correlation among bands on classification, but can also fully utilize the spectral information of hyperspectral data. Furthermore, a 1 × 1 convolutional layer is adopted to better deal with HSI information. All the convolutional layers in the proposed CNN architecture are composed of small convolutional kernels. Moreover, cascaded composite layers of the architecture consist of 1 × 1 and 3 × 3 convolutional layers. The inputs and outputs of each composite layer are stitched as the inputs of the next composite layer, thereby accomplishing feature reuse. This special module with joint alternate small convolution and feature reuse can extract high-level features from hyperspectral data meticulously and comprehensively solve the overfitting problem to an extent, in order to obtain a considerable classification effect. Finally, global average pooling is used to replace the traditional fully connected layer to reduce the model parameters and extract high-dimensional features from the hyperspectral data at the end of the architecture. Experimental results on three benchmark HSI datasets show the high classification accuracy and effectiveness of the proposed method

Channel reciprocity and time-reversed propagation for ultra-wideband communications

Abstract — This paper is the first paper of an new effort to understand UWB communications and sensor networking in RF harsh environments. Channel reciprocity and time reversed propagation are studied, using an intra-vehicle environment. Communications using time reversal is also investigated. I

Time Reversal with MISO for UltraWideband Communications: Experimental Results

Abstract — Time reversal (TR) communications marks a paradigm shift in UWB communications. The system complexity can be shifted from the receiver to the transmitter, which is ideal to UWB sensors. UWB Multiple Input Single Output (MISO) is enabled by the use of the TR scheme. Two basic problems are investigated experimentally using short UWB radio pulses (nanosecond duration). Temporal focusing and SNR increase with the number of antennas are verified. Also, reciprocity of realistic channels, the foundation of TR, is demonstrated for the first time in electromagnetics