SG-Net: Syntax-Guided Machine Reading Comprehension

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. To verify its effectiveness, the proposed SG-Net is applied to typical pre-trained language model BERT which is right based on a Transformer encoder. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-2020

Increased IL-10 mRNA expression in tumor-associated macrophage correlated with late stage of lung cancer

<p>Abstract</p> <p>Background</p> <p>Monocyte recruited into the tumor and maturation to tumor-associated macrophage (TAM). <it>Interleukin-10(IL-10) </it>is a potent immunosuppressive cytokine, which can be secreted from both primary tumor and stromal cells. However, there are controversies regarding its role in the progression of cancer. So it is important to isolate TAM from tumor cells to study the role of <it>IL-10 </it>in the progress of cancer. The aim of our study was to determine whether <it>IL-10 </it>expressed by TAM correlated with clinicopathological factors in NSCLC.</p> <p>Methods</p> <p>TAM in NSCLC was isolated by short-term culture in serum free medium with the modification to literature reports. The mRNA expression levels of <it>IL-10</it>, <it>cathepsin B</it>, <it>cathepsin S</it>, which were closely related with TAM according to the literatures, were evaluated by Quantitative real-time RT-PCR in 63 NSCLC. The relationships between their expression levels and clinicopathological features were investigated.</p> <p>Results</p> <p>We successfully achieved up to 95% purity of TAM, derived from 63 primary lung cancer tissues. TAM expressed high levels of <it>IL-10</it>, <it>cathepsin B </it>in NSCLC. High levels of <it>IL-10 </it>in TAM significantly correlated with stage, tumor size, lymph node metastasis, lymphovascular invasion or histologic poor differentiation.</p> <p>Conclusions</p> <p>Our results revealed that TAM with high levels of <it>IL-10 </it>expression may play an important role in the progression of non-small cell lung cancer. The data also suggested that TAMs may involve in tumor immunosuppression through overexpressed <it>IL-10</it>. Additionally, the phenotype of isolated TAM can be potentially used to predict clinicopathological features as well.</p

Achieving low energy consuming bio-based piezoelectric nanogenerators via modulating the inner layer thickness for a highly sensitive pedometer

Considering their drawbacks of environmental pollution, biodegradable cellulose-based materials are becoming one of the most promising alternative candidates for conventional petroleum-based polymers, which are considered the fundamental materials for dynamical units in human-machine interaction systems. Using an up-to-date hydrogen bond replacement strategy, which means using the highly electronegative F− in polyvinylidene fluoride (PVDF) to replace the intramolecular hydrogen bonds in cellulose for weakening the self-assembly behavior, herein, multilayer-structured piezoelectric nanogenerators (PENGs) composed of cellulose, a small amount of PVDF, and Ba0.7Ca0.3Zr0.2Ti0.8O3 (BCZT) fillers were fabricated via modified tape-casting technology. Due to the hydrogen bond network, which was confirmed using multiple characterization methods, the fillers dispersed uniformly in the matrix. Through changing the inner layer thickness, the output performance of the PENGs can be subtly modulated, which is revealed to be caused by the synergistic effect between the trapped electrons and the inter-squeezing between adjacent particles by employing the band theory. When applied to a pedometer, one of the essential devices for monitoring human health, such a modulation can significantly improve its sensitivity. The water contact angle test also indicates their potential for use in humid environments. Compared with some typical cellulose-based PENGs, our device shows outstanding performance in PD/F, defined as the power density triggered by unit force, indicating our PENG's low energy consumption characteristic.</p

A Novel Porous Carrier Found in Nature for Nanocomposite Materials Preparation: A Case Study of Artemia Egg Shell-Supported TiO 2 for Formaldehyde Removal

Artemia egg shells have an asymptotic sized pore structure (pore diameter: 500 nm-2500 nm), which could be used as a porous carrier for the preparation of nanocomposite materials. The objective of the present study was to prepare shell-supported TiO 2 using a naturally porous carrier, Artemia egg shell, and to exhibit a case study of shell-supported TiO 2 for formaldehyde removal. Characterization of shell-TiO 2 using SEM-EDS, TEM, and XRD proved that Artemia shell with asymptotic reduction pores (pore diameter: 500 nm-2500 nm) can be used as the carrier for nanocomposite materials. Artemia egg shell-supported TiO 2 in polycrystalline-like nanostructures can be used for the high efficiency removal (adsorption and degradation) of formaldehyde under visible light. Our results suggest that iron, one of the shell&apos;s components, should broaden the absorption of visible light and enhance the photocatalytic efficiency of nanotitanium dioxide under visible light. Due to their interesting absorption and formaldehyde removal qualities, Artemia egg shell, as a novel naturally porous carrier for nanocomposite materials preparation, especially in the preparation of nanocatalysts, is worthy of further study

To Understand Representation of Layer-aware Sequence Encoders as Multi-order-graph

In this paper, we propose a unified explanation of representation for layer-aware neural sequence encoders, which regards the representation as a revisited multigraph called multi-order-graph (MoG), so that model encoding can be viewed as a processing to capture all subgraphs in MoG. The relationship reflected by Multi-order-graph, called $n$-order dependency, can present what existing simple directed graph explanation cannot present. Our proposed MoG explanation allows to precisely observe every step of the generation of representation, put diverse relationship such as syntax into a unifiedly depicted framework. Based on the proposed MoG explanation, we further propose a graph-based self-attention network empowered Graph-Transformer by enhancing the ability of capturing subgraph information over the current models. Graph-Transformer accommodates different subgraphs into different groups, which allows model to focus on salient subgraphs. Result of experiments on neural machine translation tasks show that the MoG-inspired model can yield effective performance improvement.Comment: arXiv admin note: text overlap with arXiv:2009.0748