Tree Transformer: Integrating Tree Structures into Self-Attention

Pre-training Transformer from large-scale raw texts and fine-tuning on the desired task have achieved state-of-the-art results on diverse NLP tasks. However, it is unclear what the learned attention captures. The attention computed by attention heads seems not to match human intuitions about hierarchical structures. This paper proposes Tree Transformer, which adds an extra constraint to attention heads of the bidirectional Transformer encoder in order to encourage the attention heads to follow tree structures. The tree structures can be automatically induced from raw texts by our proposed "Constituent Attention" module, which is simply implemented by self-attention between two adjacent words. With the same training procedure identical to BERT, the experiments demonstrate the effectiveness of Tree Transformer in terms of inducing tree structures, better language modeling, and further learning more explainable attention scores.Comment: accepted by EMNLP 201

Fuzzy mathematical model for solving supply chain problem

In a real world application supply chain, there are many elements of uncertainty such as supplier performance, market demands, product price, operation time, and shipping method which increases the difficulty for manufacturers to quickly respond in order to fulfil the customer requirements. In this paper, the authors developed a fuzzy mathematical model to integrate different operational functions with the aim to provide satisfy decisions to help decision maker resolve production problem for all functions simultaneously. A triangular fuzzy number or possibilistic distribution represents all the uncertainty parameters. A comparison between a fuzzy model, a possibilistic model and a deterministic model is presented in this paper in order to distinguish the effectiveness of model in dealing the uncertain nature of supply chain. The proposed models performance is evaluated based on the operational aspect and computational aspect. The fuzzy model and the possibilistic model are expected to be more preferable to respond to the dynamic changes of the supply change network compared to the deterministic model. The developed fuzzy model seems to be more flexible in undertaking the lack of information or imprecise data of a variable in real situation whereas possibilistic model is more practical in solving an existing systems problem that has available data provided

A Support Vector Data Description Committee for Face Detection

Face detection is a crucial prestage for face recognition and is often treated as a binary (face and nonface) classification problem. While this strategy is simple to implement, face detection accuracy would drop when nonface training patterns are undersampled. To avoid these problems, we propose in this paper a one-class learning-based face detector called support vector data description (SVDD) committee, which consists of several SVDD members, each of which is trained on a subset of face patterns. Nonfaces are not required in the training of the SVDD committee. Therefore, the face detection accuracy of SVDD committee is independent of the nonface training patterns. Moreover, the proposed SVDD committee is also able to improve generalization ability of the original SVDD when the face data set has a multicluster distribution. Experiments carried out on the extended MIT face data set show that the proposed SVDD committee can achieve better face detection accuracy than the widely used SVM face detector and performs better than other one-class classifiers, including the original SVDD and the kernel principal component analysis (Kernel PCA)

Comparative functional genomic analysis of Alzheimer’s affected and naturally aging brains

Background Alzheimer’s disease (AD) is a prevalent progressive neurodegenerative human disease whose cause remains unclear. Numerous initially highly hopeful anti-AD drugs based on the amyloid-β (Aβ) hypothesis of AD have failed recent late-phase tests. Natural aging (AG) is a high-risk factor for AD. Here, we aim to gain insights in AD that may lead to its novel therapeutic treatment through conducting meta-analyses of gene expression microarray data from AG and AD-affected brain. Methods Five sets of gene expression microarray data from different regions of AD (hereafter, ALZ when referring to data)-affected brain, and one set from AG, were analyzed by means of the application of the methods of differentially expressed genes and differentially co-expressed gene pairs for the identification of putatively disrupted biological pathways and associated abnormal molecular contents. Results Brain-region specificity among ALZ cases and AG-ALZ differences in gene expression and in KEGG pathway disruption were identified. Strong heterogeneity in AD signatures among the five brain regions was observed: HC/PC/SFG showed clear and pronounced AD signatures, MTG moderately so, and EC showed essentially none. There were stark differences between ALZ and AG. OXPHOS and Proteasome were the most disrupted pathways in HC/PC/SFG, while AG showed no OXPHOS disruption and relatively weak Proteasome disruption in AG. Metabolic related pathways including TCA cycle and Pyruvate metabolism were disrupted in ALZ but not in AG. Three pathogenic infection related pathways were disrupted in ALZ. Many cancer and signaling related pathways were shown to be disrupted AG but far less so in ALZ, and not at all in HC. We identified 54 “ALZ-only” differentially expressed genes, all down-regulated and which, when used to augment the gene list of the KEGG AD pathway, made it significantly more AD-specific

Rhodiola crenulata

Exposure to hypoxia leads to impaired pulmonary sodium transport, which is associated with Na,K-ATPase dysfunction in the alveolar epithelium. The present study is designed to examine the effect and mechanism of Rhodiola crenulata extract (RCE) and its bioactive components on hypoxia-mediated Na,K-ATPase endocytosis. A549 cells were exposed to hypoxia in the presence or absence of RCE, salidroside, or tyrosol. The generation of intracellular ROS was measured by using the fluorescent probe DCFH-DA, and the endocytosis was determined by measuring the expression level of Na,K-ATPase in the PM fraction. Rats exposed to a hypobaric hypoxia chamber were used to investigate the efficacy and underlying mechanism of RCE in vivo. Our results showed that RCE and its bioactive compounds significantly prevented the hypoxia-mediated endocytosis of Na,K-ATPase via the inhibition of the ROS-AMPK-PKCζ pathway in A549 cells. Furthermore, RCE also showed a comparable preventive effect on the reduction of Na,K-ATPase endocytosis and inhibition of AMPK-PKCξ pathway in the rodent model. Our study is the first to offer substantial evidence to support the efficacy of Rhodiola products against hypoxia-associated Na,K-ATPase endocytosis and clarify the ethnopharmacological relevance of Rhodiola crenulata as a popular folk medicine for high-altitude illness

Statistical identification of gene association by CID in application of constructing ER regulatory network

<p>Abstract</p> <p>Background</p> <p>A variety of high-throughput techniques are now available for constructing comprehensive gene regulatory networks in systems biology. In this study, we report a new statistical approach for facilitating <it>in silico </it>inference of regulatory network structure. The new measure of association, coefficient of intrinsic dependence (CID), is model-free and can be applied to both continuous and categorical distributions. When given two variables X and Y, CID answers whether Y is dependent on X by examining the conditional distribution of Y given X. In this paper, we apply CID to analyze the regulatory relationships between transcription factors (TFs) (X) and their downstream genes (Y) based on clinical data. More specifically, we use estrogen receptor α (ERα) as the variable X, and the analyses are based on 48 clinical breast cancer gene expression arrays (48A).</p> <p>Results</p> <p>The analytical utility of CID was evaluated in comparison with four commonly used statistical methods, Galton-Pearson's correlation coefficient (GPCC), Student's <it>t</it>-test (STT), coefficient of determination (CoD), and mutual information (MI). When being compared to GPCC, CoD, and MI, CID reveals its preferential ability to discover the regulatory association where distribution of the mRNA expression levels on X and Y does not fit linear models. On the other hand, when CID is used to measure the association of a continuous variable (Y) against a discrete variable (X), it shows similar performance as compared to STT, and appears to outperform CoD and MI. In addition, this study established a two-layer transcriptional regulatory network to exemplify the usage of CID, in combination with GPCC, in deciphering gene networks based on gene expression profiles from patient arrays.</p> <p>Conclusion</p> <p>CID is shown to provide useful information for identifying associations between genes and transcription factors of interest in patient arrays. When coupled with the relationships detected by GPCC, the association predicted by CID are applicable to the construction of transcriptional regulatory networks. This study shows how information from different data sources and learning algorithms can be integrated to investigate whether relevant regulatory mechanisms identified in cell models can also be partially re-identified in clinical samples of breast cancers.</p> <p>Availability</p> <p>the implementation of CID in R codes can be freely downloaded from <url>http://homepage.ntu.edu.tw/~lyliu/BC/</url>.</p