AIS: A nonlinear activation function for industrial safety engineering

In the task of Chinese named entity recognition based on deep learning, activation function plays an irreplaceable role, it introduces nonlinear characteristics into neural network, so that the fitted model can be applied to various tasks. However, the information density of industrial safety analysis text is relatively high, and the correlation and similarity between the information are large, which is easy to cause the problem of high deviation and high standard deviation of the model, no specific activation function has been designed in previous studies, and the traditional activation function has the problems of gradient vanishing and negative region, which also lead to the recognition accuracy of the model can not be further improved. To solve these problems, a novel activation function AIS is proposed in this paper. AIS is an activation function applied in industrial safety engineering, which is composed of two piecewise nonlinear functions. In the positive region, the structure combining exponential function and quadratic function is used to alleviate the problem of deviation and standard deviation, and the linear function is added to modify it, which makes the whole activation function smoother and overcomes the problem of gradient vanishing. In the negative region, the cubic function structure is used to solve the negative region problem and accelerate the convergence of the model. Based on the deep learning model of BERT-BiLSTM-CRF, the performance of AIS is evaluated. The results show that, compared with other activation functions, AIS overcomes the problems of gradient vanishing and negative region, reduces the deviation of the model, speeds up the model fitting, and improves the extraction ability of the model for industrial entities

Molecular Characterization of Isoniazid and Rifampin Resistance of Mycobacterium tuberculosis Clinical Isolates from Malatya, Turkey

Molecular characterization of drug resistance of Mycobacterium tuberculosis strains of different origins can generate information useful for developing molecular methods that are widely applicable for rapid drug resistance detection. Using DNA sequencing and allele-specific polymerase chain reaction (AS-PCR), we investigated genetic mutations associated with isoniazid (INH) and rifampin (RIF) resistance among 29 drug-resistant clinical isolates of M. tuberculosis collected from Malatya, Turkey, including 19 multi-drug-resistant (MDR) isolates. Point mutations were detected at codons 531, 516, 526, and 513 of the RNA polymerase β- subunit gene (rpoB) in 10 (47.6%), five (23.8%), three (14.3%), and three (14.3%) of the 21 RIF-resistant isolates, respectively. Of the five isolates having mutations in codon 516, three also had mutations at codon 527; one had a concurrent mutation at codon 572. Mutations at codon 315 of the catalase-peroxidase-encoding gene (katG) were found in 17 (63.0%) of the 27 INH-resistant isolates. Interestingly, the katG codon 315 mutation was observed at a much higher frequency in MDR isolates than in INH-mono-resistant isolates (∼79% vs. 25%). This study provided the first molecular characterization of INH and RIF resistance of M. tuberculosis clinical isolates from Eastern Turkey, and extended our knowledge of molecular basis of M. tuberculosis drug resistance.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/63182/1/mdr.2005.11.94.pd

Recommendation with User Active Disclosing Willingness

Recommender system has been deployed in a large amount of real-world applications, profoundly influencing people's daily life and production.Traditional recommender models mostly collect as comprehensive as possible user behaviors for accurate preference estimation. However, considering the privacy, preference shaping and other issues, the users may not want to disclose all their behaviors for training the model. In this paper, we study a novel recommendation paradigm, where the users are allowed to indicate their "willingness" on disclosing different behaviors, and the models are optimized by trading-off the recommendation quality as well as the violation of the user "willingness". More specifically, we formulate the recommendation problem as a multiplayer game, where the action is a selection vector representing whether the items are involved into the model training. For efficiently solving this game, we design a tailored algorithm based on influence function to lower the time cost for recommendation quality exploration, and also extend it with multiple anchor selection vectors.We conduct extensive experiments to demonstrate the effectiveness of our model on balancing the recommendation quality and user disclosing willingness

Why KDAC? A general activation function for knowledge discovery

Deep learning oriented named entity recognition (DNER) has gradually become the paradigm of knowledge discovery, which greatly promotes domain intelligence. However, the current activation function of DNER fails to treat gradient vanishing, no negative output or non-differentiable existence, which may impede knowledge exploration caused by the omission and incomplete representation of latent semantics. To break through the dilemma, we present a novel activation function termed KDAC. Detailly, KDAC is an aggregation function with multiple conversion modes. The backbone of the activation region is the interaction between exponent and linearity, and the both ends extend through adaptive linear divergence, which surmounts the obstacle of gradient vanishing and no negative output. Crucially, the non-differentiable points are alerted and eliminated by an approximate smoothing algorithm. KDAC has a series of brilliant properties, including nonlinear, stable near-linear transformation and derivative, as well as dynamic style, etc. We perform experiments based on BERT-BiLSTM-CNN-CRF model on six benchmark datasets containing different domain knowledge, such as Weibo, Clinical, E-commerce, Resume, HAZOP and People's daily. The evaluation results show that KDAC is advanced and effective, and can provide more generalized activation to stimulate the performance of DNER. We hope that KDAC can be exploited as a promising activation function to devote itself to the construction of knowledge.Comment: Accepted by Neurocomputin

Preferential gene expression in the limbus of the vervet monkey

PurposeTo elucidate the unique molecular factors and biological processes that are differentially expressed in the limbal stem cell microenvironment by comparing directly to that of its immediate adjacent structures, the cornea and conjunctiva.MethodsTotal RNA was isolated and amplified from the limbus, cornea, and conjunctiva. A gene expression profile of each tissue type was obtained by using microarray technique. The transcripts in which the expression level was at least twofold higher than that in the other two tissue types were identified. The expression levels of selected genes were confirmed by quantitative reverse transcription polymerase chain reaction (QRT-PCR). Protein expression of selected genes were confirmed by an immunohistochemistry study in normal human ocular tissue.ResultsThere were 186 preferentially overexpressed transcripts in the limbus in direct comparison to that in the cornea and conjunctiva. Many signature genes in the cornea and conjunctiva were among the preferentially expressed transcripts obtained by the microarray data. In addition, a significant number of new genes were identified, and the expression level of all nine selected genes was verified by QRT-PCR. Protein expression levels of keratin 13, tenascin c, homeodomain only protein (HOP), and TP53 apoptosis effector (PERP) were confirmed in human ocular tissues. Functional analysis of the preferentially expressed genes in the limbus reviewed that melanin metabolism and cell-cell adhesion were among the noticeable biological processes. Chromosomal distribution of the overexpressed genes in the limbus was disproportional to that of all known human genes.ConclusionsThese findings may shed light on the unique molecular components and regulation of limbal stem cells and their niche

Law Article-Enhanced Legal Case Matching: a Causal Learning Approach

Legal case matching, which automatically constructs a model to estimate the similarities between the source and target cases, has played an essential role in intelligent legal systems. Semantic text matching models have been applied to the task where the source and target legal cases are considered as long-form text documents. These general-purpose matching models make the predictions solely based on the texts in the legal cases, overlooking the essential role of the law articles in legal case matching. In the real world, the matching results (e.g., relevance labels) are dramatically affected by the law articles because the contents and the judgments of a legal case are radically formed on the basis of law. From the causal sense, a matching decision is affected by the mediation effect from the cited law articles by the legal cases, and the direct effect of the key circumstances (e.g., detailed fact descriptions) in the legal cases. In light of the observation, this paper proposes a model-agnostic causal learning framework called Law-Match, under which the legal case matching models are learned by respecting the corresponding law articles. Given a pair of legal cases and the related law articles, Law-Match considers the embeddings of the law articles as instrumental variables (IVs), and the embeddings of legal cases as treatments. Using IV regression, the treatments can be decomposed into law-related and law-unrelated parts, respectively reflecting the mediation and direct effects. These two parts are then combined with different weights to collectively support the final matching prediction. We show that the framework is model-agnostic, and a number of legal case matching models can be applied as the underlying models. Comprehensive experiments show that Law-Match can outperform state-of-the-art baselines on three public datasets.Comment: 10 pages accepted by SIGIR202

Less Is Better: Unweighted Data Subsampling via Influence Function

In the time of Big Data, training complex models on large-scale data sets is challenging, making it appealing to reduce data volume for saving computation resources by subsampling. Most previous works in subsampling are weighted methods designed to help the performance of subset-model approach the full-set-model, hence the weighted methods have no chance to acquire a subset-model that is better than the full-set-model. However, we question that how can we achieve better model with less data? In this work, we propose a novel Unweighted Influence Data Subsampling (UIDS) method, and prove that the subset-model acquired through our method can outperform the full-set-model. Besides, we show that overly confident on a given test set for sampling is common in Influence-based subsampling methods, which can eventually cause our subset-model's failure in out-of-sample test. To mitigate it, we develop a probabilistic sampling scheme to control the worst-case risk over all distributions close to the empirical distribution. The experiment results demonstrate our methods superiority over existed subsampling methods in diverse tasks, such as text classification, image classification, click-through prediction, etc.Comment: AAAI 202

FinalMLP: An Enhanced Two-Stream MLP Model for CTR Prediction

Click-through rate (CTR) prediction is one of the fundamental tasks for online advertising and recommendation. While multi-layer perceptron (MLP) serves as a core component in many deep CTR prediction models, it has been widely recognized that applying a vanilla MLP network alone is inefficient in learning multiplicative feature interactions. As such, many two-stream interaction models (e.g., DeepFM and DCN) have been proposed by integrating an MLP network with another dedicated network for enhanced CTR prediction. As the MLP stream learns feature interactions implicitly, existing research focuses mainly on enhancing explicit feature interactions in the complementary stream. In contrast, our empirical study shows that a well-tuned two-stream MLP model that simply combines two MLPs can even achieve surprisingly good performance, which has never been reported before by existing work. Based on this observation, we further propose feature gating and interaction aggregation layers that can be easily plugged to make an enhanced two-stream MLP model, FinalMLP. In this way, it not only enables differentiated feature inputs but also effectively fuses stream-level interactions across two streams. Our evaluation results on four open benchmark datasets as well as an online A/B test in our industrial system show that FinalMLP achieves better performance than many sophisticated two-stream CTR models. Our source code will be available at MindSpore/models.Comment: Accepted by AAAI 2023. Code available at https://xpai.github.io/FinalML

Ranolazine recruits muscle microvasculature and enhances insulin action in rats: Ranolazine, microvasculature and insulin action

Ranolazine, an anti-anginal compound, has been shown to significantly improve glycaemic control in large-scale clinical trials, and short-term ranolazine treatment is associated with an improvement in myocardial blood flow. As microvascular perfusion plays critical roles in insulin delivery and action, we aimed to determine if ranolazine could improve muscle microvascular blood flow, thereby increasing muscle insulin delivery and glucose use. Overnight-fasted, anaesthetized Sprague-Dawley rats were used to determine the effects of ranolazine on microvascular recruitment using contrast-enhanced ultrasound, insulin action with euglycaemic hyperinsulinaemic clamp, and muscle insulin uptake using 125I-insulin. Ranolazine's effects on endothelial nitric oxide synthase (eNOS) phosphorylation, cAMP generation and endothelial insulin uptake were determined in cultured endothelial cells. Ranolazine-induced myographical changes in tension were determined in isolated distal saphenous artery. Ranolazine at therapeutically effective dose significantly recruited muscle microvasculature by increasing muscle microvascular blood volume (∼2-fold, P < 0.05) and increased insulin-mediated whole body glucose disposal (∼30%, P= 0.02). These were associated with an increased insulin delivery into the muscle (P < 0.04). In cultured endothelial cells, ranolazine increased eNOS phosphorylation and cAMP production without affecting endothelial insulin uptake. In ex vivo studies, ranolazine exerted a potent vasodilatatory effect on phenylephrine pre-constricted arterial rings, which was partially abolished by endothelium denudement. In conclusion, ranolazine treatment vasodilatates pre-capillary arterioles and increases microvascular perfusion, which are partially mediated by endothelium, leading to expanded microvascular endothelial surface area available for nutrient and hormone exchanges and resulting in increased muscle delivery and action of insulin. Whether these actions contribute to improved glycaemic control in patients with insulin resistance warrants further investigation