Segatron: Segment-Aware Transformer for Language Modeling and Understanding

Transformers are powerful for sequence modeling. Nearly all state-of-the-art language models and pre-trained language models are based on the Transformer architecture. However, it distinguishes sequential tokens only with the token position index. We hypothesize that better contextual representations can be generated from the Transformer with richer positional information. To verify this, we propose a segment-aware Transformer (Segatron), by replacing the original token position encoding with a combined position encoding of paragraph, sentence, and token. We first introduce the segment-aware mechanism to Transformer-XL, which is a popular Transformer-based language model with memory extension and relative position encoding. We find that our method can further improve the Transformer-XL base model and large model, achieving 17.1 perplexity on the WikiText-103 dataset. We further investigate the pre-training masked language modeling task with Segatron. Experimental results show that BERT pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla Transformer on various NLP tasks, and outperforms RoBERTa on zero-shot sentence representation learning.Comment: Accepted by AAAI 202

Adaptive structure concept factorization for multiview clustering

Most existing multiview clustering methods require that graph matrices in different views are computed beforehand and that each graph is obtained independently. However, this requirement ignores the correlation between multiple views. In this letter, we tackle the problem of multiview clustering by jointly optimizing the graph matrix to make full use of the data correlation between views. With the interview correlation, a concept factorization–based multiview clustering method is developed for data integration, and the adaptive method correlates the affinity weights of all views. This method differs from nonnegative matrix factorization–based clustering methods in that it can be applicable to data sets containing negative values. Experiments are conducted to demonstrate the effectiveness of the proposed method in comparison with state-of-the-art approaches in terms of accuracy, normalized mutual information, and purity

Communication security of autonomous ground vehicles based on networked control systems: The optimized LMI approach

The paper presents a study of networked control systems (NCSs) that are subjected to periodic denial-of-service (DoS) attacks of varying intensity. The use of appropriate Lyapunov–Krasovskii functionals (LKFs) help to reduce the constraints of the basic conditions and lower the conservatism of the criteria. An optimization problem with constraints is formulated to select the trigger threshold, which is solved using the gradient descent algorithm (GDA) to improve resource utilization. An intelligent secure event-triggered controller (ISETC) is designed to ensure the safe operation of the system under DoS attacks. The approach is validated through experiments with an autonomous ground vehicle (AGV) system based on the Simulink platform. The proposed method offers the potential for developing effective defense mechanisms against DoS attacks in NCSs

Excellent Performance of Fe78Si9B13 Metallic Glass for Activating Peroxymonosulfate in Degradation of Naphthol Green B

The functional application of metallic glasses in the catalytic field has widely attracted research attention due to its unique atomic structure compared to crystalline materials. It has been reported that metallic glasses can effectively activate H2O2 and persulfate, yet the activation of peroxymonosulfate by metallic glasses is not studied well. In this work, the metallic glass with atomic composition of Fe78Si9B13 was applied for investigating the peroxymonosulfate (PMS) activation on degradation of naphthol green B (NGB) dye. The change of surface morphology indicated the important role of oxide films during the dye degradation. The effects and first-order kinetics model of various reaction parameters were evaluated systematically, including PMS concentration, catalyst dosage, irradiation intensity, and dye concentration. The results showed that about 98% of the dye removal rate could be achieved only within 10 min under rational conditions. The reaction kinetics k of 0.1339 min−1 without ribbons was sharply improved to 0.3140 min−1 by adding 0.5 g/L ribbons, indicating the superior activation ability of Fe78Si9B13 metallic glass. The recycling experiment revealed that the Fe78Si9B13 ribbons exhibited the excellent surface stability and catalytic reusability for activating PMS even after reused for 10th run

Evaluation of phage-based decontamination in respiratory intensive care unit environments using ddPCR and 16S rRNA targeted sequencing techniques

BackgroundKlebsiella pneumoniae is a major cause of hospital-acquired infections (HAIs), primarily spread through environmental contamination in hospitals. The effectiveness of current chemical disinfectants is waning due to emerging resistance, which poses environmental hazards and fosters new resistance in pathogens. Developing environmentally friendly and effective disinfectants against multidrug-resistant organisms is increasingly important.MethodsThis study developed a bacteriophage cocktail targeting two common carbapenem-resistant Klebsiella pneumoniae (CRKP) strains, ST11 KL47 and ST11 KL64. The cocktail was used as an adjunctive disinfectant in a hospital’s respiratory intensive care unit (RICU) via ultrasonic nebulization. Digital PCR was used to quantify CRKP levels post-intervention. The microbial community composition was analyzed via 16S rRNA sequencing to assess the intervention’s impact on overall diversity.ResultsThe phage cocktail significantly reduced CRKP levels within the first 24 hours post-treatment. While a slight increase in pathogen levels was observed after 24 hours, they remained significantly lower than those treated with conventional disinfectants. 16S rRNA sequencing showed a decrease in the target pathogens’ relative abundance, while overall species diversity remained stable, confirming that phages selectively target CRKP without disrupting ecological balance.DiscussionThe findings highlight the efficacy and safety of phage-based biocleaners as a sustainable alternative to conventional disinfectants. Phages selectively reduce multidrug-resistant pathogens while preserving microbial diversity, making them a promising tool for infection control

Risk prediction models for incident type 2 diabetes in Chinese people with intermediate hyperglycemia : a systematic literature review and external validation study

Background People with intermediate hyperglycemia (IH), including impaired fasting glucose and/or impaired glucose tolerance, are at higher risk of developing type 2 diabetes (T2D) than those with normoglycemia. We aimed to evaluate the performance of published T2D risk prediction models in Chinese people with IH to inform them about the choice of primary diabetes prevention measures. Methods A systematic literature search was conducted to identify Asian-derived T2D risk prediction models, which were eligible if they were built on a prospective cohort of Asian adults without diabetes at baseline and utilized routinely-available variables to predict future risk of T2D. These Asian-derived and five prespecified non-Asian derived T2D risk prediction models were divided into BASIC (clinical variables only) and EXTENDED (plus laboratory variables) versions, with validation performed on them in three prospective Chinese IH cohorts: ACE (n = 3241), Luzhou (n = 1333), and TCLSIH (n = 1702). Model performance was assessed in terms of discrimination (C-statistic) and calibration (Hosmer-Lemeshow test). Results Forty-four Asian and five non-Asian studies comprising 21 BASIC and 46 EXTENDED T2D risk prediction models for validation were identified. The majority were at high (n = 43, 87.8%) or unclear (n = 3, 6.1%) risk of bias, while only three studies (6.1%) were scored at low risk of bias. BASIC models showed poor-to-moderate discrimination with C-statistics 0.52-0.60, 0.50-0.59, and 0.50-0.64 in the ACE, Luzhou, and TCLSIH cohorts respectively. EXTENDED models showed poor-to-acceptable discrimination with C-statistics 0.54-0.73, 0.52-0.67, and 0.59-0.78 respectively. Fifteen BASIC and 40 EXTENDED models showed poor calibration (P < 0.05), overpredicting or underestimating the observed diabetes risk. Most recalibrated models showed improved calibration but modestly-to-severely overestimated diabetes risk in the three cohorts. The NAVIGATOR model showed the best discrimination in the three cohorts but had poor calibration (P < 0.05). Conclusions In Chinese people with IH, previously published BASIC models to predict T2D did not exhibit good discrimination or calibration. Several EXTENDED models performed better, but a robust Chinese T2D risk prediction tool in people with IH remains a major unmet need.Peer reviewe

Confined FeNi alloy nanoparticles in carbon nanotubes for photothermal oxidative dehydrogenation of ethane by carbon dioxide

Oxidative dehydrogenation of ethane with CO2 (ODEC) is an attractive reaction for reduction of carbon footprints and ethene production. In this work, we present photothermal catalysis on confined bimetal catalysts for ODEC. Carbon nanotubes confined non-noble bimetal alloy (i.e., CoNi@CNTs and FeNi@CNTs) catalysts were prepared and FeNi@CNTs showed effective performance in photothermal catalytic ODEC to ethene. Experiments and simulations reveal that UV and visible lights (420 – 490 nm) are responsible for ODEC and non-oxidative dehydrogenation of ethane, respectively, to ethene. Additionally, ODEC to ethene is preferred to C-C cracking to methane on FeNi@CNTs in light ( \u3e 490 nm)-induced thermocatalysis. The photothermal effect turns more significant when introduced into thermocatalytic ODEC (500 °C), with ethene generation at one order of magnitude. This work advances new mechanism of photo-mediated catalysis and sheds light on utilization of full-spectrum solar energy and non-noble metallic catalysts for ethene production and CO2 recycling at moderate conditions

CXCL9 Is a Potential Biomarker of Immune Infiltration Associated With Favorable Prognosis in ER-Negative Breast Cancer

The chemokine CXCL9 (C-X-C motif chemokine ligand 9) has been reported to be required for antitumour immune responses following immune checkpoint blockade. In this study, we sought to investigate the potential value of CXCL9 according to immune responses in patients with breast cancer (BC). A variety of open-source databases and online tools were used to explore the expression features and prognostic significance of CXCL9 in BC and its correlation with immune-related biomarkers followed by subsequent verification with immunohistochemistry experiments. The CXCL9 mRNA level was found to be significantly higher in BC than in normal tissue and was associated with better survival outcomes in patients with ER-negative tumours. Moreover, CXCL9 is significantly correlated with immune cell infiltration and immune-related biomarkers, including CTLA4, GZMB, LAG3, PDCD1 and HAVCR2. Finally, we performed immunohistochemistry with breast cancer tissue samples and observed that CXCL9 is highly expressed in the ER-negative subgroup and positively correlated with the immune-related factors LAG3, PD1, PDL1 and CTLA4 to varying degrees. These findings suggest that CXCL9 is an underlying biomarker for predicting the status of immune infiltration in ER-negative breast cancer