DropKey

In this paper, we focus on analyzing and improving the dropout technique for self-attention layers of Vision Transformer, which is important while surprisingly ignored by prior works. In particular, we conduct researches on three core questions: First, what to drop in self-attention layers? Different from dropping attention weights in literature, we propose to move dropout operations forward ahead of attention matrix calculation and set the Key as the dropout unit, yielding a novel dropout-before-softmax scheme. We theoretically verify that this scheme helps keep both regularization and probability features of attention weights, alleviating the overfittings problem to specific patterns and enhancing the model to globally capture vital information; Second, how to schedule the drop ratio in consecutive layers? In contrast to exploit a constant drop ratio for all layers, we present a new decreasing schedule that gradually decreases the drop ratio along the stack of self-attention layers. We experimentally validate the proposed schedule can avoid overfittings in low-level features and missing in high-level semantics, thus improving the robustness and stableness of model training; Third, whether need to perform structured dropout operation as CNN? We attempt patch-based block-version of dropout operation and find that this useful trick for CNN is not essential for ViT. Given exploration on the above three questions, we present the novel DropKey method that regards Key as the drop unit and exploits decreasing schedule for drop ratio, improving ViTs in a general way. Comprehensive experiments demonstrate the effectiveness of DropKey for various ViT architectures, e.g. T2T and VOLO, as well as for various vision tasks, e.g., image classification, object detection, human-object interaction detection and human body shape recovery.Comment: Accepted by CVPR202

Changes in Wastewater Treatment Performance and the Microbial Community during the Bioaugmentation of a Denitrifying <i>Pseudomonas </i>Strain in the Low Carbon–Nitrogen Ratio Sequencing Batch Reactor

The low carbon&ndash;nitrogen ratio (C/N) of influent wastewater results in the insufficient carbon source for the process of denitrification in urban wastewater treatment plants (WWTPs). A denitrifying bacterial strain, Pseudomonas sp. JMSTP, was isolated and demonstrated effective denitrification ability under a low C/N ratio of 1.5&ndash;4 (w/w) in anoxic conditions. Sequencing batch reactor (SBR) studies were conducted to test the bioaugmentation of JMSTP on total nitrogen (TN) removal under the influent COD/N ratio of 3/1. After the second bioaugmentation, the TN of effluent in the bioaugmented SBR was significantly lower than that in the control SBR. Redundancy analysis results showed that there was a positive correlation between Pseudomonas sp. abundance and TN removal in the bioaugmented SBR. Microbial community analysis showed that, especially after the second bioaugmentation, the abundance of Pseudomonas sp. decreased rapidly, but it was still much higher than that in the control SBR. Correlation network analysis showed that after the addition, Pseudomonas sp. had no significant co-occurrence relationship with other native bacteria, owing to the quick increase and decrease. Our results suggest that JMSTP shows the potential to enhance TN removal through bioaugmentation. Since the effect of bioaugmentation gradually diminishes, further research is still needed to investigate its long-lasting applications

Modification and Expression of mRNA m6A in the Lateral Habenular of Rats after Long-Term Exposure to Blue Light during the Sleep Period

Artificial lighting, especially blue light, is becoming a public-health risk. Excessive exposure to blue light at night has been reported to be associated with brain diseases. However, the mechanisms underlying neuropathy induced by blue light remain unclear. An early anatomical tracing study described the projection of the retina to the lateral habenula (LHb), whereas more mechanistic reports are available on multiple brain functions and neuropsychiatric disorders in the LHb, which are rarely seen in epigenetic studies, particularly N6-methyladenosine (m6A). The purpose of our study was to first expose Sprague-Dawley rats to blue light (6.11 ± 0.05 mW/cm2, the same irradiance as 200 lx of white light in the control group) for 4 h, and simultaneously provide white light to the control group for the same time to enter a sleep period. The experiment was conducted over 12 weeks. RNA m6A modifications and different mRNA transcriptome profiles were observed in the LHb. We refer to this experimental group as BLS. High-throughput MeRIP-seq and mRNA-seq were performed, and we used bioinformatics to analyze the data. There were 188 genes in the LHb that overlapped between differentially m6A-modified mRNA and differentially expressed mRNA. The Kyoto Encyclopedia of Genes and Genomes and gene ontology analysis were used to enrich neuroactive ligand–receptor interaction, long-term depression, the cyclic guanosine monophosphate-dependent protein kinase G (cGMP-PKG) signaling pathway, and circadian entrainment. The m6A methylation level of the target genes in the BLS group was disordered. In conclusion, this study suggests that the mRNA expression and their m6A of the LHb were abnormal after blue light exposure during the sleep period, and the methylation levels of target genes related to synaptic plasticity were disturbed. This study offers a theoretical basis for the scientific use of light

Single-cell RNA-sequencing of cellular heterogeneity and pathogenic mechanisms in paraquat-induced Parkinson's disease with depression

Parkinson's disease (PD) is among the most prevalent neurodegenerative diseases, and approximately one third of patients with PD are estimated to have depression. Paraquat (PQ) exposure is an important environmental risk factor for PD. In this study, we established a mouse model of PQ-induced PD with depression to comprehensively investigate cellular heterogeneity and the mechanisms underlying the progression of depression in the context of PD. We utilized single-cell RNA-seq (scRNA-seq) to acquire the transcriptomic atlas of individual cells from model mice and characterize the gene expression profiles in each differentially expressed cell type. We identified a specific glutamatergic neuron cluster responsible for the development of heterogeneous depression-associated changes and established a comprehensive gene expression atlas. Furthermore, functional enrichment and cell trajectory analyses revealed that the mechanisms underlying the progression of PD with depression were associated with specific glutamatergic neurons. Together, our findings provide a valuable resource for deciphering the cellular heterogeneity of PD with depression. The suggested connection between intrinsic transcriptional states of neurons and the progression of depression can provide insight into potential biomarkers and specific targets for anti-depression treatment in patients with PD. Synopsis: Our results obtained using model mice confirm the core effects of PQ exposure on glutamatergic neurons and their potential role in the development of PD with depression

The interplay between lncRNA NR_030777 and SF3B3 in neuronal damage caused by paraquat

Paraquat (PQ) has been widely acknowledged as an environmental risk factor for Parkinson's disease (PD). However, the interaction between splicing factor and long non-coding RNA (lncRNA) in the process of PQ-induced PD has rarely been studied. Based on previous research, this study focused on splicing factor 3 subunit 3 (SF3B3) and lncRNA NR_030777. After changing the target gene expression level by lentiviral transfection technology, the related gene expression was detected by western blot and qRT-PCR. The expression of SF3B3 protein was reduced in Neuro-2a cells after PQ exposure, and the reactive oxygen species (ROS) scavenger N-acetylcysteine prevented this decline. Knockdown of SF3B3 reduced the PQ-triggered NR_030777 expression increase, and overexpression of NR_030777 reduced the transcriptional and translational level of Sf3b3. Then, knockdown of SF3B3 exacerbated the PQ-induced decrease in cell viability and aggravated the reduction of tyrosine hydroxylase (TH) protein expression. Overexpressing SF3B3 reversed the reduction of TH expression caused by PQ. Moreover, after intervention with the autophagy inhibitor Bafilomycin A1, LC3B-II protein expression was further increased in Neuro-2a cells with the knockdown of SF3B3, indicating that autophagy was enhanced. In conclusion, PQ modulated the interplay between NR_030777 and SF3B3 through ROS production, thereby impairing autophagic flux and causing neuronal damage