7 research outputs found
DropKey
In this paper, we focus on analyzing and improving the dropout technique for
self-attention layers of Vision Transformer, which is important while
surprisingly ignored by prior works. In particular, we conduct researches on
three core questions: First, what to drop in self-attention layers? Different
from dropping attention weights in literature, we propose to move dropout
operations forward ahead of attention matrix calculation and set the Key as the
dropout unit, yielding a novel dropout-before-softmax scheme. We theoretically
verify that this scheme helps keep both regularization and probability features
of attention weights, alleviating the overfittings problem to specific patterns
and enhancing the model to globally capture vital information; Second, how to
schedule the drop ratio in consecutive layers? In contrast to exploit a
constant drop ratio for all layers, we present a new decreasing schedule that
gradually decreases the drop ratio along the stack of self-attention layers. We
experimentally validate the proposed schedule can avoid overfittings in
low-level features and missing in high-level semantics, thus improving the
robustness and stableness of model training; Third, whether need to perform
structured dropout operation as CNN? We attempt patch-based block-version of
dropout operation and find that this useful trick for CNN is not essential for
ViT. Given exploration on the above three questions, we present the novel
DropKey method that regards Key as the drop unit and exploits decreasing
schedule for drop ratio, improving ViTs in a general way. Comprehensive
experiments demonstrate the effectiveness of DropKey for various ViT
architectures, e.g. T2T and VOLO, as well as for various vision tasks, e.g.,
image classification, object detection, human-object interaction detection and
human body shape recovery.Comment: Accepted by CVPR202
Changes in Wastewater Treatment Performance and the Microbial Community during the Bioaugmentation of a Denitrifying <i>Pseudomonas </i>Strain in the Low Carbon–Nitrogen Ratio Sequencing Batch Reactor
The low carbon–nitrogen ratio (C/N) of influent wastewater results in the insufficient carbon source for the process of denitrification in urban wastewater treatment plants (WWTPs). A denitrifying bacterial strain, Pseudomonas sp. JMSTP, was isolated and demonstrated effective denitrification ability under a low C/N ratio of 1.5–4 (w/w) in anoxic conditions. Sequencing batch reactor (SBR) studies were conducted to test the bioaugmentation of JMSTP on total nitrogen (TN) removal under the influent COD/N ratio of 3/1. After the second bioaugmentation, the TN of effluent in the bioaugmented SBR was significantly lower than that in the control SBR. Redundancy analysis results showed that there was a positive correlation between Pseudomonas sp. abundance and TN removal in the bioaugmented SBR. Microbial community analysis showed that, especially after the second bioaugmentation, the abundance of Pseudomonas sp. decreased rapidly, but it was still much higher than that in the control SBR. Correlation network analysis showed that after the addition, Pseudomonas sp. had no significant co-occurrence relationship with other native bacteria, owing to the quick increase and decrease. Our results suggest that JMSTP shows the potential to enhance TN removal through bioaugmentation. Since the effect of bioaugmentation gradually diminishes, further research is still needed to investigate its long-lasting applications
Modification and Expression of mRNA m6A in the Lateral Habenular of Rats after Long-Term Exposure to Blue Light during the Sleep Period
Artificial lighting, especially blue light, is becoming a public-health risk. Excessive exposure to blue light at night has been reported to be associated with brain diseases. However, the mechanisms underlying neuropathy induced by blue light remain unclear. An early anatomical tracing study described the projection of the retina to the lateral habenula (LHb), whereas more mechanistic reports are available on multiple brain functions and neuropsychiatric disorders in the LHb, which are rarely seen in epigenetic studies, particularly N6-methyladenosine (m6A). The purpose of our study was to first expose Sprague-Dawley rats to blue light (6.11 ± 0.05 mW/cm2, the same irradiance as 200 lx of white light in the control group) for 4 h, and simultaneously provide white light to the control group for the same time to enter a sleep period. The experiment was conducted over 12 weeks. RNA m6A modifications and different mRNA transcriptome profiles were observed in the LHb. We refer to this experimental group as BLS. High-throughput MeRIP-seq and mRNA-seq were performed, and we used bioinformatics to analyze the data. There were 188 genes in the LHb that overlapped between differentially m6A-modified mRNA and differentially expressed mRNA. The Kyoto Encyclopedia of Genes and Genomes and gene ontology analysis were used to enrich neuroactive ligand–receptor interaction, long-term depression, the cyclic guanosine monophosphate-dependent protein kinase G (cGMP-PKG) signaling pathway, and circadian entrainment. The m6A methylation level of the target genes in the BLS group was disordered. In conclusion, this study suggests that the mRNA expression and their m6A of the LHb were abnormal after blue light exposure during the sleep period, and the methylation levels of target genes related to synaptic plasticity were disturbed. This study offers a theoretical basis for the scientific use of light
Single-cell RNA-sequencing of cellular heterogeneity and pathogenic mechanisms in paraquat-induced Parkinson's disease with depression
Parkinson's disease (PD) is among the most prevalent neurodegenerative diseases, and approximately one third of patients with PD are estimated to have depression. Paraquat (PQ) exposure is an important environmental risk factor for PD. In this study, we established a mouse model of PQ-induced PD with depression to comprehensively investigate cellular heterogeneity and the mechanisms underlying the progression of depression in the context of PD. We utilized single-cell RNA-seq (scRNA-seq) to acquire the transcriptomic atlas of individual cells from model mice and characterize the gene expression profiles in each differentially expressed cell type. We identified a specific glutamatergic neuron cluster responsible for the development of heterogeneous depression-associated changes and established a comprehensive gene expression atlas. Furthermore, functional enrichment and cell trajectory analyses revealed that the mechanisms underlying the progression of PD with depression were associated with specific glutamatergic neurons. Together, our findings provide a valuable resource for deciphering the cellular heterogeneity of PD with depression. The suggested connection between intrinsic transcriptional states of neurons and the progression of depression can provide insight into potential biomarkers and specific targets for anti-depression treatment in patients with PD. Synopsis: Our results obtained using model mice confirm the core effects of PQ exposure on glutamatergic neurons and their potential role in the development of PD with depression
The interplay between lncRNA NR_030777 and SF3B3 in neuronal damage caused by paraquat
Paraquat (PQ) has been widely acknowledged as an environmental risk factor for Parkinson's disease (PD). However, the interaction between splicing factor and long non-coding RNA (lncRNA) in the process of PQ-induced PD has rarely been studied. Based on previous research, this study focused on splicing factor 3 subunit 3 (SF3B3) and lncRNA NR_030777. After changing the target gene expression level by lentiviral transfection technology, the related gene expression was detected by western blot and qRT-PCR. The expression of SF3B3 protein was reduced in Neuro-2a cells after PQ exposure, and the reactive oxygen species (ROS) scavenger N-acetylcysteine prevented this decline. Knockdown of SF3B3 reduced the PQ-triggered NR_030777 expression increase, and overexpression of NR_030777 reduced the transcriptional and translational level of Sf3b3. Then, knockdown of SF3B3 exacerbated the PQ-induced decrease in cell viability and aggravated the reduction of tyrosine hydroxylase (TH) protein expression. Overexpressing SF3B3 reversed the reduction of TH expression caused by PQ. Moreover, after intervention with the autophagy inhibitor Bafilomycin A1, LC3B-II protein expression was further increased in Neuro-2a cells with the knockdown of SF3B3, indicating that autophagy was enhanced. In conclusion, PQ modulated the interplay between NR_030777 and SF3B3 through ROS production, thereby impairing autophagic flux and causing neuronal damage