TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

Large pre-trained transformers are on top of contemporary semantic segmentation benchmarks, but come with high computational cost and a lengthy training. To lift this constraint, we look at efficient semantic segmentation from a perspective of comprehensive knowledge distillation and consider to bridge the gap between multi-source knowledge extractions and transformer-specific patch embeddings. We put forward the Transformer-based Knowledge Distillation (TransKD) framework which learns compact student transformers by distilling both feature maps and patch embeddings of large teacher transformers, bypassing the long pre-training process and reducing the FLOPs by >85.0%. Specifically, we propose two fundamental and two optimization modules: (1) Cross Selective Fusion (CSF) enables knowledge transfer between cross-stage features via channel attention and feature map distillation within hierarchical transformers; (2) Patch Embedding Alignment (PEA) performs dimensional transformation within the patchifying process to facilitate the patch embedding distillation; (3) Global-Local Context Mixer (GL-Mixer) extracts both global and local information of a representative embedding; (4) Embedding Assistant (EA) acts as an embedding method to seamlessly bridge teacher and student models with the teacher's number of channels. Experiments on Cityscapes, ACDC, and NYUv2 datasets show that TransKD outperforms state-of-the-art distillation frameworks and rivals the time-consuming pre-training method. Code is available at https://github.com/RuipingL/TransKD.Comment: Code is available at https://github.com/RuipingL/TransK

Co-Targeting Plk1 and DNMT3a in Advanced Prostate Cancer

Because there is no effective treatment for late-stage prostate cancer (PCa) at this moment, identifying novel targets for therapy of advanced PCa is urgently needed. A new network-based systems biology approach, XDeath, is developed to detect crosstalk of signaling pathways associated with PCa progression. This unique integrated network merges gene causal regulation networks and protein-protein interactions to identify novel co-targets for PCa treatment. The results show that polo-like kinase 1 (Plk1) and DNA methyltransferase 3A (DNMT3a)-related signaling pathways are robustly enhanced during PCa progression and together they regulate autophagy as a common death mode. Mechanistically, it is shown that Plk1 phosphorylation of DNMT3a leads to its degradation in mitosis and that DNMT3a represses Plk1 transcription to inhibit autophagy in interphase, suggesting a negative feedback loop between these two proteins. Finally, a combination of the DNMT inhibitor 5-Aza-2\u27-deoxycytidine (5-Aza) with inhibition of Plk1 suppresses PCa synergistically

3D geological modeling of deep fractured low porosity sandstone gas reservoir in the Kuqa Depression, Tarim Basin

The fractured reservoir is one of the significant petroleum reservoir types in China, representing over one-third of total reserves. The Kuqa Depression in the Tarim Basin is dominated by fractured low-porosity sandstone gas reservoirs with characteristic tight matrix, developed fractures, and edge and bottom water. However, the continued development of these reservoirs has led to various problems, including strong reservoir heterogeneity, low well control, complex gas-water relationships, and early water invasion. Addressing these issues requires a detailed understanding of the reservoir’s geological characteristics. One method for achieving a fine reservoir description is through the use of 3D geological modeling. This high-level, comprehensive characterization technique is widely used throughout the entire life cycle of oil and gas field development. A 3D geological model can accurately predict the actual underground reservoir characteristics and provide a geological basis for later numerical simulation work. Based on a study of the geological characteristics of the Kuqa Depression in the Tarim Basin, a 3D geological modeling technique was developed, which includes structural modeling, facies modeling, petrophysical modeling, and fracture modeling. This technology has been successfully applied to many deep gas reservoirs in the Kuqa Depression of the Tarim Basin, leading to enhanced gas recovery

The Role of Lactic Acid Adsorption by Ion Exchange Chromatography

Background: The polyacrylic resin Amberlite IRA-67 is a promising adsorbent for lactic acid extraction from aqueous solution, but little systematic research has been devoted to the separation efficiency of lactic acid under different operating conditions. Methodology/Principal Findings: In this paper, we investigated the effects of temperature, resin dose and lactic acid loading concentration on the adsorption of lactic acid by Amberlite IRA-67 in batch kinetic experiments. The obtained kinetic data followed the pseudo-second order model well and both the equilibrium and ultimate adsorption slightly decreased with the increase of the temperature at 293–323K and 42.5 g/liter lactic acid loading concentration. The adsorption was a chemically heterogeneous process with a mean free energy value of 12.18 kJ/mol. According to the Boyd _ plot, the lactic acid uptake process was primarily found to be an intraparticle diffusion at a lower concentration (,50 g/liter) but a film diffusion at a higher concentration (.70 g/liter). The values of effective diffusion coefficient D i increased with temperature. By using our Equation (21), the negative values of DGu and DHu revealed that the adsorption process was spontaneous and exothermic. Moreover, the negative value of DSu reflected the decrease of solid-liquid interface randomness at the solid-liquid interface when adsorbing lactic acid on IRA-67. Conclusions/Significance: With the weakly basic resin IRA-67, in situ product removal of lactic acid can be accomplishe

Localized delivery of CRISPR/dCas9 via layer-by-layer self-assembling peptide coating on nanofibers for neural tissue engineering

The clustered regularly interspaced short palindromic repeat (CRISPR) systems have a wide variety of applications besides precise genome editing. In particular, the CRISPR/dCas9 system can be used to control specific gene expression by CRISPR activation (CRISPRa) or interference (CRISPRi). However, the safety concerns associated with viral vectors and the possible off-target issues of systemic administration remain huge concerns to be safe delivery methods for CRISPR/Cas9 systems. In this study, a layer-by-layer (LbL) self-assembling peptide (SAP) coating on nanofibers is developed to mediate localized delivery of CRISPR/dCas9 systems. Specifically, an amphiphilic negatively charged SAP− is first coated onto PCL nanofibers through strong hydrophobic interactions, and the pDNA complexes and positively charged SAP+-RGD are then absorbed via electrostatic interactions. The SAP-coated scaffolds facilitate efficient loading and sustained release of the pDNA complexes, while enhancing cell adhesion and proliferation. As a proof of concept, the scaffolds are used to activate GDNF expression in mammalian cells, and the secreted GDNF subsequently promotes neurite outgrowth of rat neurons. These promising results suggest that the LbL self-assembling peptide coated nanofibers can be a new route to establish a bioactive interface, which provides a simple and efficient platform for the delivery of CRISPR/dCas9 systems for regenerative medicine.Agency for Science, Technology and Research (A*STAR)Ministry of Education (MOE)Accepted versio

Detection of the Diffuse Supernova Neutrino Background with JUNO

As an underground multi-purpose neutrino detector with 20 kton liquid scintillator, Jiangmen Underground Neutrino Observatory (JUNO) is competitive with and complementary to the water-Cherenkov detectors on the search for the diffuse supernova neutrino background (DSNB). Typical supernova models predict 2-4 events per year within the optimal observation window in the JUNO detector. The dominant background is from the neutral-current (NC) interaction of atmospheric neutrinos with 12C nuclei, which surpasses the DSNB by more than one order of magnitude. We evaluated the systematic uncertainty of NC background from the spread of a variety of data-driven models and further developed a method to determine NC background within 15\% with {\it{in}} {\it{situ}} measurements after ten years of running. Besides, the NC-like backgrounds can be effectively suppressed by the intrinsic pulse-shape discrimination (PSD) capabilities of liquid scintillators. In this talk, I will present in detail the improvements on NC background uncertainty evaluation, PSD discriminator development, and finally, the potential of DSNB sensitivity in JUNO

Potential of Core-Collapse Supernova Neutrino Detection at JUNO

JUNO is an underground neutrino observatory under construction in Jiangmen, China. It uses 20kton liquid scintillator as target, which enables it to detect supernova burst neutrinos of a large statistics for the next galactic core-collapse supernova (CCSN) and also pre-supernova neutrinos from the nearby CCSN progenitors. All flavors of supernova burst neutrinos can be detected by JUNO via several interaction channels, including inverse beta decay, elastic scattering on electron and proton, interactions on C12 nuclei, etc. This retains the possibility for JUNO to reconstruct the energy spectra of supernova burst neutrinos of all flavors. The real time monitoring systems based on FPGA and DAQ are under development in JUNO, which allow prompt alert and trigger-less data acquisition of CCSN events. The alert performances of both monitoring systems have been thoroughly studied using simulations. Moreover, once a CCSN is tagged, the system can give fast characterizations, such as directionality and light curve