A Simple and Generic Framework for Feature Distillation via Channel-wise Transformation

Knowledge distillation is a popular technique for transferring the knowledge from a large teacher model to a smaller student model by mimicking. However, distillation by directly aligning the feature maps between teacher and student may enforce overly strict constraints on the student thus degrade the performance of the student model. To alleviate the above feature misalignment issue, existing works mainly focus on spatially aligning the feature maps of the teacher and the student, with pixel-wise transformation. In this paper, we newly find that aligning the feature maps between teacher and student along the channel-wise dimension is also effective for addressing the feature misalignment issue. Specifically, we propose a learnable nonlinear channel-wise transformation to align the features of the student and the teacher model. Based on it, we further propose a simple and generic framework for feature distillation, with only one hyper-parameter to balance the distillation loss and the task specific loss. Extensive experimental results show that our method achieves significant performance improvements in various computer vision tasks including image classification (+3.28% top-1 accuracy for MobileNetV1 on ImageNet-1K), object detection (+3.9% bbox mAP for ResNet50-based Faster-RCNN on MS COCO), instance segmentation (+2.8% Mask mAP for ResNet50-based Mask-RCNN), and semantic segmentation (+4.66% mIoU for ResNet18-based PSPNet in semantic segmentation on Cityscapes), which demonstrates the effectiveness and the versatility of the proposed method. The code will be made publicly available.Comment: 13 page

SCSC: Spatial Cross-scale Convolution Module to Strengthen both CNNs and Transformers

This paper presents a module, Spatial Cross-scale Convolution (SCSC), which is verified to be effective in improving both CNNs and Transformers. Nowadays, CNNs and Transformers have been successful in a variety of tasks. Especially for Transformers, increasing works achieve state-of-the-art performance in the computer vision community. Therefore, researchers start to explore the mechanism of those architectures. Large receptive fields, sparse connections, weight sharing, and dynamic weight have been considered keys to designing effective base models. However, there are still some issues to be addressed: large dense kernels and self-attention are inefficient, and large receptive fields make it hard to capture local features. Inspired by the above analyses and to solve the mentioned problems, in this paper, we design a general module taking in these design keys to enhance both CNNs and Transformers. SCSC introduces an efficient spatial cross-scale encoder and spatial embed module to capture assorted features in one layer. On the face recognition task, FaceResNet with SCSC can improve 2.7% with 68% fewer FLOPs and 79% fewer parameters. On the ImageNet classification task, Swin Transformer with SCSC can achieve even better performance with 22% fewer FLOPs, and ResNet with CSCS can improve 5.3% with similar complexity. Furthermore, a traditional network (e.g., ResNet) embedded with SCSC can match Swin Transformer's performance.Comment: ICCV2023 Workshop (New Ideas in Vision Transformers

Human antibody VH domains targeting uPAR as candidate therapeutics for cancers

The high expression of uPAR has been linked to tumor progression, invasion, and metastasis in several types of cancer. Such overexpression of uPAR makes it a potential target for immunotherapies across common cancers such as breast, colorectal, lung, ovarian cancer, and melanoma. In our study, two high-affinity and specific human VH domain antibody candidates, designed as clones 3 and 115, were isolated from a phage-displayed human VH antibody library. Domain-based bispecific T- cell engagers (DbTE) based on these two antibodies exhibited potent killing of uPAR-positive cancer cells. Thus, these two anti-uPAR domain antibodies are promising candidates for treating uPAR positive cancers

Medication Non-adherence and Condomless Anal Intercourse Increased Substantially During the COVID-19 Pandemic Among MSM PrEP Users: A Retrospective Cohort Study in Four Chinese Metropolises

BackgroundThe coronavirus disease (COVID-19) pandemic has impacted HIV prevention strategies globally. However, changes in pre-exposure prophylaxis (PrEP) adherence and HIV-related behaviors, and their associations with medication adherence among men who have sex with men (MSM) PrEP users remain unclear since the onset of the COVID-19 pandemic.MethodsA Retrospective Cohort Study of HIV-negative MSM PrEP users was conducted in four Chinese metropolises from December 2018 to March 2020, assessing the changes in PrEP adherence and HIV-related behaviors before and during the COVID-19. The primary outcome was poor PrEP adherence determined from self-reported missing at least one PrEP dose in the previous month. We used multivariable logistic regression to determine factors correlated with poor adherence during COVID-19.ResultsWe enrolled 791 eligible participants (418 [52.8%] in daily PrEP and 373 [47.2%] in event-driven PrEP). Compared with the data conducted before the COVID-19, the proportion of PrEP users decreased from 97.9 to 64.3%, and the proportion of poor PrEP adherence increased from 23.6 to 50.1% during the COVID-19 [odds ratio (OR) 3.24, 95% confidence interval (CI) 2.62–4.02]. While the percentage of condomless anal intercourse (CAI) with regular partners (11.8 vs. 25.7%) and with casual partners (4.4 vs. 9.0%) both significantly increased. The proportion of those who were tested for HIV decreased from 50.1 to 25.9%. Factors correlated with poor PrEP adherence during the COVID-19 included not being tested for HIV (adjusted odds ratio [aOR] = 1.38 [95% CI: 1.00, 1.91]), using condoms consistently with regular partners (vs. never, aOR = 2.19 [95% CI: 1.16, 4.13]), and being married or cohabitating with a woman (vs. not married, aOR = 3.08 [95% CI: 1.60, 5.95]).ConclusionsIncreased poor PrEP adherence and CAI along with the decrease in HIV testing can lead to an increase in HIV acquisition and drug resistance to PrEP. Targeted interventions are needed to improve PrEP adherence and HIV prevention strategies

Internet-Based HIV Self-Testing Among Men Who Have Sex With Men Through Pre-exposure Prophylaxis: 3-Month Prospective Cohort Analysis From China.

BACKGROUND: Routine HIV testing accompanied with pre-exposure prophylaxis (PrEP) requires innovative support in a real-world setting. OBJECTIVE: This study aimed to determine the usage of HIV self-testing (HIVST) kits and their secondary distribution to partners among men who have sex with men (MSM) in China, who use PrEP, in an observational study between 2018 and 2019. METHODS: In 4 major cities in China, we prospectively followed-up MSM from the China Real-world oral PrEP demonstration study, which provides daily or on-demand PrEP for 12 months, to assess the usage and secondary distribution of HIVST on quarterly follow-ups. Half of the PrEP users were randomized to receive 2 HIVSTs per month in addition to quarterly facility-based HIV testing. We evaluated the feasibility of providing HIVST to PrEP users. RESULTS: We recruited 939 MSM and randomized 471 to receive HIVST, among whom 235 (49.9%) were daily and 236 (50.1%) were on-demand PrEP users. At baseline, the median age was 29 years, 390 (82.0%) men had at least college-level education, and 119 (25.3%) had never undergone facility-based HIV testing before. Three months after PrEP initiation, 341 (74.5%) men had used the HIVST provided to them and found it very easy to use. Among them, 180 of 341 (52.8%) men had distributed the HIVST kits it to other MSM, and 132 (51.6%) among the 256 men who returned HIVST results reported that used it with their sexual partners at the onset of intercourse. Participants on daily PrEP were more likely to use HIVST (adjusted hazard ratio=1.3, 95% CI 1.0-1.6) and distribute HIVST kits (adjusted hazard ratio=1.3, 95% CI 1.1-1.7) than those using on-demand PrEP. CONCLUSIONS: MSM who used PrEP had a high rate of usage and secondary distribution of HIVST kits, especially among those on daily PrEP, which suggested high feasibility and necessity for HIVST after PrEP initiation. Assuming that fourth-generation HIVST kits are available, HIVST may be able to replace facility-based HIV testing to a certain extent. TRIAL REGISTRATION: Chinese Clinical Trial Registry ChiCTR1800020374; https://www.chictr.org.cn/showprojen.aspx?proj=32481. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.1136/bmjopen-2019-036231

Real-time Monitoring for the Next Core-Collapse Supernova in JUNO

Core-collapse supernova (CCSN) is one of the most energetic astrophysical events in the Universe. The early and prompt detection of neutrinos before (pre-SN) and during the SN burst is a unique opportunity to realize the multi-messenger observation of the CCSN events. In this work, we describe the monitoring concept and present the sensitivity of the system to the pre-SN and SN neutrinos at the Jiangmen Underground Neutrino Observatory (JUNO), which is a 20 kton liquid scintillator detector under construction in South China. The real-time monitoring system is designed with both the prompt monitors on the electronic board and online monitors at the data acquisition stage, in order to ensure both the alert speed and alert coverage of progenitor stars. By assuming a false alert rate of 1 per year, this monitoring system can be sensitive to the pre-SN neutrinos up to the distance of about 1.6 (0.9) kpc and SN neutrinos up to about 370 (360) kpc for a progenitor mass of 30$M_{\odot}$ for the case of normal (inverted) mass ordering. The pointing ability of the CCSN is evaluated by using the accumulated event anisotropy of the inverse beta decay interactions from pre-SN or SN neutrinos, which, along with the early alert, can play important roles for the followup multi-messenger observations of the next Galactic or nearby extragalactic CCSN.Comment: 24 pages, 9 figure

Simple Baselines for Image Restoration

Although there have been significant advances in the field of image restoration recently, the system complexity of the state-of-the-art (SOTA) methods is increasing as well, which may hinder the convenient analysis and comparison of methods. In this paper, we propose a simple baseline that exceeds the SOTA methods and is computationally efficient. To further simplify the baseline, we reveal that the nonlinear activation functions, e.g. Sigmoid, ReLU, GELU, Softmax, etc. are not necessary: they could be replaced by multiplication or removed. Thus, we derive a Nonlinear Activation Free Network, namely NAFNet, from the baseline. SOTA results are achieved on various challenging benchmarks, e.g. 33.69 dB PSNR on GoPro (for image deblurring), exceeding the previous SOTA 0.38 dB with only 8.4% of its computational costs; 40.30 dB PSNR on SIDD (for image denoising), exceeding the previous SOTA 0.28 dB with less than half of its computational costs. The code and the pre-trained models are released at https://github.com/megvii-research/NAFNet.Comment: Accepted to ECCV 2022; Code: https://github.com/megvii-research/NAFNet