Exploring Model Transferability through the Lens of Potential Energy

Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. However, selecting the optimal pre-trained model from a diverse pool for a specific downstream task remains a challenge. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels, but they overlook the impact of underlying representation dynamics during fine-tuning, leading to unreliable results, especially for self-supervised models. In this paper, we present an insightful physics-inspired approach named PED to address these challenges. We reframe the challenge of model selection through the lens of potential energy and directly model the interaction forces that influence fine-tuning dynamics. By capturing the motion of dynamic representations to decline the potential energy within a force-driven physical model, we can acquire an enhanced and more stable observation for estimating transferability. The experimental results on 10 downstream tasks and 12 self-supervised models demonstrate that our approach can seamlessly integrate into existing ranking techniques and enhance their performances, revealing its effectiveness for the model selection task and its potential for understanding the mechanism in transfer learning. Code will be available at https://github.com/lixiaotong97/PED.Comment: Accepted by ICCV 202

mc-BEiT: Multi-choice Discretization for Image BERT Pre-training

Image BERT pre-training with masked image modeling (MIM) becomes a popular practice to cope with self-supervised representation learning. A seminal work, BEiT, casts MIM as a classification task with a visual vocabulary, tokenizing the continuous visual signals into discrete vision tokens using a pre-learned dVAE. Despite a feasible solution, the improper discretization hinders further improvements of image pre-training. Since image discretization has no ground-truth answers, we believe that the masked patch should not be assigned with a unique token id even if a better tokenizer can be obtained. In this work, we introduce an improved BERT-style image pre-training method, namely mc-BEiT, which performs MIM proxy tasks towards eased and refined multi-choice training objectives. Specifically, the multi-choice supervision for the masked image patches is formed by the soft probability vectors of the discrete token ids, which are predicted by the off-the-shelf image tokenizer and further refined by high-level inter-patch perceptions resorting to the observation that similar patches should share their choices. Extensive experiments on classification, segmentation, and detection tasks demonstrate the superiority of our method, e.g., the pre-trained ViT-B achieves 84.1% top-1 fine-tuning accuracy on ImageNet-1K classification, 50.8% mIOU on ADE20K semantic segmentation, 51.2% AP^b and 44.3% AP^m of object detection and instance segmentation on COCO, outperforming the competitive counterparts

SpikeBERT: A Language Spikformer Trained with Two-Stage Knowledge Distillation from BERT

Spiking neural networks (SNNs) offer a promising avenue to implement deep neural networks in a more energy-efficient way. However, the network architectures of existing SNNs for language tasks are too simplistic, and deep architectures have not been fully explored, resulting in a significant performance gap compared to mainstream transformer-based networks such as BERT. To this end, we improve a recently-proposed spiking transformer (i.e., Spikformer) to make it possible to process language tasks and propose a two-stage knowledge distillation method for training it, which combines pre-training by distilling knowledge from BERT with a large collection of unlabelled texts and fine-tuning with task-specific instances via knowledge distillation again from the BERT fine-tuned on the same training examples. Through extensive experimentation, we show that the models trained with our method, named SpikeBERT, outperform state-of-the-art SNNs and even achieve comparable results to BERTs on text classification tasks for both English and Chinese with much less energy consumption

Experimental quantum adversarial learning with programmable superconducting qubits

Quantum computing promises to enhance machine learning and artificial intelligence. Different quantum algorithms have been proposed to improve a wide spectrum of machine learning tasks. Yet, recent theoretical works show that, similar to traditional classifiers based on deep classical neural networks, quantum classifiers would suffer from the vulnerability problem: adding tiny carefully-crafted perturbations to the legitimate original data samples would facilitate incorrect predictions at a notably high confidence level. This will pose serious problems for future quantum machine learning applications in safety and security-critical scenarios. Here, we report the first experimental demonstration of quantum adversarial learning with programmable superconducting qubits. We train quantum classifiers, which are built upon variational quantum circuits consisting of ten transmon qubits featuring average lifetimes of 150 $\mu$s, and average fidelities of simultaneous single- and two-qubit gates above 99.94% and 99.4% respectively, with both real-life images (e.g., medical magnetic resonance imaging scans) and quantum data. We demonstrate that these well-trained classifiers (with testing accuracy up to 99%) can be practically deceived by small adversarial perturbations, whereas an adversarial training process would significantly enhance their robustness to such perturbations. Our results reveal experimentally a crucial vulnerability aspect of quantum learning systems under adversarial scenarios and demonstrate an effective defense strategy against adversarial attacks, which provide a valuable guide for quantum artificial intelligence applications with both near-term and future quantum devices.Comment: 26 pages, 17 figures, 8 algorithm

CCL21/CCR7 Prevents Apoptosis via the ERK Pathway in Human Non-Small Cell Lung Cancer Cells

Previously, we confirmed that C-C chemokine receptor 7 (CCR7) promotes cell proliferation via the extracellular signal-regulated kinase (ERK) pathway, but its role in apoptosis of non-small cell lung cancer (NSCLC) cell lines remains unknown. A549 and H460 cells of NSCLC were used to examine the effect of CCL21/CCR7 on apoptosis using flow cytometry. The results showed that activation of CCR7 by its specific ligand, exogenous chemokine ligand 21 (CCL21), was associated with a significant decline in the percent of apoptosis. Western blot and real-time PCR assays indicated that activation of CCR7 significantly caused upregulation of anti-apoptotic bcl-2 and downregulation of pro-apoptotic bax and caspase-3, but not p53, at both protein and mRNA levels. CCR7 small interfering RNA significantly attenuated these effects of exogenous CCL21. Besides, PD98059, a selective inhibitor of MEK that disrupts the activation of downstream ERK, significantly abolished these effects of CCL21/CCR7. Coimmunoprecipitation further confirmed that there was an interaction between p-ERK and bcl-2, bax, or caspase-3, particularly in the presence of CCL21. These results strongly suggest that CCL21/CCR7 prevents apoptosis by upregulating the expression of bcl-2 and by downregulating the expression of bax and caspase-3 potentially via the ERK pathway in A549 and H460 cells of NSCLC

Impact of opioid-free analgesia on pain severity and patient satisfaction after discharge from surgery: multispecialty, prospective cohort study in 25 countries

Background: Balancing opioid stewardship and the need for adequate analgesia following discharge after surgery is challenging. This study aimed to compare the outcomes for patients discharged with opioid versus opioid-free analgesia after common surgical procedures.Methods: This international, multicentre, prospective cohort study collected data from patients undergoing common acute and elective general surgical, urological, gynaecological, and orthopaedic procedures. The primary outcomes were patient-reported time in severe pain measured on a numerical analogue scale from 0 to 100% and patient-reported satisfaction with pain relief during the first week following discharge. Data were collected by in-hospital chart review and patient telephone interview 1 week after discharge.Results: The study recruited 4273 patients from 144 centres in 25 countries; 1311 patients (30.7%) were prescribed opioid analgesia at discharge. Patients reported being in severe pain for 10 (i.q.r. 1-30)% of the first week after discharge and rated satisfaction with analgesia as 90 (i.q.r. 80-100) of 100. After adjustment for confounders, opioid analgesia on discharge was independently associated with increased pain severity (risk ratio 1.52, 95% c.i. 1.31 to 1.76; P < 0.001) and re-presentation to healthcare providers owing to side-effects of medication (OR 2.38, 95% c.i. 1.36 to 4.17; P = 0.004), but not with satisfaction with analgesia (beta coefficient 0.92, 95% c.i. -1.52 to 3.36; P = 0.468) compared with opioid-free analgesia. Although opioid prescribing varied greatly between high-income and low- and middle-income countries, patient-reported outcomes did not.Conclusion: Opioid analgesia prescription on surgical discharge is associated with a higher risk of re-presentation owing to side-effects of medication and increased patient-reported pain, but not with changes in patient-reported satisfaction. Opioid-free discharge analgesia should be adopted routinely

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals <1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data

Evaporation and crystallization dynamics in evaporating saline droplets

The evaporation of saline droplets on solid surfaces has been a topic of interest, since it plays increasingly important roles in various industrial and scientific applications. Therefore, in this project, the evaporation of NaCl-water droplet on hydrophilic substrates is studied. The effects of three influencing factors, namely the initial concentration of NaCl, presence of fluorescent particles and different types of substrates, are investigated in terms of droplet dynamics, flow fields, and crystallization by experimental approaches. The droplets are deposited onto unheated substrates and evaporate at controlled room temperature and relative humidity, and PIV analysis is used to visualize the flow fields. First of all, the experimental results agree with the common trend that higher NaCl concentration leads to higher contact angle, faster evaporation rate and flow velocity, and the presence of fluorescent particles leads to higher evaporation rate. Meanwhile, the contact line pinning effect of the fluorescent particles and receding effect of NaCl are spotted, and moreover, the slip-stick behaviour of contact line has been noticed and proven to be induced by the opposing effect of pining and receding, followed by analysis showing that the slip-stick behaviour can be suppressed by increasing the initial NaCl concentration. Next, the analysis of crystallization behaviour shows that, when pinning effect occurs, higher initial contact angle/concentration would result in higher contact angle when crystallization just occurs, but this trend is reversed when pinning effect is weak. Furthermore, four clearly defined distinct stages of evaporation are found in the evaporation on the silicon wafer, and each stage has unique droplet dynamics as well as flow patterns and can be easily distinguished by certain phenomena as starting and ending points. Last but not least, a relationship is established between the flow fields and location of NaCl crystallization forming on soda lemon glass, the NaCl crystal is often generated at where asymmetric vortexes locate.Bachelor of Engineering (Mechanical Engineering

An Improved Shapley Value Method for a Green Supply Chain Income Distribution Mechanism

Low-carbon development and environmental remediation are key factors for green resource-based supply chains in China. With this aim in mind, by applying game theory under uncertain market demand, this paper incorporates low-carbon development and environmental remediation into a resource-based supply chain coordination model for decentralized and centralized markets. The results show that a centralized market can lead to improvement in total profit. Furthermore, based on an improved Shapley value method, a theoretical model for the centralized market income distribution mechanism is developed that incorporates three corrective risk factors, ecological investment, and technological level. Finally, a numerical analysis is conducted using a MATLAB simulation to obtain intuitive results, which, in turn, show the validity of incentive income distribution mechanisms for green supply chain development in China