Exploring Model Transferability through the Lens of Potential Energy

Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. However, selecting the optimal pre-trained model from a diverse pool for a specific downstream task remains a challenge. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels, but they overlook the impact of underlying representation dynamics during fine-tuning, leading to unreliable results, especially for self-supervised models. In this paper, we present an insightful physics-inspired approach named PED to address these challenges. We reframe the challenge of model selection through the lens of potential energy and directly model the interaction forces that influence fine-tuning dynamics. By capturing the motion of dynamic representations to decline the potential energy within a force-driven physical model, we can acquire an enhanced and more stable observation for estimating transferability. The experimental results on 10 downstream tasks and 12 self-supervised models demonstrate that our approach can seamlessly integrate into existing ranking techniques and enhance their performances, revealing its effectiveness for the model selection task and its potential for understanding the mechanism in transfer learning. Code will be available at https://github.com/lixiaotong97/PED.Comment: Accepted by ICCV 202

mc-BEiT: Multi-choice Discretization for Image BERT Pre-training

Image BERT pre-training with masked image modeling (MIM) becomes a popular practice to cope with self-supervised representation learning. A seminal work, BEiT, casts MIM as a classification task with a visual vocabulary, tokenizing the continuous visual signals into discrete vision tokens using a pre-learned dVAE. Despite a feasible solution, the improper discretization hinders further improvements of image pre-training. Since image discretization has no ground-truth answers, we believe that the masked patch should not be assigned with a unique token id even if a better tokenizer can be obtained. In this work, we introduce an improved BERT-style image pre-training method, namely mc-BEiT, which performs MIM proxy tasks towards eased and refined multi-choice training objectives. Specifically, the multi-choice supervision for the masked image patches is formed by the soft probability vectors of the discrete token ids, which are predicted by the off-the-shelf image tokenizer and further refined by high-level inter-patch perceptions resorting to the observation that similar patches should share their choices. Extensive experiments on classification, segmentation, and detection tasks demonstrate the superiority of our method, e.g., the pre-trained ViT-B achieves 84.1% top-1 fine-tuning accuracy on ImageNet-1K classification, 50.8% mIOU on ADE20K semantic segmentation, 51.2% AP^b and 44.3% AP^m of object detection and instance segmentation on COCO, outperforming the competitive counterparts

Gestational TSH and FT4 Reference Intervals in Chinese Women: A Systematic Review and Meta-Analysis

Background: Serum thyroid-stimulating hormone (TSH) and free thyroxine (FT4) change dynamically during pregnancy. Differences in geographic regions, populations, and manufacturer's methodologies can affect the reference intervals for thyroid function tests. The 2017 guidelines of the American Thyroid Association (ATA) recommended 4.0 mU/L as the cut-off point for the upper limit of serum TSH in early pregnancy. A systematic review is called for to establish practical, gestational-specific TSH and FT4 reference intervals for pregnant Chinese women and to explore whether the criteria are suitable for China.Methods: English and Chinese articles published from inception to Aug 2017 were searched in the PubMed, EMBASE, and SCIE English-language databases and the CNKI, WanFang, and CQVIP Chinese databases. The relative descent or ascent rates of serum TSH and FT4 were calculated, after which Comprehensive Meta-Analysis V2.0 software was used to analyze the data.Results: Eleven studies (6 in English and 5 in Chinese), five kits and 11,629 Chinese women from nine cities were considered in this meta-analysis. Compared with the reference ranges provided by manufacturers, serum TSH decreased in the first trimester, with the upper limit declining by 21.7% (5.0–36.6%), to a value close to 4.0 mU/L, and the lower limit declining by 85.7% (73.5–97.1%). It continued decreasing in the second trimester, with the upper limit declining by 24.0% (6.4–40.9%) and the lower limit declining by 40.7% (9.0–85.7%). For FT4, the upper limit fluctuated slightly, and the lower limit increased by 6.8% (1.0–14.6%) in the first trimester. Serum FT4 dropped gradually, with the upper limit declining by 21.8% (2.5–31.8%) and the lower limit declining by 12.7% (2.6–19.6%) in the second trimester. During the third trimester, the upper limit decreased by 25.1% (12.7–35.0%), while the lower limit decreased by 20.9% (14.8–27.3%).Conclusions: Various regions, kits and test methods affect the gestational TSH and FT4 levels. The non-pregnant serum TSH upper limit minus 22% is very close to 4.0 mU/L, which can be used as a sub-optimal approach to represent the cut-off value for pregnant Chinese women in the first trimester

Thyroid function and epilepsy: a two-sample Mendelian randomization study

BackgroundThyroid hormones (THs) play a crucial role in regulating various biological processes, particularly the normal development and functioning of the central nervous system (CNS). Epilepsy is a prevalent neurological disorder with multiple etiologies. Further in-depth research on the role of thyroid hormones in epilepsy is warranted.MethodsGenome-wide association study (GWAS) data for thyroid function and epilepsy were obtained from the ThyroidOmics Consortium and the International League Against Epilepsy (ILAE) Consortium cohort, respectively. A total of five indicators of thyroid function and ten types of epilepsy were included in the analysis. Two-sample Mendelian randomization (MR) analyses were conducted to investigate potential causal relations between thyroid functions and various epilepsies. Multiple testing correction was performed using Bonferroni correction. Heterogeneity was calculated with the Cochran’s Q statistic test. Horizontal pleiotropy was evaluated by the MR-Egger regression intercept. The sensitivity was also examined by leave-one-out strategy.ResultsThe findings indicated the absence of any causal relationship between abnormalities in thyroid hormone and various types of epilepsy. The study analyzed the odds ratio (OR) between thyroid hormones and various types of epilepsy in five scenarios, including free thyroxine (FT4) on focal epilepsy with hippocampal sclerosis (IVW, OR = 0.9838, p = 0.02223), hyperthyroidism on juvenile absence epilepsy (IVW, OR = 0.9952, p = 0.03777), hypothyroidism on focal epilepsy with hippocampal sclerosis (IVW, OR = 1.0075, p = 0.01951), autoimmune thyroid diseases (AITDs) on generalized epilepsy in all documented cases (weighted mode, OR = 1.0846, p = 0.0346) and on childhood absence epilepsy (IVW, OR = 1.0050, p = 0.04555). After Bonferroni correction, none of the above results showed statistically significant differences.ConclusionThis study indicates that there is no causal relationship between thyroid-related disorders and various types of epilepsy. Future research should aim to avoid potential confounding factors that might impact the study

The R2R3-MYB Factor FhMYB5 From Freesia hybrida Contributes to the Regulation of Anthocyanin and Proanthocyanidin Biosynthesis

The flavonoids are important and nourishing compounds for plants and human. The transcription regulation of anthocyanin and proanthocyanidin (PA) biosynthesis was extensively studied in dicot compared with monocot plants. In this study, we characterized the functionality of an R2R3-MYB gene FhMYB5 from the monocotyledonous flowering plant of Iridaceae, Freesia hybrida. Multiple sequence alignment and phylogenetic analysis implied that FhMYB5 was clustered into grapevine VvMYB5b subclade. Correlation analysis indicated that the spatio-temporal expression patterns of FhMYB5 coincided well with anthocyanin and PA accumulations in Freesia per se. Furthermore, transient transfection assays in Freesia protoplasts revealed that the late flavonoid biosynthetic genes (e.g., DFR and LDOX) were slightly up-regulated by FhMYB5 alone, whereas both early and late biosynthetic genes were significantly activated when FhMYB5 were co-infected with either of the two IIIf clade bHLH genes, FhTT8L and FhGL3L. Moreover, these results were further confirmed by co-transfection of FhMYB5 with either of the bHLH genes aforementioned into protoplasts expressing GUS reporter gene driven by Freesia promoters. In addition, the overexpression of FhMYB5 in tobacco and Arabidopsis could also significantly up-regulate the expression of genes participating in the general flavonoid pathway. In conclusion, FhMYB5 was proved to function in the general flavonoid pathway in Freesia. The results implied a function conservation of flavonoid biosynthesis related MYB regulators in angiosperm plants

Network Pharmacology-Based Validation of Caveolin-1 as a Key Mediator of Ai Du Qing Inhibition of Drug Resistance in Breast Cancer

Chinese formulas have been paid increasing attention in cancer multidisciplinary therapy due to their multi-targets and multi-substances property. Here, we aim to investigate the anti-breast cancer and chemosensitizing function of Ai Du Qing (ADQ) formula made up of Hedyotis diffusa, Curcuma zedoaria (Christm.) Rosc., Astragalus membranaceus (Fisch.) Bunge, and Glycyrrhiza uralensis Fisch. Our findings revealed that ADQ significantly inhibited cell proliferation in both parental and chemo-resistant breast cancer cells, but with little cytotoxcity effects on the normal cells. Besides, ADQ was found to facilitate the G2/M arresting and apoptosis induction effects of paclitaxel. Network pharmacology and bioinformatics analysis further demonstrated that ADQ yielded 132 candidate compounds and 297 potential targets, and shared 22 putative targets associating with breast cancer chemoresponse. Enrichment analysis and experimental validation demonstrated that ADQ might improve breast cancer chemosensitivity via inhibiting caveolin-1, which further triggered expression changes of cell cycle-related proteins p21/cyclinB1 and apoptosis-associated proteins PARP1, BAX and Bcl-2. Besides, ADQ enhanced in vivo paclitaxel chemosensitivity on breast cancer. Our study not only uncovers the novel function and mechanisms of ADQ in chemosensitizing breast cancer at least partly via targeting caveolin-1, but also sheds novel light in utilizing network pharmacology in Chinese Medicine research

The oyster genome reveals stress adaptation and complexity of shell formation

The Pacific oyster Crassostrea gigas belongs to one of the most species-rich but genomically poorly explored phyla, the Mollusca. Here we report the sequencing and assembly of the oyster genome using short reads and a fosmid-pooling strategy, along with transcriptomes of development and stress response and the proteome of the shell. The oyster genome is highly polymorphic and rich in repetitive sequences, with some transposable elements still actively shaping variation. Transcriptome studies reveal an extensive set of genes responding to environmental stress. The expansion of genes coding for heat shock protein 70 and inhibitors of apoptosis is probably central to the oyster's adaptation to sessile life in the highly stressful intertidal zone. Our analyses also show that shell formation in molluscs is more complex than currently understood and involves extensive participation of cells and their exosomes. The oyster genome sequence fills a void in our understanding of the Lophotrochozoa. © 2012 Macmillan Publishers Limited. All rights reserved