Exploring Model Transferability through the Lens of Potential Energy

Transfer learning has become crucial in computer vision tasks due to the vast availability of pre-trained deep learning models. However, selecting the optimal pre-trained model from a diverse pool for a specific downstream task remains a challenge. Existing methods for measuring the transferability of pre-trained models rely on statistical correlations between encoded static features and task labels, but they overlook the impact of underlying representation dynamics during fine-tuning, leading to unreliable results, especially for self-supervised models. In this paper, we present an insightful physics-inspired approach named PED to address these challenges. We reframe the challenge of model selection through the lens of potential energy and directly model the interaction forces that influence fine-tuning dynamics. By capturing the motion of dynamic representations to decline the potential energy within a force-driven physical model, we can acquire an enhanced and more stable observation for estimating transferability. The experimental results on 10 downstream tasks and 12 self-supervised models demonstrate that our approach can seamlessly integrate into existing ranking techniques and enhance their performances, revealing its effectiveness for the model selection task and its potential for understanding the mechanism in transfer learning. Code will be available at https://github.com/lixiaotong97/PED.Comment: Accepted by ICCV 202

mc-BEiT: Multi-choice Discretization for Image BERT Pre-training

Image BERT pre-training with masked image modeling (MIM) becomes a popular practice to cope with self-supervised representation learning. A seminal work, BEiT, casts MIM as a classification task with a visual vocabulary, tokenizing the continuous visual signals into discrete vision tokens using a pre-learned dVAE. Despite a feasible solution, the improper discretization hinders further improvements of image pre-training. Since image discretization has no ground-truth answers, we believe that the masked patch should not be assigned with a unique token id even if a better tokenizer can be obtained. In this work, we introduce an improved BERT-style image pre-training method, namely mc-BEiT, which performs MIM proxy tasks towards eased and refined multi-choice training objectives. Specifically, the multi-choice supervision for the masked image patches is formed by the soft probability vectors of the discrete token ids, which are predicted by the off-the-shelf image tokenizer and further refined by high-level inter-patch perceptions resorting to the observation that similar patches should share their choices. Extensive experiments on classification, segmentation, and detection tasks demonstrate the superiority of our method, e.g., the pre-trained ViT-B achieves 84.1% top-1 fine-tuning accuracy on ImageNet-1K classification, 50.8% mIOU on ADE20K semantic segmentation, 51.2% AP^b and 44.3% AP^m of object detection and instance segmentation on COCO, outperforming the competitive counterparts

Multiwavelength Analysis of a Nearby Heavily Obscured AGN in NGC 449

We presented the multiwavelength analysis of a heavily obscured active galactic nucleus (AGN) in NGC 449. We first constructed a broadband X-ray spectrum using the latest NuSTAR and XMM-Newton data. Its column density ($\simeq 10^{24} \rm{cm}^{-2}$) and photon index ($\Gamma\simeq 2.4$) were reliably obtained by analyzing the broadband X-ray spectrum. However, the scattering fraction and the intrinsic X-ray luminosity could not be well constrained. Combined with the information obtained from the mid-infrared (mid-IR) spectrum and spectral energy distribution (SED) fitting, we derived its intrinsic X-ray luminosity ($\simeq 8.54\times 10^{42} \ \rm{erg\ s}^{-1}$) and scattering fraction ($f_{\rm{scat}}\simeq 0.26\%$). In addition, we also derived the following results: (1). The mass accretion rate of central AGN is about $2.54 \times 10^{-2} \rm{M}_\odot\ \rm{yr}^{-1}$, and the Eddington ratio is $8.39\times 10^{-2}$; (2). The torus of this AGN has a high gas-to-dust ratio ($N_{\rm H}/A_{\rm V}=8.40\times 10^{22}\ \rm{cm}^{-2}\ \rm{mag}^{-1}$); (3). The host galaxy and the central AGN are both in the early stage of co-evolution.Comment: 12 pages, 5 figures, 3 tables, Accepted to PAS

Result of a year-long animal survey in a state-owned forest farm in Beijing, China

BackgroundArtificial forest can have great potential in serving as habitat to wildlife, depending on different management methods. As the state-owned forest farms now play a new role in ecological conservation in China, the biological richness of this kind of land-use type is understudied. Once owned by a mining company, a largest state-owned forest farm, Jingxi Forest Farm, has been reformed to be a state-owned forest farm with the purpose of conservation since 2017. Although this 116.4 km2 forest farm holds a near-healthy montaine ecosystem very representative in North China, a large proportion of artificial coniferous forest in the forest farm has been proven to hold less biodiversity than natural vegetation. This situation, however, provides a great opportunity for ecological restoration and biodiversity conservation. Therefore, from November 2019 to December 2020, we conducted a set of biodiversity surveys, whose results will serve as a baseline for further restoration and conservation.New informationHere, we report the result of a multi-taxa fauna diversity survey conducted in Jingxi Forest Farm mainly in year 2020 with explicit spatial information. It is the first survey of its kind conducted in this area, revealing a total of 19 species of mammals, 86 birds, four reptiles, two amphibians and one fish species, as well as 101 species of insects. Four species of mammals are identified as data-poor species as they have less than 100 occurrence records with coordination in the GBIF database. One species of insect, representing one new provincial record genus of Beijing, is reported

Current status of heart failure in China

As a serious and terminal stage of multiple cardiovascular diseases, heart failure (HF) is one of the most critical components of the current global strategies for the prevention and management of chronic diseases. With aging and the increased incidence of various cardiovascular diseases and related risk factors in the population, HF has also become a main reason for hospitalization and death of the elderly in China, which has resulted in a heavy burden on public health system and the economy. A 2003 report suggested that the prevalence of HF in China was 0.9%, higher in women than in men, and is increasing in proportion with aging. Although the mortality rate of HF seen in hospitals has declined, the long-term prognosis of HF in China is discouraging as demonstrated by a 3-year mortality rate of approximately 30%. HF diagnosis by brain natriuretic peptide, N-terminal pro-B-type natriuretic peptide, and echocardiography in China is unsatisfactory. Room for improvement is needed in drug and nondrug HF treatment in China. Intensive efforts are also needed to promote a real-world use of the current guidelines for recommended drugs that could improve HF patients' prognosis, including angiotensin-converting enzyme inhibitors, angiotensin receptor blockers, and β-blockers. Much work is also needed to embrace the concept of HF management and systems in China. In the future, more attention should also be given to domestic epidemiological and clinical study of HF, focusing on transformation of experimental results to a clinic setting, and ongoing recognition of and attention to HF preserved ejection fraction. An efficient HF management system in China should also highlight the importance of establishing the most cost-effective prevention and therapy strategies

A Hybrid Shuffled Frog Leaping Algorithm and Its Performance Assessment in Multi-Dimensional Symmetric Function

Ensemble learning of swarm intelligence evolutionary algorithm of artificial neural network (ANN) is one of the core research directions in the field of artificial intelligence (AI). As a representative member of swarm intelligence evolutionary algorithm, shuffled frog leaping algorithm (SFLA) has the advantages of simple structure, easy implementation, short operation time, and strong global optimization ability. However, SFLA is susceptible to fall into local optimas in the face of complex and multi-dimensional symmetric function optimization, which leads to the decline of convergence accuracy. This paper proposes an improved shuffled frog leaping algorithm of threshold oscillation based on simulated annealing (SA-TO-SFLA). In this algorithm, the threshold oscillation strategy and simulated annealing strategy are introduced into the SFLA, which makes the local search behavior more diversified and the ability to escape from the local optimas stronger. By using multi-dimensional symmetric function such as drop-wave function, Schaffer function N.2, Rastrigin function, and Griewank function, two groups (i: SFLA, SA-SFLA, TO-SFLA, and SA-TO-SFLA; ii: SFLA, ISFLA, MSFLA, DSFLA, and SA-TO-SFLA) of comparative experiments are designed to analyze the convergence accuracy and convergence time. The results show that the threshold oscillation strategy has strong robustness. Moreover, compared with SFLA, the convergence accuracy of SA-TO-SFLA algorithm is significantly improved, and the median of convergence time is greatly reduced as a whole. The convergence accuracy of SFLA algorithm on these four test functions are 90%, 100%, 78%, and 92.5%, respectively, and the median of convergence time is 63.67 s, 59.71 s, 12.93 s, and 8.74 s, respectively; The convergence accuracy of SA-TO-SFLA algorithm on these four test functions is 99%, 100%, 100%, and 97.5%, respectively, and the median of convergence time is 48.64 s, 32.07 s, 24.06 s, and 3.04 s, respectively

Trends in cause-specific readmissions in heart failure with preserved vs. reduced and mid-range ejection fraction

Aims The aim of this study was to investigate whether the readmission of heart failure (HF) patients has decreased over time and how it differs among HF with preserved ejection fraction (EF) (HFpEF) vs. reduced EF (HFrEF) and mid-range EF (HFmrEF). Methods and results We evaluated HF patients index hospitalized from January 2004 to December 2011 in the Swedish Heart Failure Registry with 1 year follow-up. Outcome measures were the first occurring all-cause, cardiovascular (CV), and HF readmissions. A total of 20 877 HF patients (11 064 HFrEF, 4215 HFmrEF, and 5562 HFpEF) were included in the study. All-cause readmission was the highest in patients with HFpEF, whereas CV and HF readmissions were the highest in HFrEF. From 2004 to 2011, HF readmission rates within 6 months (from 22.3% to 17.3%,P = 0.003) and 1 year (from 27.7% to 23.4%,P = 0.019) in HFpEF declined, and the risk for 1 year HF readmission in HFpEF was reduced by 7% after adjusting for age and sex (P = 0.022). Likewise, risk factors for HF readmission in HFpEF changed. However, no significant changes were observed in all-cause or CV readmission rates in HFpEF, and no significant changes in cause-specific readmissions were observed in HFrEF. Time to the first readmission did not change significantly from 2004 to 2011, regardless of EF subgroup (all P-values &amp;gt; 0.05). Conclusions Declining temporal trend in HF readmission rates was found in HFpEF, but all-cause readmission still remained the highest in HFpEF vs. HFrEF and HFmrEF. More efforts are needed to reduce the non-HF-related readmission in patients with HFpEF.Funding Agencies|Swedish National Board of Health and Welfare; Swedish Association of Local Authorities and Regions; Swedish Society of Cardiology</p

Paired box 8 suppresses tumor angiogenesis and metastasis in gastric cancer through repression of FOXM1 via induction of microRNA-612

Abstract Background Paired box 8 (PAX8) has been documented to be downregulated in gastric cancer. However, its biological function in this malignancy is poorly understood. Methods In the present work, we investigated the effects of PAX8 overexpression and knockdown on the aggressive phenotype of gastric cancer cells. We further checked the involvement of forkhead box M1 (FOXM1), a ubiquitously expressed oncogene that can facilitate gastric cancer progression, in the action of PAX8. Results Ectopic expression of PAX8 blocked the migration and invasion of both AGS and SGC-7901 cells, but had no effect on cell proliferation. Conversely, knockdown of PAX8 enhanced gastric cancer cell migration and invasion. PAX8 overexpression inhibited epithelial-mesenchymal transition (EMT) and pro-angiogenic activity of gastric cancer cells. Mechanistically, PAX8 overexpression downregulated FOXM1 by stimulating microRNA (miR)-612 expression. Ectopic expression of miR-612 recapitulated the effect of PAX8 overexpression on gastric cancer cells, causing an inhibition of migration, invasion, EMT, and angiogenesis. Knockdown of miR-612 or overexpression of FOXM1 significantly reversed the tumor-suppressive activity of PAX8. In vivo studies further demonstrated that PAX8 overexpression restrained tumor angiogenesis and metastasis in nude mice, which was accompanied by increased expression of miR-612 and decreased expression of FOXM1. Conclusions PAX8 exerts a tumor-suppressive effect against gastric cancer cells, largely through induction of miR-612 and repression of FOXM1. Therefore, restoration of PAX8 expression may offer therapeutic benefits in the treatment of gastric cancer

Evaluating Data Augmentation Effects on the Recognition of Sugarcane Leaf Spot

Research on the recognition and segmentation of plant diseases in simple environments based on deep learning has achieved relative success. However, under the conditions of a complex environment and a lack of samples, the model has difficulty recognizing disease spots, or its recognition accuracy is too low. This paper is aimed at investigating how to improve the recognition accuracy of the model when the dataset is in a complex environment and lacks samples. First, for the complex environment, this paper uses DeepLabV3+ to segment sugarcane leaves from complex backgrounds; second, focusing on the lack of training images of sugarcane leaves, two data augmentation methods are used in this paper: supervised data augmentation and deep convolutional generative adversarial networks (DCGANs) for data augmentation. MobileNetV3-large, Alexnet, Resnet, and Densenet are trained by comparing the original dataset, original dataset with supervised data augmentation, original dataset with DCGAN augmentation, background-removed dataset, background-removed dataset with supervised data augmentation, and background-removed dataset with DCGAN augmentation. Then, the recognition abilities of the trained models are compared using the same test set. The optimal network selected based on accuracy and training time is MobileNetV3-large. Classification using MobileNetV3-large trained by the original dataset yielded 53.5% accuracy. By removing the background and adding synthetic images produced by the DCGAN, the accuracy increased to 99%