AdapterGNN: Efficient Delta Tuning Improves Generalization Ability in Graph Neural Networks

Fine-tuning pre-trained models has recently yielded remarkable performance gains in graph neural networks (GNNs). In addition to pre-training techniques, inspired by the latest work in the natural language fields, more recent work has shifted towards applying effective fine-tuning approaches, such as parameter-efficient tuning (delta tuning). However, given the substantial differences between GNNs and transformer-based models, applying such approaches directly to GNNs proved to be less effective. In this paper, we present a comprehensive comparison of delta tuning techniques for GNNs and propose a novel delta tuning method specifically designed for GNNs, called AdapterGNN. AdapterGNN preserves the knowledge of the large pre-trained model and leverages highly expressive adapters for GNNs, which can adapt to downstream tasks effectively with only a few parameters, while also improving the model's generalization ability on the downstream tasks. Extensive experiments show that AdapterGNN achieves higher evaluation performance (outperforming full fine-tuning by 1.4% and 5.5% in the chemistry and biology domains respectively, with only 5% of its parameters tuned) and lower generalization gaps compared to full fine-tuning. Moreover, we empirically show that a larger GNN model can have a worse generalization ability, which differs from the trend observed in large language models. We have also provided a theoretical justification for delta tuning can improve the generalization ability of GNNs by applying generalization bounds

Time-aware Graph Structure Learning via Sequence Prediction on Temporal Graphs

Temporal Graph Learning, which aims to model the time-evolving nature of graphs, has gained increasing attention and achieved remarkable performance recently. However, in reality, graph structures are often incomplete and noisy, which hinders temporal graph networks (TGNs) from learning informative representations. Graph contrastive learning uses data augmentation to generate plausible variations of existing data and learn robust representations. However, rule-based augmentation approaches may be suboptimal as they lack learnability and fail to leverage rich information from downstream tasks. To address these issues, we propose a Time-aware Graph Structure Learning (TGSL) approach via sequence prediction on temporal graphs, which learns better graph structures for downstream tasks through adding potential temporal edges. In particular, it predicts time-aware context embedding based on previously observed interactions and uses the Gumble-Top-K to select the closest candidate edges to this context embedding. Additionally, several candidate sampling strategies are proposed to ensure both efficiency and diversity. Furthermore, we jointly learn the graph structure and TGNs in an end-to-end manner and perform inference on the refined graph. Extensive experiments on temporal link prediction benchmarks demonstrate that TGSL yields significant gains for the popular TGNs such as TGAT and GraphMixer, and it outperforms other contrastive learning methods on temporal graphs. We will release the code in the future.Comment: 10 pages,4 figures,5 table

Parameter-efficient is not sufficient: Exploring Parameter, Memory, and Time Efficient Adapter Tuning for Dense Predictions

Pre-training & fine-tuning is a prevalent paradigm in computer vision (CV). Recently, parameter-efficient transfer learning (PETL) methods have shown promising performance in adapting to downstream tasks with only a few trainable parameters. Despite their success, the existing PETL methods in CV can be computationally expensive and require large amounts of memory and time cost during training, which limits low-resource users from conducting research and applications on large models. In this work, we propose Parameter, Memory, and Time Efficient Visual Adapter ($\mathrm{E^3VA}$) tuning to address this issue. We provide a gradient backpropagation highway for low-rank adapters which eliminates the need for expensive backpropagation through the frozen pre-trained model, resulting in substantial savings of training memory and training time. Furthermore, we optimise the $\mathrm{E^3VA}$ structure for CV tasks to promote model performance. Extensive experiments on COCO, ADE20K, and Pascal VOC benchmarks show that $\mathrm{E^3VA}$ can save up to 62.2% training memory and 26.2% training time on average, while achieving comparable performance to full fine-tuning and better performance than most PETL methods. Note that we can even train the Swin-Large-based Cascade Mask RCNN on GTX 1080Ti GPUs with less than 1.5% trainable parameters.Comment: 14 pages, 4 figures, 5 tables, Submitted to NeurIPS202

MicroRNA-212-5p Prevents Dopaminergic Neuron Death by Inhibiting SIRT2 in MPTP-Induced Mouse Model of Parkinson’s Disease

Recently, emerging evidences show that sirtuins (SIRTs) modulate aging progress and affect neurodegenerative diseases. For example, inhibition of SIRT2 has been recognized to exert neuroprotective effects in Parkinson’s disease (PD). However, current SIRT2 inhibitors are lack of selective property distinguished from its homolog. In this study, we found that SIRT2 protein level was highly increased in PD model, which was negatively regulated by miR-212-5p. In detail, miR-212-5p transfection reduced SIRT2 expression and inhibited SIRT2 activity. In vivo study, miR-212-5p treatment prevented dopaminergic neuron loss and DAT reduction by targeting SIRT2, which means miR-212-5p shows neuroprotective effect in PD. Mechanismly, we found nuclear acetylated p53 was up-regulation according to p53 is a major deacetylation substrate of SIRT2. Furthermore, decreased cytoplasmic p53 promoted autophagy in PD model, which was showed as autophagosomes, autophagic flux, LC3 B and p62 expression. Meanwhile, we also found miR-212-5p treatment somehow alleviated apoptosis in PD model, which might have some underlying mechanisms. In conclusions, our study provides a direct link between miR-212-5p and SIRT2-mediated p53-dependent programmed cell death in the pathogenesis of PD. These findings will give us an insight into the development of highly specifically SIRT2 inhibitor of opening up novel therapeutic avenues for PD

Predicting nosocomial lower respiratory tract infections by a risk index based system

Although belonging to one of the most common type of nosocomial infection, there was currently no simple prediction model for lower respiratory tract infections (LRTIs). This study aims to develop a risk index based system for predicting nosocomial LRTIs based on data from a large point-prevalence survey. Among the 49328 patients included, the prevalence of nosocomial LRTIs was 1.70% (95% confidence interval [CI], 1.64% to 1.76%). The areas under the receiver operating characteristic (ROC) curve for logistic regression and fisher discriminant analysis were 0.907 (95% CI, 0.897 to 0.917) and 0.902 (95% CI, 0.892 to 0.912), respectively. The constructed risk index based system also displayed excellent discrimination (area under the ROC curve: 0.905 [95% CI, 0.895 to 0.915]) to identify LRTI in internal validation. Six risk levels were generated according to the risk score distribution of study population, ranging from 0 to 5, the corresponding prevalence of nosocomial LRTIs were 0.00%, 0.39%, 3.86%, 12.38%, 28.79% and 44.83%, respectively. The sensitivity and specificity of prediction were 0.87 and 0.79, respectively, when the best cut-off point of risk score was set to 14. Our study suggested that this newly constructed risk index based system might be applied to boost more rational infection control programs in clinical settings

High Dose Vitamin E Attenuates Diabetic Nephropathy via Alleviation of Autophagic Stress

It has been reported that autophagic stress, which is involved in many diseases, plays a key role in the development of diabetic nephropathy (DN). In this study, we investigated the effects of high dose vitamin E on renal tubular epithelial cells and autophagic stress-related mechanisms in diabetes condition. In diabetic rats, high dose vitamin E treatment significantly decreased the serum creatinine, urea nitrogen, urinary albumin and urinary protein, reduced the levels of LCN2, HAVCR1, LDH and 8-OHdG in urine, and attenuated the cellular apoptosis and interstitial fibrosis in renal cortex. In vitro, vitamin E could reduce the release of LCN2 and HAVCR1 and the protein levels of caspase 3 and TGF-β1, as well as improve the growth inhibition in cultured HK-2 cells after exposure to advanced glycation end products (AGEs). Also, LC3-II and SQSTM1-positive dots were significantly increased in the renal tubular epithelial cells of DN patients and diabetic rats, and in HK-2 cells after exposure to AGEs, which were markedly declined by vitamin E. In addition, we found that the autophagosome formation was not affected by AGEs, as assessed by the mRNA levels of LC3B, Beclin-1, and ATG7. However, AGEs blocked the lysosomal degradation of autophagosome, which was characterized by a decrease in the enzymatic activity of cathepsin B/cathepsin L and DQ-ovalbumin degradation in HK-2 cells, indicating that AGEs-induced accumulation of autophagic vacuoles was a sign of autophagic stress. Interestingly, vitamin E exerted a protective effect on lysosomes to reduce the autophagic stress. Taken together, we conclude that autophagic stress may play an important part in the progression of DN, and alleviation of autophagic stress though improvement of lysosomal function provides a promising novel approach for treating DN

Effect of external beam radiation therapy versus transcatheter arterial chemoembolization for non-diffuse hepatocellular carcinoma (≥ 5 cm): a multicenter experience over a ten-year period

BackgroundThe optimal local treatment for HCC with tumor diameter ≥ 5 cm is not well established. This research evaluated the effectiveness of external beam radiation therapy (EBRT) versus transcatheter arterial chemoembolization (TACE) for HCC with tumor diameter ≥ 5 cm.MethodsA total of 1210 HCC patients were enrolled in this study, including 302 and 908 patients that received EBRT and TACE, respectively. Propensity score matching (PSM) was used to identify patient pairs with similar baseline characteristics. Overall survival (OS) was the primary study endpoint.ResultsWe identified 428 patients using 1:1 PSM for survival comparison. Compared with the TACE group, the EBRT group had a significantly longer median OS (mOS) before (14.9 vs. 12.3 months, p = 0.0085) and after (16.8 vs. 11.4 months, p = 0.0026) matching. In the subgroup analysis, compared with the TACE group, the EBRT group had a significantly longer mOS for HCC with tumor diameters of 5-7 cm (34.1 vs. 14.3 months, p = 0.04) and 7-10 cm (34.4 vs. 10 months, p = 0.00065), whereas for HCC with tumor diameters ≥ 10 cm, no significant difference in mOS was observed (11.2 vs. 11.2 months, p = 0.83). In addition, the multivariable Cox analysis showed that Child-A, alkaline phosphatase < 125 U/L, and EBRT were independent prognostic indicators for longer survival.ConclusionEBRT is more effective than TACE as the primary local treatment for HCC with tumor diameter ≥ 5 cm, especially for HCC with tumor diameter of 5-10 cm

Economic analysis of border control policies during COVID-19 pandemic : a modelling study to inform cross-border travel policy between Singapore and Thailand

With countries progressing towards high COVID-19 vaccination rates, strategies for border reopening are required. This study focuses on Thailand and Singapore, two countries that share significant tourism visitation, to illustrate a framework for optimizing COVID-19 testing and quarantine policies for bilateral travel with a focus on economic recovery. The timeframe is the month of October 2021, when Thailand and Singapore were preparing to reopen borders for bilateral travel. This study was conducted to provide evidence for the border reopening policy decisions. Incremental net benefit (INB) compared to the pre-opening period was quantified through a willingness-to-travel model, a micro-simulation COVID-19 transmission model and an economic model accounting for medical and non-medical costs/benefits. Multiple testing and quarantine policies were examined, and Pareto optimal (PO) policies and the most influential components were identified. The highest possible INB for Thailand is US $125.94 million, under a PO policy with no quarantine but with antigen rapid tests (ARTs) pre-departure and upon arrival to enter both countries. The highest possible INB for Singapore is US$29.78 million, under another PO policy with no quarantine on both sides, no testing to enter Thailand, and ARTs pre-departure and upon arrival to enter Singapore. Tourism receipts and costs/profits of testing and quarantine have greater economic impacts than that from COVID-19 transmission. Provided healthcare systems have sufficient capacity, great economic benefits can be gained for both countries by relaxing border control measures

Study of the B−→Λc+Λˉc−K−B^{-} \to \Lambda_{c}^{+} \bar{\Lambda}_{c}^{-} K^{-} decay

The decay $B^{-} \to \Lambda_{c}^{+} \bar{\Lambda}_{c}^{-} K^{-}$ is studied in proton-proton collisions at a center-of-mass energy of $\sqrt{s}=13$ TeV using data corresponding to an integrated luminosity of 5 $\mathrm{fb}^{-1}$ collected by the LHCb experiment. In the $\Lambda_{c}^+ K^{-}$ system, the $\Xi_{c}(2930)^{0}$ state observed at the BaBar and Belle experiments is resolved into two narrower states, $\Xi_{c}(2923)^{0}$ and $\Xi_{c}(2939)^{0}$, whose masses and widths are measured to be $$m(\Xi_{c}(2923)^{0}) = 2924.5 \pm 0.4 \pm 1.1 \,\mathrm{MeV}, \\ m(\Xi_{c}(2939)^{0}) = 2938.5 \pm 0.9 \pm 2.3 \,\mathrm{MeV}, \\ \Gamma(\Xi_{c}(2923)^{0}) = \phantom{000}4.8 \pm 0.9 \pm 1.5 \,\mathrm{MeV},\\ \Gamma(\Xi_{c}(2939)^{0}) = \phantom{00}11.0 \pm 1.9 \pm 7.5 \,\mathrm{MeV},$$ where the first uncertainties are statistical and the second systematic. The results are consistent with a previous LHCb measurement using a prompt $\Lambda_{c}^{+} K^{-}$ sample. Evidence of a new $\Xi_{c}(2880)^{0}$ state is found with a local significance of $3.8\,\sigma$, whose mass and width are measured to be $2881.8 \pm 3.1 \pm 8.5\,\mathrm{MeV}$ and $12.4 \pm 5.3 \pm 5.8 \,\mathrm{MeV}$, respectively. In addition, evidence of a new decay mode $\Xi_{c}(2790)^{0} \to \Lambda_{c}^{+} K^{-}$ is found with a significance of $3.7\,\sigma$. The relative branching fraction of $B^{-} \to \Lambda_{c}^{+} \bar{\Lambda}_{c}^{-} K^{-}$ with respect to the $B^{-} \to D^{+} D^{-} K^{-}$ decay is measured to be $2.36 \pm 0.11 \pm 0.22 \pm 0.25$, where the first uncertainty is statistical, the second systematic and the third originates from the branching fractions of charm hadron decays.Comment: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2022-028.html (LHCb public pages

Measurement of the ratios of branching fractions R(D∗)\mathcal{R}(D^{*}) and R(D0)\mathcal{R}(D^{0})

The ratios of branching fractions $\mathcal{R}(D^{*})\equiv\mathcal{B}(\bar{B}\to D^{*}\tau^{-}\bar{\nu}_{\tau})/\mathcal{B}(\bar{B}\to D^{*}\mu^{-}\bar{\nu}_{\mu})$ and $\mathcal{R}(D^{0})\equiv\mathcal{B}(B^{-}\to D^{0}\tau^{-}\bar{\nu}_{\tau})/\mathcal{B}(B^{-}\to D^{0}\mu^{-}\bar{\nu}_{\mu})$ are measured, assuming isospin symmetry, using a sample of proton-proton collision data corresponding to 3.0 fb${ }^{-1}$ of integrated luminosity recorded by the LHCb experiment during 2011 and 2012. The tau lepton is identified in the decay mode $\tau^{-}\to\mu^{-}\nu_{\tau}\bar{\nu}_{\mu}$. The measured values are $\mathcal{R}(D^{*})=0.281\pm0.018\pm0.024$ and $\mathcal{R}(D^{0})=0.441\pm0.060\pm0.066$, where the first uncertainty is statistical and the second is systematic. The correlation between these measurements is $\rho=-0.43$. Results are consistent with the current average of these quantities and are at a combined 1.9 standard deviations from the predictions based on lepton flavor universality in the Standard Model.Comment: All figures and tables, along with any supplementary material and additional information, are available at https://cern.ch/lhcbproject/Publications/p/LHCb-PAPER-2022-039.html (LHCb public pages