A high-resolution dynamical view on momentum methods for over-parameterized neural networks

Due to the simplicity and efficiency of the first-order gradient method, it has been widely used in training neural networks. Although the optimization problem of the neural network is non-convex, recent research has proved that the first-order method is capable of attaining a global minimum for training over-parameterized neural networks, where the number of parameters is significantly larger than that of training instances. Momentum methods, including heavy ball method (HB) and Nesterov's accelerated method (NAG), are the workhorse first-order gradient methods owning to their accelerated convergence. In practice, NAG often exhibits better performance than HB. However, current research fails to distinguish their convergence difference in training neural networks. Motivated by this, we provide convergence analysis of HB and NAG in training an over-parameterized two-layer neural network with ReLU activation, through the lens of high-resolution dynamical systems and neural tangent kernel (NTK) theory. Compared to existing works, our analysis not only establishes tighter upper bounds of the convergence rate for both HB and NAG, but also characterizes the effect of the gradient correction term, which leads to the acceleration of NAG over HB. Finally, we validate our theoretical result on three benchmark datasets.Comment: 19 page

GraphMoco:a Graph Momentum Contrast Model that Using Multimodel Structure Information for Large-scale Binary Function Representation Learning

In the field of cybersecurity, the ability to compute similarity scores at the function level is import. Considering that a single binary file may contain an extensive amount of functions, an effective learning framework must exhibit both high accuracy and efficiency when handling substantial volumes of data. Nonetheless, conventional methods encounter several limitations. Firstly, accurately annotating different pairs of functions with appropriate labels poses a significant challenge, thereby making it difficult to employ supervised learning methods without risk of overtraining on erroneous labels. Secondly, while SOTA models often rely on pre-trained encoders or fine-grained graph comparison techniques, these approaches suffer from drawbacks related to time and memory consumption. Thirdly, the momentum update algorithm utilized in graph-based contrastive learning models can result in information leakage. Surprisingly, none of the existing articles address this issue. This research focuses on addressing the challenges associated with large-scale BCSD. To overcome the aforementioned problems, we propose GraphMoco: a graph momentum contrast model that leverages multimodal structural information for efficient binary function representation learning on a large scale. Our approach employs a CNN-based model and departs from the usage of memory-intensive pre-trained models. We adopt an unsupervised learning strategy that effectively use the intrinsic structural information present in the binary code. Our approach eliminates the need for manual labeling of similar or dissimilar information.Importantly, GraphMoco demonstrates exceptional performance in terms of both efficiency and accuracy when operating on extensive datasets. Our experimental results indicate that our method surpasses the current SOTA approaches in terms of accuracy.Comment: 22 pages,7 figure

Contralateral upper tract urothelial carcinoma after nephroureterectomy: the predictive role of DNA methylation

Abstract Background Aberrant methylation of genes is one of the most common epigenetic modifications involved in the development of urothelial carcinoma. However, it is unknown the predictive role of methylation to contralateral new upper tract urothelial carcinoma (UTUC) after radical nephroureterectomy (RNU). We retrospectively investigated the predictive role of DNA methylation and other clinicopathological factors in the contralateral upper tract urothelial carcinoma (UTUC) recurrence after radical nephroureterectomy (RNU) in a large single-center cohort of patients. Methods In a retrospective design, methylation of 10 genes was analyzed on tumor specimens belonging to 664 consecutive patients treated by RNU for primary UTUC. Median follow-up was 48 mo (range: 3–144 mo). Gene methylation was accessed by methylation-sensitive polymerase chain reaction, and we calculated the methylation index (MI), a reflection of the extent of methylation. The log-rank test and Cox regression were used to identify the predictor of contralateral UTUC recurrence. Results Thirty (4.5%) patients developed a subsequent contralateral UTUC after a median follow-up time of 27.5 (range: 2–139) months. Promoter methylation for at least one gene promoter locus was present in 88.9% of UTUC. Fewer methylation and lower MI (P = 0.001) were seen in the tumors with contralateral UTUC recurrence than the tumors without contralateral recurrence. High MI (P = 0.007) was significantly correlated with poor cancer-specific survival. Multivariate analysis indicated that unmethylated RASSF1A (P = 0.039), lack of bladder recurrence prior to contralateral UTUC (P = 0.009), history of renal transplantation (P < 0.001), and preoperative renal insufficiency (P = 0.002) are independent risk factors for contralateral UTUC recurrence after RNU. Conclusions Our data suggest a potential role of DNA methylation in predicting contralateral UTUC recurrence after RNU. Such information could help identify patients at high risk of new contralateral UTUC recurrence after RNU who need close surveillance during follow up.http://deepblue.lib.umich.edu/bitstream/2027.42/110306/1/13046_2015_Article_120.pd

Comprehensive analysis to identify a novel diagnostic marker of lung adenocarcinoma and its immune infiltration landscape

BackgroundLung cancer continues to be a problem faced by all of humanity. It is the cancer with the highest morbidity and mortality in the world, and the most common histological type of lung cancer is lung adenocarcinoma (LUAD), accounting for about 40% of lung malignant tumors. This study was conducted to discuss and explore the immune-related biomarkers and pathways during the development and progression of LUAD and their relationship with immunocyte infiltration.MethodsThe cohorts of data used in this study were downloaded from the Gene Expression Complex (GEO) database and the Cancer Genome Atlas Program (TCGA) database. Through the analysis of differential expression analysis, weighted gene co-expression network analysis (WGCNA), and least absolute shrinkage and selection operator(LASSO), selecting the module with the highest correlation with LUAD progression, and then the HUB gene was further determined. The Gene Ontology (GO), Kyoto Encyclopedia of Genes and Genomes (KEGG), and Gene Set Enrichment Analysis (GSEA) were then used to study the function of these genes. Single-sample GSEA (ssGSEA) analysis was used to investigate the penetration of 28 immunocytes and their relationship with HUB genes. Finally, the receiver operating characteristic curve (ROC) was used to evaluate these HUB genes accurately to diagnose LUAD. In addition, additional cohorts were used for external validation. Based on the TCGA database, the effect of the HUB genes on the prognosis of LUAD patients was assessed using the Kaplan-Meier curve. The mRNA levels of some HUB genes in cancer cells and normal cells were analyzed by reverse transcription-quantitative polymerase chain reaction (RT-qPCR).ResultsThe turquoise module with the highest correlation with LUAD was identified among the seven modules obtained with WGCNA. Three hundred fifty-four differential genes were chosen. After LASSO analysis, 12 HUB genes were chosen as candidate biomarkers for LUAD expression. According to the immune infiltration results, CD4 + T cells, B cells, and NK cells were high in LUAD sample tissue. The ROC curve showed that all 12 HUB genes had a high diagnostic value. Finally, the functional enrichment analysis suggested that the HUB gene is mainly related to inflammatory and immune responses. According to the RT-qPCR study, we found that the expression of DPYSL2, OCIAD2, and FABP4 in A549 was higher than BEAS-2B. The expression content of DPYSL2 was lower in H1299 than in BEAS-2B. However, the expression difference of FABP4 and OCIAD2 genes in H1299 lung cancer cells was insignificant, but both showed a trend of increase.ConclusionsThe mechanism of LUAD pathogenesis and progression is closely linked to T cells, B cells, and monocytes. 12 HUB genes(ADAMTS8, CD36, DPYSL2, FABP4, FGFR4, HBA2, OCIAD2, PARP1, PLEKHH2, STX11, TCF21, TNNC1) may participate in the progression of LUAD via immune-related signaling pathways

Development and validation of a questionnaire to measure the congenital heart disease of children’s family stressor

BackgroundFamilies of children with congenital heart disease (CHD) face tremendous stressors in the process of coping with the disease, which threatens the health of families of children with CHD. Studies have shown that nursing interventions focusing on family stress management can improve parents’ ability to cope with illness and promote family health. At present, there is no measuring tool for family stressors of CHD.MethodsThe items of the scale were generated through qualitative interviews and a literature review. Initial items were evaluated by seven experts to determine content validity. Factor analysis and reliability testing were conducted with a convenience sample of 670 family members. The criterion-related validity of the scale was calculated using scores on the Self-Rating Anxiety Scale (SAS).ResultsThe CHD Children’s Family Stressor Scale consisted of six dimensions and 41 items. In the exploratory factor analysis, the cumulative explained variance of the six factors was 61.085%. In the confirmatory factor analysis, the six factors in the EFA were well validated, indicating that the model fits well. The correlation coefficient between CHD Children’s Family Stressor Scale and SAS was r = 0.504 (p &lt; 0.001), which indicated that the criterion-related validity of the scale was good. In the reliability test, Cronbach’s α coefficients of six sub-scales were 0.774–0.940, and the scale-level Cronbach’s α coefficient value was 0.945.ConclusionThe study indicates that the CHD Children’s Family Stressor Scale is valid and reliable, and it is recommended for use in clinical practice to assess CHD children’s family stressors

OC6 Phase II: Integration and verification of a new soil–structure interaction model for offshore wind design

This paper provides a summary of the work done within the OC6 Phase II project, which was focused on the implementation and verification of an advanced soil–structure interaction model for offshore wind system design and analysis. The soil–structure interaction model comes from the REDWIN project and uses an elastoplastic, macroelement model with kinematic hardening, which captures the stiffness and damping characteristics of offshore wind foundations more accurately than more traditional and simplified soil–structure interaction modeling approaches. Participants in the OC6 project integrated this macroelement capability to coupled aero-hydro-servo-elastic offshore wind turbine modeling tools and verified the implementation by comparing simulation results across the modeling tools for an example monopile design. The simulation results were also compared to more traditional soil–structure interaction modeling approaches like apparent fixity, coupled springs, and distributed springs models. The macroelement approach resulted in smaller overall loading in the system due to both shifts in the system frequencies and increased energy dissipation. No validation work was performed, but the macroelement approach has shown increased accuracy within the REDWIN project, resulting in decreased uncertainty in the design. For the monopile design investigated here, that implies a less conservative and thus more cost-effective offshore wind design.US Department of Energy Office of Energy Efficiency and Renewable Energy Wind Energy Technologies Office, Grant/Award Number: DE-AC36-08GO2830