Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case

Adam is a commonly used stochastic optimization algorithm in machine learning. However, its convergence is still not fully understood, especially in the non-convex setting. This paper focuses on exploring hyperparameter settings for the convergence of vanilla Adam and tackling the challenges of non-ergodic convergence related to practical application. The primary contributions are summarized as follows: firstly, we introduce precise definitions of ergodic and non-ergodic convergence, which cover nearly all forms of convergence for stochastic optimization algorithms. Meanwhile, we emphasize the superiority of non-ergodic convergence over ergodic convergence. Secondly, we establish a weaker sufficient condition for the ergodic convergence guarantee of Adam, allowing a more relaxed choice of hyperparameters. On this basis, we achieve the almost sure ergodic convergence rate of Adam, which is arbitrarily close to $o(1/\sqrt{K})$. More importantly, we prove, for the first time, that the last iterate of Adam converges to a stationary point for non-convex objectives. Finally, we obtain the non-ergodic convergence rate of $O(1/K)$ for function values under the Polyak-Lojasiewicz (PL) condition. These findings build a solid theoretical foundation for Adam to solve non-convex stochastic optimization problems

Tumor promoter TPA activates Wnt/β-catenin signaling in a casein kinase 1-dependent manner.

The tumor promoter 12-O-tetra-decanoylphorbol-13-acetate (TPA) has been defined by its ability to promote tumorigenesis on carcinogen-initiated mouse skin. Activation of Wnt/β-catenin signaling has a decisive role in mouse skin carcinogenesis, but it remains unclear how TPA activates Wnt/β-catenin signaling in mouse skin carcinogenesis. Here, we found that TPA could enhance Wnt/β-catenin signaling in a casein kinase 1 (CK1) ε/δ-dependent manner. TPA stabilized CK1ε and enhanced its kinase activity. TPA further induced the phosphorylation of LRP6 at Thr1479 and Ser1490 and the formation of a CK1ε-LRP6-axin1 complex, leading to an increase in cytosolic β-catenin. Moreover, TPA increased the association of β-catenin with TCF4E in a CK1ε/δ-dependent way, resulting in the activation of Wnt target genes. Consistently, treatment with a selective CK1ε/δ inhibitor SR3029 suppressed TPA-induced skin tumor formation in vivo, probably through blocking Wnt/β-catenin signaling. Taken together, our study has identified a pathway by which TPA activates Wnt/β-catenin signaling

SPOC learner's final grade prediction based on a novel sampling batch normalization embedded neural network method

Recent years have witnessed the rapid growth of Small Private Online Courses (SPOC) which is able to highly customized and personalized to adapt variable educational requests, in which machine learning techniques are explored to summarize and predict the learner's performance, mostly focus on the final grade. However, the problem is that the final grade of learners on SPOC is generally seriously imbalance which handicaps the training of prediction model. To solve this problem, a sampling batch normalization embedded deep neural network (SBNEDNN) method is developed in this paper. First, a combined indicator is defined to measure the distribution of the data, then a rule is established to guide the sampling process. Second, the batch normalization (BN) modified layers are embedded into full connected neural network to solve the data imbalanced problem. Experimental results with other three deep learning methods demonstrates the superiority of the proposed method.Comment: 11 pages, 5 figures, ICAIS 202

A suitable method for alpine wetland delineation: An example for the headwater area of the yellow river, Tibetan Plateau

Alpine wetlands are one of the most important ecosystems in the Three Rivers Source Area, China, which plays an important role in regulating the regional hydrological cycle and carbon cycle. Accordingly, Wetland area and its distribution are of great significance for wetland management and scientific research. In our study, a new wetland classification model which based on geomorphological types and combine object-oriented and decision tree classification model (ODTC), and used a new wetland classification system to accurately extract the wetland distributed in the Headwater Area of the Yellow River (HAYR) of the Qinghai-Tibet Plateau (QTP), China. The object-oriented method was first used to segment the image into several areas according to similarity in Pixels and Textures, and then the wetland was extracted through a decision tree constructed based on geomorphological types. The wetland extracted by the model was compared with that by other seven commonly methods, such as support vector machine (SVM) and random forest (RF), and it proved the accuracy was improved by 10%–20%. The overall classification accuracy rate was 98.9%. According to our results, the HAYR’s wetland area is 3142.3 km2, accounting for 16.1% of the study area. Marsh wetlands and flood wetlands accounted for 37.7% and 16.7% respectively. A three-dimensional map of the area showed that alpine wetlands in the research region are distributed around lakes, piedmont groundwater overflow belts, and inter-mountain catchment basin. This phenomenon demonstrates that hydrogeological circumstances influence alpine wetlands’ genesis and evolution. This work provides a new approach to investigating alpine wetlands

Metrics of the Gynecologic Oncology Literature Focused on Cited Utilization and Costs

OBJECTIVE: The newest findings on literature utilization relevant to gynecologic oncology were published by Thomson Reuters during June 2013 as determinants of journal standing. Our objective was to assess the different metrics reported for relative impact and cost for journals relevant to gynecologic oncology. METHODS: 55 journals were evaluated for Impact Factor (IF), 5Year IF, Immediacy Index, Cited Half Life, Eigenfactor (EF) Score, Article Influence (AI) scores and subscription costs obtained from publisher information. RESULTS: CA-A Cancer Journal for Clinicians had the highest IF (101.78) & AI (24.502). The top EF cancer-specific journals were the Journal of Clinical Oncology, Cancer Research, Clinical Cancer Research and Oncogene. Rankings for Gynecologic Oncology (409 articles, 18,243 citations) were IF=3.929, 43/55, EF=0.038, 28/55, and AI=1.099, 44/55, all higher than the previous year. The IF improved from the 5year IF in 31 journals, including Gynecologic Oncology, 29/31. Subscription costs for Gynecologic Oncology compared favorably to other journals. CONCLUSIONS: The high utilization of review information in CA-A Cancer Journal for Clinicians and Nature Review Cancer illustrated by the IF coupled with a relatively low number of articles and short cited half life indicates that they serve as a leading source of quoted cancer statistics (CA-A Cancer Journal for Clinicians). Rankings for Gynecologic Oncology and the International Journal of Gynecologic Cancer have improved. Regardless of specialty size, the Impact Factor for Gynecologic Oncology is respectably strong. The decreased IF in 44% of the journals may reflect the international economy\u27s effect on cancer research