16 research outputs found

    Homogeneous Second-Order Descent Framework: A Fast Alternative to Newton-Type Methods

    Full text link
    This paper proposes a homogeneous second-order descent framework (HSODF) for nonconvex and convex optimization based on the generalized homogeneous model (GHM). In comparison to the Newton steps, the GHM can be solved by extremal symmetric eigenvalue procedures and thus grant an advantage in ill-conditioned problems. Moreover, GHM extends the ordinary homogeneous model (OHM) to allow adaptiveness in the construction of the aggregated matrix. Consequently, HSODF is able to recover some well-known second-order methods, such as trust-region methods and gradient regularized methods, while maintaining comparable iteration complexity bounds. We also study two specific realizations of HSODF. One is adaptive HSODM, which has a parameter-free O(Ļµāˆ’3/2)O(\epsilon^{-3/2}) global complexity bound for nonconvex second-order Lipschitz continuous objective functions. The other one is homotopy HSODM, which is proven to have a global linear rate of convergence without strong convexity. The efficiency of our approach to ill-conditioned and high-dimensional problems is justified by some preliminary numerical results.Comment: improved writin

    A Universal Trust-Region Method for Convex and Nonconvex Optimization

    Full text link
    This paper presents a universal trust-region method simultaneously incorporating quadratic regularization and the ball constraint. We introduce a novel mechanism to set the parameters in the proposed method that unifies the analysis for convex and nonconvex optimization. Our method exhibits an iteration complexity of O~(Ļµāˆ’3/2)\tilde O(\epsilon^{-3/2}) to find an approximate second-order stationary point for nonconvex optimization. Meanwhile, the analysis reveals that the universal method attains an O(Ļµāˆ’1/2)O(\epsilon^{-1/2}) complexity bound for convex optimization and can be accelerated. These results are complementary to the existing literature as the trust-region method was historically conceived for nonconvex optimization. Finally, we develop an adaptive universal method to address practical implementations. The numerical results show the effectiveness of our method in both nonconvex and convex problems

    A Homogenization Approach for Gradient-Dominated Stochastic Optimization

    Full text link
    Gradient dominance property is a condition weaker than strong convexity, yet it sufficiently ensures global convergence for first-order methods even in non-convex optimization. This property finds application in various machine learning domains, including matrix decomposition, linear neural networks, and policy-based reinforcement learning (RL). In this paper, we study the stochastic homogeneous second-order descent method (SHSODM) for gradient-dominated optimization with Ī±āˆˆ[1,2]\alpha \in [1, 2] based on a recently proposed homogenization approach. Theoretically, we show that SHSODM achieves a sample complexity of O(Ļµāˆ’7/(2Ī±)+1)O(\epsilon^{-7/(2 \alpha) +1}) for Ī±āˆˆ[1,3/2)\alpha \in [1, 3/2) and O~(Ļµāˆ’2/Ī±)\tilde{O}(\epsilon^{-2/\alpha}) for Ī±āˆˆ[3/2,2]\alpha \in [3/2, 2]. We further provide a SHSODM with a variance reduction technique enjoying an improved sample complexity of O(Ļµāˆ’(7āˆ’3Ī±)/(2Ī±))O( \epsilon ^{-( 7-3\alpha ) /( 2\alpha )}) for Ī±āˆˆ[1,3/2)\alpha \in [1,3/2). Our results match the state-of-the-art sample complexity bounds for stochastic gradient-dominated optimization without \emph{cubic regularization}. Since the homogenization approach only relies on solving extremal eigenvector problems instead of Newton-type systems, our methods gain the advantage of cheaper iterations and robustness in ill-conditioned problems. Numerical experiments on several RL tasks demonstrate the efficiency of SHSODM compared to other off-the-shelf methods

    Trust Region Methods For Nonconvex Stochastic Optimization Beyond Lipschitz Smoothness

    Full text link
    In many important machine learning applications, the standard assumption of having a globally Lipschitz continuous gradient may fail to hold. This paper delves into a more general (L0,L1)(L_0, L_1)-smoothness setting, which gains particular significance within the realms of deep neural networks and distributionally robust optimization (DRO). We demonstrate the significant advantage of trust region methods for stochastic nonconvex optimization under such generalized smoothness assumption. We show that first-order trust region methods can recover the normalized and clipped stochastic gradient as special cases and then provide a unified analysis to show their convergence to first-order stationary conditions. Motivated by the important application of DRO, we propose a generalized high-order smoothness condition, under which second-order trust region methods can achieve a complexity of O(Ļµāˆ’3.5)\mathcal{O}(\epsilon^{-3.5}) for convergence to second-order stationary points. By incorporating variance reduction, the second-order trust region method obtains an even better complexity of O(Ļµāˆ’3)\mathcal{O}(\epsilon^{-3}), matching the optimal bound for standard smooth optimization. To our best knowledge, this is the first work to show convergence beyond the first-order stationary condition for generalized smooth optimization. Preliminary experiments show that our proposed algorithms perform favorably compared with existing methods

    cuPDLP-C: A Strengthened Implementation of cuPDLP for Linear Programming by C language

    Full text link
    A recent GPU implementation of the Restarted Primal-Dual Hybrid Gradient Method for Linear Programming was proposed in Lu and Yang (2023). Its computational results demonstrate the significant computational advantages of the GPU-based first-order algorithm on certain large-scale problems. The average performance also achieves a level close to commercial solvers for the first time in history. However, due to limitations in experimental hardware and the disadvantage of implementing the algorithm in Julia compared to C language, neither the commercial solver nor cuPDLP reached their maximum efficiency. Therefore, in this report, we have re-implemented and optimized cuPDLP in C language. Utilizing state-of-the-art CPU and GPU hardware, we extensively compare cuPDLP with the best commercial solvers. The experiments further highlight its substantial computational advantages and potential for solving large-scale linear programming problems. We also discuss the profound impact this breakthrough may have on mathematical programming research and the entire operations research community.Comment: fix typos, update numerical result

    An Enhanced ADMM-based Interior Point Method for Linear and Conic Optimization

    Full text link
    The ADMM-based interior point (ABIP, Lin et al. 2021) method is a hybrid algorithm that effectively combines interior point method (IPM) and first-order methods to achieve a performance boost in large-scale linear optimization. Different from traditional IPM that relies on computationally intensive Newton steps, the ABIP method applies the alternating direction method of multipliers (ADMM) to approximately solve the barrier penalized problem. However, similar to other first-order methods, this technique remains sensitive to condition number and inverse precision. In this paper, we provide an enhanced ABIP method with multiple improvements. Firstly, we develop an ABIP method to solve the general linear conic optimization and establish the associated iteration complexity. Secondly, inspired by some existing methods, we develop different implementation strategies for ABIP method, which substantially improve its performance in linear optimization. Finally, we conduct extensive numerical experiments in both synthetic and real-world datasets to demonstrate the empirical advantage of our developments. In particular, the enhanced ABIP method achieves a 5.8x reduction in the geometric mean of run time on 105105 selected LP instances from Netlib, and it exhibits advantages in certain structured problems such as SVM and PageRank. However, the enhanced ABIP method still falls behind commercial solvers in many benchmarks, especially when high accuracy is desired. We posit that it can serve as a complementary tool alongside well-established solvers

    Diversity of Trichoderma species associated with the black rot disease of Gastrodia elata, including four new species

    Get PDF
    IntroductionTrichoderma species establish symbiotic relationships with plants through both parasitic and mutualistic mechanisms. While some Trichoderma species act as plant pathogenic fungi, others utilize various strategies to protect and enhance plant growth.MethodsPhylogenetic positions of new species of Trichoderma were determined through multi-gene analysis relying on the internal transcribed spacer (ITS) regions of the ribosomal DNA, the translation elongation factor 1-Ī± (tef1-Ī±) gene, and the RNA polymerase II (rpb2) gene. Additionally, pathogenicity experiments were conducted, and the aggressiveness of each isolate was evaluated based on the area of the cross-section of the infected site.ResultsIn this study, 13 Trichoderma species, including 9 known species and 4 new species, namely, T. delicatum, T. robustum, T. perfasciculatum, and T. subulatum were isolated from the diseased tubers of Gastrodia elata in Yunnan, China. Among the known species, T. hamatum had the highest frequency. T. delicatum belonged to the Koningii clade. T. robustum and T. perfasciculatum were assigned to the Virens clade. T. subulatum emerged as a new member of the Spirale clade. Pathogenicity experiments were conducted on the new species T. robustum, T. delicatum, and T. perfasciculatum, as well as the known species T. hamatum, T. atroviride, and T. harzianum. The infective abilities of different Trichoderma species on G. elata varied, indicating that Trichoderma was a pathogenic fungus causing black rot disease in G. elata.DiscussionThis study provided the morphological characteristics of new species and discussed the morphological differences with phylogenetically proximate species, laying the foundation for research aimed at preventing and managing diseases that affect G. elata

    Analyses of a chromosome-scale genome assembly reveal the origin and evolution of cultivated chrysanthemum

    Get PDF
    DATA AVAILABILITY : The raw sequencing data generated in this study have been deposited in the NCBI under accession PRJNA796762 and PRJNA895586 The chloroplast andmitochondrial genome were also available at GenBank under the accession number OP104251 and OP104742 respectively. The assembled genome sequences and annotations are available at Figshare [https://doi.org/10.6084/m9.figshare.21655364.v2]. The Arabidopsis ABCE and chrysanthemum CYC2 genes were used as query sequences for gene family identification, which are available at Figshare [https://doi.org/10.6084/m9.figshare.21610305]. Source data are provided with this paper.Chrysanthemum (Chrysanthemum morifolium Ramat.) is a globally important ornamental plant with great economic, cultural, and symbolic value. However, research on chrysanthemum is challenging due to its complex genetic background. Here, we report a near-complete assembly and annotation for C. morifolium comprising 27 pseudochromosomes (8.15 Gb; scaffold N50 of 303.69Mb). Comparative and evolutionary analyses reveal a whole-genome triplication (WGT) event shared by Chrysanthemum species approximately 6 million years ago (Mya) and the possible lineage-specific polyploidization of C. morifolium approximately 3 Mya. Multilevel evidence suggests that C. morifolium is likely a segmental allopolyploid. Furthermore, a combination of genomics and transcriptomics approaches demonstrate the C. morifolium genome can be used to identify genes underlying key ornamental traits. Phylogenetic analysis of CmCCD4a traces the flower colour breeding history of cultivated chrysanthemum. Genomic resources generated from this study could help to accelerate chrysanthemum genetic improvement.The National Natural Science Foundation of China, the Natural Science Fund of Jiangsu Province, China Agriculture Research System, the National Key Research and Development Program of China, the ā€œJBGSā€ Project of Seed Industry Revitalisation in Jiangsu Province, the European Unionā€™s Horizon 2020 research and innovation program from European Research Council, the Methusalem funding from Ghent University, and a project funded by the Priority Academic Program Development of Jiangsu Higher Education Institution.https://www.nature.com/ncomms/am2024BiochemistryGeneticsMicrobiology and Plant PathologySDG-15:Life on lan

    ļ»æTwo new Trichoderma species (Hypocreales, Hypocreaceae) isolated from decaying tubers of Gastrodia elate

    Get PDF
    Species of Trichoderma are widely distributed around the world. In this study, two new species in Trichoderma, named as T. albidum and T. variegatum, were introduced and illustrated. These species were isolated from diseased tubers of Gastrodia elata in China and identified based on morphological characteristics and multi-gene sequence analyses of three loci that is the internal transcribed spacer regions of the ribosomal DNA (ITS), the translation elongation factor 1-Ī± encoding gene (tef1-Ī±) and the gene encoding the second largest nuclear RNA polymerase subunit (rpb2). Distinctions between the new species and their close relatives were discussed. According to results of the phylogenetic analyses, T. albidum belonged to the Harzianum clade and T. variegatum are grouped with species of the Spirale clade. The expansion of two clades provided research foundations for the prevention and control of tuber diseases in G. elata

    Oral Delivery of a Nanocrystal Formulation of Schisantherin A with Improved Bioavailability and Brain Delivery for the Treatment of Parkinsonā€™s Disease

    No full text
    Schisantherin A (SA) is a promising anti-Parkinsonism Chinese herbal medicine but with poor water solubility and challenges to be delivered to the brain. We formulated SA as nanocrystals (SA-NC), aiming to improve its solubility and pharmacokinetic profile and thus provide a potential therapeutic agent for the treatment of Parkinsonā€™s disease (PD). The rod-shaped SA-NC had a particle size of āˆ¼160 nm with 33.3% drug loading, and the nanocrystals exhibited a fast dissolution rate <i>in vitro</i>. The intact drug nanocrystals could be internalized into Madin-Darby canine kidney (MDCK) cells, which were followed by rapid intracellular release, and most of the drug was transported to the basolateral side in its soluble form. Following oral administration of the SA-NC or an SA suspension, the accumulated concentration of the SA-NC in the plasma and brain was considerably higher than that observed for the SA suspension, but the drug targeting efficiency was similar. The SA-NC significantly reversed the 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP)-induced dopaminergic (DA) neuronal loss and locomotion deficiency in zebrafish, as well as the 1-methyl-4-phenylpyridinium ion (MPP<sup>+</sup>)-induced damage of neuronal cell culture model. Further Western blot analysis demonstrated that the stronger neuroprotective effect of SA-NC may be partially mediated by the activation of the protein kinase B (Akt)/glycogen synthase kinase-3Ī² (Gsk3Ī²) pathway. Taken together, these data provide solid evidence that the nanocrystal formulation has the potential to improve the bioavailability and brain concentration of this Biopharmaceutics Classification System (BCS) class II compound, SA, for the treatment of PD
    corecore