On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization

Extrapolation is a well-known technique for solving convex optimization and variational inequalities and recently attracts some attention for non-convex optimization. Several recent works have empirically shown its success in some machine learning tasks. However, it has not been analyzed for non-convex minimization and there still remains a gap between the theory and the practice. In this paper, we analyze gradient descent and stochastic gradient descent with extrapolation for finding an approximate first-order stationary point in smooth non-convex optimization problems. Our convergence upper bounds show that the algorithms with extrapolation can be accelerated than without extrapolation

Fast Objective & Duality Gap Convergence for Nonconvex-Strongly-Concave Min-Max Problems

This paper focuses on stochastic methods for solving smooth non-convex strongly-concave min-max problems, which have received increasing attention due to their potential applications in deep learning (e.g., deep AUC maximization, distributionally robust optimization). However, most of the existing algorithms are slow in practice, and their analysis revolves around the convergence to a nearly stationary point. We consider leveraging the Polyak-\L ojasiewicz (PL) condition to design faster stochastic algorithms with stronger convergence guarantee. Although PL condition has been utilized for designing many stochastic minimization algorithms, their applications for non-convex min-max optimization remain rare. In this paper, we propose and analyze a generic framework of proximal epoch-based method with many well-known stochastic updates embeddable. Fast convergence is established in terms of both {\bf the primal objective gap and the duality gap}. Compared with existing studies, (i) our analysis is based on a novel Lyapunov function consisting of the primal objective gap and the duality gap of a regularized function, and (ii) the results are more comprehensive with improved rates that have better dependence on the condition number under different assumptions. We also conduct deep and non-deep learning experiments to verify the effectiveness of our methods

Distribution Network Congestion Dispatch Considering Time-Spatial Diversion of Electric Vehicles Charging

With the popularization of electric vehicles, free charging behaviors of electric vehicle owners can lead to uncertainty about charging in both time and space. A time-spatial dispatching strategy for the distribution network guided by electric vehicle charging fees is proposed in this paper, which aims to solve the network congestion problem caused by the unrestrained and free charging behaviors of large numbers of electric vehicles. In this strategy, congestion severity of different lines is analyzed and the relationship between the congested lines and the charging stations is clarified. A price elastic matrix is introduced to reflect the degree of owners’ response to the charging prices. A pricing scheme for optimal real-time charging fees for multiple charging stations is designed according to the congestion severity of the lines and the charging power of the related charging stations. Charging price at different charging station at different time is different, it can influence the charging behaviors of vehicle owners. The simulation results confirmed that the proposed congestion dispatching strategy considers the earnings of the operators, charging cost to the owners and the satisfaction of the owners. Moreover, the strategy can influence owners to make judicious charging plans that help to solve congestion problems in the network and improve the safety and economy of the power grid

CREBBP Inactivation Promotes the Development of HDAC3-Dependent Lymphomas

Somatic mutations in CREBBP occur frequently in B-cell lymphoma. Here, we show that loss of CREBBP facilitates the development of germinal center (GC)-derived lymphomas in mice. In both human and murine lymphomas, CREBBP loss-of-function resulted in focal depletion of enhancer H3K27 acetylation and aberrant transcriptional silencing of genes that regulate B-cell signaling and immune responses, including class II MHC. Mechanistically, CREBBP-regulated enhancers are counter-regulated by the BCL6 transcriptional repressor in a complex with SMRT and HDAC3, which we found to bind extensively to MHC class II loci. HDAC3 loss-of-function rescued repression of these enhancers and corresponding genes, including MHC class II, and more profoundly suppressed CREBBP-mutant lymphomas in vitro and in vivo Hence, CREBBP loss-of-function contributes to lymphomagenesis by enabling unopposed suppression of enhancers by BCL6/SMRT/HDAC3 complexes, suggesting HDAC3-targeted therapy as a precision approach for CREBBP-mutant lymphomas.SignificanceOur findings establish the tumor suppressor function of CREBBP in GC lymphomas in which CREBBP mutations disable acetylation and result in unopposed deacetylation by BCL6/SMRT/HDAC3 complexes at enhancers of B-cell signaling and immune response genes. Hence, inhibition of HDAC3 can restore the enhancer histone acetylation and may serve as a targeted therapy for CREBBP-mutant lymphomas. Cancer Discov; 7(1); 38-53. ©2016 AACR.See related commentary by Höpken, p. 14This article is highlighted in the In This Issue feature, p. 1