Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization

We consider the problem of optimizing the sum of a smooth convex function and a non-smooth convex function using proximal-gradient methods, where an error is present in the calculation of the gradient of the smooth term or in the proximity operator with respect to the non-smooth term. We show that both the basic proximal-gradient method and the accelerated proximal-gradient method achieve the same convergence rate as in the error-free case, provided that the errors decrease at appropriate rates.Using these rates, we perform as well as or better than a carefully chosen fixed error level on a set of structured sparsity problems.Comment: Neural Information Processing Systems (2011

Minimizing Finite Sums with the Stochastic Average Gradient

We propose the stochastic average gradient (SAG) method for optimizing the sum of a finite number of smooth convex functions. Like stochastic gradient (SG) methods, the SAG method's iteration cost is independent of the number of terms in the sum. However, by incorporating a memory of previous gradient values the SAG method achieves a faster convergence rate than black-box SG methods. The convergence rate is improved from O(1/k^{1/2}) to O(1/k) in general, and when the sum is strongly-convex the convergence rate is improved from the sub-linear O(1/k) to a linear convergence rate of the form O(p^k) for p \textless{} 1. Further, in many cases the convergence rate of the new method is also faster than black-box deterministic gradient methods, in terms of the number of gradient evaluations. Numerical experiments indicate that the new algorithm often dramatically outperforms existing SG and deterministic gradient methods, and that the performance may be further improved through the use of non-uniform sampling strategies.Comment: Revision from January 2015 submission. Major changes: updated literature follow and discussion of subsequent work, additional Lemma showing the validity of one of the formulas, somewhat simplified presentation of Lyapunov bound, included code needed for checking proofs rather than the polynomials generated by the code, added error regions to the numerical experiment

A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method

In this note, we present a new averaging technique for the projected stochastic subgradient method. By using a weighted average with a weight of t+1 for each iterate w_t at iteration t, we obtain the convergence rate of O(1/t) with both an easy proof and an easy implementation. The new scheme is compared empirically to existing techniques, with similar performance behavior.Comment: 8 pages, 6 figures. Changes with previous version: Added reference to concurrently submitted work arXiv:1212.1824v1; clarifications added; typos corrected; title changed to 'subgradient method' as 'subgradient descent' is misnome

A review of in-situ loading conditions for mathematical modelling of asymmetric wind turbine blades

This paper reviews generalized solutions to the classical beam moment equation for solving the deflexion and strain fields of composite wind turbine blades. A generalized moment functional is presented to effectively model the moment at any point on a blade/beam utilizing in-situ load cases. Models assume that the components are constructed from inplane quasi-isotropic composite materials of an overall elastic modulus of 42 GPa. Exact solutions for the displacement and strains for an adjusted aerofoil to that presented in the literature and compared with another defined by the Joukowski transform. Models without stiffening ribs resulted in deflexions of the blades which exceeded the generally acceptable design code criteria. Each of the models developed were rigorously validated via numerical (Runge-Kutta) solutions of an identical differential equation used to derive the analytical models presented. The results obtained from the robust design codes, written in the open source Computer Aided Software (CAS) Maxima, are shown to be congruent with simulations using the ANSYS commercial finite element (FE) codes as well as experimental data. One major implication of the theoretical treatment is that these solutions can now be used in design codes to maximize the strength of analogues components, used in aerospace and most notably renewable energy sectors, while significantly reducing their weight and hence cost. The most realistic in-situ loading conditions for a dynamic blade and stationary blade are presented which are shown to be unique to the blade optimal tip speed ratio, blade dimensions and wind speed

ROBERT M. COVER — Justice Accused, Antislavery and the Judicial Process.

Fabre Michel. Robert M. Cover. — Justice Accused, Antislavery and the Judicial Process. In: Revue Française d'Etudes Américaines, N°23, février 1985. Aspects du Sud aujourd'hui. pp. 142-143