The limited memory steepest descent method (Fletcher, 2012) for unconstrained
optimization problems stores a few past gradients to compute multiple stepsizes
at once. We review this method and propose new variants. For strictly convex
quadratic objective functions, we study the numerical behavior of different
techniques to compute new stepsizes. In particular, we introduce a method to
improve the use of harmonic Ritz values. We also show the existence of a secant
condition associated with LMSD, where the approximating Hessian is projected
onto a low-dimensional space. In the general nonlinear case, we propose two new
alternatives to Fletcher's method: first, the addition of symmetry constraints
to the secant condition valid for the quadratic case; second, a perturbation of
the last differences between consecutive gradients, to satisfy multiple secant
equations simultaneously. We show that Fletcher's method can also be
interpreted from this viewpoint