2,132 research outputs found
A Non-Monotone Conjugate Subgradient Type Method for Minimization of Convex Functions
We suggest a conjugate subgradient type method without any line-search for
minimization of convex non differentiable functions. Unlike the custom methods
of this class, it does not require monotone decrease of the goal function and
reduces the implementation cost of each iteration essentially. At the same
time, its step-size procedure takes into account behavior of the method along
the iteration points. Preliminary results of computational experiments confirm
efficiency of the proposed modification.Comment: 11 page
A descent subgradient method using Mifflin line search for nonsmooth nonconvex optimization
We propose a descent subgradient algorithm for minimizing a real function,
assumed to be locally Lipschitz, but not necessarily smooth or convex. To find
an effective descent direction, the Goldstein subdifferential is approximated
through an iterative process. The method enjoys a new two-point variant of
Mifflin line search in which the subgradients are arbitrary. Thus, the line
search procedure is easy to implement. Moreover, in comparison to bundle
methods, the quadratic subproblems have a simple structure, and to handle
nonconvexity the proposed method requires no algorithmic modification. We study
the global convergence of the method and prove that any accumulation point of
the generated sequence is Clarke stationary, assuming that the objective is
weakly upper semismooth. We illustrate the efficiency and effectiveness of the
proposed algorithm on a collection of academic and semi-academic test problems
Bundle methods for regularized risk minimization with applications to robust learning
Supervised learning in general and regularized risk minimization in particular is about solving optimization problem which is jointly defined by a performance measure and a set of labeled training examples. The outcome of learning, a model, is then used mainly for predicting the labels for unlabeled examples in the testing environment. In real-world scenarios: a typical learning process often involves solving a sequence of similar problems with different parameters before a final model is identified. For learning to be successful, the final model must be produced timely, and the model should be robust to (mild) irregularities in the testing environment. The purpose of this thesis is to investigate ways to speed up the learning process and improve the robustness of the learned model. We first develop a batch convex optimization solver specialized to the regularized risk minimization based on standard bundle methods. The solver inherits two main properties of the standard bundle methods. Firstly, it is capable of solving both differentiable and non-differentiable problems, hence its implementation can be reused for different tasks with minimal modification. Secondly, the optimization is easily amenable to parallel and distributed computation settings; this makes the solver highly scalable in the number of training examples. However, unlike the standard bundle methods, the solver does not have extra parameters which need careful tuning. Furthermore, we prove that the solver has faster convergence rate. In addition to that, the solver is very efficient in computing approximate regularization path and model selection. We also present a convex risk formulation for incorporating invariances and prior knowledge into the learning problem. This formulation generalizes many existing approaches for robust learning in the setting of insufficient or noisy training examples and covariate shift. Lastly, we extend a non-convex risk formulation for binary classification to structured prediction. Empirical results show that the model obtained with this risk formulation is robust to outliers in the training examples
- …