A Primal-Dual Augmented Lagrangian Penalty-Interior-Point Algorithm for Nonlinear Programming

This thesis treats a new numerical solution method for large-scale nonlinear optimization problems. Nonlinear programs occur in a wide range of engineering and academic applications like discretized optimal control processes and parameter identification of physical systems. The most efficient and robust solution approaches for this problem class have been shown to be sequential quadratic programming and primal-dual interior-point methods. The proposed algorithm combines a variant of the latter with a special penalty function to increase its robustness due to an automatic regularization of the nonlinear constraints caused by the penalty term. In detail, a modified barrier function and a primal-dual augmented Lagrangian approach with an exact l2-penalty is used. Both share the property that for certain Lagrangian multiplier estimates the barrier and penalty parameter do not have to converge to zero or diverge, respectively. This improves the conditioning of the internal linear equation systems near the optimal solution, handles rank-deficiency of the constraint derivatives for all non-feasible iterates and helps with identifying infeasible problem formulations. Although the resulting merit function is non-smooth, a certain step direction is a guaranteed descent. The algorithm includes an adaptive update strategy for the barrier and penalty parameters as well as the Lagrangian multiplier estimates based on a sensitivity analysis. Global convergence is proven to yield a first-order optimal solution, a certificate of infeasibility or a Fritz-John point and is maintained by combining the merit function with a filter or piecewise linear penalty function. Unlike the majority of filter methods, no separate feasibility restoration phase is required. For a fixed barrier parameter the method has a quadratic order of convergence. Furthermore, a sensitivity based iterative refinement strategy is developed to approximate the optimal solution of a parameter dependent nonlinear program under parameter changes. It exploits special sensitivity derivative approximations and converges locally with a linear convergence order to a feasible point that further satisfies the perturbed complementarity condition of the modified barrier method. Thereby, active-set changes from active to inactive can be handled. Due to a certain update of the Lagrangian multiplier estimate, the refinement is suitable in the context of warmstarting the penalty-interior-point approach. A special focus of the thesis is the development of an algorithm with excellent performance in practice. Details on an implementation of the proposed primal-dual penalty-interior-point algorithm in the nonlinear programming solver WORHP and a numerical study based on the CUTEst test collection is provided. The efficiency and robustness of the algorithm is further compared to state-of-the-art nonlinear programming solvers, in particular the interior-point solvers IPOPT and KNITRO as well as the sequential quadratic programming solvers SNOPT and WORHP

Interior Point Decoding for Linear Vector Channels

In this paper, a novel decoding algorithm for low-density parity-check (LDPC) codes based on convex optimization is presented. The decoding algorithm, called interior point decoding, is designed for linear vector channels. The linear vector channels include many practically important channels such as inter symbol interference channels and partial response channels. It is shown that the maximum likelihood decoding (MLD) rule for a linear vector channel can be relaxed to a convex optimization problem, which is called a relaxed MLD problem. The proposed decoding algorithm is based on a numerical optimization technique so called interior point method with barrier function. Approximate variations of the gradient descent and the Newton methods are used to solve the convex optimization problem. In a decoding process of the proposed algorithm, a search point always lies in the fundamental polytope defined based on a low-density parity-check matrix. Compared with a convectional joint message passing decoder, the proposed decoding algorithm achieves better BER performance with less complexity in the case of partial response channels in many cases.Comment: 18 pages, 17 figures, The paper has been submitted to IEEE Transaction on Information Theor

Faster Convex Optimization: Simulated Annealing with an Efficient Universal Barrier

This paper explores a surprising equivalence between two seemingly-distinct convex optimization methods. We show that simulated annealing, a well-studied random walk algorithms, is directly equivalent, in a certain sense, to the central path interior point algorithm for the the entropic universal barrier function. This connection exhibits several benefits. First, we are able improve the state of the art time complexity for convex optimization under the membership oracle model. We improve the analysis of the randomized algorithm of Kalai and Vempala by utilizing tools developed by Nesterov and Nemirovskii that underly the central path following interior point algorithm. We are able to tighten the temperature schedule for simulated annealing which gives an improved running time, reducing by square root of the dimension in certain instances. Second, we get an efficient randomized interior point method with an efficiently computable universal barrier for any convex set described by a membership oracle. Previously, efficiently computable barriers were known only for particular convex sets