A Simple and Efficient Algorithm for Nonlinear Model Predictive Control

We present PANOC, a new algorithm for solving optimal control problems arising in nonlinear model predictive control (NMPC). A usual approach to this type of problems is sequential quadratic programming (SQP), which requires the solution of a quadratic program at every iteration and, consequently, inner iterative procedures. As a result, when the problem is ill-conditioned or the prediction horizon is large, each outer iteration becomes computationally very expensive. We propose a line-search algorithm that combines forward-backward iterations (FB) and Newton-type steps over the recently introduced forward-backward envelope (FBE), a continuous, real-valued, exact merit function for the original problem. The curvature information of Newton-type methods enables asymptotic superlinear rates under mild assumptions at the limit point, and the proposed algorithm is based on very simple operations: access to first-order information of the cost and dynamics and low-cost direct linear algebra. No inner iterative procedure nor Hessian evaluation is required, making our approach computationally simpler than SQP methods. The low-memory requirements and simple implementation make our method particularly suited for embedded NMPC applications

Modified BFGS Update (H-Version) Based on the Determinant Property of Inverse of Hessian Matrix for Unconstrained Optimization

الهدف من هذا البحث هو لتحوير التحديث BFGS النسخة H وذلك بالاعتماد على صفات المحدد لمعكوس المصفوفة هيسين ( المشتقة الثانية لدالة الهدف ) وذلك بتحديث المتجه s ( الفرق بين الحل القادم  الحل الاني ) بحيث تكون قيمة المحدد لمعكوس المصفوفة هيسين القادمة مساوي لقيمة المحدد لمعكوس المصفوفة هيسين الانية في كل تكرار , لذلك فان متتابعة التحديثات للمصفوفة هيسين المتولدة من هذه الطريقة وكذلك  معكوس المصفوفة هيسين سوف تكون قيمة المحدد لها ثابت في كل تكرار ولا تقترب من صيغة المفرد ( المحدد = صفر ) مما يؤدي الى ان البرنامج المستخدم للحسابات العددية سوف لن يتوقف بسبب اقتراب محدد المصفوفة المتولدة من الصفر وان البرنامج المذكور سوف يتوقف فقط عندما نحصل على الحل الامثل لدالة الهدف. اضافة الى ذلك فان التحوير الجديد سوف يحافظ على خاصيتي التناظرية الموجبة للمصفوفة المتولدة  وبدون شروط وفي كل تكرار. The study presents the modification of the Broyden-Flecher-Goldfarb-Shanno (BFGS) update (H-Version) based on the determinant property of inverse of Hessian matrix (second derivative of the objective function), via updating of the vector s ( the difference between the next solution and the current solution), such that the determinant of the next inverse of Hessian matrix is equal to the determinant of the current inverse of Hessian matrix at every iteration. Moreover, the sequence of inverse of Hessian matrix generated by the method would never  approach a near-singular matrix, such that the program would never break before the minimum value of the objective function is obtained. Moreover, the new modification of BFGS update (H-version) preserves the symmetric property and the positive definite property without any condition

Limited-Memory BFGS with Displacement Aggregation

A displacement aggregation strategy is proposed for the curvature pairs stored in a limited-memory BFGS method such that the resulting (inverse) Hessian approximations are equal to those that would be derived from a full-memory BFGS method. This means that, if a sufficiently large number of pairs are stored, then an optimization algorithm employing the limited-memory method can achieve the same theoretical convergence properties as when full-memory (inverse) Hessian approximations are stored and employed, such as a local superlinear rate of convergence under assumptions that are common for attaining such guarantees. To the best of our knowledge, this is the first work in which a local superlinear convergence rate guarantee is offered by a quasi-Newton scheme that does not either store all curvature pairs throughout the entire run of the optimization algorithm or store an explicit (inverse) Hessian approximation.Comment: 24 pages, 3 figure

Positive Definiteness of Symmetric Rank 1 (H-Version) Update for Unconstrained Optimization

عدة محاولات بذلت لتحوير شرط كواسي نيوتن للمثلية غير المقيدة وذلك للحصول على تقارب اسرع مع خواص كاملة ( التناظرية والموجبة) لمعكوس المصفوفة هسين (المشتقة الثانية لدالة الهدف), هناك الكثير من طرق المثلية غير المقيدة التي لا تولد معكوس مصفوفة هيسين موجبة. احد هذه الطرق هو التحديث التناظري من الرتبة الاولى (النسخة H ), حيث ان هذا التحديث يحقق شرط كواسي نيوتن وايضا يحقق  صفة التناظرية ولكنه لا يضمن خاصية الموجبة لمعكوس مصفوفة هيسين عندما تكون معكوس مصفوفة هيسين الابتدائية موجبة. ان الموجبة لمعكوس المصفوفة هيسين مهم لضمان وجود نقطة النهاية الصغرى لدالة الهدف وكذلك للحصول على اصغر قيمة لدالة الهدف.Several attempts have been made to modify the quasi-Newton condition in order to obtain rapid convergence with complete properties (symmetric and positive definite) of the inverse of  Hessian matrix (second derivative of the objective function). There are many unconstrained optimization methods that do not generate positive definiteness of the inverse of Hessian matrix. One of those methods is the symmetric rank 1( H-version) update (SR1 update), where this update satisfies the quasi-Newton condition and the symmetric property of inverse of Hessian matrix, but does not preserve the positive definite property of the inverse of Hessian matrix where the initial inverse of Hessian matrix is positive definiteness. The positive definite property for the inverse of Hessian matrix is very important to guarantee the existence of the minimum point of the objective function and determine the minimum value of the objective function

Some Unconstrained Optimization Methods

Although it is a very old theme, unconstrained optimization is an area which is always actual for many scientists. Today, the results of unconstrained optimization are applied in different branches of science, as well as generally in practice. Here, we present the line search techniques. Further, in this chapter we consider some unconstrained optimization methods. We try to present these methods but also to present some contemporary results in this area

Fast B-spline Curve Fitting by L-BFGS

We propose a novel method for fitting planar B-spline curves to unorganized data points. In traditional methods, optimization of control points and foot points are performed in two very time-consuming steps in each iteration: 1) control points are updated by setting up and solving a linear system of equations; and 2) foot points are computed by projecting each data point onto a B-spline curve. Our method uses the L-BFGS optimization method to optimize control points and foot points simultaneously and therefore it does not need to perform either matrix computation or foot point projection in every iteration. As a result, our method is much faster than existing methods