Masking Strategies for Image Manifolds

We consider the problem of selecting an optimal mask for an image manifold, i.e., choosing a subset of the pixels of the image that preserves the manifold's geometric structure present in the original data. Such masking implements a form of compressive sensing through emerging imaging sensor platforms for which the power expense grows with the number of pixels acquired. Our goal is for the manifold learned from masked images to resemble its full image counterpart as closely as possible. More precisely, we show that one can indeed accurately learn an image manifold without having to consider a large majority of the image pixels. In doing so, we consider two masking methods that preserve the local and global geometric structure of the manifold, respectively. In each case, the process of finding the optimal masking pattern can be cast as a binary integer program, which is computationally expensive but can be approximated by a fast greedy algorithm. Numerical experiments show that the relevant manifold structure is preserved through the data-dependent masking process, even for modest mask sizes

Combining filter method and dynamically dimensioned search for constrained global optimization

In this work we present an algorithm that combines the filter technique and the dynamically dimensioned search (DDS) for solving nonlinear and nonconvex constrained global optimization problems. The DDS is a stochastic global algorithm for solving bound constrained problems that in each iteration generates a randomly trial point perturbing some coordinates of the current best point. The filter technique controls the progress related to optimality and feasibility defining a forbidden region of points refused by the algorithm. This region can be given by the flat or slanting filter rule. The proposed algorithm does not compute or approximate any derivatives of the objective and constraint functions. Preliminary experiments show that the proposed algorithm gives competitive results when compared with other methods.The first author thanks a scholarship supported by the International Cooperation Program CAPES/ COFECUB at the University of Minho. The second and third authors thanks the support given by FCT (Funda¸c˜ao para Ciˆencia e Tecnologia, Portugal) in the scope of the projects: UID/MAT/00013/2013 and UID/CEC/00319/2013. The fourth author was partially supported by CNPq-Brazil grants 308957/2014-8 and 401288/2014-5.info:eu-repo/semantics/publishedVersio

Limited Memory BFGS method for Sparse and Large-Scale Nonlinear Optimization

Optimization-based control systems are used in many areas of application, including aerospace engineering, economics, robotics and automotive engineering. This work was motivated by the demand for a large-scale sparse solver for this problem class. The sparsity property of the problem is used for the computational efficiency regarding performance and memory consumption. This includes an efficient storing of the occurring matrices and vectors and an appropriate approximation of the Hessian matrix, which is the main subject of this work. Thus, a so-called the limited memory BFGS method has been developed. The limited memory BFGS method, has been implemented in a software library for solving the nonlinear optimization problems, WORHP. Its solving performance has been tested on different optimal control problems and test sets

Derivative free algorithms for nonsmooth and global optimization with application in cluster analysis

This thesis is devoted to the development of algorithms for solving nonsmooth nonconvex problems. Some of these algorithms are derivative free methods.Doctor of Philosoph

An LDLTLDL^T Trust-Region Quasi-Newton Method

For quasi-Newton methods in unconstrained minimization, it is valuable to develop methods that are robust, i.e., methods that converge on a large number of problems. Trust-region algorithms are often regarded to be more robust than line-search methods, however, because trust-region methods are computationally more expensive, the most popular quasi-Newton implementations use line-search methods. To fill this gap, we develop a trust-region method that updates an $LDL^T$ factorization, scales quadratically with the size of the problem, and is competitive with a conventional line-search method

Model Identification and Robust Nonlinear Model Predictive Control of a Twin Rotor MIMO System

PhDThis thesis presents an investigation into a number of model predictive control (MPC) paradigms for a nonlinear aerodynamics test rig, a twin rotor multi-input multi-output system (TRMS). To this end, the nonlinear dynamic model of the system is developed using various modelling techniques. A comprehensive study is made to compare these models and to select the best one to be used for control design purpose. On the basis of the selected model, a state-feedback multistep Newton-type MPC is developed and its stability is addressed using a terminal equality constraint approach. Moreover, the state-feedback control approach is combined with a nonlinear state observer to form an output-feedback MPC. Finally, a robust MPC technique is employed to address the uncertainties of the system. In the modelling stage, analytical models are developed by extracting the physical equations of the system using the Newtonian and Lagrangian approaches. In the case of the black-box modelling, artificial neural networks (ANNs) are utilised to model the TRMS. Finally, the grey-box model is used to enhance the performance of the white-box model developed earlier through the optimisation of parameters using a genetic algorithm (GA) based approach. Stability analysis of the autonomous TRMS is carried out before designing any control paradigms for the system. In the control design stage, an MPC method is proposed for constrained nonlinear systems, which is the improvement of the multistep Newton-type control strategy. The stability of the proposed state-feedback MPC is guaranteed using terminal equality constraints. Moreover, the formerly proposed MPC algorithm is combined with an unscented Kalman filter (UKF) to formulate an output-feedback MPC. An extended Kalman filter (EKF) based on a state-dependent model is also introduced, whose performance is found to be better compared to that of the UKF. Finally, a robust MPC is introduced and implemented on the TRMS based on a polytopic uncertainty that is cast into linear matrix inequalities (LMI)

Sequential quadratic programming with indefinite Hessian approximations for nonlinear optimum experimental design for parameter estimation in differential–algebraic equations

In this thesis we develop algorithms for the numerical solution of problems from nonlinear optimum experimental design (OED) for parameter estimation in differential–algebraic equations. These OED problems can be formulated as special types of path- and control- constrained optimal control (OC) problems. The objective is to minimize a functional on the covariance matrix of the model parameters that is given by first-order sensitivities of the model equations. Additionally, the objective is nonlinearly coupled in time, which make OED problems a challenging class of OC problems. For their numerical solution, we propose a direct multiple shooting parameterization to obtain a structured nonlinear programming problem (NLP). An augmented system of nominal and variational states for the model sensitivities is parameterized on multiple shooting intervals and the objective is decoupled by means of additional variables and constraints. In the resulting NLP, we identify several structures that allow to evaluate derivatives at greatly reduced costs compared to a standard OC formulation. For the solution of the block-structured NLPs, we develop a new sequential quadratic programming (SQP) method. Therein, partitioned quasi-Newton updates are used to approximate the block-diagonal Hessian of the Lagrangian. We analyze a model problem with indefinite, block-diagonal Hessian and prove that positive definite approximations of the individual blocks prevent superlinear convergence. For an OED model problem, we show that more and more negative eigenvalues appear in the Hessian as the multiple shooting grid is refined and confirm the detrimental impact of positive definite Hessian approximations. Hence, we propose indefinite SR1 updates to guarantee fast local convergence. We develop a filter line search globalization strategy that accepts indefinite Hessians based on a new criterion derived from the proof of global convergence. BFGS updates with a scaling strategy to prevent large eigenvalues are used as fallback if the SR1 update does not promote convergence. For the solution of the arising sparse and nonconvex quadratic subproblems, a parametric active set method with inertia control within a Schur complement approach is developed. It employs a symmetric, indefinite LBL T -factorization for the large, sparse KKT matrix and maintains and updates QR-factors of a small and dense Schur complement. The new methods are complemented by two C++ implementations: muse transforms an OED or OC problem instance to a structured NLP by means of direct multiple shooting. A special feature is that fully independent grids for controls, states, path constraints, and measurements are maintained. This provides higher flexibility to adapt the NLP formulation to the characteristics of the problem at hand and facilitates comparison of different formulations in the light of the lifted Newton method. The software package blockSQP is an implementation of the new SQP method that uses a newly developed variant of the quadratic programming solver qpOASES. Numerical results are presented for a benchmark collection of OED and OC problems that show how SR1 approximations improve local convergence over BFGS. The new method is then applied to two challenging OED applications from chemical engineering. Its performance compares favorably to an available existing implementation