Semidefinite relaxation based branch-and-bound method for nonconvex quadratic programming

Thesis (S.M.)--Massachusetts Institute of Technology, Computation for Design and Optimization Program, 2006.Includes bibliographical references (leaves 73-75).In this thesis, we use a semidefinite relaxation based branch-and-bound method to solve nonconvex quadratic programming problems. Firstly, we show an interval branch-and-bound method to calculate the bounds for the minimum of bounded polynomials. Then we demonstrate four SDP relaxation methods to solve nonconvex Box constrained Quadratic Programming (BoxQP) problems and the comparison of the four methods. For some lower dimensional problems, SDP relaxation methods can achieve tight bounds for the BoxQP problem; whereas for higher dimensional cases (more than 20 dimensions), the bounds achieved by the four Semidefinite programming (SDP) relaxation methods are always loose. To achieve tight bounds for higher dimensional BoxQP problems, we combine the branch-and-bound method and SDP relaxation method to develop an SDP relaxation based branch-and-bound (SDPBB) method. We introduce a sensitivity analysis method for the branching process of SDPBB. This sensitivity analysis method can improve the convergence speed significantly.(cont.) Compared to the interval branch-and-bound method and the global optimization software BARON, SDPBB can achieve better bounds and is also much more efficient. Additionally, we have developed a multisection algorithm for SDPBB and the multisection algorithm has been parallelized using Message Passing Interface (MPI). By parallelizing the program, we can significantly improve the speed of solving higher dimensional BoxQP problems.by Sha Hu.S.M

Laplacian Mixture Modeling for Network Analysis and Unsupervised Learning on Graphs

Laplacian mixture models identify overlapping regions of influence in unlabeled graph and network data in a scalable and computationally efficient way, yielding useful low-dimensional representations. By combining Laplacian eigenspace and finite mixture modeling methods, they provide probabilistic or fuzzy dimensionality reductions or domain decompositions for a variety of input data types, including mixture distributions, feature vectors, and graphs or networks. Provable optimal recovery using the algorithm is analytically shown for a nontrivial class of cluster graphs. Heuristic approximations for scalable high-performance implementations are described and empirically tested. Connections to PageRank and community detection in network analysis demonstrate the wide applicability of this approach. The origins of fuzzy spectral methods, beginning with generalized heat or diffusion equations in physics, are reviewed and summarized. Comparisons to other dimensionality reduction and clustering methods for challenging unsupervised machine learning problems are also discussed.Comment: 13 figures, 35 reference

Custom optimization algorithms for efficient hardware implementation

The focus is on real-time optimal decision making with application in advanced control systems. These computationally intensive schemes, which involve the repeated solution of (convex) optimization problems within a sampling interval, require more efficient computational methods than currently available for extending their application to highly dynamical systems and setups with resource-constrained embedded computing platforms. A range of techniques are proposed to exploit synergies between digital hardware, numerical analysis and algorithm design. These techniques build on top of parameterisable hardware code generation tools that generate VHDL code describing custom computing architectures for interior-point methods and a range of first-order constrained optimization methods. Since memory limitations are often important in embedded implementations we develop a custom storage scheme for KKT matrices arising in interior-point methods for control, which reduces memory requirements significantly and prevents I/O bandwidth limitations from affecting the performance in our implementations. To take advantage of the trend towards parallel computing architectures and to exploit the special characteristics of our custom architectures we propose several high-level parallel optimal control schemes that can reduce computation time. A novel optimization formulation was devised for reducing the computational effort in solving certain problems independent of the computing platform used. In order to be able to solve optimization problems in fixed-point arithmetic, which is significantly more resource-efficient than floating-point, tailored linear algebra algorithms were developed for solving the linear systems that form the computational bottleneck in many optimization methods. These methods come with guarantees for reliable operation. We also provide finite-precision error analysis for fixed-point implementations of first-order methods that can be used to minimize the use of resources while meeting accuracy specifications. The suggested techniques are demonstrated on several practical examples, including a hardware-in-the-loop setup for optimization-based control of a large airliner.Open Acces

High performance implementation of MPC schemes for fast systems

In recent years, the number of applications of model predictive control (MPC) is rapidly increasing due to the better control performance that it provides in comparison to traditional control methods. However, the main limitation of MPC is the computational e ort required for the online solution of an optimization problem. This shortcoming restricts the use of MPC for real-time control of dynamic systems with high sampling rates. This thesis aims to overcome this limitation by implementing high-performance MPC solvers for real-time control of fast systems. Hence, one of the objectives of this work is to take the advantage of the particular mathematical structures that MPC schemes exhibit and use parallel computing to improve the computational e ciency. Firstly, this thesis focuses on implementing e cient parallel solvers for linear MPC (LMPC) problems, which are described by block-structured quadratic programming (QP) problems. Speci cally, three parallel solvers are implemented: a primal-dual interior-point method with Schur-complement decomposition, a quasi-Newton method for solving the dual problem, and the operator splitting method based on the alternating direction method of multipliers (ADMM). The implementation of all these solvers is based on C++. The software package Eigen is used to implement the linear algebra operations. The Open Message Passing Interface (Open MPI) library is used for the communication between processors. Four case-studies are presented to demonstrate the potential of the implementation. Hence, the implemented solvers have shown high performance for tackling large-scale LMPC problems by providing the solutions in computation times below milliseconds. Secondly, the thesis addresses the solution of nonlinear MPC (NMPC) problems, which are described by general optimal control problems (OCPs). More precisely, implementations are done for the combined multiple-shooting and collocation (CMSC) method using a parallelization scheme. The CMSC method transforms the OCP into a nonlinear optimization problem (NLP) and de nes a set of underlying sub-problems for computing the sensitivities and discretized state values within the NLP solver. These underlying sub-problems are decoupled on the variables and thus, are solved in parallel. For the implementation, the software package IPOPT is used to solve the resulting NLP problems. The parallel solution of the sub-problems is performed based on MPI and Eigen. The computational performance of the parallel CMSC solver is tested using case studies for both OCPs and NMPC showing very promising results. Finally, applications to autonomous navigation for the SUMMIT robot are presented. Specially, reference tracking and obstacle avoidance problems are addressed using an NMPC approach. Both simulation and experimental results are presented and compared to a previous work on the SUMMIT, showing a much better computational e ciency and control performance.Tesi