Augmented Lagrangian and differentiable exact penalty methods

Bibliography: leaves 13-14."July 1981""National Science Foundation Grant no. NSF/ECS 79-20834."Dimitri P. Bertsekas

An investigation of new methods for estimating parameter sensitivities

The method proposed for estimating sensitivity derivatives is based on the Recursive Quadratic Programming (RQP) method and in conjunction a differencing formula to produce estimates of the sensitivities. This method is compared to existing methods and is shown to be very competitive in terms of the number of function evaluations required. In terms of accuracy, the method is shown to be equivalent to a modified version of the Kuhn-Tucker method, where the Hessian of the Lagrangian is estimated using the BFS method employed by the RQP algorithm. Initial testing on a test set with known sensitivities demonstrates that the method can accurately calculate the parameter sensitivity

L0 Constraint Optimization, Homogeneity Fusion, and Mediation Analyses

The focus of this dissertation is to develop a framework of L0 regularized statistical procedures to identify subgroups among regression coefficients and estimation of subgroup-specific parameters. The proposed constrained discrete optimization methodology estimates group labels by solving mixed integer programming problems (MIP) via efficient algorithms. I develop key large-sample theories for the proposed methods, including subgroup selection consistency and estimation consistency using some new non-asymptotic bounds. Also, the R statistical software is made available to the public for the proposed methods. In the first project presented in Chapter II, I consider a high-dimensional regression setting. The primary objective is to develop a dimension reduction method that can identify homogeneous subgroups among regression coefficients and sparse feature selection simultaneously. The resulting estimates of regression coefficients in each subgroup share the same value. To encourage sparsity, a large subgroup of coefficients is allowed to be estimated exactly as zero. To achieve this objective, I propose a new L0 constrained optimization method, which is formulated as a MIP problem. To implement this MIP method, I develop a novel algorithm with warm start via both a discrete first-order method and segment neighborhood method, and establish its convergence properties. This new approach is able to solve the MIP problem with satisfactory accuracy in short time. To attain global optimality of the MIP method, I reformulate the constrained optimality as another MIP problem that can then be solved efficiently by Kelley's cutting plane method. A sufficient condition for consistent estimation of group labels is affirmed, which is proved to be the necessary condition under which any method attains consistency of subgroup clustering up to a constant factor. Surprisingly, to achieve the clustering consistency, the sample size only needs to grow at the same rate as the sum of logarithm of the number of regression coefficients and the logarithm of the true number of subgroups. A real data analysis is used to illustrate the performance of the proposed method and algorithms. In the second project presented in Chapter III, I consider a structural equation model, and aim to estimate model parameters in causal mediation pathways in the presence of high-dimensional potential mediators. I develop statistical procedures to select sparse important mediators and to identify sparse causal pathways simultaneously. To address the technical challenge, I propose a new L0 constrained optimization method, which leads to an MIP formulation. To solve this MIP problem, I develop a new warm start algorithm by using the discrete first-order method and establish its convergence properties. This new algorithm is able to quickly attain a near-optimal solution. To achieve the global optimality of the MIP problem, I reformulate it, so that I can solve this MIP problem efficiently using Kelley's cutting plane method. I present a sufficient condition for the proposed method for the selection consistency of causal pathways, which is proved as the necessary condition under which any method can achieve the causal pathway selection consistency up to a constant factor. Simulation studies and real-world data analyses are used to demonstrate the performance of the proposed method and algorithms.PHDBiostatisticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/162877/1/wangwen_1.pd

Automation of reverse engineering process in aircraft modeling and related optimization problems

During the year of 1994, the engineering problems in aircraft modeling were studied. The initial concern was to obtain a surface model with desirable geometric characteristics. Much of the effort during the first half of the year was to find an efficient way of solving a computationally difficult optimization model. Since the smoothing technique in the proposal 'Surface Modeling and Optimization Studies of Aerodynamic Configurations' requires solutions of a sequence of large-scale quadratic programming problems, it is important to design algorithms that can solve each quadratic program in a few interactions. This research led to three papers by Dr. W. Li, which were submitted to SIAM Journal on Optimization and Mathematical Programming. Two of these papers have been accepted for publication. Even though significant progress has been made during this phase of research and computation times was reduced from 30 min. to 2 min. for a sample problem, it was not good enough for on-line processing of digitized data points. After discussion with Dr. Robert E. Smith Jr., it was decided not to enforce shape constraints in order in order to simplify the model. As a consequence, P. Dierckx's nonparametric spline fitting approach was adopted, where one has only one control parameter for the fitting process - the error tolerance. At the same time the surface modeling software developed by Imageware was tested. Research indicated a substantially improved fitting of digitalized data points can be achieved if a proper parameterization of the spline surface is chosen. A winning strategy is to incorporate Dierckx's surface fitting with a natural parameterization for aircraft parts. The report consists of 4 chapters. Chapter 1 provides an overview of reverse engineering related to aircraft modeling and some preliminary findings of the effort in the second half of the year. Chapters 2-4 are the research results by Dr. W. Li on penalty functions and conjugate gradient methods for quadratic programming problems

Optimization by nonhierarchical asynchronous decomposition

Large scale optimization problems are tractable only if they are somehow decomposed. Hierarchical decompositions are inappropriate for some types of problems and do not parallelize well. Sobieszczanski-Sobieski has proposed a nonhierarchical decomposition strategy for nonlinear constrained optimization that is naturally parallel. Despite some successes on engineering problems, the algorithm as originally proposed fails on simple two dimensional quadratic programs. The algorithm is carefully analyzed for quadratic programs, and a number of modifications are suggested to improve its robustness

Recent Experiences in Multidisciplinary Analysis and Optimization, part 2

The papers presented at the NASA Symposium on Recent Experiences in Multidisciplinary Analysis and Optimization held at NASA Langley Research Center, Hampton, Virginia, April 24 to 26, 1984 are given. The purposes of the symposium were to exchange information about the status of the application of optimization and the associated analyses in industry or research laboratories to real life problems and to examine the directions of future developments

Exploring novel designs of NLP solvers: Architecture and Implementation of WORHP

Mathematical Optimization in general and Nonlinear Programming in particular, are applied by many scientific disciplines, such as the automotive sector, the aerospace industry, or the space agencies. With some established NLP solvers having been available for decades, and with the mathematical community being rather conservative in this respect, many of their programming standards are severely outdated. It is safe to assume that such usability shortcomings impede the wider use of NLP methods; a representative example is the use of static workspaces by legacy FORTRAN codes. This dissertation gives an account of the construction of the European NLP solver WORHP by using and combining software standards and techniques that have not previously been applied to mathematical software to this extent. Examples include automatic code generation, a consistent reverse communication architecture and the elimination of static workspaces. The result is a novel, industrial-grade NLP solver that overcomes many technical weaknesses of established NLP solvers and other mathematical software