A New Exact Penalty Function Method for Continuous Inequality Constrained Optimization Problems

publisher-authenticated version online. Alternate Location: Permanent Link: The attached document may provide the author's accepted version of a published work. See Citation for details of the published work

A study of optimization and optimal control computation : exact penalty function approach

In this thesis, We propose new computational algorithms and methods for solving four classes of constrained optimization and optimal control problems. In Chapter 1, we present a brief review on optimization and optimal control. In Chapter 2, we consider a class of continuous inequality constrained optimization problems. The continuous inequality constraints are first approximated by smooth function in integral form. Then, we construct a new exact penalty function, where the summation of all these approximate smooth functions in integral form, called the constraint violation, is appended to the objective function. In this way, we obtain a sequence of approximate unconstrained optimization problems. It is shown that if the value of the penalty parameter is sufficiently large, then any local minimizer of the corresponding unconstrained optimization problem is a local minimizer of the original problem. For illustration, three examples are solved using the proposed method.From the solutions obtained, we observe that the values of their objective functions are amongst the smallest when compared with those obtained by other existing methods available in the literature. More importantly, our method finds solutions which satisfy the continuous inequality constraints.In Chapter 3, we consider a general class of nonlinear mixed discrete programming problems. By introducing continuous variables to replace the discrete variables, the problem is first transformed into an equivalent nonlinear continuous optimization problem subject to original constraints and additional linear and quadratic constraints. However, the existing gradient-based optimization techniques have difficulty to solve this equivalent nonlinear optimization problem effectively due to the new quadratic inequality constraint. Thus, an exact penalty function is employed to construct a sequence of unconstrained optimization problems, each of which can be solved effectively by unconstrained optimization techniques, such as conjugate gradient or quasi-Newton types of methods.It is shown that any local optimal solution of the unconstrained optimization problem is a local optimal solution of the transformed nonlinear constrained continuous optimization problem when the penalty parameter is sufficiently large. Numerical experiments are carried out to test the efficiency of the proposed method.In Chapter 4, we investigate the optimal design of allpass variable fractional delay (VFD) filters with coefficients expressed as sums of signed powers-of-two terms, where the weighted integral squared error is minimized. A new optimization procedure is proposed to generate a reduced discrete search region. Then, a new exact penalty function method is developed to solve the optimal design of allpass variable fractional delay filter with signed powers-of-two coefficients. Design examples show that the proposed method is highly effective. Compared with the conventional quantization method, the solutions obtained by our method are of much higher accuracy. Furthermore, the computational complexity is low.In Chapter 5, we consider an optimal control problem in which the control takes values from a discrete set and the state and control are subject to continuous inequality constraints. By introducing auxiliary controls and applying a time-scaling transformation, we transform this optimal control problem into an equivalent optimal control problem subject to original constraints and additional linear and quadratic constraints, where the decision variables are taking values from a feasible region, which is the union of some continuous sets. However, due to the new quadratic constraints, standard optimization techniques do not perform well when they are applied to solve the transformed problem directly.We introduce a novel exact penalty function to penalize constraint violations, and then append this penalty function to the objective function, forming a penalized objective function. This leads to a sequence of approximate optimal control problems, each of which can be solved by using optimal control techniques, and consequently, many optimal control software packages, such as MISER 3.4, can be used. Convergence results how that when the penalty parameter is sufficiently large, any local solution of the approximate problem is also a local solution of the original problem. We conclude this chapter with some numerical results for two train control problems.In Chapter 6, some concluding remarks and suggestions for future research directions are made

A new exact penalty method for semi-infinite programming problems

In this paper, we consider a class of nonlinear semi-infinite optimization problems. These problems involve continuous inequality constraints that need to be satisfied at every point in an infinite index set, as well as conventional equality and inequality constraints. By introducing a novel penalty function to penalize constraint violations, we form an approximate optimization problem in which the penalty function is minimized subject to only bound constraints. We then show that this penalty function is exact—that is, when the penalty parameter is sufficiently large, any local solution of the approximate problem can be used to generate a corresponding local solution of the original problem. On this basis, the original problem can be solved as a sequence of approximate nonlinear programming problems. We conclude the paper with some numerical results demonstrating the applicability of our approach to PID control and filter design

Distributed Coupled Multi-Agent Stochastic Optimization

This work develops effective distributed strategies for the solution of constrained multi-agent stochastic optimization problems with coupled parameters across the agents. In this formulation, each agent is influenced by only a subset of the entries of a global parameter vector or model, and is subject to convex constraints that are only known locally. Problems of this type arise in several applications, most notably in disease propagation models, minimum-cost flow problems, distributed control formulations, and distributed power system monitoring. This work focuses on stochastic settings, where a stochastic risk function is associated with each agent and the objective is to seek the minimizer of the aggregate sum of all risks subject to a set of constraints. Agents are not aware of the statistical distribution of the data and, therefore, can only rely on stochastic approximations in their learning strategies. We derive an effective distributed learning strategy that is able to track drifts in the underlying parameter model. A detailed performance and stability analysis is carried out showing that the resulting coupled diffusion strategy converges at a linear rate to an $O(\mu)-$neighborhood of the true penalized optimizer