Electromagnetism-like augmented lagrangian algorithm for global optimization

This paper presents an augmented Lagrangian algorithm to solve continuous constrained global optimization problems. The algorithm approximately solves a sequence of bound constrained subproblems whose objective function penalizes equality and inequality constraints violation and depends on the Lagrange multiplier vectors and a penalty parameter. Each subproblem is solved by a population-based method that uses an electromagnetism-like mechanism to move points towards optimality. Benchmark problems are solved in a performance evaluation of the proposed augmented Lagrangian methodology. A comparison with a well-known technique is also reported

IST Austria Technical Report

We study the problem of developing efficient approaches for proving termination of recursive programs with one-dimensional arrays. Ranking functions serve as a sound and complete approach for proving termination of non-recursive programs without array operations. First, we generalize ranking functions to the notion of measure functions, and prove that measure functions (i) provide a sound method to prove termination of recursive programs (with one-dimensional arrays), and (ii) is both sound and complete over recursive programs without array operations. Our second contribution is the synthesis of measure functions of specific forms in polynomial time. More precisely, we prove that (i) polynomial measure functions over recursive programs can be synthesized in polynomial time through Farkas’ Lemma and Handelman’s Theorem, and (ii) measure functions involving logarithm and exponentiation can be synthesized in polynomial time through abstraction of logarithmic or exponential terms and Handelman’s Theorem. A key application of our method is the worst-case analysis of recursive programs. While previous methods obtain worst-case polynomial bounds of the form O(n^k), where k is an integer, our polynomial time methods can synthesize bounds of the form O(n log n), as well as O(n^x), where x is not an integer. We show the applicability of our automated technique to obtain worst-case complexity of classical recursive algorithms such as (i) Merge-Sort, the divideand- conquer algorithm for the Closest-Pair problem, where we obtain O(n log n) worst-case bound, and (ii) Karatsuba’s algorithm for polynomial multiplication and Strassen’s algorithm for matrix multiplication, where we obtain O(n^x) bound, where x is not an integer and close to the best-known bounds for the respective algorithms. Finally, we present experimental results to demonstrate the effectiveness of our approach

Numerical study of augmented lagrangian algorithms for constrained global optimization

To cite this article: Ana Maria A.C. Rocha & Edite M.G.P. Fernandes (2011): Numerical study of augmented Lagrangian algorithms for constrained global optimization, Optimization, 60:10-11, 1359-1378This article presents a numerical study of two augmented Lagrangian algorithms to solve continuous constrained global optimization problems. The algorithms approximately solve a sequence of bound constrained subproblems whose objective function penalizes equality and inequality constraints violation and depends on the Lagrange multiplier vectors and a penalty parameter. Each subproblem is solved by a population-based method that uses an electromagnetism-like (EM) mechanism to move points towards optimality. Three local search procedures are tested to enhance the EM algorithm. Benchmark problems are solved in a performance evaluation of the proposed augmented Lagrangian methodologies. A comparison with other techniques presented in the literature is also reported

Augmented Lagrangian Algorithms under Constraint Partitioning

We present a novel constraint-partitioning approach for solving continuous nonlinear optimization based on augmented Lagrange method. In contrast to previous work, our approach is based on a new constraint partitioning theory and can handle global constraints. We employ a hyper-graph partitioning method to recognize the problem structure. We prove global convergence under assumptions that are much more relaxed than previous work and solve problems as large as 40,000 variables that other solvers such as IPOPT [11] cannot solve

An ADMM Based Framework for AutoML Pipeline Configuration

We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervised learning pipelines. This black-box (gradient-free) optimization with mixed integer & continuous variables is a challenging problem. We propose a novel AutoML scheme by leveraging the alternating direction method of multipliers (ADMM). The proposed framework is able to (i) decompose the optimization problem into easier sub-problems that have a reduced number of variables and circumvent the challenge of mixed variable categories, and (ii) incorporate black-box constraints along-side the black-box optimization objective. We empirically evaluate the flexibility (in utilizing existing AutoML techniques), effectiveness (against open source AutoML toolkits),and unique capability (of executing AutoML with practically motivated black-box constraints) of our proposed scheme on a collection of binary classification data sets from UCI ML& OpenML repositories. We observe that on an average our framework provides significant gains in comparison to other AutoML frameworks (Auto-sklearn & TPOT), highlighting the practical advantages of this framework

Handling Group Fairness in Federated Learning Using Augmented Lagrangian Approach

Federated learning (FL) has garnered considerable attention due to its privacy-preserving feature. Nonetheless, the lack of freedom in managing user data can lead to group fairness issues, where models might be biased towards sensitive factors such as race or gender, even if they are trained using a legally compliant process. To redress this concern, this paper proposes a novel FL algorithm designed explicitly to address group fairness issues. We show empirically on CelebA and ImSitu datasets that the proposed method can improve fairness both quantitatively and qualitatively with minimal loss in accuracy in the presence of statistical heterogeneity and with different numbers of clients. Besides improving fairness, the proposed FL algorithm is compatible with local differential privacy (LDP), has negligible communication costs, and results in minimal overhead when migrating existing FL systems from the common FL protocol such as FederatedAveraging (FedAvg). We also provide the theoretical convergence rate guarantee for the proposed algorithm and the required noise level of the Gaussian mechanism to achieve desired LDP. This innovative approach holds significant potential to enhance the fairness and effectiveness of FL systems, particularly in sensitive applications such as healthcare or criminal justice.Comment: the main paper has 8 pages and the supplementary material has 12 pages. At the time of uploading, it is currently under review in ECA

Predictive Control for Alleviation of Gust Loads on Very Flexible Aircraft

In this work the dynamics of very flexible aircraft are described by a set of non-linear, multi-disciplinary equations of motion. Primary structural components are represented by a geometrically-exact composite beam model which captures the large dynamic deformations of the aircraft and the interaction between rigid-body and elastic degrees-of-freedom. In addition, an implementation of the unsteady vortex-lattice method capable of handling arbitrary kinematics is used to capture the unsteady, three-dimensional flow-eld around the aircraft as it deforms. Linearization of this coupled nonlinear description, which can in general be about a nonlinear reference state, is performed to yield relatively high-order linear time-invariant state-space models. Subsequent reduction of these models using standard balanced truncation results in low-order models suitable for the synthesis of online, optimization-based control schemes that incorporate actuator constraints. Predictive controllers are synthesized using these reduced-order models and applied to nonlinear simulations of the plant dynamics where they are shown to be superior to equivalent optimal linear controllers (LQR) for problems in which constraints are active