Developing an Enhanced Algorithms to Solve Mixed Integer Non-Linear Programming Problems Based on a Feasible Neighborhood Search Strategy

Engineering optimization problems often involve nonlinear objective functions, which can capture complex relationships and dependencies between variables. This study focuses on a unique nonlinear mathematics programming problem characterized by a subset of variables that can only take discrete values and are linearly separable from the continuous variables. The combination of integer variables and non-linearities makes this problem much more complex than traditional nonlinear programming problems with only continuous variables. Furthermore, the presence of integer variables can result in a combinatorial explosion of potential solutions, significantly enlarging the search space and making it challenging to explore effectively. This issue becomes especially challenging for larger problems, leading to long computation times or even infeasibility. To address these challenges, we propose a method that employs the "active constraint" approach in conjunction with the release of nonbasic variables from their boundaries. This technique compels suitable non-integer fundamental variables to migrate to their neighboring integer positions. Additionally, we have researched selection criteria for choosing a nonbasic variable to use in the integerizing technique. Through implementation and testing on various problems, these techniques have proven to be successful

Extended Trust-Tech Methodology For Nonlinear Optimization: Analyses, Methods And Applications

Many theoretical and practical problems can be formulated as a global optimization problem. Traditional local optimization methods can only attain a local optimal solution and be entrapped in the local optimal solution; while existing global optimization algorithms usually sparsely approximates the global optimal solution in a stochastic manner. In contrast, the transformation under stability-retaining equilibrium characterization (TRUST-TECH) methodology prevails over existing algorithms due to its capability of locating multiple, if not all, local optimal solutions to the optimization problem deterministically and systematically in a tier-by-tier manner. The TRUST-TECH methodology was developed to solve unconstrained and constrained nonlinear optimization problems. This work extends the TRUST-TECH methodology by incorporating new analytical results, developing new solution methods and solving new problems in practical applications. This work first provides analytical results regarding the invariance of partial stability region in quasi-gradient systems. Our motivation is to resolve numerical difficulties arising in implementations of trajectory based methods, including TRUST-TECH. Improved algorithms were developed to resolve these issues by altering the original problem to speed-up movement of the trajectory. However, such operations can lead the trajectory converge to a different solution, which could be undesired under specific situations. This work attempts to answer the question regarding invariant convergence for a special class of numerical operations whose dynamical behaviours can be characterized by a quasi-gradient dynamical system. To this end, we study relationship between a gradient dynamical system and its associated quasi-gradient system and reveal the invariance of partial stability region in the quasi-gradient system. These analytical results lead to methods for checking invariant convergence of the trajectory starting from a given point in the quasi-gradient system and the algorithm to maintain invariant convergence. This work also develops new solution methods to enhance TRUST-TECH's capability of solving constrained nonlinear optimization problems and applies them to solve practical problems arising in different applications. Specifically, TRUST-TECH based methods are first developed for feasibility computation and restoration and are applied to power system applications, including power flow computation and feasibility restoration for infeasible optimal power flow problems. Indeed, a unified framework based on TRUST-TECH is introduced for analysing feasibility and infeasibility for nonlinear problems. Secondly, the TRUST-TECH based interior point method (TT-IPM) and the reduced projected gradient method are developed to better tackle constrained nonlinear optimization problems. As application, the TT-IPM method is used to solve mixed-integer nonlinear programs (MINLPs). Finally, this work develops the ensemble of optimal, input-pruned neural networks using TRUST-TECH (ELITE) method for constructing high-quality neural network ensembles and applies ELITE to build a short-term load forecaster named ELITE-STLF with promising performance. Possible extensions of the TRUST-TECH methodology to a much broader range of optimization models, including multi-objective optimization and variational optimization, are suggested for future research efforts

An algorithm for the global resolution of linear stochastic bilevel programs

The aim of this thesis is to find a technique that allows for the use of decomposition methods known from stochastic programming in the framework of linear stochastic bilevel problems. The uncertainty is modeled as a discrete, finite distribution on some probability space. Two approaches are made, one using the optimal value function of the lower level, whereas the second technique uses the Karush-Kuhn-Tucker conditions of the lower level. Using the latter approach, an integer-programming based algorithm for the global resolution of these problems is presented and evaluated

Hinge-Loss Markov Random Fields and Probabilistic Soft Logic: A Scalable Approach to Structured Prediction

A fundamental challenge in developing impactful artificial intelligence technologies is balancing the ability to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge graphs and the Web, to images, video, and natural language. In this thesis I introduce two new formalisms for modeling structured data, distinguished from previous approaches by their ability to both capture rich structure and scale to big data. The first, hinge-loss Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model that generalizes different approaches to convex inference. I unite three views of inference from the randomized algorithms, probabilistic graphical models, and fuzzy logic communities, showing that all three views lead to the same inference objective. I then derive HL-MRFs by generalizing this unified objective. The second new formalism, probabilistic soft logic (PSL), is a probabilistic programming language that makes HL-MRFs easy to define, refine, and reuse for relational data. PSL uses a syntax based on first-order logic to compactly specify complex models. I next introduce an algorithm for inferring most-probable variable assignments (MAP inference) for HL-MRFs that is extremely scalable, much more so than commercially available software, because it uses message passing to leverage the sparse dependency structures common in inference tasks. I then show how to learn the parameters of HL-MRFs using a number of learning objectives. The learned HL-MRFs are as accurate as traditional, discrete models, but much more scalable. To enable HL-MRFs and PSL to capture even richer dependencies, I then extend learning to support latent variables, i.e., variables without training labels. To overcome the bottleneck of repeated inferences required during learning, I introduce paired-dual learning, which interleaves inference and parameter updates. Paired-dual learning learns accurate models and is also scalable, often completing before traditional methods make even one parameter update. Together, these algorithms enable HL-MRFs and PSL to model rich, structured data at scales not previously possible

International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book

The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions. This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more

The Third Air Force/NASA Symposium on Recent Advances in Multidisciplinary Analysis and Optimization

The third Air Force/NASA Symposium on Recent Advances in Multidisciplinary Analysis and Optimization was held on 24-26 Sept. 1990. Sessions were on the following topics: dynamics and controls; multilevel optimization; sensitivity analysis; aerodynamic design software systems; optimization theory; analysis and design; shape optimization; vehicle components; structural optimization; aeroelasticity; artificial intelligence; multidisciplinary optimization; and composites

Feasible Form Parameter Design of Complex Ship Hull Form Geometry

This thesis introduces a new methodology for robust form parameter design of complex hull form geometry via constraint programming, automatic differentiation, interval arithmetic, and truncated hierarchical B- splines. To date, there has been no clearly stated methodology for assuring consistency of general (equality and inequality) constraints across an entire geometric form parameter ship hull design space. In contrast, the method to be given here can be used to produce guaranteed narrowing of the design space, such that infeasible portions are eliminated. Furthermore, we can guarantee that any set of form parameters generated by our method will be self consistent. It is for this reason that we use the title feasible form parameter design. In form parameter design, a design space is represented by a tuple of design parameters which are extended in each design space dimension. In this representation, a single feasible design is a consistent set of real valued parameters, one for every component of the design space tuple. Using the methodology to be given here, we pick out designs which consist of consistent parameters, narrowed to any desired precision up to that of the machine, even for equality constraints. Furthermore, the method is developed to enable the generation of complex hull forms using an extension of the basic rules idea to allow for automated generation of rules networks, plus the use of the truncated hierarchical B-splines, a wavelet-adaptive extension of standard B-splines and hierarchical B-splines. The adaptive resolution methods are employed in order to allow an automated program the freedom to generate complex B-spline representations of the geometry in a robust manner across multiple levels of detail. Thus two complementary objectives are pursued: ensuring feasible starting sets of form parameters, and enabling the generation of complex hull form geometry