Economic inexact restoration for derivative-free expensive function minimization and applications

The Inexact Restoration approach has proved to be an adequate tool for handling the problem of minimizing an expensive function within an arbitrary feasible set by using different degrees of precision in the objective function. The Inexact Restoration framework allows one to obtain suitable convergence and complexity results for an approach that rationally combines low- and high-precision evaluations. In the present research, it is recognized that many problems with expensive objective functions are nonsmooth and, sometimes, even discontinuous. Having this in mind, the Inexact Restoration approach is extended to the nonsmooth or discontinuous case. Although optimization phases that rely on smoothness cannot be used in this case, basic convergence and complexity results are recovered. A derivative-free optimization phase is defined and the subproblems that arise at this phase are solved using a regularization approach that take advantage of different notions of stationarity. The new methodology is applied to the problem of reproducing a controlled experiment that mimics the failure of a dam

Assessing the reliability of general-purpose Inexact Restoration methods

Inexact Restoration methods have been proved to be effective to solve constrained optimization problems in which some structure of the feasible set induces a natural way of recovering feasibility from arbitrary infeasible points. Sometimes natural ways of dealing with minimization over tangent approximations of the feasible set are also employed. A recent paper Banihashemi and Kaya (2013)] suggests that the Inexact Restoration approach can be competitive with well-established nonlinear programming solvers when applied to certain control problems without any problem-oriented procedure for restoring feasibility. This result motivated us to revisit the idea of designing general-purpose Inexact Restoration methods, especially for large-scale problems. In this paper we introduce affordable algorithms of Inexact Restoration type for solving arbitrary nonlinear programming problems and we perform the first experiments that aim to assess their reliability. Initially, we define a purely local Inexact Restoration algorithm with quadratic convergence. Then, we modify the local algorithm in order to increase the chances of success of both the restoration and the optimization phase. This hybrid algorithm is intermediate between the local algorithm and a globally convergent one for which, under suitable assumptions, convergence to KKT points can be proved28

A dai-liao hybrid hestenes-stiefel and fletcher-revees methods for unconstrained optimization

Some problems have no analytical solution or too difficult to solve by scientists, engineers, and mathematicians, so the development of numerical methods to obtain approximate solutions became necessary. Gradient methods are more efficient when the function to be minimized continuously in its first derivative. Therefore, this article presents a new hybrid Conjugate Gradient (CG) method to solve unconstrained optimization problems. The method requires the first-order derivatives but overcomes the steepest descent method’s shortcoming of slow convergence and needs not to save or compute the second-order derivatives needed by the Newton method. The CG update parameter is suggested from the Dai-Liao conjugacy condition as a convex combination of Hestenes-Stiefel and Fletcher-Revees algorithms by employing an optimal modulating choice parameterto avoid matrix storage. Numerical computation adopts an inexact line search to obtain the step-size that generates a decent property, showing that the algorithm is robust and efficient. The scheme converges globally under Wolfe line search, and it’s like is suitable in compressive sensing problems and M-tensor systems

Investigating Task-driven Latent Feasibility for Nonconvex Image Modeling

Properly modeling latent image distributions plays an important role in a variety of image-related vision problems. Most exiting approaches aim to formulate this problem as optimization models (e.g., Maximum A Posterior, MAP) with handcrafted priors. In recent years, different CNN modules are also considered as deep priors to regularize the image modeling process. However, these explicit regularization techniques require deep understandings on the problem and elaborately mathematical skills. In this work, we provide a new perspective, named Task-driven Latent Feasibility (TLF), to incorporate specific task information to narrow down the solution space for the optimization-based image modeling problem. Thanks to the flexibility of TLF, both designed and trained constraints can be embedded into the optimization process. By introducing control mechanisms based on the monotonicity and boundedness conditions, we can also strictly prove the convergence of our proposed inference process. We demonstrate that different types of image modeling problems, such as image deblurring and rain streaks removals, can all be appropriately addressed within our TLF framework. Extensive experiments also verify the theoretical results and show the advantages of our method against existing state-of-the-art approaches.Comment: 11 pages, Accepted at IEEE TI

Matematické metody výpočtů elektronové struktury velkých systémů

This thesis focuses on mathematical methods of the quantum chemistry. It consists of several thematic parts. The first part focuses on tensor numerical methods which serve as a~tool for storing and numerical treatment of large multidimensional data. We focus on an efficient numerical representation of several types of basis functions that can be used in electronic structure calculations. In the second part we present our development of an~optimization algorithm intended for solving Kohn-Sham equations. It can be understood as an alternative to standard iterative methods, where problems with the convergence to the ground state energy occur. Finally, our parallel software for electronic structure calculations based on the Hartree-Fock approximation and the density functional theory together with achieved results is presented.Tato disertační práce se zaměřuje na matematické metody kvantové chemie a skládá se z několika tematických okruhů. V první části se věnuje tenzorovým numerickým metodám jakožto efektivnímu nástroji pro práci s rozsáhlýmí vícedimenzionálními numerickými daty. V rámci našeho výzkumu se zabýváme numerickou reprezentací bázových funkcí využívaných ve výpočtech elektronových struktur. Ve druhé části se věnujeme vývoji optimalizačního algoritmu, který je určen pro řešení Kohnovy-Shamovy rovnice. Algoritmus je primárně určen jako alternativa ke standardním metodám, u nichž jsou známy problémy s konvergencí k energii základního stavu. Na závěr se věnujeme námi vyvíjenému paralelnímu softwaru pro výpočty elektronových struktur založených na Hartreeho-Fockově aproximaci a teorii funkcionálu hustoty. V této části jsou rovněž prezentovány vybrané dosažené výsledky.470 - Katedra aplikované matematikyvyhově

The workshop on iterative methods for large scale nonlinear problems

The aim of the workshop was to bring together researchers working on large scale applications with numerical specialists of various kinds. Applications that were addressed included reactive flows (combustion and other chemically reacting flows, tokamak modeling), porous media flows, cardiac modeling, chemical vapor deposition, image restoration, macromolecular modeling, and population dynamics. Numerical areas included Newton iterative (truncated Newton) methods, Krylov subspace methods, domain decomposition and other preconditioning methods, large scale optimization and optimal control, and parallel implementations and software. This report offers a brief summary of workshop activities and information about the participants. Interested readers are encouraged to look into an online proceedings available at http://www.usi.utah.edu/logan.proceedings. In this, the material offered here is augmented with hypertext abstracts that include links to locations such as speakers` home pages, PostScript copies of talks and papers, cross-references to related talks, and other information about topics addresses at the workshop

Particle Swarm Optimization on Grassmann Manifolds with Applications in Electronic Structure Calculations

Cílem této bakalářské práce je úprava metody Optimalizace hejnem částic tak, aby byla vhodná pro optimalizační problémy s omezením používané při výpočtech elektronových struktur. Tato práce nejprve představí problém optimalizace i původní metodu. Poté je navržena dvoukroková modifikace Optimalizace hejnem částic, která se skládá z projekce částic na tečnou množinu a následné restorace zpět na množinu danou omezeními. Upravená metoda byla testována. Výpočty potenciální křivky molekuly $Be_2$ přinesly uspokojivé výsledky ve srovnání s referenčními daty. Hlavním přínosem této práce je nová modifikace metody Optimalizace hejnem částic vhodná pro použití na optimalizační problémy s omezením.The goal of this bachelor thesis is the modification of the Particle Swarm Optimization method in a way that is suitable for constrained optimization problems used in Electronic Structures Calculations. This work firstly introduces both the optimization problem and the original method. Then a two-step modification of the Particle Swarm Optimization is proposed, which consists of particle projection onto the tangent set and the following restoration back to the feasible set. The modified method was tested. Computations of the potential curve of $Be_2$ molecule yielded satisfactory results in comparison with benchmark data. The main contribution of this work is the invention of a new modification of Particle Swarm Optimization suitable for constrained optimization problems.470 - Katedra aplikované matematikyvýborn

Iterative Methods for the Elasticity Imaging Inverse Problem

Cancers of the soft tissue reign among the deadliest diseases throughout the world and effective treatments for such cancers rely on early and accurate detection of tumors within the interior of the body. One such diagnostic tool, known as elasticity imaging or elastography, uses measurements of tissue displacement to reconstruct the variable elasticity between healthy and unhealthy tissue inside the body. This gives rise to a challenging parameter identification inverse problem, that of identifying the Lamé parameter μ in a system of partial differential equations in linear elasticity. Due to the near incompressibility of human tissue, however, common techniques for solving the direct and inverse problems are rendered ineffective due to a phenomenon known as the “locking effect”. Alternative methods, such as mixed finite element methods, must be applied to overcome this complication. Using these methods, this work reposes the problem as a generalized saddle point problem along with a presentation of several optimization formulations, including the modified output least squares (MOLS), energy output least squares (EOLS), and equation error (EE) frameworks, for solving the elasticity imaging inverse problem. Subsequently, numerous iterative optimization methods, including gradient, extragradient, and proximal point methods, are explored and applied to solve the related optimization problem. Implementations of all of the iterative techniques under consideration are applied to all of the developed optimization frameworks using a representative numerical example in elasticity imaging. A thorough analysis and comparison of the methods is subsequently presented

Global convection in Earth's mantle : advanced numerical methods and extreme-scale simulations

The thermal convection of rock in Earth's mantle and associated plate tectonics are modeled by nonlinear incompressible Stokes and energy equations. This dissertation focuses on the development of advanced, scalable linear and nonlinear solvers for numerical simulations of realistic instantaneous mantle flow, where we must overcome several computational challenges. The most notable challenges are the severe nonlinearity, heterogeneity, and anisotropy due to the mantle's rheology as well as a wide range of spatial scales and highly localized features. Resolving the crucial small scale features efficiently necessitates adaptive methods, while computational results greatly benefit from a high accuracy per degree of freedom and local mass conservation. Consequently, the discretization of Earth's mantle is carried out by high-order finite elements on aggressively adaptively refined hexahedral meshes with a continuous, nodal velocity approximation and a discontinuous, modal pressure approximation. These velocity--pressure pairings yield optimal asymptotic convergence rates of the finite element approximation to the infinite-dimensional solution with decreasing mesh element size, are inf-sup stable on general, non-conforming hexahedral meshes with "hanging nodes,'' and have the advantage of preserving mass locally at the element level due to the discontinuous pressure. However, because of the difficulties cited above and the desired accuracy, the large implicit systems to be solved are extremely poorly conditioned and sophisticated linear and nonlinear solvers including powerful preconditioning techniques are required. The nonlinear Stokes system is solved using a grid continuation, inexact Newton--Krylov method. We measure the residual of the momentum equation in the H⁻¹-norm for backtracking line search to avoid overly conservative update steps that are significantly reduced from one. The Newton linearization is augmented by a perturbation of a highly nonlinear term in mantle's rheology, resulting in dramatically improved nonlinear convergence. We present a new Schur complement-based Stokes preconditioner, weighted BFBT, that exhibits robust fast convergence for Stokes problems with smooth but highly varying (up to 10 orders of magnitude) viscosities, optimal algorithmic scalability with respect to mesh refinement, and only a mild dependence on the polynomial order of high-order finite element discretizations. In addition, we derive theoretical eigenvalue bounds to prove spectral equivalence of our inverse Schur complement approximation. Finally, we present a parallel hybrid spectral--geometric--algebraic multigrid (HMG) to approximate the inverses of the Stokes system's viscous block and variable-coefficient pressure Poisson operators within weighted BFBT. Building on the parallel scalability of HMG, our Stokes solver demonstrates excellent parallel scalability to 1.6 million CPU cores without sacrificing algorithmic optimality.Computational Science, Engineering, and Mathematic