A dynamical view of nonlinear conjugate gradient methods with applications to FFT-based computational micromechanics

For fast Fourier transform (FFT)-based computational micromechanics, solvers need to be fast, memory-efficient, and independent of tedious parameter calibration. In this work, we investigate the benefits of nonlinear conjugate gradient (CG) methods in the context of FFT-based computational micromechanics. Traditionally, nonlinear CG methods require dedicated line-search procedures to be efficient, rendering them not competitive in the FFT-based context. We contribute to nonlinear CG methods devoid of line searches by exploiting similarities between nonlinear CG methods and accelerated gradient methods. More precisely, by letting the step-size go to zero, we exhibit the Fletcher–Reeves nonlinear CG as a dynamical system with state-dependent nonlinear damping. We show how to implement nonlinear CG methods for FFT-based computational micromechanics, and demonstrate by numerical experiments that the Fletcher–Reeves nonlinear CG represents a competitive, memory-efficient and parameter-choice free solution method for linear and nonlinear homogenization problems, which, in addition, decreases the residual monotonically

A Family of Hybrid Stochastic Conjugate Gradient Algorithms for Local and Global Minimization Problems

This paper contains two main parts, Part I and Part II, which discuss the local and global minimization problems, respectively. In Part I, a fresh conjugate gradient (CG) technique is suggested and then combined with a line-search technique to obtain a globally convergent algorithm. The finite difference approximations approach is used to compute the approximate values of the first derivative of the function f. The convergence analysis of the suggested method is established. The comparisons between the performance of the new CG method and the performance of four other CG methods demonstrate that the proposed CG method is promising and competitive for finding a local optimum point. In Part II, three formulas are designed by which a group of solutions are generated. This set of random formulas is hybridized with the globally convergent CG algorithm to obtain a hybrid stochastic conjugate gradient algorithm denoted by HSSZH. The HSSZH algorithm finds the approximate value of the global solution of a global optimization problem. Five combined stochastic conjugate gradient algorithms are constructed. The performance profiles are used to assess and compare the rendition of the family of hybrid stochastic conjugate gradient algorithms. The comparison results between our proposed HSSZH algorithm and four other hybrid stochastic conjugate gradient techniques demonstrate that the suggested HSSZH method is competitive with, and in all cases superior to, the four algorithms in terms of the efficiency, reliability and effectiveness to find the approximate solution of the global optimization problem that contains a non-convex function

Robust parallel nonlinear solvers for implicit time discretizations of the Bidomain equations

In this work, we study the convergence and performance of nonlinear solvers for the Bidomain equations after decoupling the ordinary and partial differential equations of the cardiac system. Firstly, we provide a rigorous proof of the global convergence of Quasi-Newton methods, such as BFGS, and nonlinear Conjugate-Gradient methods, such as Fletcher--Reeves, for the Bidomain system, by analyzing an auxiliary variational problem under physically reasonable hypotheses. Secondly, we compare several nonlinear Bidomain solvers in terms of execution time, robustness with respect to the data and parallel scalability. Our findings indicate that Quasi-Newton methods are the best choice for nonlinear Bidomain systems, since they exhibit faster convergence rates compared to standard Newton-Krylov methods, while maintaining robustness and scalability. Furthermore, first-order methods also demonstrate competitiveness and serve as a viable alternative, particularly for matrix-free implementations that are well-suited for GPU computing

A Taylor polynomial expansion line search for large-scale optimization

In trying to cope with the Big Data deluge, the landscape of distributed computing has changed. Large commodity hardware clusters, typically operating in some form of MapReduce framework, are becoming prevalent for organizations that require both tremendous storage capacity and fault tolerance. However, the high cost of communication can dominate the computation time in large-scale optimization routines in these frameworks. This thesis considers the problem of how to efficiently conduct univariate line searches in commodity clusters in the context of gradient-based batch optimization algorithms, like the staple limited-memory BFGS (LBFGS) method. In it, a new line search technique is proposed for cases where the underlying objective function is analytic, as in logistic regression and low rank matrix factorization. The technique approximates the objective function by a truncated Taylor polynomial along a fixed search direction. The coefficients of this polynomial may be computed efficiently in parallel with far less communication than needed to transmit the high-dimensional gradient vector, after which the polynomial may be minimized with high accuracy in a neighbourhood of the expansion point without distributed operations. This Polynomial Expansion Line Search (PELS) may be invoked iteratively until the expansion point and minimum are sufficiently accurate, and can provide substantial savings in time and communication costs when multiple iterations in the line search procedure are required. Three applications of the PELS technique are presented herein for important classes of analytic functions: (i) logistic regression (LR), (ii) low-rank matrix factorization (MF) models, and (iii) the feedforward multilayer perceptron (MLP). In addition, for LR and MF, implementations of PELS in the Apache Spark framework for fault-tolerant cluster computing are provided. These implementations conferred significant convergence enhancements to their respective algorithms, and will be of interest to Spark and Hadoop practitioners. For instance, the Spark PELS technique reduced the number of iterations and time required by LBFGS to reach terminal training accuracies for LR models by factors of 1.8--2. Substantial acceleration was also observed for the Nonlinear Conjugate Gradient algorithm for MLP models, which is an interesting case for future study in optimization for neural networks. The PELS technique is applicable to a broad class of models for Big Data processing and large-scale optimization, and can be a useful component of batch optimization routines

Optimization for Decision Making II

In the current context of the electronic governance of society, both administrations and citizens are demanding the greater participation of all the actors involved in the decision-making process relative to the governance of society. This book presents collective works published in the recent Special Issue (SI) entitled “Optimization for Decision Making II”. These works give an appropriate response to the new challenges raised, the decision-making process can be done by applying different methods and tools, as well as using different objectives. In real-life problems, the formulation of decision-making problems and the application of optimization techniques to support decisions are particularly complex and a wide range of optimization techniques and methodologies are used to minimize risks, improve quality in making decisions or, in general, to solve problems. In addition, a sensitivity or robustness analysis should be done to validate/analyze the influence of uncertainty regarding decision-making. This book brings together a collection of inter-/multi-disciplinary works applied to the optimization of decision making in a coherent manner

Accélération d'une approche régularisée de reconstruction en tomographie à rayons X avec réduction des artéfacts métalliques

Résumé Ce travail porte sur l'imagerie par tomographie à rayons X des vaisseaux périphériques traités par angioplastie avec implantation d'un tuteur endovasculaire métallique. On cherche à détecter le développement de la resténose en mesurant la lumière du vaisseau sanguin imagé. Cette application nécessite la reconstruction d'images de haute résolution. De plus, la présence du tuteur métallique cause l'apparition d'artéfacts qui nuisent à la précision de la mesure dans les images reconstruites dans les appareils tomographiques utilisés en milieu clinique. On propose donc de réaliser la reconstruction à l'aide d'un algorithme axé sur la maximisation pénalisée de la log-vraisemblance conditionnelle de l'image. Cet algorithme est déduit d'un modèle de formation des données qui tient compte de la variation non linéaire de l'atténuation des photons X dans l'objet selon leur énergie, ainsi que du caractère polychromatique du faisceau X. L'algorithme réduit donc effectivement les artéfacts causés spécifiquement par le tuteur métallique. De plus, il peut être configuré de manière à obtenir un compromis satisfaisant entre la résolution de l'image et la variance de l'image reconstruite, selon le niveau de bruit des données. Cette méthode de reconstruction est reconnue pour donner des images d'excellente qualité. Toutefois, le temps de calcul nécessaire à la convergence de cet algorithme est excessivement long. Le but de ce travail est donc de réduire le temps de calcul de cet algorithme de reconstruction itératif. Cette réduction passe par la critique de la formulation du problème et de la méthode de reconstruction, ainsi que par la mise en oeuvre d'approches alternatives.---------- Abstract This thesis is concerned with X-ray tomography of peripheral vessels that have undergone angioplasty with implantation of an endovascular metal stent. We seek to detect the onset of restenosis by measuring the lumen of the imaged blood vessel. This application requires the reconstruction of high-resolution images. In addition, the presence of a metal stent causes streak artifacts that complicate the lumen measurements in images obtained with the usual algorithms, like those implemented in clinical scanners. A regularized statistical reconstruction algorithm, hinged on the maximization of the conditional log-likelihood of the image, is preferable in this case. We choose a variant deduced from a data formation model that takes into account the nonlinear variation of X~photon attenuation to photon energy, as well as the polychromatic character of the X-ray beam. This algorithm effectively reduces the artifacts specifically caused by the metal structures. Moreover, the algorithm may be set to determine a good compromise between image resolution and variance, according to data noise. This reconstruction method is thus known to yield images of excellent quality. However, the runtime to convergence is excessively long. The goal of this work is to reduce the reconstruction runtime

Development and application of 2D and 3D transient electromagnetic inverse solutions based on adjoint Green functions: A feasibility study for the spatial reconstruction of conductivity distributions by means of sensitivities

To enhance interpretation capabilities of transient electromagnetic (TEM) methods, a multidimensional inverse solution is introduced, which allows for a explicit sensitivity calculation with reduced computational effort. The main conservation of computational load is obtained by solving Maxwell's equations directly in time domain. This is achieved by means of a high efficient Krylov-subspace technique that is particularly developed for the fast computation of EM fields in the diffusive regime. Traditional modeling procedures for Maxwell's equations yields solutions independently for every frequency or, in the time domain, at a given time through explicit time stepping. Because of this, frequency domain methods are rendered extremely time consuming for multi-frequency simulations. Likewise the stability conditions required by explicit time stepping techniques often result in highly inefficient calculations for large diffusion times and conductivity contrasts. The computation of sensitivities is carried out using the adjoint Green functions approach. For time domain applications, it is realized by convolution of the background electrical field information, originating from the primary signal, with the impulse response of the receiver acting as secondary source. In principle, the adjoint formulation may be extended allowing for a fast gradient calculation without calculating and storing the whole sensitivity matrix but just the gradient of the data residual. This technique, which is also known as migration, is widely used for seismic and, to some extend, for EM methods as well. However, the sensitivity matrix, which is not easily given by migration techniques, plays a central role in resolution analysis and would therefore be discarded. But, since it allows one to discriminate features in the a posteriori model which are data or regularization driven, it would therefore be very likely additional information to have. The additional cost of its storage and explicit computation is comparable low disbursement to the gain of a posteriori model resolution analysis. Inversion of TEM data arising from various types of sources is approached by two different methods. Both methods reconstruct the subsurface electrical conductivity properties directly in the time domain. A principal difference is given by the space dimensions of the inversion problems to be solved and the type of the optimization procedure. For two-dimensional (2D) models, the ill-posed and non-linear inverse problem is solved by means of a regularized Gauss-Newton type of optimization. For three-dimensional (3D) problems, due to the increase of complexity, a simpler, gradient based minimization scheme is presented. The 2D inversion is successfully applied to a long offset (LO)TEM survey conducted in the Arava basin (Jordan), where the joint interpretation of 168 transient soundings support the same subsurface conductivity structure as the one derived by inversion of a Magnetotelluric (MT) experiment. The 3D application to synthetic data demonstrates, that the spatial conductivity distribution can be reconstructed either by deep or shallow TEM sounding methods

a feasibility study for the spatial reconstruction of conductivity distributions by means of sensitivities

To enhance interpretation capabilities of transient electromagnetic (TEM) methods, a multidimensional inverse solution is introduced, which allows for a explicit sensitivity calculation with reduced computational effort. The main conservation of computational load is obtained by solving Maxwell's equations directly in time domain. This is achieved by means of a high efficient Krylov-subspace technique that is particularly developed for the fast computation of EM fields in the diffusive regime. Traditional modeling procedures for Maxwell's equations yields solutions independently for every frequency or, in the time domain, at a given time through explicit time stepping. Because of this, frequency domain methods are rendered extremely time consuming for multi-frequency simulations. Likewise the stability conditions required by explicit time stepping techniques often result in highly inefficient calculations for large diffusion times and conductivity contrasts. The computation of sensitivities is carried out using the adjoint Green functions approach. For time domain applications, it is realized by convolution of the background electrical field information, originating from the primary signal, with the impulse response of the receiver acting as secondary source. In principle, the adjoint formulation may be extended allowing for a fast gradient calculation without calculating and storing the whole sensitivity matrix but just the gradient of the data residual. This technique, which is also known as migration, is widely used for seismic and, to some extend, for EM methods as well. However, the sensitivity matrix, which is not easily given by migration techniques, plays a central role in resolution analysis and would therefore be discarded ...thesi

Optimierung thermischer Verhältnisse bei der Bahnplanung für das thermische Spritzen mit Industrierobotern

Diese Arbeit befasst sich mit der Erzeugung und Optimierung von neuartigen Bahnen für Industrieroboter beim thermischen Spritzen auf komplexen Freiformoberflächen unter besonderer Berücksichtigung der thermischen Verhältnisse in dem Werkstück. Thermisches Spritzen ist ein Produktionsprozess, bei dem eine Werkstückoberfläche mit geschmolzenem Material beschichtet wird, so dass die Oberfläche die gewünschten Oberflächeneigenschaften aufweist. Ein Alleinstellungsmerkmal des präsentierten Systems ist der modulare Aufbau, der vor allem eine in diesem Bereich unübliche Trennung zwischen der Initialbahnplanung und der Bahnoptimierung vorsieht. Die Basis des Gesamtsystems bilden verschiedene Simulationskomponenten, wie die Beschichtungssimulation, die thermische Simulation und die Robotersimulation. Die Initialbahnplanung erzeugt flächenüberdeckende Bahnen auf einem Werkstück unter Berücksichtigung verschiedener Qualitätsmerkmale. Dazu werden die Bahnen über flexible Bahnstrukturen repräsentiert, darunter neuartige Strukturen, wie die Rand-zu-Rand Bahnen und die Punkt-zu-Punkt Bahnen. Die Qualität der Bahnen wird über verschiedene Zielfunktionen bewertet, die neben der Schichtqualität vor allem die thermischen Varianzen berücksichtigen, welche bisher nur selten in Betracht gezogen wurden, obwohl sie großen Einfluss auf die endgültige Schichtqualität haben. Weitere praxisrelevante Zielkriterien, wie die Roboterachsbeschleunigungen und der Overspray, welcher das Material beschreibt, das nicht auf der funktionalen Fläche abgelagert wird, werden ebenfalls beachtet. Das Problem der Initialbahnplanung wird als mehrkriterielles Optimierungsproblem formuliert und mit Hilfe eines Evolutionären Algorithmus optimiert. Verschiedene Varianten für die Operatoren des Evolutionären Algorithmus werden verwendet und gegeneinander evaluiert. Hieraus wird die Kombination von Operatoren bestimmt, mit der der Algorithmus mit hoher Konvergenzgeschwindigkeit strukturell gute Bahnen für den anschließenden Bahnoptimierungsprozess erzeugt. Die Bahnoptimierung wird für die Verbesserung vorhandener Bahnen bezüglich der Beschichtungsfehler und der Ausführbarkeit mit Robotern verwendet. Ein neuartiges Konzept zur kombinierten Anwendung des in der Arbeit entwickelten, analytischen Auftragsmodells mit einer externen Blackbox Simulation wird verwendet, um die Bahnen mit Hilfe des Verfahrens der nichtlinearen konjugierten Gradienten zu optimieren. Die Fehler werden hierbei über die externe Simulation und die Gradienten über das analytische Auftragsmodell bestimmt. Die Verwendung der Bahnoptimierung beschränkt sich nicht nur auf die Optimierung der Bahnen, die von der Initialbahnplanung erstellt worden sind, sondern kann ebenfalls genutzt werden, um bereits erstellte Bahnen an andere Spritzprozesse oder ähnliche Werkstückgeometrien anzupassen. Hierdurch lässt sich der erhebliche Aufwand zur Generierung neuer Bahnen stark reduzieren. Zum Abschluss der Arbeit wird ein Verfahren vorgestellt, das die bisher unberücksichtigte Roboterdynamik in das System miteinbezieht. Dazu wird eine Dynamikkorrektur präsentiert, die die Bahnen mit Hilfe einer Roboterherstellersoftware in den dynamisch zulässigen Bereich projiziert. Diese Projektion wird in einer weiteren Optimierungsschleife alternierend mit der Bahnoptimierung genutzt, um eine dynamisch zulässige Bahn zu erzeugen, die sehr gute Ergebnisse bezüglich der Qualitätsmaße liefert