Simplex-stochastic collocation method with improved scalability

The Simplex-Stochastic Collocation (SSC) method is a robust tool used to propagate uncertain input distributions through a computer code. However, it becomes prohibitively expensive for problems with dimensions higher than 5. The main purpose of this paper is to identify bottlenecks, and to improve upon this bad scalability. In order to do so, we propose an alternative interpolation stencil technique based upon the Set-Covering problem, and we integrate the SSC method in the High-Dimensional Model-Reduction framework. In addition, we address the issue of ill-conditioned sample matrices, and we present an analytical map to facilitate uniformly-distributed simplex sampling

Efficient Optimization Algorithms for Nonlinear Data Analysis

Identification of low-dimensional structures and main sources of variation from multivariate data are fundamental tasks in data analysis. Many methods aimed at these tasks involve solution of an optimization problem. Thus, the objective of this thesis is to develop computationally efficient and theoretically justified methods for solving such problems. Most of the thesis is based on a statistical model, where ridges of the density estimated from the data are considered as relevant features. Finding ridges, that are generalized maxima, necessitates development of advanced optimization methods. An efficient and convergent trust region Newton method for projecting a point onto a ridge of the underlying density is developed for this purpose. The method is utilized in a differential equation-based approach for tracing ridges and computing projection coordinates along them. The density estimation is done nonparametrically by using Gaussian kernels. This allows application of ridge-based methods with only mild assumptions on the underlying structure of the data. The statistical model and the ridge finding methods are adapted to two different applications. The first one is extraction of curvilinear structures from noisy data mixed with background clutter. The second one is a novel nonlinear generalization of principal component analysis (PCA) and its extension to time series data. The methods have a wide range of potential applications, where most of the earlier approaches are inadequate. Examples include identification of faults from seismic data and identification of filaments from cosmological data. Applicability of the nonlinear PCA to climate analysis and reconstruction of periodic patterns from noisy time series data are also demonstrated. Other contributions of the thesis include development of an efficient semidefinite optimization method for embedding graphs into the Euclidean space. The method produces structure-preserving embeddings that maximize interpoint distances. It is primarily developed for dimensionality reduction, but has also potential applications in graph theory and various areas of physics, chemistry and engineering. Asymptotic behaviour of ridges and maxima of Gaussian kernel densities is also investigated when the kernel bandwidth approaches infinity. The results are applied to the nonlinear PCA and to finding significant maxima of such densities, which is a typical problem in visual object tracking.Siirretty Doriast

Nature-inspired algorithms for solving some hard numerical problems

Optimisation is a branch of mathematics that was developed to find the optimal solutions, among all the possible ones, for a given problem. Applications of optimisation techniques are currently employed in engineering, computing, and industrial problems. Therefore, optimisation is a very active research area, leading to the publication of a large number of methods to solve specific problems to its optimality. This dissertation focuses on the adaptation of two nature inspired algorithms that, based on optimisation techniques, are able to compute approximations for zeros of polynomials and roots of non-linear equations and systems of non-linear equations. Although many iterative methods for finding all the roots of a given function already exist, they usually require: (a) repeated deflations, that can lead to very inaccurate results due to the problem of accumulating rounding errors, (b) good initial approximations to the roots for the algorithm converge, or (c) the computation of first or second order derivatives, which besides being computationally intensive, it is not always possible. The drawbacks previously mentioned served as motivation for the use of Particle Swarm Optimisation (PSO) and Artificial Neural Networks (ANNs) for root-finding, since they are known, respectively, for their ability to explore high-dimensional spaces (not requiring good initial approximations) and for their capability to model complex problems. Besides that, both methods do not need repeated deflations, nor derivative information. The algorithms were described throughout this document and tested using a test suite of hard numerical problems in science and engineering. Results, in turn, were compared with several results available on the literature and with the well-known Durand–Kerner method, depicting that both algorithms are effective to solve the numerical problems considered.A Optimização é um ramo da matemática desenvolvido para encontrar as soluções óptimas, de entre todas as possíveis, para um determinado problema. Actualmente, são várias as técnicas de optimização aplicadas a problemas de engenharia, de informática e da indústria. Dada a grande panóplia de aplicações, existem inúmeros trabalhos publicados que propõem métodos para resolver, de forma óptima, problemas específicos. Esta dissertação foca-se na adaptação de dois algoritmos inspirados na natureza que, tendo como base técnicas de optimização, são capazes de calcular aproximações para zeros de polinómios e raízes de equações não lineares e sistemas de equações não lineares. Embora já existam muitos métodos iterativos para encontrar todas as raízes ou zeros de uma função, eles usualmente exigem: (a) deflações repetidas, que podem levar a resultados muito inexactos, devido ao problema da acumulação de erros de arredondamento a cada iteração; (b) boas aproximações iniciais para as raízes para o algoritmo convergir, ou (c) o cálculo de derivadas de primeira ou de segunda ordem que, além de ser computacionalmente intensivo, para muitas funções é impossível de se calcular. Estas desvantagens motivaram o uso da Optimização por Enxame de Partículas (PSO) e de Redes Neurais Artificiais (RNAs) para o cálculo de raízes. Estas técnicas são conhecidas, respectivamente, pela sua capacidade de explorar espaços de dimensão superior (não exigindo boas aproximações iniciais) e pela sua capacidade de modelar problemas complexos. Além disto, tais técnicas não necessitam de deflações repetidas, nem do cálculo de derivadas. Ao longo deste documento, os algoritmos são descritos e testados, usando um conjunto de problemas numéricos com aplicações nas ciências e na engenharia. Os resultados foram comparados com outros disponíveis na literatura e com o método de Durand–Kerner, e sugerem que ambos os algoritmos são capazes de resolver os problemas numéricos considerados

Bayesian nonparametric inference in mechanistic models of complex biological systems

Parameter estimation in expensive computational models is a problem that commonly arises in science and engineering. With the increase in computational power, modellers started developing simulators of real life phenomena that are computationally intensive to evaluate. This, however, makes inference prohibitive due to the unit cost of a single function evaluation. This thesis focuses on computational models of biological and biomechanical processes such as the left-ventricular dynamics or the human pulmonary blood circulatory system. In the former model a single forward simulation is in the order of 11 minutes CPU time, while the latter takes approximately 23 seconds in our machines. Markov chain Monte Carlo methods or likelihood maximization using iterative algorithms would take days or weeks to provide a result. This makes them not suitable for clinical decision support systems, where a decision must be taken in a reasonable time frame. I discuss how to accelerate the inference by using the concept of emulation, i.e. by replacing a computationally expensive function with a statistical approximation based on a finite set of expensive training runs. The emulation target could be either the output-domain, representing the standard approach in the emulation literature, or the loss-domain, which is an alternative and different perspective. Then, I demonstrate how this approach can be used to estimate the parameters of expensive simulators. First I apply loss-emulation to a nonstandard variant of the Lotka-Volterra model of prey-predator interactions, in order to assess if the approach is approximately unbiased. Next, I present a comprehensive comparison between output-emulation and loss-emulation on a computational model of left ventricular dynamics, with the goal of inferring the constitutive law relating the myocardial stretch to its strain. This is especially relevant for assessing cardiac function post myocardial infarction. The results show how it is possible to estimate the stress-strain curve in just 15 minutes, compared to the one week required by the current best literature method. This means a reduction in the computational costs of 3 orders of magnitude. Next, I review Bayesian optimization (BO), an algorithm to optimize a computationally expensive function by adaptively improving the emulator. This method is especially useful in scenarios where the simulator is not considered to be a ``stable release''. For example, the simulator could still be undergoing further developments, bug fixing, and improvements. I develop a new framework based on BO to estimate the parameters of a partial differential equation (PDE) model of the human pulmonary blood circulation. The parameters, being related to the vessel structure and stiffness, represent important indicators of pulmonary hypertension risk, which need to be estimated as they can only be measured with invasive experiments. The results using simulated data show how it is possible to estimate a patient's vessel properties in a time frame suitable for clinical applications. I demonstrate a limitation of standard improvement-based acquisition functions for Bayesian optimization. The expected improvement (EI) policy recommends query points where the improvement is on average high. However, it does not account for the variance of the random variable Improvement. I define a new acquisition function, called ScaledEI, which recommends query points where the improvement on the incumbent minimum is expected to be high, with high confidence. This new BO algorithm is compared to acquisition functions from the literature on a large set of benchmark functions for global optimization, where it turns out to be a powerful default choice for Bayesian optimization. ScaledEI is then compared to standard non-Bayesian optimization solvers, to confirm that the policy still leads to a reduction in the number of forward simulations required to reach a given tolerance level on the function value. Finally, the new algorithm is applied to the problem of estimating the PDE parameters of the pulmonary circulation model previously discussed

Computer Science for Continuous Data:Survey, Vision, Theory, and Practice of a Computer Analysis System

Building on George Boole's work, Logic provides a rigorous foundation for the powerful tools in Computer Science that underlie nowadays ubiquitous processing of discrete data, such as strings or graphs. Concerning continuous data, already Alan Turing had applied "his" machines to formalize and study the processing of real numbers: an aspect of his oeuvre that we transform from theory to practice.The present essay surveys the state of the art and envisions the future of Computer Science for continuous data: natively, beyond brute-force discretization, based on and guided by and extending classical discrete Computer Science, as bridge between Pure and Applied Mathematics

Sublinear Computation Paradigm

This open access book gives an overview of cutting-edge work on a new paradigm called the “sublinear computation paradigm,” which was proposed in the large multiyear academic research project “Foundations of Innovative Algorithms for Big Data.” That project ran from October 2014 to March 2020, in Japan. To handle the unprecedented explosion of big data sets in research, industry, and other areas of society, there is an urgent need to develop novel methods and approaches for big data analysis. To meet this need, innovative changes in algorithm theory for big data are being pursued. For example, polynomial-time algorithms have thus far been regarded as “fast,” but if a quadratic-time algorithm is applied to a petabyte-scale or larger big data set, problems are encountered in terms of computational resources or running time. To deal with this critical computational and algorithmic bottleneck, linear, sublinear, and constant time algorithms are required. The sublinear computation paradigm is proposed here in order to support innovation in the big data era. A foundation of innovative algorithms has been created by developing computational procedures, data structures, and modelling techniques for big data. The project is organized into three teams that focus on sublinear algorithms, sublinear data structures, and sublinear modelling. The work has provided high-level academic research results of strong computational and algorithmic interest, which are presented in this book. The book consists of five parts: Part I, which consists of a single chapter on the concept of the sublinear computation paradigm; Parts II, III, and IV review results on sublinear algorithms, sublinear data structures, and sublinear modelling, respectively; Part V presents application results. The information presented here will inspire the researchers who work in the field of modern algorithms

Surrogate based Optimization and Verification of Analog and Mixed Signal Circuits

Nonlinear Analog and Mixed Signal (AMS) circuits are very complex and expensive to design and verify. Deeper technology scaling has made these designs susceptible to noise and process variations which presents a growing concern due to the degradation in the circuit performances and risks of design failures. In fact, due to process parameters, AMS circuits like phase locked loops may present chaotic behavior that can be confused with noisy behavior. To design and verify circuits, current industrial designs rely heavily on simulation based verification and knowledge based optimization techniques. However, such techniques lack mathematical rigor necessary to catch up with the growing design constraints besides being computationally intractable. Given all aforementioned barriers, new techniques are needed to ensure that circuits are robust and optimized despite process variations and possible chaotic behavior. In this thesis, we develop a methodology for optimization and verification of AMS circuits advancing three frontiers in the variability-aware design flow. The first frontier is a robust circuit sizing methodology wherein a multi-level circuit optimization approach is proposed. The optimization is conducted in two phases. First, a global sizing phase powered by a regional sensitivity analysis to quickly scout the feasible design space that reduces the optimization search. Second, nominal sizing step based on space mapping of two AMS circuits models at different levels of abstraction is developed for the sake of breaking the re-design loop without performance penalties. The second frontier concerns a dynamics verification scheme of the circuit behavior (i.e., study the chaotic vs. stochastic circuit behavior). It is based on a surrogate generation approach and a statistical proof by contradiction technique using Gaussian Kernel measure in the state space domain. The last frontier focus on quantitative verification approaches to predict parametric yield for both a single and multiple circuit performance constraints. The single performance approach is based on a combination of geometrical intertwined reachability analysis and a non-parametric statistical verification scheme. On the other hand, the multiple performances approach involves process parameter reduction, state space based pattern matching, and multiple hypothesis testing procedures. The performance of the proposed methodology is demonstrated on several benchmark analog and mixed signal circuits. The optimization approach greatly improves computational efficiency while locating a comparable/better design point than other approaches. Moreover, great improvements were achieved using our verification methods with many orders of speedup compared to existing techniques

Interactive computer graphics in non-linear optimization.

The thesis surveys the current state of knowledge in the field of both interactive computer graphics and non-linear optimization. The potential contribution of interactive computer graphics in nonlinear optimization is then evaluated from the points of view of model formulation and solution, and the requirements of an interactive system for realizing this potential are outlined. Such a system is developed and described, together with a full account of its applications to both real and standard test problems. A novel application is the direct optimization of N-dimensional (N > 2) problems by visual analysis of 1 and 2 dimensional sub-problems, following on the formal development of an algorithm for this approach. The interactive control of conventional methods through computer graphics is also featured. On the basis of these accounts an evaluation is given of the contribution which interactive computer graphics is capable of making in both teaching and research in the general field of non-linear optimization

An Intelligent Time and Performance Efficient Algorithm for Aircraft Design Optimization

Die Optimierung des Flugzeugentwurfs erfordert die Beherrschung der komplexen Zusammenhänge mehrerer Disziplinen. Trotz seiner Abhängigkeit von einer Vielzahl unabhängiger Variablen zeichnet sich dieses komplexe Entwurfsproblem durch starke indirekte Verbindungen und eine daraus resultierende geringe Anzahl lokaler Minima aus. Kürzlich entwickelte intelligente Methoden, die auf selbstlernenden Algorithmen basieren, ermutigten die Suche nach einer diesem Bereich zugeordneten neuen Methode. Tatsächlich wird der in dieser Arbeit entwickelte Hybrid-Algorithmus (Cavus) auf zwei Hauptdesignfälle im Luft- und Raumfahrtbereich angewendet: Flugzeugentwurf- und Flugbahnoptimierung. Der implementierte neue Ansatz ist in der Lage, die Anzahl der Versuchspunkte ohne große Kompromisse zu reduzieren. Die Trendanalyse zeigt, dass der Cavus-Algorithmus für die komplexen Designprobleme, mit einer proportionalen Anzahl von Prüfpunkten konservativer ist, um die erfolgreichen Muster zu finden. Aircraft Design Optimization requires mastering of the complex interrelationships of multiple disciplines. Despite its dependency on a diverse number of independent variables, this complex design problem has favourable nature as having strong indirect links and as a result a low number of local minimums. Recently developed intelligent methods that are based on self-learning algorithms encouraged finding a new method dedicated to this area. Indeed, the hybrid (Cavus) algorithm developed in this thesis is applied two main design cases in aerospace area: aircraft design optimization and trajectory optimization. The implemented new approach is capable of reducing the number of trial points without much compromise. The trend analysis shows that, for the complex design problems the Cavus algorithm is more conservative with a proportional number of trial points in finding the successful patterns

Industrial Robotics

This book covers a wide range of topics relating to advanced industrial robotics, sensors and automation technologies. Although being highly technical and complex in nature, the papers presented in this book represent some of the latest cutting edge technologies and advancements in industrial robotics technology. This book covers topics such as networking, properties of manipulators, forward and inverse robot arm kinematics, motion path-planning, machine vision and many other practical topics too numerous to list here. The authors and editor of this book wish to inspire people, especially young ones, to get involved with robotic and mechatronic engineering technology and to develop new and exciting practical applications, perhaps using the ideas and concepts presented herein