A Collection of Challenging Optimization Problems in Science, Engineering and Economics

Function optimization and finding simultaneous solutions of a system of nonlinear equations (SNE) are two closely related and important optimization problems. However, unlike in the case of function optimization in which one is required to find the global minimum and sometimes local minima, a database of challenging SNEs where one is required to find stationary points (extrama and saddle points) is not readily available. In this article, we initiate building such a database of important SNE (which also includes related function optimization problems), arising from Science, Engineering and Economics. After providing a short review of the most commonly used mathematical and computational approaches to find solutions of such systems, we provide a preliminary list of challenging problems by writing the Mathematical formulation down, briefly explaning the origin and importance of the problem and giving a short account on the currently known results, for each of the problems. We anticipate that this database will not only help benchmarking novel numerical methods for solving SNEs and function optimization problems but also will help advancing the corresponding research areas.Comment: Accepted as an invited contribution to the special session on Evolutionary Computation for Nonlinear Equation Systems at the 2015 IEEE Congress on Evolutionary Computation (at Sendai International Center, Sendai, Japan, from 25th to 28th May, 2015.

Efficient Covariance Matrix Update for Variable Metric Evolution Strategies

International audienceRandomized direct search algorithms for continuous domains, such as Evolution Strategies, are basic tools in machine learning. They are especially needed when the gradient of an objective function (e.g., loss, energy, or reward function) cannot be computed or estimated efficiently. Application areas include supervised and reinforcement learning as well as model selection. These randomized search strategies often rely on normally distributed additive variations of candidate solutions. In order to efficiently search in non-separable and ill-conditioned landscapes the covariance matrix of the normal distribution must be adapted, amounting to a variable metric method. Consequently, Covariance Matrix Adaptation (CMA) is considered state-of-the-art in Evolution Strategies. In order to sample the normal distribution, the adapted covariance matrix needs to be decomposed, requiring in general $\Theta(n^3)$ operations, where $n$ is the search space dimension. We propose a new update mechanism which can replace a rank-one covariance matrix update and the computationally expensive decomposition of the covariance matrix. The newly developed update rule reduces the computational complexity of the rank-one covariance matrix adaptation to $\Theta(n^2)$ without resorting to outdated distributions. We derive new versions of the elitist Covariance Matrix Adaptation Evolution Strategy (CMA-ES) and the multi-objective CMA-ES. These algorithms are equivalent to the original procedures except that the update step for the variable metric distribution scales better in the problem dimension. We also introduce a simplified variant of the non-elitist CMA-ES with the incremental covariance matrix update and investigate its performance. Apart from the reduced time-complexity of the distribution update, the algebraic computations involved in all new algorithms are simpler compared to the original versions. The new update rule improves the performance of the CMA-ES for large scale machine learning problems in which the objective function can be evaluated fast

Representations of molecules and materials for interpolation of quantum-mechanical simulations via machine learning

Computational study of molecules and materials from first principles is a cornerstone of physics, chemistry and materials science, but limited by the cost of accurate and precise simulations. In settings involving many simulations, machine learning can reduce these costs, sometimes by orders of magnitude, by interpolating between reference simulations. This requires representations that describe any molecule or material and support interpolation. We review, discuss and benchmark state-of-the-art representations and relations between them, including smooth overlap of atomic positions, many-body tensor representation, and symmetry functions. For this, we use a unified mathematical framework based on many-body functions, group averaging and tensor products, and compare energy predictions for organic molecules, binary alloys and Al-Ga-In sesquioxides in numerical experiments controlled for data distribution, regression method and hyper-parameter optimization

Inverse Kinematic Analysis of Robot Manipulators

An important part of industrial robot manipulators is to achieve desired position and orientation of end effector or tool so as to complete the pre-specified task. To achieve the above stated goal one should have the sound knowledge of inverse kinematic problem. The problem of getting inverse kinematic solution has been on the outline of various researchers and is deliberated as thorough researched and mature problem. There are many fields of applications of robot manipulators to execute the given tasks such as material handling, pick-n-place, planetary and undersea explorations, space manipulation, and hazardous field etc. Moreover, medical field robotics catches applications in rehabilitation and surgery that involve kinematic, dynamic and control operations. Therefore, industrial robot manipulators are required to have proper knowledge of its joint variables as well as understanding of kinematic parameters. The motion of the end effector or manipulator is controlled by their joint actuator and this produces the required motion in each joints. Therefore, the controller should always supply an accurate value of joint variables analogous to the end effector position. Even though industrial robots are in the advanced stage, some of the basic problems in kinematics are still unsolved and constitute an active focus for research. Among these unsolved problems, the direct kinematics problem for parallel mechanism and inverse kinematics for serial chains constitute a decent share of research domain. The forward kinematics of robot manipulator is simpler problem and it has unique or closed form solution. The forward kinematics can be given by the conversion of joint space to Cartesian space of the manipulator. On the other hand inverse kinematics can be determined by the conversion of Cartesian space to joint space. The inverse kinematic of the robot manipulator does not provide the closed form solution. Hence, industrial manipulator can achieve a desired task or end effector position in more than one configuration. Therefore, to achieve exact solution of the joint variables has been the main concern to the researchers. A brief introduction of industrial robot manipulators, evolution and classification is presented. The basic configurations of robot manipulator are demonstrated and their benefits and drawbacks are deliberated along with the applications. The difficulties to solve forward and inverse kinematics of robot manipulator are discussed and solution of inverse kinematic is introduced through conventional methods. In order to accomplish the desired objective of the work and attain the solution of inverse kinematic problem an efficient study of the existing tools and techniques has been done. A review of literature survey and various tools used to solve inverse kinematic problem on different aspects is discussed. The various approaches of inverse kinematic solution is categorized in four sections namely structural analysis of mechanism, conventional approaches, intelligence or soft computing approaches and optimization based approaches. A portion of important and more significant literatures are thoroughly discussed and brief investigation is made on conclusions and gaps with respect to the inverse kinematic solution of industrial robot manipulators. Based on the survey of tools and techniques used for the kinematic analysis the broad objective of the present research work is presented as; to carry out the kinematic analyses of different configurations of industrial robot manipulators. The mathematical modelling of selected robot manipulator using existing tools and techniques has to be made for the comparative study of proposed method. On the other hand, development of new algorithm and their mathematical modelling for the solution of inverse kinematic problem has to be made for the analysis of quality and efficiency of the obtained solutions. Therefore, the study of appropriate tools and techniques used for the solution of inverse kinematic problems and comparison with proposed method is considered. Moreover, recommendation of the appropriate method for the solution of inverse kinematic problem is presented in the work. Apart from the forward kinematic analysis, the inverse kinematic analysis is quite complex, due to its non-linear formulations and having multiple solutions. There is no unique solution for the inverse kinematics thus necessitating application of appropriate predictive models from the soft computing domain. Artificial neural network (ANN) can be gainfully used to yield the desired results. Therefore, in the present work several models of artificial neural network (ANN) are used for the solution of the inverse kinematic problem. This model of ANN does not rely on higher mathematical formulations and are adept to solve NP-hard, non-linear and higher degree of polynomial equations. Although intelligent approaches are not new in this field but some selected models of ANN and their hybridization has been presented for the comparative evaluation of inverse kinematic. The hybridization scheme of ANN and an investigation has been made on accuracies of adopted algorithms. On the other hand, any Optimization algorithms which are capable of solving various multimodal functions can be implemented to solve the inverse kinematic problem. To overcome the problem of conventional tool and intelligent based method the optimization based approach can be implemented. In general, the optimization based approaches are more stable and often converge to the global solution. The major problem of ANN based approaches are its slow convergence and often stuck in local optimum point. Therefore, in present work different optimization based approaches are considered. The formulation of the objective function and associated constrained are discussed thoroughly. The comparison of all adopted algorithms on the basis of number of solutions, mathematical operations and computational time has been presented. The thesis concludes the summary with contributions and scope of the future research work

Multilevel assimilation of inverted seismic data

I ensemble-basert data-assimilering (DA) er størrelsen på ensemblet vanligvis begrenset til hundre medlemmer. Rett frem bruk av ensemble-basert DA kan resultere i betydelig Monte Carlo-feil, som ofte viser seg som alvorlig undervurdering av parameterusikkerheter. Assimilering av store mengder samtidige data forsterker de negative effektene av Monte Carlo-feilen. Avstandsbasert lokalisering er det konvensjonelle middelet for å begrense dette problemet. Denne metoden har imidlertid sine egne ulemper. Den vil, f.eks., fjerne sanne korrelasjoner over lange distanser og det er svært vanskelig å benytte på data som ikke har en unik fysisk plassering. Bruk av modeller med lavere kvalitet reduserer beregningskostnadene per ensemble-medlem og gir derfor muligheten til å redusere Monte Carlo-feilen ved å øke ensemble-størrelsen. Men, modeller med lavere kvalitet øker også modelleringsfeilen. Data-assimilering på flere nivåer (MLDA) bruker et utvalg av modeller som danner hierarkier av både beregningskostnad og beregningsnøyaktighet, og prøver åå oppnå en bedre balanse mellom Monte Carlo-feil og modelleringsfeil. I dette PhD-prosjektet ble flere MLDA-algoritmer utviklet og deres kvalitet for assimilering av inverterte seismiske data ble vurdert på forenklede reservoarproblemer. Bruk av modeller på flere nivå innebærer introduksjon av noen numeriske feil (multilevel modeling error, MLME), i tillegg til de allerede eksisterende numeriske feilene. Flere beregningsmessig rimelige metoder ble utviklet for delvis å kompansere for MLME i gjennomføring av data-assimilering på flere nivåer. Metodene ble også undersøkt under historie tilpassing på forenklede reservoar problemer. Til slutt ble en av de nye MLDA-algoritmene valgt og ytelsen ble vurdert på et historie tilpassings problem med en realistisk reservoar modell.In ensemble-based data assimilation (DA), the ensemble size is usually limited to around one hundred. Straightforward application of ensemble-based DA can therefore result in significant Monte Carlo errors, often manifesting themselves as severe underestimation of parameter uncertainties. Assimilation of large amounts of simultaneous data enhances the negative effects of Monte Carlo errors. Distance-based localization is the conventional remedy for this problem. However, it has its own drawbacks, e.g. not allowing for true long-range correlations and difficulty in assimilation of data which do not have a specific physical location. Use of lower-fidelity models reduces the computational cost per ensemble member and therefore renders the possibility to reduce Monte Carlo errors by increasing the ensemble size, but it also adds to the modeling error. Multilevel data assimilation (MLDA) uses a selection of models forming hierarchies of both computational cost and computational accuracy, and tries to obtain a better balance between Monte Carlo errors and modeling errors. In this PhD project, several MLDA algorithms were developed and their quality for assimilation of inverted seismic data was assessed in simplistic reservoir problems. Utilization of multilevel models entails introduction of some numerical errors (multilevel modeling error, MLME) to the problem in addition to the already existing numerical errors. Several computationally inexpensive methods were devised for partially accounting for MLME in the context of multilevel data assimilation. They were also investigated in simplistic reservoir history-matching problems. Finally, one of the novel MLDA algorithms was chosen and its performance was assessed in a realistic reservoir history-matching problem.Doktorgradsavhandlin

Advances in Spacecraft Attitude Control

Spacecraft attitude maneuvers comply with Euler's moment equations, a set of three nonlinear, coupled differential equations. Nonlinearities complicate the mathematical treatment of the seemingly simple action of rotating, and these complications lead to a robust lineage of research. This book is meant for basic scientifically inclined readers, and commences with a chapter on the basics of spaceflight and leverages this remediation to reveal very advanced topics to new spaceflight enthusiasts. The topics learned from reading this text will prepare students and faculties to investigate interesting spaceflight problems in an era where cube satellites have made such investigations attainable by even small universities. It is the fondest hope of the editor and authors that readers enjoy this book

Parameterization using Generative Adversarial Networks for Control Space Reduction in Data Assimilation.

This thesis examines the use of generative adversarial networks (GANs) as a parameterization tool for inverse problems solved with ensemble-based data assimilation methods. Ensemble methods often rely on the assumption of Gaussian distributed parameters in cases where this assumption is not valid, the parameter estimation can be invalid. Parameterization methods allow the transformation of these non-Gaussian parameters into a better suited distribution, and optimally reduce their dimension. Another limitation of ensemble methods is the injection of prior information of the physical relation as a constraint between parameters such as spatial coherence or physical balances. Optimal parameterization should encompass these different properties to facilitate the estimation. The novel approach presented in this work relies on GANs to achieve these objectives. Two application domains are tackled through the present work. In a first application, subsurface reservoir characterization, the objective is to determine geological properties of a numerical reservoir model from the observation of the reservoir dynamical response by the way of data assimilation. Rock facies, that describe the type of rock present in each cell of the numerical model, have to be determined due to their strong influence on the dynamical response. The rock facies spatial distribution is ruled by geological phenomena such as sedimentation and forms well known patterns, like channels, called heterogeneities. The noncontinuous property and their spatial coherence make their characterization by ensemble-based data assimilation algorithms difficult, and requires parameterization. Parameterization is a challenge for numerous heterogeneities, notably channels, due to the numerical cost or the statistical representation of their spatial distribution. A Second application domain is the atmospheric balance in the context of numerical weather prediction. When new observations are available, correction of the atmospheric state is done using ensemblebased data assimilation methods. This correction step can introduce imbalance in the physical state and cause numerical instability during the integration in time of the atmosphere, deteriorating the information brought by the previous observations. The importance of generating or correcting balanced climate, also called initialized atmospheric state, during data assimilation is then a key step in numerical weather prediction. This work aims at presenting the performance of GAN parameterization and its multi-disciplinary applicability to researchers who are not familiar with the domain of deep learning. GAN is an unsupervised deep learning method belonging to the deep generative network family, able to learn a dataset distribution and generate new samples from the learned distribution in an unsupervised way. These synthetic samples are encoded in a low-dimensional latent space that can be sampled from a Gaussian distribution that is suited to perform ensemble data assimilation. Their recent ability to generate complex images led us to consider them as a good candidate for parameterization method. The unsupervised property of this type of parameterization makes it applicable to several diverse domains such as learning the pattern of geological heterogeneities or learning the physical constraints that makes an atmospheric state balanced. This study shows how to train GANs for two different applications : subsurface reservoir and climate data. The use of the parameterization in an ensemble based data assimilation such as ensemble smoother with multiple data assimilation (ES-MDA) is demonstrated for subsurface reservoir. Finally, a posteriori conditioning of the GAN function is examined using derivative free optimization