Modeling and Analysis Generic Interface for eXternal numerical codes (MAGIX)

The modeling and analysis generic interface for external numerical codes (MAGIX) is a model optimizer developed under the framework of the coherent set of astrophysical tools for spectroscopy (CATS) project. The MAGIX package provides a framework of an easy interface between existing codes and an iterating engine that attempts to minimize deviations of the model results from available observational data, constraining the values of the model parameters and providing corresponding error estimates. Many models (and, in principle, not only astrophysical models) can be plugged into MAGIX to explore their parameter space and find the set of parameter values that best fits observational/experimental data. MAGIX complies with the data structures and reduction tools of ALMA (Atacama Large Millimeter Array), but can be used with other astronomical and with non-astronomical data.Comment: 12 pages, 15 figures, 2 tables, paper is also available at http://www.aanda.org/articles/aa/pdf/forth/aa20063-12.pd

Particle Swarm Optimization and Uncertainty Assessment in Inverse Problems

Most inverse problems in the industry (and particularly in geophysical exploration) are highly underdetermined because the number of model parameters too high to achieve accurate data predictions and because the sampling of the data space is scarce and incomplete; it is always affected by different kinds of noise. Additionally, the physics of the forward problem is a simplification of the reality. All these facts result in that the inverse problem solution is not unique; that is, there are different inverse solutions (called equivalent), compatible with the prior information that fits the observed data within similar error bounds. In the case of nonlinear inverse problems, these equivalent models are located in disconnected flat curvilinear valleys of the cost-function topography. The uncertainty analysis consists of obtaining a representation of this complex topography via different sampling methodologies. In this paper, we focus on the use of a particle swarm optimization (PSO) algorithm to sample the region of equivalence in nonlinear inverse problems. Although this methodology has a general purpose, we show its application for the uncertainty assessment of the solution of a geophysical problem concerning gravity inversion in sedimentary basins, showing that it is possible to efficiently perform this task in a sampling-while-optimizing mode. Particularly, we explain how to use and analyze the geophysical models sampled by exploratory PSO family members to infer different descriptors of nonlinear uncertainty

Strategies for optimal design of biomagnetic sensor systems

Magnetic field imaging (MFI) is a technique to record contact free the magnetic field distribution and estimate the underlying source distribution in the heart. Currently, the cardiomagnetic fields are recorded with superconducting quantum interference devices (SQUIDs), which are restricted to the inside of a cryostat filled with liquid helium or nitrogen. New room temperature optical magnetometers allow less restrictive sensor positioning, which raises the question of how to optimally place the sensors for robust field reconstruction. The objective in this study is to develop a generic object-oriented framework for optimizing sensor arrangements (sensor positions and orientations) which supports the necessary constraints of a limited search volume (only outside the body) and the technical minimum distance of sensors (e.g. 1 cm). In order to test the framework, a new quasi-continuous particle swarm optimizer (PSO) component is developed as well as an exemplary goal function component using the condition number (CN) of the leadfield matrix. Generic constraint handling algorithms are designed and implemented, that decompose complex constraints into basic ones. The constraint components interface to an operational exemplary optimization strategy which is validated on the magnetocardiographic sensor arrangement problem. The simulation setup includes a three compartment boundary element model of a torso with a fitted multi-dipole heart model. The results show that the CN, representing the reconstruction robustness of the inverse problem, can be reduced with our optimization by one order of magnitude within a sensor plane (the cryostat bottom) in front of the torso compared to a regular sensor grid. Reduction of another order of magnitude is achieved by optimizing sensor positions on the entire torso surface. Results also indicate that the number of sensors may be reduced to 20-30 without loss of robustness in terms of CN. The original contributions are the generic reusable framework and exemplary components, the quasicontinuous PSO algorithm with constraint support and the composite constraint handling algorithms

On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice

Machine learning algorithms have been used widely in various applications and areas. To fit a machine learning model into different problems, its hyper-parameters must be tuned. Selecting the best hyper-parameter configuration for machine learning models has a direct impact on the model's performance. It often requires deep knowledge of machine learning algorithms and appropriate hyper-parameter optimization techniques. Although several automatic optimization techniques exist, they have different strengths and drawbacks when applied to different types of problems. In this paper, optimizing the hyper-parameters of common machine learning models is studied. We introduce several state-of-the-art optimization techniques and discuss how to apply them to machine learning algorithms. Many available libraries and frameworks developed for hyper-parameter optimization problems are provided, and some open challenges of hyper-parameter optimization research are also discussed in this paper. Moreover, experiments are conducted on benchmark datasets to compare the performance of different optimization methods and provide practical examples of hyper-parameter optimization. This survey paper will help industrial users, data analysts, and researchers to better develop machine learning models by identifying the proper hyper-parameter configurations effectively.Comment: 69 Pages, 10 tables, accepted in Neurocomputing, Elsevier. Github link: https://github.com/LiYangHart/Hyperparameter-Optimization-of-Machine-Learning-Algorithm

Efficient multiobjective optimization employing Gaussian processes, spectral sampling and a genetic algorithm

Many engineering problems require the optimization of expensive, black-box functions involving multiple conflicting criteria, such that commonly used methods like multiobjective genetic algorithms are inadequate. To tackle this problem several algorithms have been developed using surrogates. However, these often have disadvantages such as the requirement of a priori knowledge of the output functions or exponentially scaling computational cost with respect to the number of objectives. In this paper a new algorithm is proposed, TSEMO, which uses Gaussian processes as surrogates. The Gaussian processes are sampled using spectral sampling techniques to make use of Thompson sampling in conjunction with the hypervolume quality indicator and NSGA-II to choose a new evaluation point at each iteration. The reference point required for the hypervolume calculation is estimated within TSEMO. Further, a simple extension was proposed to carry out batch-sequential design. TSEMO was compared to ParEGO, an expected hypervolume implementation, and NSGA-II on 9 test problems with a budget of 150 function evaluations. Overall, TSEMO shows promising performance, while giving a simple algorithm without the requirement of a priori knowledge, reduced hypervolume calculations to approach linear scaling with respect to the number of objectives, the capacity to handle noise and lastly the ability for batch-sequential usage

Optimization of convolutional neural networks for image classification using genetic algorithms and bayesian optimization

Notwithstanding the recent successes of deep convolutional neural networks for classification tasks, they are sensitive to the selection of their hyperparameters, which impose an exponentially large search space on modern convolutional models. Traditional hyperparameter selection methods include manual, grid, or random search, but these require expert knowledge or are computationally burdensome. Divergently, Bayesian optimization and evolutionary inspired techniques have surfaced as viable alternatives to the hyperparameter problem. Thus, an alternative hybrid approach that combines the advantages of these techniques is proposed. Specifically, the search space is partitioned into discrete-architectural, and continuous and categorical hyperparameter subspaces, which are respectively traversed by a stochastic genetic search, followed by a genetic-Bayesian search. Simulations on a prominent image classification task reveal that the proposed method results in an overall classification accuracy improvement of 0.87% over unoptimized baselines, and a greater than 97% reduction in computational costs compared to a commonly employed brute force approach.Electrical and Mining EngineeringM. Tech. (Electrical Engineering

A New Approach to Model Parameter Determination of Self-Potential Data using Memory-based Hybrid Dragonfly Algorithm

A new approach based on global optimization technique is applied to invert Self-Potential (SP) data which is a highly nonlinear inversion problem. This technique is called Memory-based Hybrid Dragonfly Algorithm (MHDA). This algorithm is proposed to balance out the high exploration behavior of Dragonfly Algorithm (DA), which causes a low convergence rate and often leads to the local optimum solution. MHDA was developed by adding internal memory and iterative level hybridization into DA which successfully balanced the exploration and exploitation behaviors of DA. In order to assess the performance of MHDA, it is firstly implemented to invert the single and multiple noises contaminated in synthetic SP data, which were caused by several simple geometries of buried anomalies: sphere and inclined sheet. MHDA is subsequently implemented to invert the field SP data for several cases: buried metallic drum, landslide, and Lumpur Sidoarjo (LUSI) embankment anomalies. As a stochastic method, MHDA is able to provide Posterior Distribution Model (PDM), which contains possible solutions of the SP data inversion. PDM is obtained from the exploration behavior of MHDA. All accepted models as PDM have a lower misfit value than the specified tolerance value of the objective function in the inversion process. In this research, solutions of the synthetic and field SP data inversions are estimated by the median value of PDM. Furthermore, the uncertainty value of obtained solutions can be estimated by the standard deviation value of PDM. The inversion results of synthetic and field SP data show that MHDA is able to estimate the solutions and the uncertainty values of solutions well. It indicates that MHDA is a good and an innovative technique to be implemented in solving the SP data inversion problem