8,322 research outputs found
Metaheuristic design of feedforward neural networks: a review of two decades of research
Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era
On-the-fly adaptivity for nonlinear twoscale simulations using artificial neural networks and reduced order modeling
A multi-fidelity surrogate model for highly nonlinear multiscale problems is
proposed. It is based on the introduction of two different surrogate models and
an adaptive on-the-fly switching. The two concurrent surrogates are built
incrementally starting from a moderate set of evaluations of the full order
model. Therefore, a reduced order model (ROM) is generated. Using a hybrid
ROM-preconditioned FE solver, additional effective stress-strain data is
simulated while the number of samples is kept to a moderate level by using a
dedicated and physics-guided sampling technique. Machine learning (ML) is
subsequently used to build the second surrogate by means of artificial neural
networks (ANN). Different ANN architectures are explored and the features used
as inputs of the ANN are fine tuned in order to improve the overall quality of
the ML model. Additional ANN surrogates for the stress errors are generated.
Therefore, conservative design guidelines for error surrogates are presented by
adapting the loss functions of the ANN training in pure regression or pure
classification settings. The error surrogates can be used as quality indicators
in order to adaptively select the appropriate -- i.e. efficient yet accurate --
surrogate. Two strategies for the on-the-fly switching are investigated and a
practicable and robust algorithm is proposed that eliminates relevant technical
difficulties attributed to model switching. The provided algorithms and ANN
design guidelines can easily be adopted for different problem settings and,
thereby, they enable generalization of the used machine learning techniques for
a wide range of applications. The resulting hybrid surrogate is employed in
challenging multilevel FE simulations for a three-phase composite with
pseudo-plastic micro-constituents. Numerical examples highlight the performance
of the proposed approach
A machine learning approach for efficient uncertainty quantification using multiscale methods
Several multiscale methods account for sub-grid scale features using coarse
scale basis functions. For example, in the Multiscale Finite Volume method the
coarse scale basis functions are obtained by solving a set of local problems
over dual-grid cells. We introduce a data-driven approach for the estimation of
these coarse scale basis functions. Specifically, we employ a neural network
predictor fitted using a set of solution samples from which it learns to
generate subsequent basis functions at a lower computational cost than solving
the local problems. The computational advantage of this approach is realized
for uncertainty quantification tasks where a large number of realizations has
to be evaluated. We attribute the ability to learn these basis functions to the
modularity of the local problems and the redundancy of the permeability patches
between samples. The proposed method is evaluated on elliptic problems yielding
very promising results.Comment: Journal of Computational Physics (2017
The Voice of Optimization
We introduce the idea that using optimal classification trees (OCTs) and
optimal classification trees with-hyperplanes (OCT-Hs), interpretable machine
learning algorithms developed by Bertsimas and Dunn [2017, 2018], we are able
to obtain insight on the strategy behind the optimal solution in continuous and
mixed-integer convex optimization problem as a function of key parameters that
affect the problem. In this way, optimization is not a black box anymore.
Instead, we redefine optimization as a multiclass classification problem where
the predictor gives insights on the logic behind the optimal solution. In other
words, OCTs and OCT-Hs give optimization a voice. We show on several realistic
examples that the accuracy behind our method is in the 90%-100% range, while
even when the predictions are not correct, the degree of suboptimality or
infeasibility is very low. We compare optimal strategy predictions of OCTs and
OCT-Hs and feedforward neural networks (NNs) and conclude that the performance
of OCT-Hs and NNs is comparable. OCTs are somewhat weaker but often
competitive. Therefore, our approach provides a novel insightful understanding
of optimal strategies to solve a broad class of continuous and mixed-integer
optimization problems
- …