4,102 research outputs found
Bayesian Neural Architecture Search using A Training-Free Performance Metric
Recurrent neural networks (RNNs) are a powerful approach for time series
prediction. However, their performance is strongly affected by their
architecture and hyperparameter settings. The architecture optimization of RNNs
is a time-consuming task, where the search space is typically a mixture of
real, integer and categorical values. To allow for shrinking and expanding the
size of the network, the representation of architectures often has a variable
length. In this paper, we propose to tackle the architecture optimization
problem with a variant of the Bayesian Optimization (BO) algorithm. To reduce
the evaluation time of candidate architectures the Mean Absolute Error Random
Sampling (MRS), a training-free method to estimate the network performance, is
adopted as the objective function for BO. Also, we propose three fixed-length
encoding schemes to cope with the variable-length architecture representation.
The result is a new perspective on accurate and efficient design of RNNs, that
we validate on three problems. Our findings show that 1) the BO algorithm can
explore different network architectures using the proposed encoding schemes and
successfully designs well-performing architectures, and 2) the optimization
time is significantly reduced by using MRS, without compromising the
performance as compared to the architectures obtained from the actual training
procedure
Computational Optimizations for Machine Learning
The present book contains the 10 articles finally accepted for publication in the Special Issue “Computational Optimizations for Machine Learning” of the MDPI journal Mathematics, which cover a wide range of topics connected to the theory and applications of machine learning, neural networks and artificial intelligence. These topics include, among others, various types of machine learning classes, such as supervised, unsupervised and reinforcement learning, deep neural networks, convolutional neural networks, GANs, decision trees, linear regression, SVM, K-means clustering, Q-learning, temporal difference, deep adversarial networks and more. It is hoped that the book will be interesting and useful to those developing mathematical algorithms and applications in the domain of artificial intelligence and machine learning as well as for those having the appropriate mathematical background and willing to become familiar with recent advances of machine learning computational optimization mathematics, which has nowadays permeated into almost all sectors of human life and activity
Model Predictive Evolutionary Temperature Control via Neural-Network-Based Digital Twins
In this study, we propose a population-based, data-driven intelligent controller that leverages neural-network-based digital twins for hypothesis testing. Initially, a diverse set of control laws is generated using genetic programming with the digital twin of the system, facilitating a robust response to unknown disturbances. During inference, the trained digital twin is utilized to virtually test alternative control actions for a multi-objective optimization task associated with each control action. Subsequently, the best policy is applied to the system. To evaluate the proposed model predictive control pipeline, experiments are conducted on a multi-mode heat transfer test rig. The objective is to achieve homogeneous cooling over the surface, minimizing the occurrence of hot spots and energy consumption. The measured variable vector comprises high dimensional infrared camera measurements arranged as a sequence (655,360 inputs), while the control variable includes power settings for fans responsible for convective cooling (3 outputs). Disturbances are induced by randomly altering the local heat loads. The findings reveal that by utilizing an evolutionary algorithm on measured data, a population of control laws can be effectively learned in the virtual space. This empowers the system to deliver robust performance. Significantly, the digital twin-assisted, population-based model predictive control (MPC) pipeline emerges as a superior approach compared to individual control models, especially when facing sudden and random changes in local heat loads. Leveraging the digital twin to virtually test alternative control policies leads to substantial improvements in the controller’s performance, even with limited training data
- …