24 research outputs found
A view of Estimation of Distribution Algorithms through the lens of Expectation-Maximization
We show that a large class of Estimation of Distribution Algorithms,
including, but not limited to, Covariance Matrix Adaption, can be written as a
Monte Carlo Expectation-Maximization algorithm, and as exact EM in the limit of
infinite samples. Because EM sits on a rigorous statistical foundation and has
been thoroughly analyzed, this connection provides a new coherent framework
with which to reason about EDAs
Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling
Deep neural networks (DNNs) are powerful machine learning models and have
succeeded in various artificial intelligence tasks. Although various
architectures and modules for the DNNs have been proposed, selecting and
designing the appropriate network structure for a target problem is a
challenging task. In this paper, we propose a method to simultaneously optimize
the network structure and weight parameters during neural network training. We
consider a probability distribution that generates network structures, and
optimize the parameters of the distribution instead of directly optimizing the
network structure. The proposed method can apply to the various network
structure optimization problems under the same framework. We apply the proposed
method to several structure optimization problems such as selection of layers,
selection of unit types, and selection of connections using the MNIST,
CIFAR-10, and CIFAR-100 datasets. The experimental results show that the
proposed method can find the appropriate and competitive network structures.Comment: To appear in the Thirty-Second AAAI Conference on Artificial
Intelligence (AAAI-18), 9 page