12,221 research outputs found
BayesNAS: A Bayesian Approach for Neural Architecture Search
One-Shot Neural Architecture Search (NAS) is a promising method to
significantly reduce search time without any separate training. It can be
treated as a Network Compression problem on the architecture parameters from an
over-parameterized network. However, there are two issues associated with most
one-shot NAS methods. First, dependencies between a node and its predecessors
and successors are often disregarded which result in improper treatment over
zero operations. Second, architecture parameters pruning based on their
magnitude is questionable. In this paper, we employ the classic Bayesian
learning approach to alleviate these two issues by modeling architecture
parameters using hierarchical automatic relevance determination (HARD) priors.
Unlike other NAS methods, we train the over-parameterized network for only one
epoch then update the architecture. Impressively, this enabled us to find the
architecture on CIFAR-10 within only 0.2 GPU days using a single GPU.
Competitive performance can be also achieved by transferring to ImageNet. As a
byproduct, our approach can be applied directly to compress convolutional
neural networks by enforcing structural sparsity which achieves extremely
sparse networks without accuracy deterioration.Comment: International Conference on Machine Learning 201
Critical fluctuations in proteins native states
We study a large data set of protein structure ensembles of very diverse
sizes determined by nuclear magnetic resonance. By examining the
distance-dependent correlations in the displacement of residues pairs and
conducting finite size scaling analysis it was found that the correlations and
susceptibility behave as in systems near a critical point implying that, at the
native state, the motion of each amino acid residue is felt by every other
residue up to the size of the protein molecule. Furthermore certain protein's
shapes corresponding to maximum susceptibility were found to be more probable
than others. Overall the results suggest that the protein's native state is
critical, implying that despite being posed near the minimum of the energy
landscape, they still preserve their dynamic flexibility
- …