799 research outputs found
Improved Dropout for Shallow and Deep Learning
Dropout has been witnessed with great success in training deep neural
networks by independently zeroing out the outputs of neurons at random. It has
also received a surge of interest for shallow learning, e.g., logistic
regression. However, the independent sampling for dropout could be suboptimal
for the sake of convergence. In this paper, we propose to use multinomial
sampling for dropout, i.e., sampling features or neurons according to a
multinomial distribution with different probabilities for different
features/neurons. To exhibit the optimal dropout probabilities, we analyze the
shallow learning with multinomial dropout and establish the risk bound for
stochastic optimization. By minimizing a sampling dependent factor in the risk
bound, we obtain a distribution-dependent dropout with sampling probabilities
dependent on the second order statistics of the data distribution. To tackle
the issue of evolving distribution of neurons in deep learning, we propose an
efficient adaptive dropout (named \textbf{evolutional dropout}) that computes
the sampling probabilities on-the-fly from a mini-batch of examples. Empirical
studies on several benchmark datasets demonstrate that the proposed dropouts
achieve not only much faster convergence and but also a smaller testing error
than the standard dropout. For example, on the CIFAR-100 data, the evolutional
dropout achieves relative improvements over 10\% on the prediction performance
and over 50\% on the convergence speed compared to the standard dropout.Comment: In NIPS 201
Dark energy imprints on the kinematic Sunyaev-Zel'dovich signal
We investigate the imprint of dark energy on the kinetic Sunyaev-Zel'dovich
(kSZ) angular power spectrum on scales of to , and find that
the kSZ signal is sensitive to the dark energy parameter. For example, varying
the constant by 20\% around results in a change on the
kSZ spectrum; changing the dark energy dynamics parametrized by by
, a 30\% change on the kSZ spectrum is expected. We discuss the
observational aspects and develop a fitting formula for the kSZ power spectrum.
Finally, we discuss how the precise modeling of the post-reionization signal
would help the constraints on patchy reionization signal, which is crucial for
measuring the duration of reionization.Comment: 12 pages, 9 figures, 2 table
Cosmic Mach Number: A Sensitive Probe for the Growth of Structure
In this Letter, we investigate the potential power of the Cosmic Mach Number
(CMN), which is the ratio between the mean velocity and the velocity dispersion
of galaxies as a function of cosmic scales, to constrain cosmologies. We first
measure the CMN from 5 catalogues of galaxy peculiar velocity surveys at low
redshift (0.002<z<0.03), and use them to contrast cosmological models. Overall,
current data is consistent with the WMAP7 LCDM model. We find that the CMN is
highly sensitive to the growth of structure on scales 0.01<k<0.1 h/Mpc in
Fourier space. Therefore, modified gravity models, and models with massive
neutrinos, in which the structure growth generally deviates from that in the
LCDM model in a scale-dependent way, can be well differentiated from the LCDM
model using future CMN data.Comment: 7 pages, matches the version accepted to JCA
- β¦