300 research outputs found
Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing
Variational autoencoders (VAEs) face a notorious problem wherein the
variational posterior often aligns closely with the prior, a phenomenon known
as posterior collapse, which hinders the quality of representation learning. To
mitigate this problem, an adjustable hyperparameter and a strategy for
annealing this parameter, called KL annealing, are proposed. This study
presents a theoretical analysis of the learning dynamics in a minimal VAE. It
is rigorously proved that the dynamics converge to a deterministic process
within the limit of large input dimensions, thereby enabling a detailed
dynamical analysis of the generalization error. Furthermore, the analysis shows
that the VAE initially learns entangled representations and gradually acquires
disentangled representations. A fixed-point analysis of the deterministic
process reveals that when exceeds a certain threshold, posterior
collapse becomes inevitable regardless of the learning period. Additionally,
the superfluous latent variables for the data-generative factors lead to
overfitting of the background noise; this adversely affects both generalization
and learning convergence. The analysis further unveiled that appropriately
tuned KL annealing can accelerate convergence.Comment: 24 pages, 5 figure
Dataset Size Dependence of Rate-Distortion Curve and Threshold of Posterior Collapse in Linear VAE
In the Variational Autoencoder (VAE), the variational posterior often aligns
closely with the prior, which is known as posterior collapse and hinders the
quality of representation learning. To mitigate this problem, an adjustable
hyperparameter beta has been introduced in the VAE. This paper presents a
closed-form expression to assess the relationship between the beta in VAE, the
dataset size, the posterior collapse, and the rate-distortion curve by
analyzing a minimal VAE in a high-dimensional limit. These results clarify that
a long plateau in the generalization error emerges with a relatively larger
beta. As the beta increases, the length of the plateau extends and then becomes
infinite beyond a certain beta threshold. This implies that the choice of beta,
unlike the usual regularization parameters, can induce posterior collapse
regardless of the dataset size. Thus, beta is a risky parameter that requires
careful tuning. Furthermore, considering the dataset-size dependence on the
rate-distortion curve, a relatively large dataset is required to obtain a
rate-distortion curve with high rates. Extensive numerical experiments support
our analysis.Comment: 16 pages, 3 figure
Wino Dark Matter and Future dSph Observations
We discuss the indirect detection of the wino dark matter utilizing gamma-ray
observations of dwarf spheroidal galaxies (dSphs). After carefully reviewing
current limits with particular attention to astrophysical uncertainties, we
show prospects of the wino mass limit in future gamma-ray observation by the
Fermi-LAT and the GAMMA-400 telescopes. We find that the improvement of the
so-called -factor of both the classical and the ultra-faint dSphs will play
a crucial role to cover whole mass range of the wino dark matter. For example,
with for both the classical and the ultra-faint
dSphs, whole wino dark matter mass range can be covered by 15 years and 10
years data at the Fermi-LAT and GAMMA-400 telescopes, respectively.Comment: 32 pages, 13 figure
High-Scale SUSY Breaking Models in light of the BICEP2 Result
The large value of the tensor-to-scalar ratio in the cosmic microwave
background radiation reported by the BICEP2 collaboration gives strong impact
on models of supersymmetry (SUSY). The large ratio indicates inflation with a
high-energy scale and thus a high reheating temperature in general, and various
SUSY models suffer from the serious gravitino and Polonyi problems. In this
article, we discuss a class of the high-scale SUSY breaking models which are
completely free from those problems. With especially focusing on the dark
matter relic abundance, we examine how the BICEP2 result narrows down the
parameter space of the models, assuming the simplest chaotic inflation model.
We find that the mass of the dark matter is predicted to be less than about 1
TeV thanks to the non-thermal production in the early universe through the
decay of abundant gravitinos produced after the reheating process. We also
discuss implications in some details to dark matter searches at collider and
indirect dark matter detection experiments.Comment: 17 pages, 6 figure
Indirect Probe of Electroweak-Interacting Particles at Future Lepton Colliders
Various types of electroweak-interacting particles, which have non-trivial
charges under the gauge symmetry,
appear in various extensions of the Standard Model. These particles are good
targets of future lepton colliders, such as the International Linear Collider
(ILC), the Compact LInear Collider (CLIC) and the Future Circular Collider of
electrons and positrons (FCC-ee). An advantage of the experiments is that, even
if their beam energies are below the threshold of the production of the new
particles, quantum effects of the particles can be detected through high
precision measurements. We estimate the capability of future lepton colliders
to probe electroweak-interacting particles through the quantum effects, with
particular focus on the wino, the Higgsino and the so-called minimal dark
matters, and found that a particle whose mass is greater than the beam energy
by 100-1000 GeV is detectable by measuring di-fermion production cross sections
with \% accuracy. In addition, with the use of the same analysis, we
also discuss the sensitivity of the future colliders to model independent
higher dimensional operators, and found that the cutoff scales corresponding to
the operators can be probed up to a few ten TeV
Heavy Fermion Bound States for Diphoton Excess at 750GeV Collider and Cosmological Constraints
A colored heavy particle with sufficiently small width may form
non-relativistic bound states when they are produced at the large hadron
collider\,(LHC), and they can annihilate into a diphoton final state. The
invariant mass of the diphoton would be around twice of the colored particle
mass. In this paper, we study if such bound state can be responsible for the
750 GeV diphoton excess reported by ATLAS and CMS. We found that the best-fit
signal cross section is obtained for the SU(2) singlet colored fermion
with . Having such an exotic hypercharge, the particle is expected to
decay through some higher dimensional operators, consistent with the small
width assumption. The decay of may involve a stable particle , if
both and are odd under some conserved symmetry. In that case,
the particle suffers from the constraints of jets + missing searches
by ATLAS and CMS at 8 TeV and 13 TeV. We found that such a scenario still
survives if the mass difference between and is above 30 GeV
for GeV. Even assuming pair annihilation of is small, the
relic density of is small enough if the mass difference between and
is smaller than 40 GeV
Metal Hydride-Based Materials as Negative Electrode for All- Solid-State Lithium-Ion Batteries
The recently developed metal hydride (MH)-based material is considered to be a potential negative material for lithium-ion batteries, owing to its high theoretical Li storage capacity, relatively low volume expansion, and suitable working potential with very small polarization. However, it suffers from the slow kinetics, poor reversibility, and unfavourable cyclability in conventional organic liquid electrolyte systems, which enormously limit its practical application. In this chapter, we describe an all-solid-state battery system consisting of MH working electrode, LiBH4 solid electrolyte, and Li metal counter electrode. The electrochemical properties of MgH2 and TiH2 composites are investigated, which showed much better performances using LiBH4 as solid-state electrolyte than using conventional organic liquid electrolyte
- …