Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealing

Variational autoencoders (VAEs) face a notorious problem wherein the variational posterior often aligns closely with the prior, a phenomenon known as posterior collapse, which hinders the quality of representation learning. To mitigate this problem, an adjustable hyperparameter $\beta$ and a strategy for annealing this parameter, called KL annealing, are proposed. This study presents a theoretical analysis of the learning dynamics in a minimal VAE. It is rigorously proved that the dynamics converge to a deterministic process within the limit of large input dimensions, thereby enabling a detailed dynamical analysis of the generalization error. Furthermore, the analysis shows that the VAE initially learns entangled representations and gradually acquires disentangled representations. A fixed-point analysis of the deterministic process reveals that when $\beta$ exceeds a certain threshold, posterior collapse becomes inevitable regardless of the learning period. Additionally, the superfluous latent variables for the data-generative factors lead to overfitting of the background noise; this adversely affects both generalization and learning convergence. The analysis further unveiled that appropriately tuned KL annealing can accelerate convergence.Comment: 24 pages, 5 figure

Dataset Size Dependence of Rate-Distortion Curve and Threshold of Posterior Collapse in Linear VAE

In the Variational Autoencoder (VAE), the variational posterior often aligns closely with the prior, which is known as posterior collapse and hinders the quality of representation learning. To mitigate this problem, an adjustable hyperparameter beta has been introduced in the VAE. This paper presents a closed-form expression to assess the relationship between the beta in VAE, the dataset size, the posterior collapse, and the rate-distortion curve by analyzing a minimal VAE in a high-dimensional limit. These results clarify that a long plateau in the generalization error emerges with a relatively larger beta. As the beta increases, the length of the plateau extends and then becomes infinite beyond a certain beta threshold. This implies that the choice of beta, unlike the usual regularization parameters, can induce posterior collapse regardless of the dataset size. Thus, beta is a risky parameter that requires careful tuning. Furthermore, considering the dataset-size dependence on the rate-distortion curve, a relatively large dataset is required to obtain a rate-distortion curve with high rates. Extensive numerical experiments support our analysis.Comment: 16 pages, 3 figure

Wino Dark Matter and Future dSph Observations

We discuss the indirect detection of the wino dark matter utilizing gamma-ray observations of dwarf spheroidal galaxies (dSphs). After carefully reviewing current limits with particular attention to astrophysical uncertainties, we show prospects of the wino mass limit in future gamma-ray observation by the Fermi-LAT and the GAMMA-400 telescopes. We find that the improvement of the so-called $J$-factor of both the classical and the ultra-faint dSphs will play a crucial role to cover whole mass range of the wino dark matter. For example, with $\delta (\log_{10}J) = 0.1$ for both the classical and the ultra-faint dSphs, whole wino dark matter mass range can be covered by 15 years and 10 years data at the Fermi-LAT and GAMMA-400 telescopes, respectively.Comment: 32 pages, 13 figure

High-Scale SUSY Breaking Models in light of the BICEP2 Result

The large value of the tensor-to-scalar ratio in the cosmic microwave background radiation reported by the BICEP2 collaboration gives strong impact on models of supersymmetry (SUSY). The large ratio indicates inflation with a high-energy scale and thus a high reheating temperature in general, and various SUSY models suffer from the serious gravitino and Polonyi problems. In this article, we discuss a class of the high-scale SUSY breaking models which are completely free from those problems. With especially focusing on the dark matter relic abundance, we examine how the BICEP2 result narrows down the parameter space of the models, assuming the simplest chaotic inflation model. We find that the mass of the dark matter is predicted to be less than about 1 TeV thanks to the non-thermal production in the early universe through the decay of abundant gravitinos produced after the reheating process. We also discuss implications in some details to dark matter searches at collider and indirect dark matter detection experiments.Comment: 17 pages, 6 figure

Indirect Probe of Electroweak-Interacting Particles at Future Lepton Colliders

Various types of electroweak-interacting particles, which have non-trivial charges under the $\mathrm{SU}(2)_L \times \mathrm{U}(1)_Y$ gauge symmetry, appear in various extensions of the Standard Model. These particles are good targets of future lepton colliders, such as the International Linear Collider (ILC), the Compact LInear Collider (CLIC) and the Future Circular Collider of electrons and positrons (FCC-ee). An advantage of the experiments is that, even if their beam energies are below the threshold of the production of the new particles, quantum effects of the particles can be detected through high precision measurements. We estimate the capability of future lepton colliders to probe electroweak-interacting particles through the quantum effects, with particular focus on the wino, the Higgsino and the so-called minimal dark matters, and found that a particle whose mass is greater than the beam energy by 100-1000 GeV is detectable by measuring di-fermion production cross sections with $O(0.1)$\% accuracy. In addition, with the use of the same analysis, we also discuss the sensitivity of the future colliders to model independent higher dimensional operators, and found that the cutoff scales corresponding to the operators can be probed up to a few ten TeV

Heavy Fermion Bound States for Diphoton Excess at 750GeV ∼\sim Collider and Cosmological Constraints ∼\sim

A colored heavy particle with sufficiently small width may form non-relativistic bound states when they are produced at the large hadron collider\,(LHC), and they can annihilate into a diphoton final state. The invariant mass of the diphoton would be around twice of the colored particle mass. In this paper, we study if such bound state can be responsible for the 750 GeV diphoton excess reported by ATLAS and CMS. We found that the best-fit signal cross section is obtained for the SU(2)$_L$ singlet colored fermion $X$ with $Y_X=4/3$. Having such an exotic hypercharge, the particle is expected to decay through some higher dimensional operators, consistent with the small width assumption. The decay of $X$ may involve a stable particle $\chi$, if both $X$ and $\chi$ are odd under some conserved $Z_2$ symmetry. In that case, the particle $X$ suffers from the constraints of jets + missing $E_T$ searches by ATLAS and CMS at 8 TeV and 13 TeV. We found that such a scenario still survives if the mass difference between $X$ and $\chi$ is above $\sim$ 30 GeV for $m_X \sim 375$ GeV. Even assuming pair annihilation of $\chi$ is small, the relic density of $\chi$ is small enough if the mass difference between $X$ and $\chi$ is smaller than $\sim$ 40 GeV

Metal Hydride-Based Materials as Negative Electrode for All- Solid-State Lithium-Ion Batteries

The recently developed metal hydride (MH)-based material is considered to be a potential negative material for lithium-ion batteries, owing to its high theoretical Li storage capacity, relatively low volume expansion, and suitable working potential with very small polarization. However, it suffers from the slow kinetics, poor reversibility, and unfavourable cyclability in conventional organic liquid electrolyte systems, which enormously limit its practical application. In this chapter, we describe an all-solid-state battery system consisting of MH working electrode, LiBH4 solid electrolyte, and Li metal counter electrode. The electrochemical properties of MgH2 and TiH2 composites are investigated, which showed much better performances using LiBH4 as solid-state electrolyte than using conventional organic liquid electrolyte