Alternating Back-Propagation for Generator Network

This paper proposes an alternating back-propagation algorithm for learning the generator network model. The model is a non-linear generalization of factor analysis. In this model, the mapping from the continuous latent factors to the observed signal is parametrized by a convolutional neural network. The alternating back-propagation algorithm iterates the following two steps: (1) Inferential back-propagation, which infers the latent factors by Langevin dynamics or gradient descent. (2) Learning back-propagation, which updates the parameters given the inferred latent factors by gradient descent. The gradient computations in both steps are powered by back-propagation, and they share most of their code in common. We show that the alternating back-propagation algorithm can learn realistic generator models of natural images, video sequences, and sounds. Moreover, it can also be used to learn from incomplete or indirect training data

On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models

This study investigates the effects of Markov chain Monte Carlo (MCMC) sampling in unsupervised Maximum Likelihood (ML) learning. Our attention is restricted to the family of unnormalized probability densities for which the negative log density (or energy function) is a ConvNet. We find that many of the techniques used to stabilize training in previous studies are not necessary. ML learning with a ConvNet potential requires only a few hyper-parameters and no regularization. Using this minimal framework, we identify a variety of ML learning outcomes that depend solely on the implementation of MCMC sampling. On one hand, we show that it is easy to train an energy-based model which can sample realistic images with short-run Langevin. ML can be effective and stable even when MCMC samples have much higher energy than true steady-state samples throughout training. Based on this insight, we introduce an ML method with purely noise-initialized MCMC, high-quality short-run synthesis, and the same budget as ML with informative MCMC initialization such as CD or PCD. Unlike previous models, our energy model can obtain realistic high-diversity samples from a noise signal after training. On the other hand, ConvNet potentials learned with non-convergent MCMC do not have a valid steady-state and cannot be considered approximate unnormalized densities of the training data because long-run MCMC samples differ greatly from observed images. We show that it is much harder to train a ConvNet potential to learn a steady-state over realistic images. To our knowledge, long-run MCMC samples of all previous models lose the realism of short-run samples. With correct tuning of Langevin noise, we train the first ConvNet potentials for which long-run and steady-state MCMC samples are realistic images.Comment: Code available at: https://github.com/point0bar1/ebm-anatom

Gate-Tunable Tunneling Resistance in Graphene/Topological Insulator Vertical Junctions

Graphene-based vertical heterostructures, particularly stacks incorporated with other layered materials, are promising for nanoelectronics. The stacking of two model Dirac materials, graphene and topological insulator, can considerably enlarge the family of van der Waals heterostructures. Despite well understanding of the two individual materials, the electron transport properties of a combined vertical heterojunction are still unknown. Here we show the experimental realization of a vertical heterojunction between Bi2Se3 nanoplate and monolayer graphene. At low temperatures, the electron transport through the vertical heterojunction is dominated by the tunneling process, which can be effectively tuned by gate voltage to alter the density of states near the Fermi surface. In the presence of a magnetic field, quantum oscillations are observed due to the quantized Landau levels in both graphene and the two-dimensional surface states of Bi2Se3. Furthermore, we observe an exotic gate-tunable tunneling resistance under high magnetic field, which displays resistance maxima when the underlying graphene becomes a quantum Hall insulator

Modeling the pulse signal by wave-shape function and analyzing by synchrosqueezing transform

We apply the recently developed adaptive non-harmonic model based on the wave-shape function, as well as the time-frequency analysis tool called synchrosqueezing transform (SST) to model and analyze oscillatory physiological signals. To demonstrate how the model and algorithm work, we apply them to study the pulse wave signal. By extracting features called the spectral pulse signature, {and} based on functional regression, we characterize the hemodynamics from the radial pulse wave signals recorded by the sphygmomanometer. Analysis results suggest the potential of the proposed signal processing approach to extract health-related hemodynamics features