Using Synthetic Data to Train Neural Networks is Model-Based Reasoning

We draw a formal connection between using synthetic training data to optimize neural network parameters and approximate, Bayesian, model-based reasoning. In particular, training a neural network using synthetic data can be viewed as learning a proposal distribution generator for approximate inference in the synthetic-data generative model. We demonstrate this connection in a recognition task where we develop a novel Captcha-breaking architecture and train it using synthetic data, demonstrating both state-of-the-art performance and a way of computing task-specific posterior uncertainty. Using a neural network trained this way, we also demonstrate successful breaking of real-world Captchas currently used by Facebook and Wikipedia. Reasoning from these empirical results and drawing connections with Bayesian modeling, we discuss the robustness of synthetic data results and suggest important considerations for ensuring good neural network generalization when training with synthetic data.Comment: 8 pages, 4 figure

Deep Variational Reinforcement Learning for POMDPs

Many real-world sequential decision making problems are partially observable by nature, and the environment model is typically unknown. Consequently, there is great need for reinforcement learning methods that can tackle such problems given only a stream of incomplete and noisy observations. In this paper, we propose deep variational reinforcement learning (DVRL), which introduces an inductive bias that allows an agent to learn a generative model of the environment and perform inference in that model to effectively aggregate the available information. We develop an n-step approximation to the evidence lower bound (ELBO), allowing the model to be trained jointly with the policy. This ensures that the latent state representation is suitable for the control task. In experiments on Mountain Hike and flickering Atari we show that our method outperforms previous approaches relying on recurrent neural networks to encode the past

Auto-Encoding Sequential Monte Carlo

We build on auto-encoding sequential Monte Carlo (AESMC): a method for model and proposal learning based on maximizing the lower bound to the log marginal likelihood in a broad family of structured probabilistic models. Our approach relies on the efficiency of sequential Monte Carlo (SMC) for performing inference in structured probabilistic models and the flexibility of deep neural networks to model complex conditional probability distributions. We develop additional theoretical insights and introduce a new training procedure which improves both model and proposal learning. We demonstrate that our approach provides a fast, easy-to-implement and scalable means for simultaneous model learning and proposal adaptation in deep generative models

The shapes of light curves of Mira-type variables

Using a sample of 454 mira light curves from the ASAS survey we study the shape of the light variations in this kind of variable stars. Opposite to earlier studies, we choose a general approach to identify any deviation from a sinusoidal light change. We find that about 30% of the studied light curves show a significant deviation from the sinusoidal reference shape. Among these stars two characteristic light curve shapes of comparable frequency could be identified. Some hint for a connection between atmospheric chemistry and light curve shape was found, but beside that no or only very weak relations between light curve shape and other stellar parameters seem to exist.Comment: 7 pages, 7 figures, accepted for publication in A

Why metallic surfaces with grooves a few nanometers deep and wide may strongly absorb visible light

It is theoretically shown that nanometric silver lamellar gratings present very strong visible light absorption inside the grooves, leading to electric field intensities by several orders of magnitude larger than that of the impinging light. This effect, due to the excitation of long wave vector surface plasmon polaritons with particular small penetration depth in the metal, may explain the abnormal optical absorption observed a long time ago on almost flat Ag films. Surface enhanced Raman scattering in rough metallic films could also be due to the excitation of such plasmon polaritons in the grain boundaries or notches of the films.Comment: 5 pages, 5 figure, submitted to Phys. Rev. Let

Bayesian Optimization for Probabilistic Programs

We present the first general purpose framework for marginal maximum a posteriori estimation of probabilistic program variables. By using a series of code transformations, the evidence of any probabilistic program, and therefore of any graphical model, can be optimized with respect to an arbitrary subset of its sampled variables. To carry out this optimization, we develop the first Bayesian optimization package to directly exploit the source code of its target, leading to innovations in problem-independent hyperpriors, unbounded optimization, and implicit constraint satisfaction; delivering significant performance improvements over prominent existing packages. We present applications of our method to a number of tasks including engineering design and parameter optimization

Variability and spectral classification of LMC giants: results from DENIS and EROS

We present the first cross-identifications of sources in the near-infrared DENIS survey and in the micro-lensing EROS survey in a field of about 0.5 square degrees around the optical center (OC) of the Large Magellanic Cloud. We analyze the photometric history of these stars in the EROS data base and obtain light-curves for about 800 variables. Most of the stars are long period variables (Miras and Semi-Regulars), a few Cepheids are also present. We also present new spectroscopic data on 126 asymptotic giant branch stars in the OC field, 30 previously known and 96 newly discovered by the DENIS survey. The visible spectra are used to assign a carbon- (C-) or oxygen-rich (O-rich) nature to the observed stars on the basis of the presence of molecular bands of TiO, VO, CN, C2. For the remaining of the stars we used the (J-Ks) color to determine whether they are O-rich or C-rich. Plotting Log(period) versus Ks we find three very distinct period-luminosity relations, mainly populated by Semi-Regular of type a (SRa), b (SRb) and Mira variables. Carbon-rich stars occupy mostly the upper part of these relations. We find that 65% of the asymptotic giant branch population are long period variables (LPVs).Comment: 9 pages, 7 figures, 4 tables (2 via CDS), accepted by A&A journa