Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods that have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, and on real-world video frames. We present analyses of the learned network representations, showing it is implicitly learning a compact encoding of object appearance and motion. We also demonstrate a few of its applications, including visual analogy-making and video extrapolation.Comment: Journal preprint of arXiv:1607.02586 (IEEE TPAMI, 2019). The first two authors contributed equally to this work. Project page: http://visualdynamics.csail.mit.ed

Quantum features derived from the classical model of a bouncer-walker coupled to a zero-point field

In our bouncer-walker model a quantum is a nonequilibrium steady-state maintained by a permanent throughput of energy. Specifically, we consider a "particle" as a bouncer whose oscillations are phase-locked with those of the energy-momentum reservoir of the zero-point field (ZPF), and we combine this with the random-walk model of the walker, again driven by the ZPF. Starting with this classical toy model of the bouncer-walker we were able to derive fundamental elements of quantum theory. Here this toy model is revisited with special emphasis on the mechanism of emergence. Especially the derivation of the total energy hbar.omega and the coupling to the ZPF are clarified. For this we make use of a sub-quantum equipartition theorem. It can further be shown that the couplings of both bouncer and walker to the ZPF are identical. Then we follow this path in accordance with previous work, expanding the view from the particle in its rest frame to a particle in motion. The basic features of ballistic diffusion are derived, especially the diffusion constant D, thus providing a missing link between the different approaches of our previous works.Comment: 14 pages, based on a talk given at "Emergent Quantum Mechanics (Heinz von Foerster Conference 2011)", see http://www.univie.ac.at/hvf11/congress/EmerQuM.htm

Motional effects on the efficiency of excitation transfer

Energy transfer plays a vital role in many natural and technological processes. In this work, we study the effects of mechanical motion on the excitation transfer through a chain of interacting molecules with application to biological scenarios of transfer processes. Our investigation demonstrates that, for various types of mechanical oscillations, the transfer efficiency is significantly enhanced over that of comparable static configurations. This enhancement is a genuine quantum signature, and requires the collaborative interplay between the quantum-coherent evolution of the excitation and the mechanical motion of the molecules; it has no analogue in the classical incoherent energy transfer. This effect may not only occur naturally, but it could be exploited in artificially designed systems to optimize transport processes. As an application, we discuss a simple and hence robust control technique.Comment: 25 pages, 11 figures; completely revised; version accepted for publicatio

UltraSwarm: A Further Step Towards a Flock of Miniature Helicopters

We describe further progress towards the development of a MAV (micro aerial vehicle) designed as an enabling tool to investigate aerial flocking. Our research focuses on the use of low cost off the shelf vehicles and sensors to enable fast prototyping and to reduce development costs. Details on the design of the embedded electronics and the modification of the chosen toy helicopter are presented, and the technique used for state estimation is described. The fusion of inertial data through an unscented Kalman filter is used to estimate the helicopter’s state, and this forms the main input to the control system. Since no detailed dynamic model of the helicopter in use is available, a method is proposed for automated system identification, and for subsequent controller design based on artificial evolution. Preliminary results obtained with a dynamic simulator of a helicopter are reported, along with some encouraging results for tackling the problem of flocking

The AdS/QCD Correspondence: Still Undelivered

We consider the particle spectrum and event shapes in large N gauge theories in different regimes of the short-distance 't Hooft coupling, lambda. The mesons in the small lambda limit should have a Regge spectrum in order to agree with perturbation theory, while generically the large lambda theories with gravity duals produce spectra reminiscent of KK modes. We argue that these KK-like states are qualitatively different from QCD modes: they are deeply bound states which are sensitive to short distance interactions rather than the flux tube-like states expected in asymptotically free, confining gauge theories. In addition, we also find that the characteristic event shapes for the large lambda theories with gravity duals are close to spherical, very different from QCD-like (small lambda, small N) and Nambu-Goto-like (small lambda, large N) theories which have jets. This observation is in agreement with the conjecture of Strassler on event shapes in large 't Hooft coupling theories, which was recently proved by Hofman and Maldacena for the conformal case. This conclusion does not change even when considering soft-wall backgrounds in the gravity dual. The picture that emerges is the following: theories with small and large lambda are qualitatively different, while theories with small and large N are qualitatively similar. Thus it seems that it is the relative smallness of the 't Hooft coupling in QCD that prevents a reliable AdS/QCD correspondence from emerging, and that reproducing characteristic QCD-like behavior will require genuine stringy dynamics to be incorporated into any putative dual theory.Comment: 32 pages, 15 figures; references added, minor changes, history clarifie

Hamiltonian GAN

A growing body of work leverages the Hamiltonian formalism as an inductive bias for physically plausible neural network based video generation. The structure of the Hamiltonian ensures conservation of a learned quantity (e.g., energy) and imposes a phase-space interpretation on the low-dimensional manifold underlying the input video. While this interpretation has the potential to facilitate the integration of learned representations in downstream tasks, existing methods are limited in their applicability as they require a structural prior for the configuration space at design time. In this work, we present a GAN-based video generation pipeline with a learned configuration space map and Hamiltonian neural network motion model, to learn a representation of the configuration space from data. We train our model with a physics-inspired cyclic-coordinate loss function which encourages a minimal representation of the configuration space and improves interpretability. We demonstrate the efficacy and advantages of our approach on the Hamiltonian Dynamics Suite Toy Physics dataset