Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods that have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, and on real-world video frames. We present analyses of the learned network representations, showing it is implicitly learning a compact encoding of object appearance and motion. We also demonstrate a few of its applications, including visual analogy-making and video extrapolation.Comment: Journal preprint of arXiv:1607.02586 (IEEE TPAMI, 2019). The first two authors contributed equally to this work. Project page: http://visualdynamics.csail.mit.ed

Wavelet-based response spectrum compatible synthesis of accelerograms-Eurocode application (EC8)

An integrated approach for addressing the problem of synthesizing artificial seismic accelerograms compatible with a given displacement design/target spectrum is presented in conjunction with aseismic design applications. Initially, a stochastic dynamics solution is used to obtain a family of simulated non-stationary earthquake records whose response spectrum is on the average in good agreement with the target spectrum. The degree of the agreement depends significantly on the adoption of an appropriate parametric evolutionary power spectral form, which is related to the target spectrum in an approximate manner. The performance of two commonly used spectral forms along with a newly proposed one is assessed with respect to the elastic displacement design spectrum defined by the European code regulations (EC8). Subsequently, the computational versatility of the family of harmonic wavelets is employed to modify iteratively the simulated records to satisfy the compatibility criteria for artificial accelerograms prescribed by EC8. In the process, baseline correction steps, ordinarily taken to ensure that the obtained accelerograms are characterized by physically meaningful velocity and displacement traces, are elucidated. Obviously, the presented approach can be used not only in the case of the EC8, for which extensive numerical results/examples are included, but also for any code provisions mandated by regulatory agencies. In any case, the presented numerical results can be quite useful in any aseismic design process dominated by the EC8 specifications

Thermochemistry of Alane Complexes for Hydrogen Storage: A Theoretical and Experimental Comparison

Knowledge of the relative stabilities of alane (AlH3) complexes with electron donors is essential for identifying hydrogen storage materials for vehicular applications that can be regenerated by off-board methods; however, almost no thermodynamic data are available to make this assessment. To fill this gap, we employed the G4(MP2) method to determine heats of formation, entropies, and Gibbs free energies of formation for thirty-eight alane complexes with NH3-nRn (R = Me, Et; n = 0-3), pyridine, pyrazine, triethylenediamine (TEDA), quinuclidine, OH2-nRn (R = Me, Et; n = 0-2), dioxane, and tetrahydrofuran (THF). Monomer, bis, and selected dimer complex geometries were considered. Using these data, we computed the thermodynamics of the key formation and dehydrogenation reactions that would occur during hydrogen delivery and alane regeneration, from which trends in complex stability were identified. These predictions were tested by synthesizing six amine-alane complexes involving trimethylamine, triethylamine, dimethylethylamine, TEDA, quinuclidine, and hexamine, and obtaining upper limits of delta G for their formation from metallic aluminum. Combining these computational and experimental results, we establish a criterion for complex stability relevant to hydrogen storage that can be used to assess potential ligands prior to attempting synthesis of the alane complex. Based on this, we conclude that only a subset of the tertiary amine complexes considered and none of the ether complexes can be successfully formed by direct reaction with aluminum and regenerated in an alane-based hydrogen storage system.Comment: Accepted by the Journal of Physical Chemistry

Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging

A variety of techniques such as light field, structured illumination, and time-of-flight (TOF) are commonly used for depth acquisition in consumer imaging, robotics and many other applications. Unfortunately, each technique suffers from its individual limitations preventing robust depth sensing. In this paper, we explore the strengths and weaknesses of combining light field and time-of-flight imaging, particularly the feasibility of an on-chip implementation as a single hybrid depth sensor. We refer to this combination as depth field imaging. Depth fields combine light field advantages such as synthetic aperture refocusing with TOF imaging advantages such as high depth resolution and coded signal processing to resolve multipath interference. We show applications including synthesizing virtual apertures for TOF imaging, improved depth mapping through partial and scattering occluders, and single frequency TOF phase unwrapping. Utilizing space, angle, and temporal coding, depth fields can improve depth sensing in the wild and generate new insights into the dimensions of light's plenoptic function.Comment: 9 pages, 8 figures, Accepted to 3DV 201

Machine Learning Models that Remember Too Much

Machine learning (ML) is becoming a commodity. Numerous ML frameworks and services are available to data holders who are not ML experts but want to train predictive models on their data. It is important that ML models trained on sensitive inputs (e.g., personal images or documents) not leak too much information about the training data. We consider a malicious ML provider who supplies model-training code to the data holder, does not observe the training, but then obtains white- or black-box access to the resulting model. In this setting, we design and implement practical algorithms, some of them very similar to standard ML techniques such as regularization and data augmentation, that "memorize" information about the training dataset in the model yet the model is as accurate and predictive as a conventionally trained model. We then explain how the adversary can extract memorized information from the model. We evaluate our techniques on standard ML tasks for image classification (CIFAR10), face recognition (LFW and FaceScrub), and text analysis (20 Newsgroups and IMDB). In all cases, we show how our algorithms create models that have high predictive power yet allow accurate extraction of subsets of their training data

Effect of Polydispersity and Anisotropy in Colloidal and Protein Solutions: an Integral Equation Approach

Application of integral equation theory to complex fluids is reviewed, with particular emphasis to the effects of polydispersity and anisotropy on their structural and thermodynamic properties. Both analytical and numerical solutions of integral equations are discussed within the context of a set of minimal potential models that have been widely used in the literature. While other popular theoretical tools, such as numerical simulations and density functional theory, are superior for quantitative and accurate predictions, we argue that integral equation theory still provides, as in simple fluids, an invaluable technique that is able to capture the main essential features of a complex system, at a much lower computational cost. In addition, it can provide a detailed description of the angular dependence in arbitrary frame, unlike numerical simulations where this information is frequently hampered by insufficient statistics. Applications to colloidal mixtures, globular proteins and patchy colloids are discussed, within a unified framework.Comment: 17 pages, 7 figures, to appear in Interdiscip. Sci. Comput. Life Sci. (2011), special issue dedicated to Prof. Lesser Blu