Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods, which have tackled this problem in a deterministic or non-parametric way, we propose a novel approach that models future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. Future frame synthesis is challenging, as it involves low- and high-level image and motion understanding. We propose a novel network structure, namely a Cross Convolutional Network to aid in synthesizing future frames; this network structure encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, as well as on real-wold videos. We also show that our model can be applied to tasks such as visual analogy-making, and present an analysis of the learned network representations.Comment: The first two authors contributed equally to this wor

Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks

We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods that have tackled this problem in a deterministic or non-parametric way, we propose to model future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. To synthesize realistic movement of objects, we propose a novel network structure, namely a Cross Convolutional Network; this network encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, and on real-world video frames. We present analyses of the learned network representations, showing it is implicitly learning a compact encoding of object appearance and motion. We also demonstrate a few of its applications, including visual analogy-making and video extrapolation.Comment: Journal preprint of arXiv:1607.02586 (IEEE TPAMI, 2019). The first two authors contributed equally to this work. Project page: http://visualdynamics.csail.mit.ed

Provable Probabilistic Imaging using Score-Based Generative Priors

Estimating high-quality images while also quantifying their uncertainty are two desired features in an image reconstruction algorithm for solving ill-posed inverse problems. In this paper, we propose plug-and-play Monte Carlo (PMC) as a principled framework for characterizing the space of possible solutions to a general inverse problem. PMC is able to incorporate expressive score-based generative priors for high-quality image reconstruction while also performing uncertainty quantification via posterior sampling. In particular, we introduce two PMC algorithms which can be viewed as the sampling analogues of the traditional plug-and-play priors (PnP) and regularization by denoising (RED) algorithms. We also establish a theoretical analysis for characterizing the convergence of the PMC algorithms. Our analysis provides non-asymptotic stationarity guarantees for both algorithms, even in the presence of non-log-concave likelihoods and imperfect score networks. We demonstrate the performance of the PMC algorithms on multiple representative inverse problems with both linear and nonlinear forward models. Experimental results show that PMC significantly improves reconstruction quality and enables high-fidelity uncertainty quantification

Prenatal and Early Postnatal Exposure to Cigarette Smoke Decreases BDNF/TrkB Signaling and Increases Abnormal Behaviors Later in Life

Background: Cigarette smoke exposure during prenatal and early postnatal periods increases the incidence of a variety of abnormal behaviors later in life. The purpose of this study was to identify the possible critical period of susceptibility to cigarette smoke exposure and evaluate the possibe effects of cigarette smoke during early life on brain-derived neurotrophic factor/neurotrophic tyrosine kinase receptor B signaling in the brain. Methods: Three different age of imprinting control region mice were exposed to cigarette smoke or filtered air for 10 consecutive days beginning on either gestational day 7 by maternal exposure, or postnatal days 2 or 21 by direct inhalation. A series of behavioral profiles and neurotrophins in brain were measured 24 hours after mice received acute restraint stress for 1 hour on postnatal day 59. Results: Cigarette smoke exposure in gestational day 7 and postnatal day 2 produced depression-like behaviors as evidenced by significantly increased immobility in both tail suspension and forced-swim test. Increased entry latencies, but not ambulation in the open field test, were also observed in the gestational day 7 and postnatal day 2 cigarette smoke exposure groups. Genetic analysis showed that gestational day 7 cigarette smoke exposure significantly altered mRNA level of brain derived neurotrophic factor/tyrosine kinase receptor B in the hippocampus. However, behavioral profiles and brain-derived neurotrophic factor/tyrosine kinase receptor B signaling were not significantly changed in PND21 cigarette smoke exposure group compared with FA group. Conclusions: These results suggest that a critical period of susceptibility to cigarette smoke exposure exists in the prenatal and early postnatal period, which results a downregulation in brain-derived neurotrophic factor/tyrosine kinase receptor B signaling in the hippocampus and enhances depression-like behaviors later in life

Cancer-associated fibroblast exosomes regulate survival and proliferation of pancreatic cancer cells

Cancer associated fibroblasts (CAFs) comprise the majority of the tumor bulk of pancreatic adenocarcinomas (PDACs). Current efforts to eradicate these tumors focus predominantly on targeting the proliferation of rapidly growing cancer epithelial cells. We know that this is largely ineffective with resistance arising in most tumors following exposure to chemotherapy. Despite the long-standing recognition of the prominence of CAFs in PDAC, the effect of chemotherapy on CAFs and how they may contribute to drug resistance in neighboring cancer cells is not well characterized. Here we show that CAFs exposed to chemotherapy play an active role in regulating the survival and proliferation of cancer cells. We found that CAFs are intrinsically resistant to gemcitabine, the chemotherapeutic standard of care for PDAC. Further, CAFs exposed to gemcitabine significantly increase the release of extracellular vesicles called exosomes. These exosomes increased chemoresistance-inducing factor, Snail, in recipient epithelial cells and promote proliferation and drug resistance. Finally, treatment of gemcitabine-exposed CAFs with an inhibitor of exosome release, GW4869, significantly reduces survival in co-cultured epithelial cells, signifying an important role of CAF exosomes in chemotherapeutic drug resistance. Collectively, these findings show the potential for exosome inhibitors as treatment options alongside chemotherapy for overcoming PDAC chemoresistance

Prenatal and Early, but Not Late, Postnatal Exposure of Mice to Sidestream Tobacco Smoke Increases Airway Hyperresponsiveness Later in Life

Background Cigarette smoke exposure in utero and during early postnatal development increases the incidence of asthma and airway hyperresponsiveness (AHR) later in life, suggesting that a possible critical period of developmental sensitivity exists in the prenatal and early postnatal periods. Objective We investigated mechanisms of susceptibility during critical developmental periods to sidestream smoke (SS) exposure and evaluated the possible effects of SS on neural responses. Methods We exposed three different age groups of mice to either SS or filtered air (FA) for 10 consecutive days beginning on gestation day (GD) 7 by maternal exposure or beginning on postnatal day (PND) 2 or PND21 by direct inhalation. Lung function, airway substance P (SP) innervation, and nerve growth factor (NGF) levels in broncho alveolar lavage fluid were measured after a single SS exposure on PND59. Results Methacholine (MCh) dose response for lung resistance (RL) was significantly elevated, and dynamic pulmonary compliance (Cdyn) was significantly decreased, in the GD7 and PND2 SS exposure groups compared with the FA groups after SS exposure on PND59. At the same time points, the percent area of SP nerve fibers in tracheal smooth muscle and the levels of NGF were significantly elevated. MCh dose–response curves for RL and Cdyn, SP nerve fiber density, and the level of NGF were not significantly changed in the PND21 exposure group after SS exposure on PND59. Conclusions These results suggest that a critical period of susceptibility to SS exposure exists in the prenatal and early postnatal period of development in mice that results in increased SP innervation, increased NGF levels in the airway, and enhanced MCh AHR later in life

Prenatal and Early, but Not Late, Postnatal Exposure of Mice to Sidestream Tobacco Smoke Increases Airway Hyperresponsiveness Later in Life

Background Cigarette smoke exposure in utero and during early postnatal development increases the incidence of asthma and airway hyperresponsiveness (AHR) later in life, suggesting that a possible critical period of developmental sensitivity exists in the prenatal and early postnatal periods. Objective We investigated mechanisms of susceptibility during critical developmental periods to sidestream smoke (SS) exposure and evaluated the possible effects of SS on neural responses. Methods We exposed three different age groups of mice to either SS or filtered air (FA) for 10 consecutive days beginning on gestation day (GD) 7 by maternal exposure or beginning on postnatal day (PND) 2 or PND21 by direct inhalation. Lung function, airway substance P (SP) innervation, and nerve growth factor (NGF) levels in broncho alveolar lavage fluid were measured after a single SS exposure on PND59. Results Methacholine (MCh) dose response for lung resistance (RL) was significantly elevated, and dynamic pulmonary compliance (Cdyn) was significantly decreased, in the GD7 and PND2 SS exposure groups compared with the FA groups after SS exposure on PND59. At the same time points, the percent area of SP nerve fibers in tracheal smooth muscle and the levels of NGF were significantly elevated. MCh dose–response curves for RL and Cdyn, SP nerve fiber density, and the level of NGF were not significantly changed in the PND21 exposure group after SS exposure on PND59. Conclusions These results suggest that a critical period of susceptibility to SS exposure exists in the prenatal and early postnatal period of development in mice that results in increased SP innervation, increased NGF levels in the airway, and enhanced MCh AHR later in life

The DEEP Groth Strip Galaxy Redshift Survey. III. Redshift Catalog and Properties of Galaxies

The Deep Extragalactic Evolutionary Probe (DEEP) is a series of spectroscopic surveys of faint galaxies, targeted at the properties and clustering of galaxies at redshifts z ~ 1. We present the redshift catalog of the DEEP 1 GSS pilot phase of this project, a Keck/LRIS survey in the HST/WFPC2 Groth Survey Strip. The redshift catalog and data, including reduced spectra, are publicly available through a Web-accessible database. The catalog contains 658 secure galaxy redshifts with a median z=0.65, and shows large-scale structure walls to z = 1. We find a bimodal distribution in the galaxy color-magnitude diagram which persists to z = 1. A similar color division has been seen locally by the SDSS and to z ~ 1 by COMBO-17. For red galaxies, we find a reddening of only 0.11 mag from z ~ 0.8 to now, about half the color evolution measured by COMBO-17. We measure structural properties of the galaxies from the HST imaging, and find that the color division corresponds generally to a structural division. Most red galaxies, ~ 75%, are centrally concentrated, with a red bulge or spheroid, while blue galaxies usually have exponential profiles. However, there are two subclasses of red galaxies that are not bulge-dominated: edge-on disks and a second category which we term diffuse red galaxies (DIFRGs). The distant edge-on disks are similar in appearance and frequency to those at low redshift, but analogs of DIFRGs are rare among local red galaxies. DIFRGs have significant emission lines, indicating that they are reddened mainly by dust rather than age. The DIFRGs in our sample are all at z>0.64, suggesting that DIFRGs are more prevalent at high redshifts; they may be related to the dusty or irregular extremely red objects (EROs) beyond z>1.2 that have been found in deep K-selected surveys. (abridged)Comment: ApJ in press. 24 pages, 17 figures (12 color). The DEEP public database is available at http://saci.ucolick.org

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods, which have tackled this problem in a deterministic or non-parametric way, we propose a novel approach which models future frames in a probabilistic manner. Our proposed method is therefore able to synthesize multiple possible next frames using the same model. Solving this challenging problem involves low- and high-level image and motion understanding for successful image synthesis. Here, we propose a novel network structure, namely a Cross Convolutional Network, that encodes images as feature maps and motion information as convolutional kernels to aid in synthesizing future frames. In experiments, our model performs well on both synthetic data, such as 2D shapes and animated game sprites, as well as on real-wold video data. We show that our model can also be applied to tasks such as visual analogy-making, and present analysis of the learned network representations