Learning Temporal Transformations From Time-Lapse Videos

Based on life-long observations of physical, chemical, and biologic phenomena in the natural world, humans can often easily picture in their minds what an object will look like in the future. But, what about computers? In this paper, we learn computational models of object transformations from time-lapse videos. In particular, we explore the use of generative models to create depictions of objects at future times. These models explore several different prediction tasks: generating a future state given a single depiction of an object, generating a future state given two depictions of an object at different times, and generating future states recursively in a recurrent framework. We provide both qualitative and quantitative evaluations of the generated results, and also conduct a human evaluation to compare variations of our models.Comment: ECCV201

Predicting Future Instance Segmentation by Forecasting Convolutional Features

Anticipating future events is an important prerequisite towards intelligent behavior. Video forecasting has been studied as a proxy task towards this goal. Recent work has shown that to predict semantic segmentation of future frames, forecasting at the semantic level is more effective than forecasting RGB frames and then segmenting these. In this paper we consider the more challenging problem of future instance segmentation, which additionally segments out individual objects. To deal with a varying number of output labels per image, we develop a predictive model in the space of fixed-sized convolutional features of the Mask R-CNN instance segmentation model. We apply the "detection head'" of Mask R-CNN on the predicted features to produce the instance segmentation of future frames. Experiments show that this approach significantly improves over strong baselines based on optical flow and repurposed instance segmentation architectures

Survey on Vision-based Path Prediction

Path prediction is a fundamental task for estimating how pedestrians or vehicles are going to move in a scene. Because path prediction as a task of computer vision uses video as input, various information used for prediction, such as the environment surrounding the target and the internal state of the target, need to be estimated from the video in addition to predicting paths. Many prediction approaches that include understanding the environment and the internal state have been proposed. In this survey, we systematically summarize methods of path prediction that take video as input and and extract features from the video. Moreover, we introduce datasets used to evaluate path prediction methods quantitatively.Comment: DAPI 201

CAR-Net: Clairvoyant Attentive Recurrent Network

We present an interpretable framework for path prediction that leverages dependencies between agents' behaviors and their spatial navigation environment. We exploit two sources of information: the past motion trajectory of the agent of interest and a wide top-view image of the navigation scene. We propose a Clairvoyant Attentive Recurrent Network (CAR-Net) that learns where to look in a large image of the scene when solving the path prediction task. Our method can attend to any area, or combination of areas, within the raw image (e.g., road intersections) when predicting the trajectory of the agent. This allows us to visualize fine-grained semantic elements of navigation scenes that influence the prediction of trajectories. To study the impact of space on agents' trajectories, we build a new dataset made of top-view images of hundreds of scenes (Formula One racing tracks) where agents' behaviors are heavily influenced by known areas in the images (e.g., upcoming turns). CAR-Net successfully attends to these salient regions. Additionally, CAR-Net reaches state-of-the-art accuracy on the standard trajectory forecasting benchmark, Stanford Drone Dataset (SDD). Finally, we show CAR-Net's ability to generalize to unseen scenes.Comment: The 2nd and 3rd authors contributed equall

Charge distribution and electroluminescence in cross-linked polyethylene under dc field

The intent of this paper is to cross-correlate the information obtained by space charge distribution analysis and electroluminescence (EL) detection in cross-linked polyethylene samples submitted to dc fields, with the objective to make a link between space charge phenomena and energy release as revealed by the detection of visible photons. Space charge measurements carried out at different field levels by the pulsed electro-acoustic method show the presence of a low-field threshold, close to 15-20 kV mm-1, above which considerable space charge begins to accumulate in the insulation. Charges are seen to cross the insulation thickness through a packet-like behaviour at higher fields, starting at about 60-70 kV mm-1. EL measurements show the existence of two distinct thresholds, one related to the continuous excitation of EL under voltage, the other being transient EL detected upon specimen short circuit. The former occurs at values of field corresponding to charge packet formation and the latter to the onset of space charge accumulation. The correspondence between pertinent values of the electric field obtained through space charge and EL analyses provides support for the existence of degradation thresholds in insulating materials. Special emphasis is given to the relationship between charge packet formation and propagation, and EL. Although the two phenomena are observed in the same field range, it is found that the onset of continuous EL follows the formation at the electrodes of positive and negative space charge regions that extend into the bulk prior to the propagation of charge packets. Charge recombination appears to be the excitation process of EL since oppositely charged domains meet in the material bulk. To gain an insight into specific light-excitation processes associated with charge packet propagation, EL has been recorded for several hours under fields at which charge packet dynamics were evidenced. It is shown that current and luminescence oscillations are detected during charge packet propagation, and that they are in phase. The mechanisms underlying EL and charge packets are further considered on the basis of these results

Nitrogen yield advantage from grass-legume mixtures is robust over a wide range of legume proportions and environmental conditions

Coordination of this project was supported by the EU Commission through COST Action 852 ‘Quality legume-based forage systems for contrasting environments‘. A636 contribution to the research leading to these results has been conducted as part of the Animal Change project which received funding from the European Union’s Seventh Framework Programme (FP7/2007-20 13) under the grant agreement no. 266018.peer-reviewedCurrent challenges to global food security require sustainable intensification of agriculture through initiatives that include more efficient use of nitrogen (N), increased protein self-sufficiency through home-grown crops, and reduced N losses to the environment. Such challenges were addressed in a continental-scale field experiment conducted over three years, in which the amount of total nitrogen yield (Ntot) and the gain of N yield in mixtures as compared to grass monocultures (Ngainmix) was quantified from four-species grass-legume stands with greatly varying legume proportions. Stands consisted of monocultures and mixtures of two N2 fixing legumes and two non-fixing grasses.The amount of Ntot of mixtures was significantly greater (P ≤ 0.05) than that of grass monocultures at the majority of evaluated sites in all three years. Ntot and thus Ngainmix increased with increasing legume proportion up to one third of legumes. With higher legume percentages, Ntot and Ngainmix did not continue to increase. Thus, across sites and years, mixtures with one third proportion of legumes attained ~95% of the maximum Ntot acquired by any stand and had 57% higher Ntot than grass monocultures.Realized legume proportion in stands and the relative N gain in mixture (Ngainmix/Ntot in mixture) were most severely impaired by minimum site temperature (R = 0.70, P = 0.003 for legume proportion; R = 0.64, P = 0.010 for Ngainmix/Ntot in mixture). Nevertheless, the relative N gain in mixture was not correlated to site productivity (P = 0.500), suggesting that, within climatic restrictions, balanced grass-legume mixtures can benefit from comparable relative gains in N yield across largely differing productivity levels.We conclude that the use of grass-legume mixtures can substantially contribute to resource-efficient agricultural grassland systems over a wide range of productivity levels, implying important savings in N fertilizers and thus greenhouse gas emissions and a considerable potential for climate change mitigation.European Unio

Globally Continuous and Non-Markovian Crowd Activity Analysis from Videos

Automatically recognizing activities in video is a classic problem in vision and helps to understand behaviors, describe scenes and detect anomalies. We propose an unsupervised method for such purposes. Given video data, we discover recurring activity patterns that appear, peak, wane and disappear over time. By using non-parametric Bayesian methods, we learn coupled spatial and temporal patterns with minimum prior knowledge. To model the temporal changes of patterns, previous works compute Markovian progressions or locally continuous motifs whereas we model time in a globally continuous and non-Markovian way. Visually, the patterns depict flows of major activities. Temporally, each pattern has its own unique appearance-disappearance cycles. To compute compact pattern representations, we also propose a hybrid sampling method. By combining these patterns with detailed environment information, we interpret the semantics of activities and report anomalies. Also, our method fits data better and detects anomalies that were difficult to detect previously

Forecasting Human-Object Interaction: Joint Prediction of Motor Attention and Actions in First Person Video

We address the challenging task of anticipating human-object interaction in first person videos. Most existing methods ignore how the camera wearer interacts with the objects, or simply consider body motion as a separate modality. In contrast, we observe that the international hand movement reveals critical information about the future activity. Motivated by this, we adopt intentional hand movement as a future representation and propose a novel deep network that jointly models and predicts the egocentric hand motion, interaction hotspots and future action. Specifically, we consider the future hand motion as the motor attention, and model this attention using latent variables in our deep model. The predicted motor attention is further used to characterise the discriminative spatial-temporal visual features for predicting actions and interaction hotspots. We present extensive experiments demonstrating the benefit of the proposed joint model. Importantly, our model produces new state-of-the-art results for action anticipation on both EGTEA Gaze+ and the EPIC-Kitchens datasets. Our project page is available at https://aptx4869lm.github.io/ForecastingHOI

Hyperthermophilic Aquifex aeolicus initiates primer synthesis on a limited set of trinucleotides comprised of cytosines and guanines

The placement of the extreme thermophile Aquifex aeolicus in the bacterial phylogenetic tree has evoked much controversy. We investigated whether adaptations for growth at high temperatures would alter a key functional component of the replication machinery, specifically DnaG primase. Although the structure of bacterial primases is conserved, the trinucleotide initiation specificity for A. aeolicus was hypothesized to differ from other microbes as an adaptation to a geothermal milieu. To determine the full range of A. aeolicus primase activity, two oligonucleotides were designed that comprised all potential trinucleotide initiation sequences. One of the screening templates supported primer synthesis and the lengths of the resulting primers were used to predict possible initiation trinucleotides. Use of trinucleotide-specific templates demonstrated that the preferred initiation trinucleotide sequence for A. aeolicus primase was 5′-d(CCC)-3′. Two other sequences, 5′-d(GCC)-3′ and d(CGC)-3′, were also capable of supporting initiation, but to a much lesser degree. None of these trinucleotides were known to be recognition sequences used by other microbial primases. These results suggest that the initiation specificity of A. aeolicus primase may represent an adaptation to a thermophilic environment

Production of scFv-Conjugated Affinity Silk Powder by Transgenic Silkworm Technology

Bombyx mori (silkworm) silk proteins are being utilized as unique biomaterials for medical applications. Chemical modification or post-conjugation of bioactive ligands expand the applicability of silk proteins; however, the processes are elaborate and costly. In this study, we used transgenic silkworm technology to develop single-chain variable fragment (scFv)-conjugated silk fibroin. The cocoons of the transgenic silkworm contain fibroin L-chain linked with scFv as a fusion protein. After dissolving the cocoons in lithium bromide, the silk solution was dialyzed, concentrated, freeze-dried, and crushed into powder. Immunoprecipitation analyses demonstrate that the scFv domain retains its specific binding activity to the target molecule after multiple processing steps. These results strongly suggest the promise of scFv-conjugated silk fibroin as an alternative affinity reagent, which can be manufactured using transgenic silkworm technology at lower cost than traditional affinity carriers