Multilevel Language and Vision Integration for Text-to-Clip Retrieval

We address the problem of text-based activity retrieval in video. Given a sentence describing an activity, our task is to retrieve matching clips from an untrimmed video. To capture the inherent structures present in both text and video, we introduce a multilevel model that integrates vision and language features earlier and more tightly than prior work. First, we inject text features early on when generating clip proposals, to help eliminate unlikely clips and thus speed up processing and boost performance. Second, to learn a fine-grained similarity metric for retrieval, we use visual features to modulate the processing of query sentences at the word level in a recurrent neural network. A multi-task loss is also employed by adding query re-generation as an auxiliary task. Our approach significantly outperforms prior work on two challenging benchmarks: Charades-STA and ActivityNet Captions.Comment: AAAI 201

HOW PARTS CONNECT TO WHOLE IN BUILDING DIGITAL GENERATIVITY IN DIGITAL PLATFORM ECOSYSTEMS

Generativity drives digital innovation and platform growth by engaging many other businesses with diverse digital skills and resources in a digital platform. As the proliferation of generativity research grows, the Information Systems (IS) literature demonstrates the basic understanding of this notion in the areas of properties of digital technologies, social events, and/or the interaction between these two without an integrated view of how generativity is raised to enable the digital innovation. Therefore, considering that digital platforms are a kind of ecosystem, we aim to develop a new understanding of this emerging phenomenon by employing a holistic perspective. Through the information ecology theoretical lens, we develop a digital generativity process model that explains how the technological and social resources interact to generate perpetual digital innovation in digital platform ecosystems (DPE). This study contributes to generativity research by providing a dynamic and holistic view of generativity formalization in DPEs

Dynamics of allosteric transitions in GroEL

The chaperonin GroEL-GroES, a machine which helps some proteins to fold, cycles through a number of allosteric states, the $T$ state, with high affinity for substrate proteins (SPs), the ATP-bound $R$ state, and the $R^{\prime\prime}$ ($GroEL-ADP-GroES$) complex. Structures are known for each of these states. Here, we use a self-organized polymer (SOP) model for the GroEL allosteric states and a general structure-based technique to simulate the dynamics of allosteric transitions in two subunits of GroEL and the heptamer. The $T \to R$ transition, in which the apical domains undergo counter-clockwise motion, is mediated by a multiple salt-bridge switch mechanism, in which a series of salt-bridges break and form. The initial event in the $R \to R^{\prime\prime}$ transition, during which GroEL rotates clockwise, involves a spectacular outside-in movement of helices K and L that results in K80-D359 salt-bridge formation. In both the transitions there is considerable heterogeneity in the transition pathways. The transition state ensembles (TSEs) connecting the $T$, $R$, and $R^{\prime\prime}$ states are broad with the the TSE for the $T \to R$ transition being more plastic than the $R\to R^{\prime\prime}$ TSE. The results suggest that GroEL functions as a force-transmitting device in which forces of about (5-30) pN may act on the SP during the reaction cycle.Comment: 32 pages, 10 figures (Longer version than the one published

Integration of Pre-trained Protein Language Models into Geometric Deep Learning Networks

Geometric deep learning has recently achieved great success in non-Euclidean domains, and learning on 3D structures of large biomolecules is emerging as a distinct research area. However, its efficacy is largely constrained due to the limited quantity of structural data. Meanwhile, protein language models trained on substantial 1D sequences have shown burgeoning capabilities with scale in a broad range of applications. Several previous studies consider combining these different protein modalities to promote the representation power of geometric neural networks, but fail to present a comprehensive understanding of their benefits. In this work, we integrate the knowledge learned by well-trained protein language models into several state-of-the-art geometric networks and evaluate a variety of protein representation learning benchmarks, including protein-protein interface prediction, model quality assessment, protein-protein rigid-body docking, and binding affinity prediction. Our findings show an overall improvement of 20% over baselines. Strong evidence indicates that the incorporation of protein language models' knowledge enhances geometric networks' capacity by a significant margin and can be generalized to complex tasks

State control can result in good performance for firms

State firms are now hybrid organisations, say Ciprian Stan, David Ahlstrom, Mike W. Peng, Kehan Xu and Garry D. Bruto

Conditional Local Convolution for Spatio-temporal Meteorological Forecasting

Spatio-temporal forecasting is challenging attributing to the high nonlinearity in temporal dynamics as well as complex location-characterized patterns in spatial domains, especially in fields like weather forecasting. Graph convolutions are usually used for modeling the spatial dependency in meteorology to handle the irregular distribution of sensors' spatial location. In this work, a novel graph-based convolution for imitating the meteorological flows is proposed to capture the local spatial patterns. Based on the assumption of smoothness of location-characterized patterns, we propose conditional local convolution whose shared kernel on nodes' local space is approximated by feedforward networks, with local representations of coordinate obtained by horizon maps into cylindrical-tangent space as its input. The established united standard of local coordinate system preserves the orientation on geography. We further propose the distance and orientation scaling terms to reduce the impacts of irregular spatial distribution. The convolution is embedded in a Recurrent Neural Network architecture to model the temporal dynamics, leading to the Conditional Local Convolution Recurrent Network (CLCRN). Our model is evaluated on real-world weather benchmark datasets, achieving state-of-the-art performance with obvious improvements. We conduct further analysis on local pattern visualization, model's framework choice, advantages of horizon maps and etc.Comment: 14 page

Differences in the Hydrological Cycle and Sensitivity Between Multiscale Modeling Frameworks with and Without a HigherOrder Turbulence Closure

Current conventional global climate models (GCMs) produce a weak increase in globalmean precipitation with anthropogenic warming in comparison with the lower tropospheric moisture increases. The motive of this study is to understand the differences in the hydrological sensitivity between two multiscale modeling frameworks (MMFs) that arise from the different treatments of turbulence and low clouds in order to aid to the understanding of the model spread among conventional GCMs. We compare the hydrological sensitivity and its energetic constraint from MMFs with (SPCAMIPHOC) or without (SPCAM) an advanced higherorder turbulence closure. SPCAMIPHOC simulates higher global hydrological sensitivity for the slow response but lower sensitivity for the fast response than SPCAM. Their differences are comparable to the spreads of conventional GCMs. The higher sensitivity in SPCAMIPHOC is associated with the higher ratio of the changes in latent heating to those in net atmospheric radiative cooling, which is further related to a stronger decrease in the Bowen ratio with warming than in SPCAM. The higher sensitivity of cloud radiative cooling resulting from the lack of low clouds in SPCAM is another major factor in contributing to the lower precipitation sensitivity. The two MMFs differ greatly in the hydrological sensitivity over the tropical lands, where the simulated sensitivity of surface sensible heat fluxes to surface warming and CO2 increase in SPCAMIPHOC is weaker than in SPCAM. The difference in divergences of dry static energy flux simulated by the two MMFs also contributes to the difference in land precipitation sensitivity between the two models

Toward a Mechanistic Modeling of Nitrogen Limitation on Vegetation Dynamics

Nitrogen is a dominant regulator of vegetation dynamics, net primary production, and terrestrial carbon cycles; however, most ecosystem models use a rather simplistic relationship between leaf nitrogen content and photosynthetic capacity. Such an approach does not consider how patterns of nitrogen allocation may change with differences in light intensity, growing-season temperature and CO2 concentration. To account for this known variability in nitrogen-photosynthesis relationships, we develop a mechanistic nitrogen allocation model based on a trade-off of nitrogen allocated between growth and storage, and an optimization of nitrogen allocated among light capture, electron transport, carboxylation, and respiration. The developed model is able to predict the acclimation of photosynthetic capacity to changes in CO2 concentration, temperature, and radiation when evaluated against published data of Vc,max (maximum carboxylation rate) and Jmax (maximum electron transport rate). A sensitivity analysis of the model for herbaceous plants, deciduous and evergreen trees implies that elevated CO2 concentrations lead to lower allocation of nitrogen to carboxylation but higher allocation to storage. Higher growing-season temperatures cause lower allocation of nitrogen to carboxylation, due to higher nitrogen requirements for light capture pigments and for storage. Lower levels of radiation have a much stronger effect on allocation of nitrogen to carboxylation for herbaceous plants than for trees, resulting from higher nitrogen requirements for light capture for herbaceous plants. As far as we know, this is the first model of complete nitrogen allocation that simultaneously considers nitrogen allocation to light capture, electron transport, carboxylation, respiration and storage, and the responses of each to altered environmental conditions. We expect this model could potentially improve our confidence in simulations of carbon-nitrogen interactions and the vegetation feedbacks to climate in Earth system models

Evidence for Superfluidity of Ultracold Fermions in an Optical Lattice

The study of superfluid fermion pairs in a periodic potential has important ramifications for understanding superconductivity in crystalline materials. Using cold atomic gases, various condensed matter models can be studied in a highly controllable environment. Weakly repulsive fermions in an optical lattice could undergo d-wave pairing at low temperatures, a possible mechanism for high temperature superconductivity in the cuprates. The lattice potential could also strongly increase the critical temperature for s-wave superfluidity. Recent experimental advances in the bulk include the observation of fermion pair condensates and high-temperature superfluidity. Experiments with fermions and bosonic bound pairs in optical lattices have been reported, but have not yet addressed superfluid behavior. Here we show that when a condensate of fermionic atom pairs was released from an optical lattice, distinct interference peaks appear, implying long range order, a property of a superfluid. Conceptually, this implies that strong s-wave pairing and superfluidity have now been established in a lattice potential, where the transport of atoms occurs by quantum mechanical tunneling and not by simple propagation. These observations were made for unitarity limited interactions on both sides of a Feshbach resonance. For larger lattice depths, the coherence was lost in a reversible manner, possibly due to a superfluid to insulator transition. Such strongly interacting fermions in an optical lattice can be used to study a new class of Hamiltonians with interband and atom-molecule couplings.Comment: accepted for publication in Natur