Single Shot Temporal Action Detection

Temporal action detection is a very important yet challenging problem, since videos in real applications are usually long, untrimmed and contain multiple action instances. This problem requires not only recognizing action categories but also detecting start time and end time of each action instance. Many state-of-the-art methods adopt the "detection by classification" framework: first do proposal, and then classify proposals. The main drawback of this framework is that the boundaries of action instance proposals have been fixed during the classification step. To address this issue, we propose a novel Single Shot Action Detector (SSAD) network based on 1D temporal convolutional layers to skip the proposal generation step via directly detecting action instances in untrimmed video. On pursuit of designing a particular SSAD network that can work effectively for temporal action detection, we empirically search for the best network architecture of SSAD due to lacking existing models that can be directly adopted. Moreover, we investigate into input feature types and fusion strategies to further improve detection accuracy. We conduct extensive experiments on two challenging datasets: THUMOS 2014 and MEXaction2. When setting Intersection-over-Union threshold to 0.5 during evaluation, SSAD significantly outperforms other state-of-the-art systems by increasing mAP from 19.0% to 24.6% on THUMOS 2014 and from 7.4% to 11.0% on MEXaction2.Comment: ACM Multimedia 201

First principles calculation of vibrational Raman spectra in large systems: signature of small rings in crystalline SiO2

We present an approach for the efficient calculation of vibrational Raman intensities in periodic systems within density functional theory. The Raman intensities are computed from the second order derivative of the electronic density matrix with respect to a uniform electric field. In contrast to previous approaches, the computational effort required by our method for the evaluation of the intensities is negligible compared to that required for the calculation of vibrational frequencies. As a first application, we study the signature of 3- and 4-membered rings in the the Raman spectra of several polymorphs of SiO2, including a zeolite having 102 atoms per unit cell.Comment: 4 pages, 2 figures, revtex4 Minor corrections; accepted in Phys. Rev. Let

SchNet - a deep learning architecture for molecules and materials

Deep learning has led to a paradigm shift in artificial intelligence, including web, text and image search, speech recognition, as well as bioinformatics, with growing impact in chemical physics. Machine learning in general and deep learning in particular is ideally suited for representing quantum-mechanical interactions, enabling to model nonlinear potential-energy surfaces or enhancing the exploration of chemical compound space. Here we present the deep learning architecture SchNet that is specifically designed to model atomistic systems by making use of continuous-filter convolutional layers. We demonstrate the capabilities of SchNet by accurately predicting a range of properties across chemical space for \emph{molecules and materials} where our model learns chemically plausible embeddings of atom types across the periodic table. Finally, we employ SchNet to predict potential-energy surfaces and energy-conserving force fields for molecular dynamics simulations of small molecules and perform an exemplary study of the quantum-mechanical properties of C$_{20}$-fullerene that would have been infeasible with regular ab initio molecular dynamics

Ab initio studies of phonon softening and high pressure phase transitions of alpha-quartz SiO2

Density functional perturbation theory calculations of alpha-quartz using extended norm conserving pseudopotentials have been used to study the elastic properties and phonon dispersion relations along various high symmetry directions as a function of bulk, uniaxial and non-hydrostatic pressure. The computed equation of state, elastic constants and phonon frequencies are found to be in good agreement with available experimental data. A zone boundary (1/3, 1/3, 0) K-point phonon mode becomes soft for pressures above P=32 GPa. Around the same pressure, studies of the Born stability criteria reveal that the structure is mechanically unstable. The phonon and elastic softening are related to the high pressure phase transitions and amorphization of quartz and these studies suggest that the mean transition pressure is lowered under non-hydrostatic conditions. Application of uniaxial pressure, results in a post-quartz crystalline monoclinic C2 structural transition in the vicinity of the K-point instability. This structure, intermediate between quartz and stishovite has two-thirds of the silicon atoms in octahedral coordination while the remaining silicon atoms remain tetrahedrally coordinated. This novel monoclinic C2 polymorph of silica, which is found to be metastable under ambient conditions, is possibly one of the several competing dense forms of silica containing octahedrally coordinated silicon. The possible role of high pressure ferroelastic phases in causing pressure induced amorphization in silica are discussed.Comment: 17 pages, 8 figs., 8 Table

Neural Networks for Information Retrieval

Machine learning plays a role in many aspects of modern IR systems, and deep learning is applied in all of them. The fast pace of modern-day research has given rise to many different approaches for many different IR problems. The amount of information available can be overwhelming both for junior students and for experienced researchers looking for new research topics and directions. Additionally, it is interesting to see what key insights into IR problems the new technologies are able to give us. The aim of this full-day tutorial is to give a clear overview of current tried-and-trusted neural methods in IR and how they benefit IR research. It covers key architectures, as well as the most promising future directions.Comment: Overview of full-day tutorial at SIGIR 201

Anharmonic effects in the A15 compounds induced by sublattice distortions

We demonstrate that elastic anomalies and lattice instabilities in the the A15 compounds are describable in terms of first-principles LDA electronic structure calculations. We show that at T=0 V_3Si, V_3Ge, and Nb_3Sn are intrinsically unstable against shears with elastic moduli C_11-C_12 and C_44, and that the zone center phonons, Gamma_2 and Gamma_12, are either unstable or extremely soft. We demonstrate that sublattice relaxation (internal strain) effects are key to understanding the behavior of the A15 materials.Comment: 5 pages, RevTex, 3 postscript figures, Submitted to Phys. Rev. Lett. Apr. 23, 1997 July 7, 1997: minor corrections, final accepted versio

Geometric deep learning

The goal of these course notes is to describe the main mathematical ideas behind geometric deep learning and to provide implementation details for several applications in shape analysis and synthesis, computer vision and computer graphics. The text in the course materials is primarily based on previously published work. With these notes we gather and provide a clear picture of the key concepts and techniques that fall under the umbrella of geometric deep learning, and illustrate the applications they enable. We also aim to provide practical implementation details for the methods presented in these works, as well as suggest further readings and extensions of these ideas

On Sampling Strategies for Neural Network-based Collaborative Filtering

Recent advances in neural networks have inspired people to design hybrid recommendation algorithms that can incorporate both (1) user-item interaction information and (2) content information including image, audio, and text. Despite their promising results, neural network-based recommendation algorithms pose extensive computational costs, making it challenging to scale and improve upon. In this paper, we propose a general neural network-based recommendation framework, which subsumes several existing state-of-the-art recommendation algorithms, and address the efficiency issue by investigating sampling strategies in the stochastic gradient descent training for the framework. We tackle this issue by first establishing a connection between the loss functions and the user-item interaction bipartite graph, where the loss function terms are defined on links while major computation burdens are located at nodes. We call this type of loss functions "graph-based" loss functions, for which varied mini-batch sampling strategies can have different computational costs. Based on the insight, three novel sampling strategies are proposed, which can significantly improve the training efficiency of the proposed framework (up to $\times 30$ times speedup in our experiments), as well as improving the recommendation performance. Theoretical analysis is also provided for both the computational cost and the convergence. We believe the study of sampling strategies have further implications on general graph-based loss functions, and would also enable more research under the neural network-based recommendation framework.Comment: This is a longer version (with supplementary attached) of the KDD'17 pape

Assessing the Potential of Deep Learning for Emulating Cloud Superparameterization in Climate Models with Real-Geography Boundary Conditions

We explore the potential of feed-forward deep neural networks (DNNs) for emulating cloud superparameterization in realistic geography, using offline fits to data from the Super Parameterized Community Atmospheric Model. To identify the network architecture of greatest skill, we formally optimize hyperparameters using ~250 trials. Our DNN explains over 70 percent of the temporal variance at the 15-minute sampling scale throughout the mid-to-upper troposphere. Autocorrelation timescale analysis compared against DNN skill suggests the less good fit in the tropical, marine boundary layer is driven by neural network difficulty emulating fast, stochastic signals in convection. However, spectral analysis in the temporal domain indicates skillful emulation of signals on diurnal to synoptic scales. A close look at the diurnal cycle reveals correct emulation of land-sea contrasts and vertical structure in the heating and moistening fields, but some distortion of precipitation. Sensitivity tests targeting precipitation skill reveal complementary effects of adding positive constraints vs. hyperparameter tuning, motivating the use of both in the future. A first attempt to force an offline land model with DNN emulated atmospheric fields produces reassuring results further supporting neural network emulation viability in real-geography settings. Overall, the fit skill is competitive with recent attempts by sophisticated Residual and Convolutional Neural Network architectures trained on added information, including memory of past states. Our results confirm the parameterizability of superparameterized convection with continents through machine learning and we highlight advantages of casting this problem locally in space and time for accurate emulation and hopefully quick implementation of hybrid climate models.Comment: 32 Pages, 13 Figures, Revised Version Submitted to Journal of Advances in Modeling Earth Systems April 202