Convolutional Neural Networks over Tree Structures for Programming Language Processing

Programming language processing (similar to natural language processing) is a hot research topic in the field of software engineering; it has also aroused growing interest in the artificial intelligence community. However, different from a natural language sentence, a program contains rich, explicit, and complicated structural information. Hence, traditional NLP models may be inappropriate for programs. In this paper, we propose a novel tree-based convolutional neural network (TBCNN) for programming language processing, in which a convolution kernel is designed over programs' abstract syntax trees to capture structural information. TBCNN is a generic architecture for programming language processing; our experiments show its effectiveness in two different program analysis tasks: classifying programs according to functionality, and detecting code snippets of certain patterns. TBCNN outperforms baseline methods, including several neural models for NLP.Comment: Accepted at AAAI-1

Distilling Word Embeddings: An Encoding Approach

Distilling knowledge from a well-trained cumbersome network to a small one has recently become a new research topic, as lightweight neural networks with high performance are particularly in need in various resource-restricted systems. This paper addresses the problem of distilling word embeddings for NLP tasks. We propose an encoding approach to distill task-specific knowledge from a set of high-dimensional embeddings, which can reduce model complexity by a large margin as well as retain high accuracy, showing a good compromise between efficiency and performance. Experiments in two tasks reveal the phenomenon that distilling knowledge from cumbersome embeddings is better than directly training neural networks with small embeddings.Comment: Accepted by CIKM-16 as a short paper, and by the Representation Learning for Natural Language Processing (RL4NLP) Workshop @ACL-16 for presentatio

A Comparative Study on Regularization Strategies for Embedding-based Neural Networks

This paper aims to compare different regularization strategies to address a common phenomenon, severe overfitting, in embedding-based neural networks for NLP. We chose two widely studied neural models and tasks as our testbed. We tried several frequently applied or newly proposed regularization strategies, including penalizing weights (embeddings excluded), penalizing embeddings, re-embedding words, and dropout. We also emphasized on incremental hyperparameter tuning, and combining different regularizations. The results provide a picture on tuning hyperparameters for neural NLP models.Comment: EMNLP '1

Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path

Relation classification is an important research arena in the field of natural language processing (NLP). In this paper, we present SDP-LSTM, a novel neural network to classify the relation of two entities in a sentence. Our neural architecture leverages the shortest dependency path (SDP) between two entities; multichannel recurrent neural networks, with long short term memory (LSTM) units, pick up heterogeneous information along the SDP. Our proposed model has several distinct features: (1) The shortest dependency paths retain most relevant information (to relation classification), while eliminating irrelevant words in the sentence. (2) The multichannel LSTM networks allow effective information integration from heterogeneous sources over the dependency paths. (3) A customized dropout strategy regularizes the neural network to alleviate overfitting. We test our model on the SemEval 2010 relation classification task, and achieve an $F_1$-score of 83.7\%, higher than competing methods in the literature.Comment: EMNLP '1

Assessment of Severe Accident Depressurization Valve Activation Strategy for Chinese Improved 1000 MWe PWR

To prevent HPME and DCH, SADV is proposed to be added to the pressurizer for Chinese improved 1000 MWe PWR NPP with the reference of EPR design. Rapid depressurization capability is assessed using the mechanical analytical code. Three typical severe accident sequences of TMLB’, SBLOCA, and LOFW are selected. It shows that with activation of the SADV the RCS pressure is low enough to prevent HPME and DCH. Natural circulation at upper RPV and hot leg is considered for the rapid depressurization capacity analysis. The result shows that natural circulation phenomenon results in heat transfer from the core to the pipes in RCS which may cause the creep rupture of pipes in RCS and delays the severe accident progression. Different SADV valve areas are investigated to the influence of depressurization of RCS. Analysis shows that the introduction of SADV with right valve area will delay progression of core degradation to RPV failure. Valve area is to be optimized since smaller SADV area will reduce its effect and too large valve area will lead to excessive loss of water inventory in RCS and makes core degradation progression to RPV failure faster without additional core cooling water sources

catena-Poly[[diaqua­cobalt(II)]bis­[μ-2-(4-carboxyl­atophen­yl)-4,4,5,5-tetra­methyl-4,5-dihydro-1H-imidazol-1-oxyl 3-oxide]]

In the title compound, [Co(C14H16N2O4)2(H2O)2]n, the CoII atom, lying on an inversion center, is coordinated by six O atoms in a distorted octa­hedral geometry. The CoII atoms are bridged by the nitronyl nitroxide ligands into a tape-like structure along the b axis. The tapes are further connected by O—H⋯O hydrogen bonds, forming a layer parallel to the bc plane

Building Program Vector Representations for Deep Learning

Deep learning has made significant breakthroughs in various fields of artificial intelligence. Advantages of deep learning include the ability to capture highly complicated features, weak involvement of human engineering, etc. However, it is still virtually impossible to use deep learning to analyze programs since deep architectures cannot be trained effectively with pure back propagation. In this pioneering paper, we propose the "coding criterion" to build program vector representations, which are the premise of deep learning for program analysis. Our representation learning approach directly makes deep learning a reality in this new field. We evaluate the learned vector representations both qualitatively and quantitatively. We conclude, based on the experiments, the coding criterion is successful in building program representations. To evaluate whether deep learning is beneficial for program analysis, we feed the representations to deep neural networks, and achieve higher accuracy in the program classification task than "shallow" methods, such as logistic regression and the support vector machine. This result confirms the feasibility of deep learning to analyze programs. It also gives primary evidence of its success in this new field. We believe deep learning will become an outstanding technique for program analysis in the near future.Comment: This paper was submitted to ICSE'1