Understanding Hidden Memories of Recurrent Neural Networks

Recurrent neural networks (RNNs) have been successfully applied to various natural language processing (NLP) tasks and achieved better results than conventional methods. However, the lack of understanding of the mechanisms behind their effectiveness limits further improvements on their architectures. In this paper, we present a visual analytics method for understanding and comparing RNN models for NLP tasks. We propose a technique to explain the function of individual hidden state units based on their expected response to input texts. We then co-cluster hidden state units and words based on the expected response and visualize co-clustering results as memory chips and word clouds to provide more structured knowledge on RNNs' hidden states. We also propose a glyph-based sequence visualization based on aggregate information to analyze the behavior of an RNN's hidden state at the sentence-level. The usability and effectiveness of our method are demonstrated through case studies and reviews from domain experts.Comment: Published at IEEE Conference on Visual Analytics Science and Technology (IEEE VAST 2017

A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning

Knowledge Graph Embedding (KGE) has proven to be an effective approach to solving the Knowledge Graph Completion (KGC) task. Relational patterns which refer to relations with specific semantics exhibiting graph patterns are an important factor in the performance of KGE models. Though KGE models' capabilities are analyzed over different relational patterns in theory and a rough connection between better relational patterns modeling and better performance of KGC has been built, a comprehensive quantitative analysis on KGE models over relational patterns remains absent so it is uncertain how the theoretical support of KGE to a relational pattern contributes to the performance of triples associated to such a relational pattern. To address this challenge, we evaluate the performance of 7 KGE models over 4 common relational patterns on 2 benchmarks, then conduct an analysis in theory, entity frequency, and part-to-whole three aspects and get some counterintuitive conclusions. Finally, we introduce a training-free method Score-based Patterns Adaptation (SPA) to enhance KGE models' performance over various relational patterns. This approach is simple yet effective and can be applied to KGE models without additional training. Our experimental results demonstrate that our method generally enhances performance over specific relational patterns. Our source code is available from GitHub at https://github.com/zjukg/Comprehensive-Study-over-Relational-Patterns.Comment: This paper is accepted by ISWC 202

Standard Biological Part Automatic Modeling Database Language (MoDeL)

This BioBricks Foundation Request for Comments (BBF RFC) describes the Standard Biological Part Automatic Modeling Database Language (MoDeL). MoDeL provides a language and syntax standard for automatic modeling databases used by synthetic biology software. Meanwhile, MoDeL allows detailed description of biological complex, and presents the concept of Chain-Node Model

The Three-body Force and the Tetraquark Interpretation of Light Scalar Mesons

We study the possible tetraquark interpretation of light scalar meson states $a_0(980)$, $f_0(980)$, $\kappa$, $\sigma$ within the framework of the non-relativistic potential model. The wave functions of tetraquark states are obtained in a space spanned by multiple Gaussian functions. We find that the mass spectra of the light scalar mesons can be well accommodated in the tetraquark picture if we introduce a three-body quark interaction in the quark model. Using the obtained multiple Gaussian wave functions, the decay constants of tetraquarks are also calculated within the ``fall apart'' mechanism