Understanding Hidden Memories of Recurrent Neural Networks

Recurrent neural networks (RNNs) have been successfully applied to various natural language processing (NLP) tasks and achieved better results than conventional methods. However, the lack of understanding of the mechanisms behind their effectiveness limits further improvements on their architectures. In this paper, we present a visual analytics method for understanding and comparing RNN models for NLP tasks. We propose a technique to explain the function of individual hidden state units based on their expected response to input texts. We then co-cluster hidden state units and words based on the expected response and visualize co-clustering results as memory chips and word clouds to provide more structured knowledge on RNNs' hidden states. We also propose a glyph-based sequence visualization based on aggregate information to analyze the behavior of an RNN's hidden state at the sentence-level. The usability and effectiveness of our method are demonstrated through case studies and reviews from domain experts.Comment: Published at IEEE Conference on Visual Analytics Science and Technology (IEEE VAST 2017

Improving Fine-grained Entity Typing with Entity Linking

Fine-grained entity typing is a challenging problem since it usually involves a relatively large tag set and may require to understand the context of the entity mention. In this paper, we use entity linking to help with the fine-grained entity type classification process. We propose a deep neural model that makes predictions based on both the context and the information obtained from entity linking results. Experimental results on two commonly used datasets demonstrates the effectiveness of our approach. On both datasets, it achieves more than 5\% absolute strict accuracy improvement over the state of the art.Comment: EMNLP 201

Peer-inspired student performance prediction in interactive online question pools with graph neural network

Student performance prediction is critical to online education. It can benefit many downstream tasks on online learning platforms, such as estimating dropout rates, facilitating strategic intervention, and enabling adaptive online learning. Interactive online question pools provide students with interesting interactive questions to practice their knowledge in online education. However, little research has been done on student performance prediction in interactive online question pools. Existing work on student performance prediction targets at online learning platforms with predefined course curriculum and accurate knowledge labels like MOOC platforms, but they are not able to fully model knowledge evolution of students in interactive online question pools. In this paper, we propose a novel approach using Graph Neural Networks (GNNs) to achieve better student performance prediction in interactive online question pools. Specifically, we model the relationship between students and questions using student interactions to construct the student-interaction-question network and further present a new GNN model, called R^2GCN, which intrinsically works for the heterogeneous networks, to achieve generalizable student performance prediction in interactive online question pools. We evaluate the effectiveness of our approach on a real-world dataset consisting of 104,113 mouse trajectories generated in the problem-solving process of over 4000 students on 1631 questions. The experiment results show that our approach can achieve a much higher accuracy of student performance prediction than both traditional machine learning approaches and GNN models.Comment: 8 pages, 8 figures. Accepted at CIKM 202

Multi-step Jailbreaking Privacy Attacks on ChatGPT

With the rapid progress of large language models (LLMs), many downstream NLP tasks can be well solved given good prompts. Though model developers and researchers work hard on dialog safety to avoid generating harmful content from LLMs, it is still challenging to steer AI-generated content (AIGC) for the human good. As powerful LLMs are devouring existing text data from various domains (e.g., GPT-3 is trained on 45TB texts), it is natural to doubt whether the private information is included in the training data and what privacy threats can these LLMs and their downstream applications bring. In this paper, we study the privacy threats from OpenAI's model APIs and New Bing enhanced by ChatGPT and show that application-integrated LLMs may cause more severe privacy threats ever than before. To this end, we conduct extensive experiments to support our claims and discuss LLMs' privacy implications.Comment: Work in progres

Peer-inspired Student Performance Prediction in Interactive Online Question Pools with Graph Neural Network

Student performance prediction is critical to online education. It can benefit many downstream tasks on online learning platforms, such as estimating dropout rates, facilitating strategic intervention, and enabling adaptive online learning. Interactive online question pools provide students with interesting interactive questions to practice their knowledge in online education. However, little research has been done on student performance prediction in interactive online question pools. Existing work on student performance prediction targets at online learning platforms with predefined course curriculum and accurate knowledge labels like MOOC platforms, but they are not able to fully model knowledge evolution of students in interactive online question pools. In this paper, we propose a novel approach using Graph Neural Networks (GNNs) to achieve better student performance prediction in interactive online question pools. Specifically, we model the relationship between students and questions using student interactions to construct the student-interaction-question network and further present a new GNN model, called R^2GCN, which intrinsically works for the heterogeneous networks, to achieve generalizable student performance prediction in interactive online question pools. We evaluate the effectiveness of our approach on a real-world dataset consisting of 104,113 mouse trajectories generated in the problem-solving process of over 4000 students on 1631 questions. The experiment results show that our approach can achieve a much higher accuracy of student performance prediction than both traditional machine learning approaches and GNN models.Comment: 8 pages, 8 figures. Accepted at CIKM 202

Knowledge Graph Reasoning over Entities and Numerical Values

A complex logic query in a knowledge graph refers to a query expressed in logic form that conveys a complex meaning, such as where did the Canadian Turing award winner graduate from? Knowledge graph reasoning-based applications, such as dialogue systems and interactive search engines, rely on the ability to answer complex logic queries as a fundamental task. In most knowledge graphs, edges are typically used to either describe the relationships between entities or their associated attribute values. An attribute value can be in categorical or numerical format, such as dates, years, sizes, etc. However, existing complex query answering (CQA) methods simply treat numerical values in the same way as they treat entities. This can lead to difficulties in answering certain queries, such as which Australian Pulitzer award winner is born before 1927, and which drug is a pain reliever and has fewer side effects than Paracetamol. In this work, inspired by the recent advances in numerical encoding and knowledge graph reasoning, we propose numerical complex query answering. In this task, we introduce new numerical variables and operations to describe queries involving numerical attribute values. To address the difference between entities and numerical values, we also propose the framework of Number Reasoning Network (NRN) for alternatively encoding entities and numerical values into separate encoding structures. During the numerical encoding process, NRN employs a parameterized density function to encode the distribution of numerical values. During the entity encoding process, NRN uses established query encoding methods for the original CQA problem. Experimental results show that NRN consistently improves various query encoding methods on three different knowledge graphs and achieves state-of-the-art results

Evolution of T-cell clonality in a patient with Ph-negative acute lymphocytic leukemia occurring after interferon and imatinib therapy for Ph-positive chronic myeloid leukemia

<p>Abstract</p> <p>Introduction</p> <p>The development of Philadelphia chromosome (Ph) negative acute leukemia/myelodysplastic syndrome (MDS) in patients with Ph-positive chronic myeloid leukemia (CML) is very rare. The features of restrictive usage and absence of partial T cell clones have been found in patients with CML. However, the T-cell clonal evolution of Ph-negative malignancies during treatment for CML is still unknown.</p> <p>Objective</p> <p>To investigate the dynamic change of clonal proliferation of T cell receptor (TCR) Vα and Vβ subfamilies in one CML patient who developed Ph-negative acute lymphoblastic leukemia (ALL) after interferon and imatinib therapy.</p> <p>Methods</p> <p>The peripheral blood mononuclear cells (PBMC) samples were collected at the 3 time points (diagnosis of Ph-positive chronic phase (CP) CML, developing Ph-negative ALL and post inductive chemotherapy (CT) for Ph-negative ALL, respectively). The CDR3 size of TCR Vα and Vβ repertoire were detected by RT-PCR. The PCR products were further analyzed by genescan to identify T cell clonality.</p> <p>Results</p> <p>The CML patient who achieved complete cytogenetic remission (CCR) after 5 years of IFN-α therapy suddenly developed Ph-negative ALL 6 months following switch to imatinib therapy. The expression pattern and clonality of TCR Vα/Vβ T cells changed in different disease stages. The restrictive expression of Vα/Vβ subfamilies could be found in all three stages, and partial subfamily of T cells showed clonal proliferation. Additionally, there have been obvious differences in Vα/Vβ subfamily of T cells between the stages of Ph-positive CML-CP and Ph-negative ALL. The Vα10 and Vβ3 T cells evolved from oligoclonality to polyclonality, the Vβ13 T cells changed from bioclonality to polyclonality, when Ph-negative ALL developed.</p> <p>Conclusions</p> <p>Restrictive usage and clonal proliferation of different Vα/Vβ subfamily T cells between the stages of Ph-positive CP and Ph-negative ALL were detected in one patient. These changes may play a role in Ph- negative leukemogenesis.</p