IRIS: Learning the Underlying Information of Scientific Research Interests Using Heterogeneous Network Representation

Understanding scientific research fields and finding potential relations between seemingly distinct fields can help researchers rapidly grasp their most interested topics with expertises. In this study, we construct a heterogeneous network which contains authors, keywords, papers and institutions, and built an “Integrated Research Interest Space (IRIS)” which can represent both author and keyword nodes. Similar keywords in the sense of research interest and research manner can obvious aggregate together. Authors that are interested in different keywords distributed in different IRIS areas, with strongly associated with research objectives and methodologies of the keywords. The average similarities between authors and their real used keywords is significantly higher than that of randomly chosen author-keyword pairs. Based on these observations, we propose a simple algorithm which attempts to recommend potential interested keywords for researchers, and got meaningful results. Our study may also give useful hints for understanding research interests and discovering potential cross disciplines

News Text Classification Based on an Improved Convolutional Neural Network

With the explosive growth in Internet news media and the disorganized status of news texts, this paper puts forward an automatic classification model for news based on a Convolutional Neural Network (CNN). In the model, Word2vec is firstly merged with Latent Dirichlet Allocation (LDA) to generate an effective text feature representation. Then when an attention mechanism is combined with the proposed model, higher attention probability values are given to key features to achieve an accurate judgment. The results show that the precision rate, the recall rate and the F1 value of the model in this paper reach 96.4%, 95.9% and 96.2% respectively, which indicates that the improved CNN, through a unique framework, can extract deep semantic features of the text and provide a strong support for establishing an efficient and accurate news text classification model

Employing Deep Learning and Structured Information Retrieval to Answer Clarification Questions on Bug Reports

Software bug reports reported on bug-tracking systems often lack crucial information for the developers to promptly resolve them, costing companies billions of dollars. There has been significant research on effectively eliciting information from bug reporters in bug tracking systems using different templates that bug reporters need to use. However, the need for asking follow-up questions persists. Recent studies propose techniques to suggest these follow-up questions to help developers obtain the missing details, but there has been little research on answering these follow up questions, which are often unanswered. In this paper, we propose a novel approach that uses CodeT5 in combination with Lucene, an information retrieval technique that leverages the relevance of different bug reports, their components, and follow-up questions to recommend answers. These top-performing answers, along with their bug report, serve as additional context apart from the deficient bug report to the deep learning model for generating an answer. We evaluate our recommended answers with the manually annotated answers using similarity metrics like Normalized Smooth BLEU Score, METEOR, Word Mover's Distance, and Semantic Similarity. We achieve a BLEU Score of up to 34 and Semantic Similarity of up to 64 which shows that the answers generated are understandable and good according to Google's standard and can outperform multiple baselines.Comment: Fixed formatting and typographical error

Learning semantic representations through multimodal graph neural networks

Proyecto de Graduación (Licenciatura en Ingeniería Mecatrónica) Instituto Tecnológico de Costa Rica. Área Académica de Ingeniería Mecatrónica, 2021Para proporcionar del conocimiento semántico sobre los objetos con los que van a interactuar los sistemas robóticos, se debe abordar el problema del aprendizaje de las representaciones semánticas a partir de las modalidades del lenguaje y la visión. El conocimiento semántico se refiere a la información conceptual, incluida la información semántica (significado) y léxica (palabra), y que proporciona la base para muchos de nuestros comportamientos no verbales cotidianos. Por lo tanto, es necesario desarrollar métodos que permitan a los robots procesar oraciones en un entorno del mundo real, por lo que este proyecto presenta un enfoque novedoso que utiliza Redes Convolucionales Gráficas para aprender representaciones de palabras basadas en el significado. El modelo propuesto consta de una primera capa que codifica representaciones unimodales y una segunda capa que integra estas representaciones unimodales en una para aprender una representación desde ambas modalidades. Los resultados experimentales muestran que el modelo propuesto supera al estado del arte en similitud semántica y que tiene la capacidad de simular juicios de similitud humana. Hasta donde sabemos, este enfoque es novedoso en el uso de Redes Convolucionales Gráficas para mejorar la calidad de las representaciones de palabras.To provide semantic knowledge about the objects that robotic systems are going to interact with, you must address the problem of learning semantic representations from modalities of language and vision. Semantic knowledge refers to conceptual information, including semantic (meaning) and lexical (word) information, and that provides the basis for many of our everyday non-verbal behaviors. Therefore, it is necessary to develop methods that enable robots to process sentences in a real-world environment, so this project introduces a novel approach that uses Graph Convolutional Networks to learn grounded meaning representations of words. The proposed model consists of a first layer that encodes unimodal representations, and a second layer that integrates these unimodal representations into one to learn a representation from both modalities. Experimental results show that the proposed model outperforms that state-of-the-art in semantic similarity and that can simulate human similarity judgments. To the best of our knowledge, this approach is novel in its use of Graph Convolutional Networks to enhance the quality of word representations

Word Embedding Driven Concept Detection in Philosophical Corpora

During the course of research, scholars often explore large textual databases for segments of text relevant to their conceptual analyses. This study proposes, develops and evaluates two algorithms for automated concept detection in theoretical corpora: ACS and WMD retrieval. Both novel algorithms are compared to key word retrieval, using a test set from the Digital Ricoeur corpus tagged by scholarly experts. WMD retrieval outperforms key word search on the concept detection task. Thus, WMD retrieval is a promising tool for concept detection and information retrieval systems focused on theoretical corpora

Seismic Foundation Model (SFM): a new generation deep learning model in geophysics

While computer science has seen remarkable advancements in foundation models, which remain underexplored in geoscience. Addressing this gap, we introduce a workflow to develop geophysical foundation models, including data preparation, model pre-training, and adaption to downstream tasks. From 192 globally collected 3-D seismic volumes, we create a carefully curated dataset of 2,286,422 2-D seismic images. Fully using these unlabeled images, we employ the self-supervised learning to pre-train a Transformer-based Seismic Foundation Model (SFM) for producing all-purpose seismic features that work across various tasks and surveys. Through experiments on seismic facies classification, geobody identification, interpolation, denoising, and inversion, our pre-trained model demonstrates versatility, generalization, scalability, and superior performance over baseline models. Conclusively, we provide a foundation model and vast dataset to advance AI in geophysics, addressing challenges (poor generalization, lacking labels, and repetitive training for task-specified models) of applying AI in geophysics and paving the way for future innovations in geoscience.Comment: 27 pages, 9 figures, and 4 table

지식 증류를위한 다단계 교사

학위논문(석사)--서울대학교 대학원 :공과대학 전기·컴퓨터공학부,2019. 8. Lee, Kyoung Mu.지식 증류 (Knowledge Distillation, KD)는 교사로부터 학생 모델로 지식을 전 달하는 잘 알려진 방법입니다. 본 논문에서는 계층 적 진보적 교사 (Layer-wise Pro- gressive Teacher)를 도입하여 지식 증류를위한 새로운 틀을 제안하고자한다. 이와 관련하여 우리는 교사의 중간 계층에서 확률을 구함으로써 서로 다른 경도 수준에 서 부드러운 목표를 만드는 방법을 제안합니다. 우리의 방법은 교사와 학생 사이에 큰 차이가있어 학생이 교사를 모방하는 것을 더 어렵게하는 경우를 위해 특별히 고 안되었습니다. 우리는 또한 학생의 온도를 제거하고 교사의 온도를 유지하는 것이 좋습니다. 실험 결과는 기존의 증류법과 비교할 때 우리의 방법이 훨씬 더 우수한 결과를 얻음을 보여줍니다.Knowledge Distillation (KD) is a well-known method for transferring knowledge from a teacher to a student model. In this thesis, we propose a new framework for Knowledge Distillation by introducing a Layer-wise Progressive Teacher. In this regard, we propose a method to create soft targets in different levels of complexity by obtaining the probabilities from the intermediate layers of the teacher network. Our method is specially designed for the cases that there is a large gap between the teacher and the student which makes it harder for the student to mimic the teacher. In addition, we proposed focalized teacher as a method to train a better teacher for the student. The experimental results show that our method gets significantly better results in comparison with existing knowledge distillation methods.1 Introduction 1 1.1 Background. 1 1.2 Motivation 3 1.3 ProposedMethod 4 1.4 Datasets. 5 2 Related Work 7 2.1 Theory of Transfer Learning 7 2.2 Applications. 8 3 Focalized Teacher 10 3.1 Overview 10 3.2 LabelCorrection 11 3.3 FocalizedTeacher 12 3.4 Experimental Results 13 4 Layer-wise Progressive Knowledge Distillation 16 4.1 BackgroundandNotations . 16 4.2 Layer-wiseKnowledgeDistillation. 17 4.3 ProgressiveTeacher. 19 4.4 Experimental Results 20 4.4.1 TemperatureAnalysis . 21 4.4.2 DistanceMetric. 23 4.4.3 Distilled Knowledge from an intermediate layer . . . . . . . . 24 4.4.4 ProgressiveTeacher 27 4.4.5 ComparisonwithotherKDmethods 29 5 Conclusion 5.1 SummaryoftheThesis . 32 5.2 FutureWorks 32 5.2.1 Progressive Teacher Assistant based Knowledge Distillation . 33Maste