External Reasoning: Towards Multi-Large-Language-Models Interchangeable Assistance with Human Feedback

Memory is identified as a crucial human faculty that allows for the retention of visual and linguistic information within the hippocampus and neurons in the brain, which can subsequently be retrieved to address real-world challenges that arise through a lifetime of learning. The resolution of complex AI tasks through the application of acquired knowledge represents a stride toward the realization of artificial general intelligence. However, despite the prevalence of Large Language Models (LLMs) like GPT-3.5 and GPT-4 , which have displayed remarkable capabilities in language comprehension, generation, interaction, and reasoning, they are inhibited by constraints on context length that preclude the processing of extensive, continually evolving knowledge bases. This paper proposes that LLMs could be augmented through the selective integration of knowledge from external repositories, and in doing so, introduces a novel methodology for External Reasoning, exemplified by ChatPDF. Central to this approach is the establishment of a tiered policy for \textbf{External Reasoning based on Multiple LLM Interchange Assistance}, where the level of support rendered is modulated across entry, intermediate, and advanced tiers based on the complexity of the query, with adjustments made in response to human feedback. A comprehensive evaluation of this methodology is conducted using multiple LLMs and the results indicate state-of-the-art performance, surpassing existing solutions including ChatPDF.com. Moreover, the paper emphasizes that this approach is more efficient compared to the direct processing of full text by LLMs.Comment: technical repor

지식 기반 대화에서의 대화 특성을 활용한 지식 선택 및 랭킹 방법

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·컴퓨터공학부, 2022. 8. 이상구.Knowledge grounded conversation (KGC) model aims to generate informative responses relevant to both conversation history and external knowledge. One of the most important parts of KGC models is to find the knowledge which provides the basis on which the responses are grounded. If the model selects inappropriate knowledge, it may produce responses that are irrelevant or lack knowledge. In this dissertation, we study the methods of leveraging conversational characteristics to select or rank the knowledge for knowledge grounded conversation. In particular, this dissertation provides novel two methods, where one of which focuses on the sequential structure of multi-turn conversation, and the other focuses on utilizing local context and topic of a long conversation. We first propose two knowledge selection strategies of which one preserves the sequential matching features and the other encodes the sequential nature of the conversation. Second, we propose a novel knowledge ranking model that composes an appropriate range of relevant documents by exploiting both the topic keywords and local context of a conversation. In addition, we apply the knowledge ranking model in quote recommendation with our new quote recommendation framework that provides hard negative samples to the model. Our experimental results show that the KGC models based on our proposed knowledge selection and ranking methods outperform the competitive models in terms of groundness and relevance.지식 기반 대화 모델은 대화 기록과 외부 지식 이 두 가지 모두에 관련된 응답을 생성하는 것을 목표로 한다. 지식 기반 대화 모델의 가장 중요한 부분 중 하나는 응답의 기반을 제공하는 지식을 찾는 것이다. 지식 기반 모델이 주어진 문맥에 부적합한 지식을 찾는 경우 관련성이 떨어지거나 지식이 부족한 응답이 생성될 수 있다. 이 문제를 해결하기 위해 이 논문에서는 지식 기반 대화를 위해 대화 여러 특성을 활용하여 지식을 선정하는 지식 선택 모델과 지식 순위 모델을 제시한다. 구체적으로 본 논문에서는 다중 턴 대화에서의 순차적 구조 또는 응답 이전 문맥과 대화의 주제를 활용하는 새로운 두 가지 방법을 제시한다. 첫 번째 방법으로써 본 논문은 두 가지 지식 선택 전략을 제안한다. 제안하는 전략 중 하나는 지식과 대화 턴 간의 순차적 매칭 특징을 보존하는 방법이고 다른 전략은 대화의 순차적 특성을 인코딩하여 지식을 선택하는 방법이다. 두 번째로 본 논문은 대화의 주제 키워드와 응답 바로 이전의 문맥을 모두 활용하여 적절한 범위의 관련 문서들로 검색 결과를 구성하는 새로운 지식 순위 모델을 제안한다. 마지막으로 지식 순위 모델의 적응성 검증을 위해 정답 인용구와 의미적으로 유사하지만 정답은 아닌 인용구의 집합을 인용구 순위 모델에 제공하는 인용구 추천 프레임워크를 제안한다. 제안된 지식 선택 및 순위 모델을 기반으로 하는 지식 기반 대화 모델이 경쟁 모델보다 외부 지식 및 대화 문맥과의 관련성 측면에서 우수하다는 것을 사람 간의 대화 데이터를 이용한 다수의 실험을 통해 검증하였다.Abstract 1 1. Introduction 17 2. Background and Related Works 25 2.1 Terminology 25 2.2 Overview of Technologies for Conversational Systems 27 2.2.1 Open-domain Dialogue System 27 2.2.2 Task-oriented Dialogue System 29 2.2.3 Question Answering System 29 2.3 Components of Knowledge Grounded Conversation Model 31 2.4 Related Works 36 2.4.1 KGC datasets 36 2.4.2 Soft Selection-based KGC Model 36 2.4.3 Hard Selection-based KGC Model 37 2.4.4 Retrieval-based KGC Models 39 2.4.5 Response Generation with Knowledge Integration 39 2.4.6 Quote Recommendation 42 2.5 Evaluation Methods 44 2.6 Problem Statements 47 3. Knowledge Selection with Sequential Structure of Conversation 48 3.1 Motivation 48 3.2 Reduce-Match Strategy & Match-Reduce Strategy 49 3.2.1 Backbone architecture 51 3.2.2 Reduce-Match Strategy-based Models 52 3.2.3 Match-Reduce Strategy-based Models 56 3.3 Experiments 62 3.3.1 Experimental Setup 62 3.3.2 Experimental Results 70 3.4 Analysis 72 3.4.1 Case Study 72 3.4.2 Impact of Matching Difficulty 75 3.4.3 Impact of Length of Context 77 3.4.4 Impact of Dialogue Act of Message 78 4. Knowledge Ranking with Local Context and Topic Keywords 81 4.1 Motivation 81 4.2 Retrieval-Augmented Knowledge Grounded Conversation Model 85 4.2.1 Base Model 86 4.2.2 Topic-aware Dual Matching for Knowledge Re-ranking 86 4.2.3 Data Weighting Scheme for Retrieval Augmented Generation Models 89 4.3 Experiments 90 4.3.1 Experimental Setup 90 4.3.2 Experimental Results 94 4.4 Analysis 98 4.4.1 Case Study 98 4.4.2 Ablation Study 99 4.4.3 Model Variations 104 4.4.4 Error Analysis 105 5. Application: Quote Recommendation with Knowledge Ranking 110 5.1 Motivation 110 5.2 CAGAR: A Framework for Quote Recommendation 112 5.2.1 Conversation Encoder 114 5.2.2 Quote Encoder 114 5.2.3 Candidate Generator 115 5.2.4 Re-ranker 116 5.2.5 Training and Inference 116 5.3 Experiments 117 5.3.1 Experimental Setup 117 5.3.2 Experimental Results 119 5.4 Analysis 120 5.4.1 Ablation Study 120 5.4.2 Case Study 121 5.4.3 Impact of Length of Context 121 5.4.4 Impact of Training Set Size per Quote 123 6. Conclusion 125 6.1 Contributions and Limitations 126 6.2 Future Works 128 Appendix A. Preliminary Experiments for Quote Recommendations 131 A.1 Methods 131 A.1.1 Matching Granularity Adjustment 131 A.1.2 Random Forest 133 A.1.3 Convolutional Neural Network 133 A.1.4 Recurrent Neural Network 134 A.2 Experiments 135 A.2.1 Baselines and Implementation Details 135 A.2.2 Datasets 136 A.2.3 Results and Discussions 137 초록 162박

Deep latent-variable models for neural text generation

Text generation aims to produce human-like natural language output for down-stream tasks. It covers a wide range of applications like machine translation, document summarization, dialogue generation and so on. Recently deep neural network-based end-to-end architectures are known to be data-hungry, and text generated from them usually suffer from low diversity, interpretability and controllability. As a result, it is difficult to trust the output from them in real-life applications. Deep latent-variable models, by specifying the probabilistic distribution over an intermediate latent process, provide a potential way of addressing these problems while maintaining the expressive power of deep neural networks. This presentation will explain how deep latent-variable models can improve over the standard encoder-decoder model for text generation. We will start from an introduction of encoder-decoder and deep latent-variable models, then go over popular optimization strategies, and finally elaborate on how latent variable models can help improve the diversity, interpretability and data efficiency in different applications of text generation tasks.Textgenerierung zielt darauf ab, eine menschenähnliche Textausgabe in natürlicher Sprache für Anwendungen zu erzeugen. Es deckt eine breite Palette von Anwendungen ab, wie maschinelle Übersetzung, Zusammenfassung von Dokumenten, Generierung von Dialogen usw. In letzter Zeit werden dafür hauptsächlich Endto- End-Architekturen auf der Basis von tiefen neuronalen Netzwerken verwendet. Der End-to-End-Ansatz fasst alle Submodule, die früher nach komplexen handgefertigten Regeln entworfen wurden, zu einer ganzheitlichen Codierungs- Decodierungs-Architektur zusammen. Bei ausreichenden Trainingsdaten kann eine Leistung auf dem neuesten Stand der Technik erzielt werden, ohne dass sprach- und domänenabhängiges Wissen erforderlich ist. Deep-Learning-Modelle sind jedoch als extrem datenhungrig bekannt und daraus generierter Text leidet normalerweise unter geringer Diversität, Interpretierbarkeit und Kontrollierbarkeit. Infolgedessen ist es schwierig, der Ausgabe von ihnen in realen Anwendungen zu vertrauen. Tiefe Modelle mit latenten Variablen bieten durch Angabe der Wahrscheinlichkeitsverteilung über einen latenten Zwischenprozess eine potenzielle Möglichkeit, diese Probleme zu lösen und gleichzeitig die Ausdruckskraft tiefer neuronaler Netze zu erhalten. Diese Dissertation zeigt, wie tiefe Modelle mit latenten Variablen Texterzeugung verbessern gegenüber dem üblichen Encoder-Decoder-Modell. Wir beginnen mit einer Einführung in Encoder-Decoder- und Deep Latent Variable-Modelle und gehen dann auf gängige Optimierungsstrategien wie Variationsinferenz, dynamische Programmierung, Soft Relaxation und Reinforcement Learning ein. Danach präsentieren wir Folgendes: 1. Wie latente Variablen Vielfalt der Texterzeugung verbessern können, indem ganzheitliche, latente Darstellungen auf Satzebene gelernt werden. Auf diese Weise kann zunächst eine latente Darstellung ausgewählt werden, aus der verschiedene Texte generiert werden können. Wir präsentieren effektive Algorithmen, um gleichzeitig das Lernen der Repräsentation und die Texterzeugung durch Variationsinferenz zu trainieren. Um die Einschränkungen der Variationsinferenz bezüglich Uni-Modalität und Inkonsistenz anzugehen, schlagen wir eine Wake-Sleep-Variation und ein auf Transinformation basierendes Trainingsziel vor. Experimente zeigen, dass sie sowohl die übliche Variationsinferenz als auch nicht-latente Variablenmodelle bei der Dialoggenerierung übertreffen. 2. Wie latente Variablen die Steuerbarkeit und Interpretierbarkeit der Texterzeugung verbessern können, indem feinkörnigere latente Spezifikationen zum Zwischengenerierungsprozess hinzugefügt werden. Wir veranschaulichen die Verwendung latenter Variablen für Wortausrichtung, Inhaltsauswahl, Textsegmentierung und Feldsegmentkorrespondenz. Wir leiten für sie effiziente Trainingsalgorithmen ab, damit die Texterzeugung explizit gesteuert werden kann, indem die latente Variable, die durch ihre Definition vom Menschen interpretiert werden kann, manipuliert wird. 3. Überwindung der Seltenheit von Trainingsmustern durch Behandlung von nicht parallelem Text als latente Variablen. Das Training kann wie beim Standard-EM-Algorithmus durchgeführt werden, der stabil konvergiert. Wir zeigen, dass es bei der Dialoggenerierung erfolgreich angewendet werden kann und den Generierungsraum durch die Verwendung von nicht-konversativem Text erheblich bereichert

Towards structured neural spoken dialogue modelling.

195 p.In this thesis, we try to alleviate some of the weaknesses of the current approaches to dialogue modelling,one of the most challenging areas of Artificial Intelligence. We target three different types of dialogues(open-domain, task-oriented and coaching sessions), and use mainly machine learning algorithms to traindialogue models. One challenge of open-domain chatbots is their lack of response variety, which can betackled using Generative Adversarial Networks (GANs). We present two methodological contributions inthis regard. On the one hand, we develop a method to circumvent the non-differentiability of textprocessingGANs. On the other hand, we extend the conventional task of discriminators, which oftenoperate at a single response level, to the batch level. Meanwhile, two crucial aspects of task-orientedsystems are their understanding capabilities because they need to correctly interpret what the user islooking for and their constraints), and the dialogue strategy. We propose a simple yet powerful way toimprove spoken understanding and adapt the dialogue strategy by explicitly processing the user's speechsignal through audio-processing transformer neural networks. Finally, coaching dialogues shareproperties of open-domain and task-oriented dialogues. They are somehow task-oriented but, there is norush to complete the task, and it is more important to calmly converse to make the users aware of theirown problems. In this context, we describe our collaboration in the EMPATHIC project, where a VirtualCoach capable of carrying out coaching dialogues about nutrition was built, using a modular SpokenDialogue System. Second, we model such dialogues with an end-to-end system based on TransferLearning

Ahead of the Curve: Insights for the International NGO of the Future

International NGOs have a unique and important role to play in addressing today's complex global challenges. But few of them are living up to their full potential. With support from the Hewlett Foundation, FSG researched how the most innovative INGOs are adapting to the disruptions in the global development sector and embracing four approaches to create greater impact