Using domain specific language and sequence to sequence models as a hybrid framework for a natural language interface to a database solution

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThe aim of this project is to provide a new approach to solving the problem of converting natural language into a language capable of querying a database or data repository. This problem has been around for a while, in the 1970's the US Navy developed a solution called LADDER and since then there have been an array of solutions, approaches and tweaks that have kept the research community busy. The introduction of electronic assistants into the smart phone in 2010 has given new impetus to this problem. With the increasingly pervasive nature of data and its ever expanding use to answer questions within business science, medicine extracting data is becoming more important. The idea behind this project is to make data more democratised by allowing access to it without the need for specialist languages. The performance and reliability of converting natural language into structured query language can be problematic in handling nuances that are prevalent in natural language. Relational databases are not designed to understand language nuance. This project introduces the following components as part of a holistic approach to improving the conversion of a natural language statement into a language capable of querying a data repository. ● The idea proposed in this project combines the use of sequence to sequence models in conjunction with the natural language part of speech technologies and domain specific languages to convert natural language queries into SQL. The approach being proposed by this chapter is to use natural language processing to perform an initial shallow pass of the incoming query and then use Google's Tensor Flow to refine the query with the use of a sequence to sequence model. ● This thesis is also proposing to use a Domain Specific Language (DSL) as part of the conversion process. The use of the DSL has the potential to allow the natural language query to be translated into more than just an SQL statement, but any query language such as NoSQL or XQuery

Learning to Map Natural Language to Executable Programs Over Databases

Natural language is a fundamental form of information and communication and is becoming the next frontier in computer interfaces. As the amount of data available online has increased exponentially, so has the need for Natural Language Interfaces (NLIs, which is not used for natural language inference in this thesis) to connect the data and the user by easily using natural language, significantly promoting the possibility and efficiency of information access for many users besides data experts. All consumer-facing software will one day have a dialogue interface, and this is the next vital leap in the evolution of search engines. Such intelligent dialogue systems should understand the meaning of language grounded in various contexts and generate effective language responses in different forms for information requests and human-computer communication.Developing these intelligent systems is challenging due to (1) limited benchmarks to drive advancements, (2) alignment mismatches between natural language and formal programs, (3) lack of trustworthiness and interpretability, (4) context dependencies in both human conversational interactions and the target programs, and (5) joint language understanding between dialog questions and NLI environments (e.g. databases and knowledge graphs). This dissertation presents several datasets, neural algorithms, and language models to address these challenges for developing deep learning technologies for conversational natural language interfaces (more specifically, NLIs to Databases or NLIDB). First, to drive advancements towards neural-based conversational NLIs, we design and propose several complex and cross-domain NLI benchmarks, along with introducing several datasets. These datasets enable training large, deep learning models. The evaluation is done on unseen databases. (e.g., about course arrangement). Systems must generalize well to not only new SQL queries but also to unseen database schemas to perform well on these tasks. Furthermore, in real-world applications, users often access information in a multi-turn interaction with the system by asking a sequence of related questions. The users may explicitly refer to or omit previously mentioned entities and constraints and may introduce refinements, additions, or substitutions to what has already been said. Therefore, some of them require systems to model dialog dynamics and generate natural language explanations for user verification. The full dialogue interaction with the system’s responses is also important as this supports clarifying ambiguous questions, verifying returned results, and notifying users of unanswerable or unrelated questions. A robust dialogue-based NLI system that can engage with users by forming its responses has thus become an increasingly necessary component for the query process. Moreover, this thesis presents the development of scalable algorithms designed to parse complex and sequential questions to formal programs (e.g., mapping questions to SQL queries that can execute against databases). We propose a novel neural model that utilizes type information from knowledge graphs to better understand rare entities and numbers in natural language questions. We also introduce a neural model based on syntax tree neural networks, which was the first methodology proposed for generating complex programs from language. Finally, language modeling creates contextualized vector representations of words by training a model to predict the next word given context words, which are the basis of deep learning for NLP. Recently, pre-trained language models such as BERT and RoBERTa achieve tremendous success in many natural language processing tasks such as text understanding and reading comprehension. However, most language models are pre-trained only on free-text such as Wikipedia articles and Books. Given that language in semantic parsing is usually related to some formal representations such as logic forms and SQL queries and has to be grounded in structural environments (e.g., databases), we propose better language models for NLIs by enforcing such compositional interpolation in them. To show they could better jointly understand dialog questions and NLI environments (e.g. databases and knowledge graphs), we show that these language models achieve new state-of-the-art results for seven representative tasks on semantic parsing, dialogue state tracking, and question answering. Also, our proposed pre-training method is much more effective than other prior work

Facilitating Information Access for Heterogeneous Data Across Many Languages

Information access, which enables people to identify, retrieve, and use information freely and effectively, has attracted interest from academia and industry. Systems for document retrieval and question answering have helped people access information in powerful and useful ways. Recently, natural language technologies based on neural network have been applied to various tasks for information access. Specifically, transformer-based pre-trained models have pushed tasks such as document and passage retrieval to new state-of-the-art effectiveness. (1) Most of the research has focused on helping people access passages and documents on the web. However, there is abundant information stored in other formats such as semi-structured tables and domain-specific relational databases in companies. Development of the models and frameworks that support access information from these data formats is also essential. (2) Moreover, most of the advances in information access research are based on English, leaving other languages less explored. It is insufficient and inequitable in our globalized and connected world to serve only speakers of English. In this thesis, we explore and develop models and frameworks that could alleviate the aforementioned challenges. This dissertation consists of three parts. We begin with a discussion on developing models designed for accessing data in formats other than passages and documents. We mainly focus on two data formats, namely semi-structured tables and relational databases. In the second part, we discuss methods that can enhance the user experience for non-English speakers when using information access systems. Specifically, we first introduce model development for multilingual knowledge graph integration, which can benefit many information access applications such as cross-lingual question answering systems and other knowledge-driven cross-lingual NLP applications. We further focus on multilingual document dense retrieval and reranking that boost the effectiveness of search engines for non-English information access. Last but not least, we take a step further based on the aforementioned two parts by investigating models and frameworks that can facilitate non-English speakers to access structured data. In detail, we present cross-lingual Text-to-SQL semantic parsing systems that enable non-English speakers to query relational databases with queries in their languages

Learning natural language interfaces with neural models

Language is the primary and most natural means of communication for humans. The learning curve of interacting with various devices and services (e.g., digital assistants, and smart appliances) would be greatly reduced if we could talk to machines using human language. However, in most cases computers can only interpret and execute formal languages. In this thesis, we focus on using neural models to build natural language interfaces which learn to map naturally worded expressions onto machineinterpretable representations. The task is challenging due to (1) structural mismatches between natural language and formal language, (2) the well-formedness of output representations, (3) lack of uncertainty information and interpretability, and (4) the model coverage for language variations. In this thesis, we develop several flexible neural architectures to address these challenges. We propose a model based on attention-enhanced encoder-decoder neural networks for natural language interfaces. Beyond sequence modeling, we propose a tree decoder to utilize the compositional nature and well-formedness of meaning representations, which recursively generates hierarchical structures in a top-down manner. To model meaning at different levels of granularity, we present a structure-aware neural architecture which decodes semantic representations following a coarse-to-fine procedure. The proposed neural models remain difficult to interpret, acting in most cases as a black box. We explore ways to estimate and interpret the model’s confidence in its predictions, which we argue can provide users with immediate and meaningful feedback regarding uncertain outputs. We estimate confidence scores that indicate whether model predictions are likely to be correct. Moreover, we identify which parts of the input contribute to uncertain predictions allowing users to interpret their model. Model coverage is one of the major reasons resulting in uncertainty of natural language interfaces. Therefore, we develop a general framework to handle the many different ways natural language expresses the same information need. We leverage external resources to generate felicitous paraphrases for the input, and then feed them to a neural paraphrase scoring model which assigns higher weights to linguistic expressions most likely to yield correct answers. The model components are trained end-to-end using supervision signals provided by the target task. Experimental results show that the proposed neural models can be easily ported across tasks. Moreover, the robustness of natural language interfaces can be enhanced by considering the output well-formedness, confidence modeling, and improving model coverage

Graph Neural Networks for Natural Language Processing: A Survey

Deep learning has become the dominant approach in coping with various tasks in Natural LanguageProcessing (NLP). Although text inputs are typically represented as a sequence of tokens, there isa rich variety of NLP problems that can be best expressed with a graph structure. As a result, thereis a surge of interests in developing new deep learning techniques on graphs for a large numberof NLP tasks. In this survey, we present a comprehensive overview onGraph Neural Networks(GNNs) for Natural Language Processing. We propose a new taxonomy of GNNs for NLP, whichsystematically organizes existing research of GNNs for NLP along three axes: graph construction,graph representation learning, and graph based encoder-decoder models. We further introducea large number of NLP applications that are exploiting the power of GNNs and summarize thecorresponding benchmark datasets, evaluation metrics, and open-source codes. Finally, we discussvarious outstanding challenges for making the full use of GNNs for NLP as well as future researchdirections. To the best of our knowledge, this is the first comprehensive overview of Graph NeuralNetworks for Natural Language Processing.Comment: 127 page

Highly Interactive Web-Based Courseware

Zukünftige Lehr-/Lernprogramme sollen als vernetzte Systeme die Lernenden befähigen, Lerninhalte zu erforschen und zu konstruieren, sowie Verständnisschwierigkeiten und Gedanken in der Lehr-/Lerngemeinschaft zu kommunizieren. Lehrmaterial soll dabei in digitale Lernobjekte übergeführt, kollaborativ von Programmierern, Pädagogen und Designern entwickelt und in einer Datenbank archiviert werden, um von Lehrern und Lernenden eingesetzt, angepasst und weiterentwickelt zu werden. Den ersten Schritt in diese Richtung machte die Lerntechnologie, indem sie Wiederverwendbarkeit und Kompabilität für hypermediale Kurse spezifizierte. Ein größeres Maß an Interaktivität wird bisher allerdings noch nicht in Betracht gezogen. Jedes interaktive Lernobjekt wird als autonome Hypermedia-Einheit angesehen, aufwändig in der Erstellung, und weder mehrstufig verschränk- noch anpassbar, oder gar adäquat spezifizierbar. Dynamische Eigenschaften, Aussehen und Verhalten sind fest vorgegeben. Die vorgestellte Arbeit konzipiert und realisiert Lerntechnologie für hypermediale Kurse unter besonderer Berücksichtigung hochgradig interaktiver Lernobjekte. Innovativ ist dabei zunächst die mehrstufige, komponenten-basierte Technologie, die verschiedenste strukturelle Abstufungen von kompletten Lernobjekten und Werkzeugsätzen bis hin zu Basiskomponenten und Skripten, einzelnen Programmanweisungen, erlaubt. Zweitens erweitert die vorgeschlagene Methodik Kollaboration und individuelle Anpassung seitens der Teilnehmer eines hypermedialen Kurses auf die Software-Ebene. Komponenten werden zu verknüpfbaren Hypermedia-Objekten, die in der Kursdatenbank verwaltet und von allen Kursteilnehmern bewertet, mit Anmerkungen versehen und modifiziert werden. Neben einer detaillierten Beschreibung der Lerntechnologie und Entwurfsmuster für interaktive Lernobjekte sowie verwandte hypermediale Kurse wird der Begriff der Interaktivität verdeutlicht, indem eine kombinierte technologische und symbolische Definition von Interaktionsgraden vorgestellt und daraus ein visuelles Skriptschema abgeleitet wird, welches Funktionalität übertragbar macht. Weiterhin wird die Evolution von Hypermedia und Lehr-/Lernprogrammen besprochen, um wesentliche Techniken für interaktive, hypermediale Kurse auszuwählen. Die vorgeschlagene Architektur unterstützt mehrsprachige, alternative Inhalte, bietet konsistente Referenzen und ist leicht zu pflegen, und besitzt selbst für interaktive Inhalte Online-Assistenten. Der Einsatz hochgradiger Interaktivität in Lehr-/Lernprogrammen wird mit hypermedialen Kursen im Bereich der Computergraphik illustriert.The grand vision of educational software is that of a networked system enabling the learner to explore, discover, and construct subject matters and communicate problems and ideas with other community members. Educational material is transformed into reusable learning objects, created collaboratively by developers, educators, and designers, preserved in a digital library, and utilized, adapted, and evolved by educators and learners. Recent advances in learning technology specified reusability and interoperability in Web-based courseware. However, great interactivity is not yet considered. Each interactive learning object represents an autonomous hypermedia entity, laborious to create, impossible to interlink and to adapt in a graduated manner, and hard to specify. Dynamic attributes, the look and feel, and functionality are predefined. This work designs and realizes learning technology for Web-based courseware with special regard to highly interactive learning objects. The innovative aspect initially lies in the multi-level, component-based technology providing a graduated structuring. Components range from complex learning objects to toolkits to primitive components and scripts. Secondly, the proposed methodologies extend community support in Web-based courseware – collaboration and personalization – to the software layer. Components become linkable hypermedia objects and part of the courseware repository, rated, annotated, and modified by all community members. In addition to a detailed description of technology and design patterns for interactive learning objects and matching Web-based courseware, the thesis clarifies the denotation of interactivity in educational software formulating combined levels of technological and symbolical interactivity, and deduces a visual scripting metaphor for transporting functionality. Further, it reviews the evolution of hypermedia and educational software to extract substantial techniques for interactive Web-based courseware. The proposed framework supports multilingual, alternative content, provides link consistency and easy maintenance, and includes state-driven online wizards also for interactive content. The impact of great interactivity in educational software is illustrated with courseware in the Computer Graphics domain

First CLIPS Conference Proceedings, volume 2

The topics of volume 2 of First CLIPS Conference are associated with following applications: quality control; intelligent data bases and networks; Space Station Freedom; Space Shuttle and satellite; user interface; artificial neural systems and fuzzy logic; parallel and distributed processing; enchancements to CLIPS; aerospace; simulation and defense; advisory systems and tutors; and intelligent control

Complex adaptive systems based data integration : theory and applications

Data Definition Languages (DDLs) have been created and used to represent data in programming languages and in database dictionaries. This representation includes descriptions in the form of data fields and relations in the form of a hierarchy, with the common exception of relational databases where relations are flat. Network computing created an environment that enables relatively easy and inexpensive exchange of data. What followed was the creation of new DDLs claiming better support for automatic data integration. It is uncertain from the literature if any real progress has been made toward achieving an ideal state or limit condition of automatic data integration. This research asserts that difficulties in accomplishing integration are indicative of socio-cultural systems in general and are caused by some measurable attributes common in DDLs. This research’s main contributions are: (1) a theory of data integration requirements to fully support automatic data integration from autonomous heterogeneous data sources; (2) the identification of measurable related abstract attributes (Variety, Tension, and Entropy); (3) the development of tools to measure them. The research uses a multi-theoretic lens to define and articulate these attributes and their measurements. The proposed theory is founded on the Law of Requisite Variety, Information Theory, Complex Adaptive Systems (CAS) theory, Sowa’s Meaning Preservation framework and Zipf distributions of words and meanings. Using the theory, the attributes, and their measures, this research proposes a framework for objectively evaluating the suitability of any data definition language with respect to degrees of automatic data integration. This research uses thirteen data structures constructed with various DDLs from the 1960\u27s to date. No DDL examined (and therefore no DDL similar to those examined) is designed to satisfy the law of requisite variety. No DDL examined is designed to support CAS evolutionary processes that could result in fully automated integration of heterogeneous data sources. There is no significant difference in measures of Variety, Tension, and Entropy among DDLs investigated in this research. A direction to overcome the common limitations discovered in this research is suggested and tested by proposing GlossoMote, a theoretical mathematically sound description language that satisfies the data integration theory requirements. The DDL, named GlossoMote, is not merely a new syntax, it is a drastic departure from existing DDL constructs. The feasibility of the approach is demonstrated with a small scale experiment and evaluated using the proposed assessment framework and other means. The promising results require additional research to evaluate GlossoMote’s approach commercial use potential