Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook

In recent years, reinforcement learning and bandits have transformed a wide range of real-world applications including healthcare, finance, recommendation systems, robotics, and last but not least, the speech and natural language processing. While most speech and language applications of reinforcement learning algorithms are centered around improving the training of deep neural networks with its flexible optimization properties, there are still many grounds to explore to utilize the benefits of reinforcement learning, such as its reward-driven adaptability, state representations, temporal structures and generalizability. In this survey, we present an overview of recent advancements of reinforcement learning and bandits, and discuss how they can be effectively employed to solve speech and natural language processing problems with models that are adaptive, interactive and scalable.Comment: To appear in Expert Systems with Applications. Accompanying INTERSPEECH 2022 Tutorial on the same topic. Including latest advancements in large language models (LLMs

Confusion Modelling - An Estimation by Semantic Embeddings

Approaching the task of coherence assessment of a conversation from its negative perspective ‘confusion’ rather than coherence itself, has been attempted by very few research works. Influencing Embeddings to learn from similarity/dissimilarity measures such as distance, cosine similarity between two utterances will equip them with the semantics to differentiate a coherent and an incoherent conversation through the detection of negative entity, ‘confusion’. This research attempts to measure coherence of conversation between a human and a conversational agent by means of such semantic embeddings trained from scratch by an architecture centralising the learning from the distance between the embeddings. State of the art performance of general BERT’s embeddings and state of the art performance of ConveRT’s conversation specific embeddings in addition to the GLOVE embeddings are also tested upon the laid architecture. Confusion, being a more sensible entity, real human labelling performance is set as the baseline to evaluate the models. The base design resulted in not such a good performance against the human score but the pre-trained embeddings when plugged into the base architecture had performance boosts in a particular order from lowest to highest, through BERT, GLOVE and ConveRT. The intuition and the efficiency of the base conceptual design is proved of its success when the variant having the ConveRT embeddings plugged into the base design, outperformed the original ConveRT’s state of art performance on generating similarity scores. Though a performance comparable to real human performance was not achieved by the models, there witnessed a considerable overlapping between the ConveRT variant and the human scores which is really a great positive inference to be enjoyed as achieving human performance is always the state of art in any research domain. Also, from the results, this research joins the group of works claiming BERT to be unsuitable for conversation specific modelling and embedding works

An Object-oriented methodology for modern user interface development.

by Lam Siu Hong.Thesis (M.Phil.)--Chinese University of Hong Kong, 1991.Includes bibliographical references.Chapter Chapter1 --- Introduction --- p.1Chapter 1.1 --- Software Development Crisis of User Interface --- p.1Chapter 1.2 --- Objectives and Scope of Interests --- p.1Chapter 1.3 --- Overview of the Thesis --- p.2Chapter Chapter2 --- Background and Problems --- p.4Chapter 2.1 --- Categories of User Interfaces --- p.4Chapter 2.2 --- Trends of User Interfaces --- p.6Chapter 2.3 --- Some other Desirable Features and Problems of UI Development --- p.7Chapter 2.3.1 --- Separating UI from Application --- p.7Chapter 2.3.1.1 --- Benefits of Separable UIs and Applications --- p.7Chapter 2.3.1.2 --- Requirements of Complete Separation --- p.10Chapter 2.3.2 --- Instant Continuous Feedback --- p.12Chapter 2.3.2.1 --- Problems of Linguistic Model on World Model Type UIs --- p.12Chapter 2.3.3 --- Undo and Recovery --- p.15Chapter 2.3.4 --- Iterative Design through Rapid Protyping --- p.16Chapter Chapter3 --- An Object-Oriented Model for Model World User Interfaces Development --- p.18Chapter 3.1 --- Features of UIs to be supported by the Model --- p.18Chapter 3.2 --- A Linkage Model for Separating UI from Application --- p.19Chapter 3.2.1 --- Communication Messages Modeled using an Object Oriented Approach --- p.20Chapter 3.2.2 --- A Sample Message --- p.22Chapter 3.2.3 --- Linkage in a Distributed Heterogenous Environment --- p.24Chapter 3.2.4 --- Comparing the Linkage Model with the Application Interface Model in Seeheim's UI Model --- p.25Chapter 3.3 --- An Object-Oriented Model for Supporting Multiple Feedbacks and Multi-thread dialogue --- p.26Chapter 3.3.1 --- An Overview of the Model --- p.27Chapter 3.3.2 --- Objects on the Lexical Layer --- p.28Chapter 3.3.3 --- Roles of Presentation Objects --- p.29Chapter 3.3.4 --- Syntactic Objects --- p.31Chapter 3.3.5 --- Interaction Objects --- p.32Chapter 3.3.6 --- Interaction between objetcs and Linkage Component --- p.33Chapter 3.3.7 --- Multiple U-tubes Ladder for Supporting Multiple Feedbacks --- p.33Chapter 3.3.8 --- Recovery through a Generic UNDO stack --- p.35Chapter 3.3.9 --- Dialogue Control in an Object --- p.37Chapter 3.3.10 --- Interactive Objects --- p.39Chapter 3.3.11 --- An Architecture for Supporting Multi-thread Dialogue --- p.40Chapter 3.4 --- Basic Object Structure --- p.42Chapter 3.4.1 --- An Event Model for Dialogue Control --- p.43Chapter 3.4.2 --- Maintain Consistency through ε-rules --- p.45Chapter 3.4.3 --- An Example of an Inner Object Specification --- p.47Chapter 3.4.4 --- Pre and Post Condition of Action --- p.49Chapter 3.4.5 --- Automatic Message Routing --- p.49Chapter 3.5 --- Systematic Approach to UI Specification --- p.50Chapter Chapter4 --- User Interface Framework Design --- p.52Chapter 4.1 --- A Framework for UI Development --- p.52Chapter 4.1.1 --- Abstract Base Class for Each Object Type --- p.54Chapter 4.1.2 --- A Kernel for Message Routing --- p.60Chapter 4.1.3 --- Interaction Knowledge Base --- p.63Chapter 4.1.4 --- A Dynamic View of UI Objects --- p.64Chapter 4.1.5 --- Switch Box Mechanism for Dialogue Switching --- p.66Chapter 4.1.6 --- Software IC Construction --- p.68Chapter 4.2 --- Summaries of Object-Object UI Model and UI Framework --- p.70Chapter 4.2.1 --- A New Approach to User Interface Development 、 --- p.70Chapter 4.2.2 --- Feautures of UI Development provided by the Object-Object UI Model and UI Framework --- p.71Chapter Chapter5 --- Implementation --- p.73Chapter 5.1 --- Implementation of Framework in Microsoft Window Environment --- p.73Chapter 5.1.1 --- Implementation of automatic message routing through dynamic binding --- p.73Chapter 5.1.2 --- A generic message structure --- p.75Chapter 5.1.3 --- A meta class for object communication --- p.76Chapter 5.1.4 --- Software component of UI framework in Microsoft Window environment --- p.76Chapter 5.2 --- A Simple Stock Market Decision Support System (SSMDSS) --- p.77Chapter 5.2.1 --- UI Specification --- p.81Chapter 5.2.2 --- UI features supported by SSMDSS --- p.87Chapter Chapter6 --- Results --- p.89Chapter 6.1 --- Facts discovered --- p.89Chapter 6.1.1 --- Asynchronous and synchronous communication among objects --- p.89Chapter 6.1.2 --- Flexibility of the C+ + language --- p.90Chapter 6.2 --- Technical Problems Encountered --- p.91Chapter 6.2.1 --- Problems from Implementation Platform --- p.91Chapter 6.2.2 --- Problems due to Object Decomposition in an Interactive Object in SSMDSS --- p.92Chapter 6.3 --- Objectives accomplished by the Object-Oriented UI Model indicated by SSMDSS --- p.93Chapter Chapter7 --- Conclusion --- p.95Chapter 7.1 --- Thesis Summary --- p.95Chapter 7.2 --- Merits and Demerit of the Object-Oriented UI Model --- p.96Chapter 7.3 --- Cost of the Object-Oriented UI Model --- p.96Chapter 7.4 --- Future work --- p.97AppendixChapter A1 --- An Alogrithm for Converting Transition Network Diagram to Event Response Language --- p.A1Chapter A2 --- An Object-Oriented Software Development --- p.A4Chapter A2.1 --- Traditional Non Object-Oriented Software Development --- p.A4Chapter A2.2 --- An Object-Oriented Software Development --- p.A6Chapter A3 --- Vienna Development Method (VDM) --- p.A8Chapter A3.1 --- An Overview of VDM --- p.A8Chapter A3.2 --- Apply VDM to Object-Oriented UI model --- p.A10Chapter A4 --- Glossaries and Terms --- p.A12Referenc

An Analysis of Current Trends and Potential Applications of Sociohydrology

Humans live in an ever-changing, increasingly-complicated world. As the fundamental understanding of systems changes, those tasked with managing these may find themselves faced with new situations with problems more complex than previously thought. Recently, an increased interest in fully integrating society into hydrologic research has given rise to a new subfield of hydrology: sociohydrology. I performed a meta-analysis of the sociohydrologic literature from its coinage in 2012 until early August 2017. There has been a steady increase in the number of sociohydrology-related publications since 2012. Articles constituted over 75% of all publications. Multidisciplinary collaborations were common for sociohydrologic publications; however, authorship was heavily biased towards engineering and the natural sciences. Studies were largely conceptual, and the most common foci included modeling, flooding, land use-land cover change, agriculture, water security, and rivers or streams. I developed a conceptual framework for constructing a model capable of analyzing long-term success of rural infrastructure projects. I did so in the context of the flood-reducing capabilities of drainage infrastructure on Texas colonias. This model was designed to estimate long-term flood risk on development. I developed a conceptual framework for constructing a model capable of analyzing long-term urban natural disaster vulnerability. I did so in the context of potential contaminant risk in the event of a rainfall-induced industrial contaminant spillage in the Beaumont-Port Arthur metropolitan area. This model was designed to estimate two varieties of storm hazards: risk of inundation by flood waters and risk of contamination by industrial plant spillage. This study provides information on the development of sociohydrology and conceptualizes potential applications of its methodology