Deep Reinforcement Learning for Chatbots Using Clustered Actions and Human-Likeness Rewards

Training chatbots using the reinforcement learning paradigm is challenging due to high-dimensional states, infinite action spaces and the difficulty in specifying the reward function. We address such problems using clustered actions instead of infinite actions, and a simple but promising reward function based on human-likeness scores derived from human-human dialogue data. We train Deep Reinforcement Learning (DRL) agents using chitchat data in raw text—without any manual annotations. Experimental results using different splits of training data report the following. First, that our agents learn reasonable policies in the environments they get familiarised with, but their performance drops substantially when they are exposed to a test set of unseen dialogues. Second, that the choice of sentence embedding size between 100 and 300 dimensions is not significantly different on test data. Third, that our proposed human-likeness rewards are reasonable for training chatbots as long as they use lengthy dialogue histories of ≥10 sentences

A Study on Dialogue Reward Prediction for Open-Ended Conversational Agents

The amount of dialogue history to include in a conversational agent is often underestimated and/or set in an empirical and thus possibly naive way. This suggests that principled investigations into optimal context windows are urgently needed given that the amount of dialogue history and corresponding representations can play an important role in the overall performance of a conversational system. This paper studies the amount of history required by conversational agents for reliably predicting dialogue rewards. The task of dialogue reward prediction is chosen for investigating the effects of varying amounts of dialogue history and their impact on system performance. Experimental results using a dataset of 18K human-human dialogues report that lengthy dialogue histories of at least 10 sentences are preferred (25 sentences being the best in our experiments) over short ones, and that lengthy histories are useful for training dialogue reward predictors with strong positive correlations between target dialogue rewards and predicted ones

Ensemble-Based Deep Reinforcement Learning for Chatbots

Trainable chatbots that exhibit fluent and human-like conversations remain a big challenge in artificial intelligence. Deep Reinforcement Learning (DRL) is promising for addressing this challenge, but its successful application remains an open question. This article describes a novel ensemble-based approach applied to value-based DRL chatbots, which use finite action sets as a form of meaning representation. In our approach, while dialogue actions are derived from sentence clustering, the training datasets in our ensemble are derived from dialogue clustering. The latter aim to induce specialised agents that learn to interact in a particular style. In order to facilitate neural chatbot training using our proposed approach, we assume dialogue data in raw text only – without any manually-labelled data. Experimental results using chitchat data reveal that (1) near human-like dialogue policies can be induced, (2) generalisation to unseen data is a difficult problem, and (3) training an ensemble of chatbot agents is essential for improved performance over using a single agent. In addition to evaluations using held-out data, our results are further supported by a human evaluation that rated dialogues in terms of fluency, engagingness and consistency – which revealed that our proposed dialogue rewards strongly correlate with human judgements

Roadmap on measurement technologies for next generation structural health monitoring systems

Structural health monitoring (SHM) is the automation of the condition assessment process of an engineered system. When applied to geometrically large components or structures, such as those found in civil and aerospace infrastructure and systems, a critical challenge is in designing the sensing solution that could yield actionable information. This is a difficult task to conduct cost-effectively, because of the large surfaces under consideration and the localized nature of typical defects and damages. There have been significant research efforts in empowering conventional measurement technologies for applications to SHM in order to improve performance of the condition assessment process. Yet, the field implementation of these SHM solutions is still in its infancy, attributable to various economic and technical challenges. The objective of this Roadmap publication is to discuss modern measurement technologies that were developed for SHM purposes, along with their associated challenges and opportunities, and to provide a path to research and development efforts that could yield impactful field applications. The Roadmap is organized into four sections: distributed embedded sensing systems, distributed surface sensing systems, multifunctional materials, and remote sensing. Recognizing that many measurement technologies may overlap between sections, we define distributed sensing solutions as those that involve or imply the utilization of numbers of sensors geometrically organized within (embedded) or over (surface) the monitored component or system. Multi-functional materials are sensing solutions that combine multiple capabilities, for example those also serving structural functions. Remote sensing are solutions that are contactless, for example cell phones, drones, and satellites. It also includes the notion of remotely controlled robots

Corrugated Photoactive Thin Films for Flexible Strain Sensor

In this study, a flexible strain sensor is devised using corrugated bilayer thin films consisting of poly(3-hexylthiophene) (P3HT) and poly(3,4-ethylenedioxythiophene)-polystyrene(sulfonate) (PEDOT:PSS). In previous studies, the P3HT-based photoactive non-corrugated thin film was shown to generate direct current (DC) under broadband light, and the generated DC voltage varied with applied tensile strain. Yet, the mechanical resiliency and strain sensing range of the P3HT-based thin film strain sensor were limited due to brittle non-corrugated thin film constituents. To address this issue, it is aimed to design a mechanically resilient strain sensor using corrugated thin film constituents. Buckling is induced to form corrugation in the thin films by applying pre-strain to the substrate, where the thin films are deposited, and releasing the pre-strain afterwards. It is known that corrugated thin film constituents exhibit different optical and electronic properties from non-corrugated ones. Therefore, to design the flexible strain sensor, it was studied to understand how the applied pre-strain and thickness of the PEDOT:PSS conductive thin film affects the optical and electrical properties. In addition, strain effect was investigated on the optical and electrical properties of the corrugated thin film constituents. Finally, flexible strain sensors are fabricated by following the design guideline, which is suggested from the studies on the corrugated thin film constituents, and the DC voltage strain sensing capability of the flexible strain sensors was validated. As a result, the flexible strain sensor exhibited a tensile strain sensing range up to 5% at a frequency up to 15 Hz with a maximum gauge factor ~7

The 2016 ASME conference on smart materials, adaptive structures and intelligent systems: Symposium on structural health monitoring

The 2016 ASME Conference on Smart Materials, Adaptive Structures and Intelligent Systems (SMASIS) was held from 28 to 30 September 2016, in Stowe, Vermont. SMASIS is a leading conference on adaptive structures and materials systems that attracts approximately 300 attendees each year with over 250 technical presentations. The conference provides a unique forum bringing together scientists, faculty, and graduate students from mechanical, aerospace, electrical and civil engineering, materials science, and mathematics, whose research spans topics in adaptive structures and smart material systems