Evaluating Variable-Length Multiple-Option Lists in Chatbots and Mobile Search

In recent years, the proliferation of smart mobile devices has lead to the gradual integration of search functionality within mobile platforms. This has created an incentive to move away from the "ten blue links'' metaphor, as mobile users are less likely to click on them, expecting to get the answer directly from the snippets. In turn, this has revived the interest in Question Answering. Then, along came chatbots, conversational systems, and messaging platforms, where the user needs could be better served with the system asking follow-up questions in order to better understand the user's intent. While typically a user would expect a single response at any utterance, a system could also return multiple options for the user to select from, based on different system understandings of the user's intent. However, this possibility should not be overused, as this practice could confuse and/or annoy the user. How to produce good variable-length lists, given the conflicting objectives of staying short while maximizing the likelihood of having a correct answer included in the list, is an underexplored problem. It is also unclear how to evaluate a system that tries to do that. Here we aim to bridge this gap. In particular, we define some necessary and some optional properties that an evaluation measure fit for this purpose should have. We further show that existing evaluation measures from the IR tradition are not entirely suitable for this setup, and we propose novel evaluation measures that address it satisfactorily.Comment: 4 pages, in Proceeding of SIGIR 201

DART: A Lightweight Quality-Suggestive Data-to-Text Annotation Tool

We present a lightweight annotation tool, the Data AnnotatoR Tool (DART), for the general task of labeling structured data with textual descriptions. The tool is implemented as an interactive application that reduces human efforts in annotating large quantities of structured data, e.g. in the format of a table or tree structure. By using a backend sequence-to-sequence model, our system iteratively analyzes the annotated labels in order to better sample unlabeled data. In a simulation experiment performed on annotating large quantities of structured data, DART has been shown to reduce the total number of annotations needed with active learning and automatically suggesting relevant labels.Comment: Accepted to COLING 2020 (selected as outstanding paper

Fast and easy language understanding for dialog systems with Microsoft Language Understanding Intelligent Service (LUIS)

Abstract With Language Understanding Intelligent Service (LUIS), developers without machine learning expertise can quickly build and use language understanding models specific to their task. LUIS is entirely cloud-based: developers log into a website, enter a few example utterances and their labels, and then deploy a model to an HTTP endpoint. Utterances sent to the endpoint are logged and can be efficiently labeled using active learning. Visualizations help identify issues, which can be resolved by either adding more labels or by giving hints to the machine learner in the form of features. Altogether, a developer can create and deploy an initial language understanding model in minutes, and easily maintain it as usage of their application grows

Proceedings of the 1st joint workshop on Smart Connected and Wearable Things 2016

These are the Proceedings of the 1st joint workshop on Smart Connected and Wearable Things (SCWT'2016, Co-located with IUI 2016). The SCWT workshop integrates the SmartObjects and IoWT workshops. It focusses on the advanced interactions with smart objects in the context of the Internet-of-Things (IoT), and on the increasing popularity of wearables as advanced means to facilitate such interactions

Implementación de un chatbot y su influencia en el proceso de atención a las unidades descentralizadas de la SUTRAN 2016

RESUMEN La presente investigación tiene como Problema general ¿Cuál es la influencia de la implementación de un chatbot en el proceso de atención a las unidades descentralizadas de la SUTRAN, 2016? El objetivo general es determinar la influencia de la implementación de un chatbot en el proceso de atención a las unidades descentralizadas de la SUTRAN, 2016. El estudio fue de tipo Experimental y el diseño Cuasi - Experimental, La población en estudio estuvo conformada por todos los procesos de atención a las unidades descentralizadas de la Superintendencia de Transporte Terrestre de Personas, Carga y Mercancía, 2016 y el muestreo es no probabilístico por conveniencia de tamaño igual a 55, los datos se obtuvieron a través de la realización de un cuestionario a partir de una escala tipo Likert. Se tabularon y se procesaron los datos en el paquete estadístico SPSS Versión 24.0. Los resultados indican una influencia positiva de la implementación de un chatbot en el proceso de atención a las unidades descentralizadas de la SUTRAN, 2016. (sig. bilateral = 0.018 < 0.05; Rho = 0.317*). PALABRAS CLAVE: chatbot, IA, PLN, ML, proceso.ABSTRACT The current research has a general problem. What is the influence of the implementation of a chatbot in the process of attention to the decentralized units of SUTRAN 2016? The general objective is to determine the influence of the implementation of a chatbot in the process of attention to the decentralized units of the SUTRAN 2016. The study was of an Experimental type and a quasi - Experimental design, the population in study was composed of all the processes of attention to decentralized units of the Superintendency of road transport of people, goods and cargo 2016 and the sampling is not probabilistic for convenience of a size equal to 55, the data was obtained through the completion of a questionnaire from a Likert type scale, the data was tabulated and processed in the statistical package SPSS Version 24.0. The results indicate a positive influence of the implementation of a chatbot in the process of attention to the decentralized units of SUTRAN 2016. (sig. bilateral = 0.018 < 0.05; Rho = 0.317*). KEYWORDS: chatbot, AI, NLP, ML, process