An Evaluation of Three Online Chatbots

Chatbots enable machines to emulate human conversation. While research has been done to examine how human-like communication with chatbots can be, heretofore comparisons of the systems with humans have not accounted for abnormal behavior from the users. For example, the people using the chatbot might be lying or trying to, in turn, imitate a computer’s response. Results of a study comparing transcripts from three chatbots and two humans show that student evaluators were able to correctly identify two computer transcripts, but failed on one. Further, they incorrectly guessed that one of the humans was a chatbot. The study also presents a detailed analysis of the 11 responses from the agents

A Statistical Turing Test for Generative Models

The emergence of human-like abilities of AI systems for content generation in domains such as text, audio, and vision has prompted the development of classifiers to determine whether content originated from a human or a machine. Implicit in these efforts is an assumption that the generation properties of a human are different from that of the machine. In this work, we provide a framework in the language of statistical pattern recognition that quantifies the difference between the distributions of human and machine-generated content conditioned on an evaluation context. We describe current methods in the context of the framework and demonstrate how to use the framework to evaluate the progression of generative models towards human-like capabilities, among many axes of analysis

A Photo-realistic Voice-bot

Technology is at the point where systems are capable of synthesizing video of human actors indistinguishably from ones in which the actor is present. This research investigates whether or not it is possible to use this technology in order to create a system which, allows video generation of a human actor, that is able to interact with a user through speech in real-time, whilst also remaining indistinguishable from a real human actor. In other words, a photo-realistic voicebot. The work discusses the motivations and ethics, but also presents and tests a prototype system. The prototype aims to take advantage of the latest in real-time video manipulation software to create a natural sounding conversation with an artificially synthesized video

Chatbot for Learning English

Cieľom tejto práce bolo vytvoriť konverzačný systém ktorého účel bude zlepšovanie jazykových schopností používateľa. Systém je zložený zo štyroch hlavných podsystémov kde má každý samostatnú funkciu. Získavanie dát používaných na trénovanie konverzácií, konverzácia samotná, korekcia gramatického stavu používateľského vstupu a samostatné cvičenia. Vytvorený systém 26 testujúcich používateľov ohodnotilo kladne až neutrálne a 80.8% z nich považovalo systém za použiteľný pri učení angličtiny.Goal of this thesis was to create conversational system whose purpose will be improving users foreign language level. System is composed of four main subsystems, where each one has standalone funcionality. Retrieving data used for training purposes, conversation, grammar correction of users inputs and exerises. Created system was tested by 26 users who scored system from possitive to neutral. 80.8% users considered created system to be usable in process of learning English.

A Primer on Seq2Seq Models for Generative Chatbots

The recent spread of Deep Learning-based solutions for Artificial Intelligence and the development of Large Language Models has pushed forwards significantly the Natural Language Processing area. The approach has quickly evolved in the last ten years, deeply affecting NLP, from low-level text pre-processing tasks –such as tokenisation or POS tagging– to high-level, complex NLP applications like machine translation and chatbots. This paper examines recent trends in the development of open-domain data-driven generative chatbots, focusing on the Seq2Seq architectures. Such architectures are compatible with multiple learning approaches, ranging from supervised to reinforcement and, in the last years, allowed to realise very engaging open-domain chatbots. Not only do these architectures allow to directly output the next turn in a conversation but, to some extent, they also allow to control the style or content of the response. To offer a complete view on the subject, we examine possible architecture implementations as well as training and evaluation approaches. Additionally, we provide information about the openly available corpora to train and evaluate such models and about the current and past chatbot competitions. Finally, we present some insights on possible future directions, given the current research status

Desenvolvimento de Chatbots para responder a perguntas frequentes

Hoje em dia, seja qual for o sítio que visitamos na Internet, dificilmente não nos deparamos com um Chatbot: desde as pequenas caixas de diálogo que nos assistem em compras online, até aos agentes virtuais que executam os pedidos que fazemos através de comandos de voz. Nesta dissertação, o objetivo principal é a implementação de um Chatbot para responder a perguntas frequentes. Para tal, foi realizado um estudo sobre o Processamento de Linguagem Natural, formas de preparar o texto para que fique pronto a ser utilizado pelo computador, distâncias que podem ser aplicadas a texto, similaridade semântica e, por fim, qual o impacto da remoção das palavras vazias. Após a realização deste estudo e obtenção da informação necessária, foi implementado um Chatbot para responder a perguntas frequentes, independentemente do tema dessas perguntas. Esse Chatbot foi, posteriormente, testado em perguntas frequentes relacionadas com o Instituto da Vinha e do Vinho; Abstract: Chatbots development to answer frequently asked questions Nowadays, whatever the site we visit on the Internet, we hardly encounter a Chatbot: from the small dialog boxes that assist us in online shopping to the virtual agents that execute the orders we make through voice commands. In this dissertation, the main goal is to implement a Chatbot to answer frequently asked questions. With this in mind, a study was carried out on natural language processing, ways to prepare the text so that it is ready for use by the computer, distances that can be applied to text, semantic similarity and, finally, what is the impact of removing stopwords. After conducting this and obtaining the necessary information, a Chatbot was implemented to answer frequently asked questions, regardless of the domain of those questions. This Chatbot was subsequently tested on frequently asked questions related to the Institute of Vine and Wine

Turingův test: filozofické aspekty umělé inteligence

Disertační práce se zabývá problematikou připisování myšlení jiným entitám, a to pomocí imitační hry navržené v roce 1950 britským filosofem Alanem Turingem. Jeho kritérium, známé v dějinách filosofie jako Turingův test, je podrobeno detailní analýze. Práce popisuje nejen původní námitky samotného Turinga, ale především pozdější diskuse v druhé polovině 20. století. Největší pozornost je věnována těmto kritikám: Lucasova matematická námitka využívající Gödelovu větu o neúplnosti, Searlův argument čínského pokoje konstatující nedostatečnost syntaxe pro sémantiku, Blockův návrh na použití brutální síly pro řešení imitační hry, Frenchova teorie subkognitivních informací a Michieho skepticismus ohledně možnosti umělého vědomí. Závěr práce zachycuje současný stav recepce Turingova testu a představuje pokusy o jeho praktickou realizaci, například v každoroční soutěži o Loebnerovu cenu. Autor práce zastává názor, že ani po více než šedesáti letech od uveřejnění Turingova paradigmatického eseje stále neexistují žádné vážné důvody pro zamítnutí jeho tvrzení. Tradiční komputační funkcionalismus možná není ideální teorií vysvětlující činnost myslí a jako slibnější se může jevit vývoj v neurálních vědách, ale Turingův test je přesto užitečným a snad i jediným nástrojem pro detekci inteligence u lidmi vytvořených strojů

The Total Turing Test and the Loebner Prize

is the first, and only regular, competition based on the Turing Test, but in order to stage the competition various modifications to the original test have been made. In particular, the Grand Prize has a controversial and as yet undefined Audio-Visual condition attached to it. This paper discusses the value of the test with and without the A/V condition, and makes a proposal about what the general nature of the A/V test should be. 1