Чатбот на основі навчання з підкріпленням

Дипломна робота: 136 с., 15 рис., 21табл., 6 додатки, 19 джерел. Об’єктом дослідження є задача симулювання дискусії чи розмови з людиною відповідно до заданого практичного завдання. Предметом дослідження є методи і алгоритми навчання з підкріпленням для побудови діалогової системи у вигляді чат-боту. Метою дипломної роботи є розробка та навчання програмного продукту для реалізації діалогу природньою мовою між діалоговою системою та симулятором користувача з використанням методів глибокого навчання з підкріпленням. У роботі розроблено і навчено програмний продукт у вигляді чат-бота на основі глибокого навчання з підкріпленням, який демонструє роботу системи замовлення квитків в кінотеатрах. Побудовано симулятор користувача, який відіграв роль середовища для агента, який здатний завершувати 89% відсотків діалогів успіхів. Для реалізації інтерфейсу створено бот в месенджері Telegram. Чат-бот розгорнуто за допомогою сервісу Heroku для забезпечення безперервного доступу та можливості використання програми з будь-якого гаджету. Програма написана мовою Python з використанням PyCharm та Google Colaboratory. При написанні роботи використано наукові статті в галузі машинного навчання, глибокого навчання та глибокого навчання з підкріпленням.The work consist of 136 pages 15 images 21 tables 19 sources The theme: «A deep reinforcement learning chat-bot». .The object of research is the task of simulating a discussion or conversation with a person in accordance with a given practical task. The subject of the study is methods and algorithms of reinforcements learning for building a dialogue system in the form of chat-bot. The purpose of the thesis is to develop and teach the software product to implement a dialogue in the natural language between the dialogue system and the user's simulator using the methods of deep reinforcement learning. In the work, the software product in the form of a chat-bot was developed and trained on the basis of deep reinforcement learning, which demonstrates the work of the system of booking tickets in cinemas. A user simulator has been constructed, which has played the role of environment for agent that can complete 89% of the success dialogs. To implement the interface, a bot has been created in Telegram Messenger. The Chat-box is hosted with the help of Heroku to provide continuous access and the ability to use the program from any gadget. The program is written in Python using PyCharm and Google Colaboratory. Scientific articles in the field of machine learning, deep learning and deep reinforcement learning were used during writing the work

A case study on the importance of belief state representation for dialogue policy management

A key component of task-oriented dialogue systems is the belief state representation, since it directly affects the policy learning efficiency. In this paper, we propose a novel, binary, compact, yet scalable belief state representation. We compare the stan- dard verbose belief state representation (268 dimensions) with the domain-independent representation (57 dimensions) and the proposed representation (13 or 4 dimensions). To test those representations, the recently introduced Advantage Actor Critic (A2C) algorithm is exploited. The latter has not been tested before for any representation apart from the verbose one. We study the effect of the belief state representation within A2C un- der 0%, 15%, 30%, and 45% semantic error rate and conclude that the novel binary representation in general outperforms both the domain-independent and the verbose belief state represen- tation. Further, the robustness of the binary representation is tested under more realistic scenarios with mismatched semantic error rates, within the A2C and DQN algorithms. The results indicate that the proposed compact, binary representation per- forms better or similarly to the other representations, being an efficient and promising alternative to the full belief

A case study on the importance of belief state representation for dialogue policy management

Summarization: A key component of task-oriented dialogue systems is the belief state representation, since it directly affects the policy learning efficiency. In this paper, we propose a novel, binary, compact, yet scalable belief state representation. We compare the standard verbose belief state representation (268 dimensions) with the domain-independent representation (57 dimensions) and the proposed representation (13 or 4 dimensions). To test those representations, the recently introduced Advantage Actor Critic (A2C) algorithm is exploited. The latter has not been tested before for any representation apart from the verbose one. We study the effect of the belief state representation within A2C under 0%, 15%, 30%, and 45% semantic error rate and conclude that the novel binary representation in general outperforms both the domain-independent and the verbose belief state representation. Further, the robustness of the binary representation is tested under more realistic scenarios with mismatched semantic error rates, within the A2C and DQN algorithms. The results indicate that the proposed compact, binary representation performs better or similarly to the other representations, being an efficient and promising alternative to the full belief.Παρουσιάστηκε στο: 19th Annual Conference of the International Speech Communicatio