1 research outputs found
MutaBot: A Mutation Testing Approach for Chatbots
Mutation testing is a technique aimed at assessing the effectiveness of test
suites by seeding artificial faults into programs. Although available for many
platforms and languages, no mutation testing tool is currently available for
conversational chatbots, which represent an increasingly popular solution to
design systems that can interact with users through a natural language
interface. Note that since conversations must be explicitly engineered by the
developers of conversational chatbots, these systems are exposed to specific
types of faults not supported by existing mutation testing tools.
In this paper, we present MutaBot, a mutation testing tool for conversational
chatbots. MutaBot addresses mutations at multiple levels, including
conversational flows, intents, and contexts. We designed the tool to
potentially target multiple platforms, while we implemented initial support for
Google Dialogflow chatbots. We assessed the tool with three Dialogflow chatbots
and test cases generated with Botium, revealing weaknesses in the test suites.Comment: 5 pages, 2 figures, 2024 IEEE/ACM 46th International Conference on
Software Engineering: Companion Proceedings (ICSE-Companion '24