2 research outputs found
Chatbot integration in few patterns
Chatbots are software agents that are able to interact with humans in natural
language. Their intuitive interaction paradigm is expected to significantly
reshape the software landscape of tomorrow, while already today chatbots are
invading a multitude of scenarios and contexts. This article takes a
developer's perspective, identifies a set of architectural patterns that
capture different chatbot integration scenarios, and reviews state-of-the-art
development aids.Comment: prior version was an incomplete early draft; current version includes
changes in references, appropriate acknowledgement; and minor revision
VASTA: A Vision and Language-assisted Smartphone Task Automation System
We present VASTA, a novel vision and language-assisted Programming By
Demonstration (PBD) system for smartphone task automation. Development of a
robust PBD automation system requires overcoming three key challenges: first,
how to make a particular demonstration robust to positional and visual changes
in the user interface (UI) elements; secondly, how to recognize changes in the
automation parameters to make the demonstration as generalizable as possible;
and thirdly, how to recognize from the user utterance what automation the user
wishes to carry out. To address the first challenge, VASTA leverages
state-of-the-art computer vision techniques, including object detection and
optical character recognition, to accurately label interactions demonstrated by
a user, without relying on the underlying UI structures. To address the second
and third challenges, VASTA takes advantage of advanced natural language
understanding algorithms for analyzing the user utterance to trigger the VASTA
automation scripts, and to determine the automation parameters for
generalization. We run an initial user study that demonstrates the
effectiveness of VASTA at clustering user utterances, understanding changes in
the automation parameters, detecting desired UI elements, and, most
importantly, automating various tasks. A demo video of the system is available
here: http://y2u.be/kr2xE-FixjIComment: Submitted to ACM IUI'20, 10 figures, 11 page