A grammatical specification of human-computer dialogue

The Seeheim Model of human-computer interaction partitions an interactive application into a user-interface, a dialogue controller and the application itself. One of the formal techniques of implementing the dialogue controller is based on context-free grammars and automata. In this work, we modify an off-the-shelf compiler generator (YACC) to generate the dialogue controller. The dialogue controller is then integrated into the popular X-window system, to create an interactive-application generator. The actions of the user drive the automaton, which in turn controls the application

A Notation and Framework for Dialog Flow Control in Web Applications

The usability of web applications today often suffers from the page-based medium’s lack of intrinsic support for hierarchical dialog sequences mirroring the parent-child relationships between dialog boxes in window-based user interfaces. For multi-channel applications, an additional challenge lies in reconciling the device-independent business logic with the device-specific interaction patterns necessitated by different clients’ input/output capabilities.We therefore present a graphical Dialog Flow Notation that allows the specification of nestable dialog sequences for different presentation channels. These specifications serve as input for a Dialog Control Framework that controls the dialog flows of complex web applications

DESIGNING CONCEPTUAL MODELS OF DIALOGS: A CASE FOR DIALOG CHARTS

The conceptual design of user interfaces focuses on the specification of the structure of the dialog, independent of any particular implementation approach. While there is common agreement with respect to the importance of this activity, adequate methods and tools to support it are generally unavailable. The Dialog Charts (DCs) presented in this paper address this problem -- they support the conceptual design of dialog control structures. The DCs combine visual modeling (i.e., diagraming) with widely accepted design principles and an explicit model of dialog structures. As no clear evaluation criteria exist in this evolving area of dialog design, the preliminary assessment of the DCs takes the form of contrasting them with representative alternative design tools based on Augmented Transition Networks or Backus-Naur Form grammars. The DCs overcome some of the problems that seem to limit the usefulness of comparable approaches. An empirical investigation of the usable power of the DCs is currently underway at New York University, and a summary of this research activity concludes the paper.Information Systems Working Papers Serie

CSE: U: Mixed-initiative Personal Assistant Agents

Specification and implementation of flexible human-computer dialogs is challenging because of the complexity involved in rendering the dialog responsive to a vast number of varied paths through which users might desire to complete the dialog. To address this problem, we developed a toolkit for modeling and implementing task-based, mixed-initiative dialogs based on metaphors from lambda calculus. Our toolkit can automatically operationalize a dialog that involves multiple prompts and/or sub-dialogs, given a high-level dialog specification of it. The use of natural language with the resulting dialogs makes the flexibility in communicating user utterances commensurate with that in dialog completion paths—an aspect missing from commercial assistants like Siri. Our results demonstrate that the dialogs authored with our toolkit support the end user’s completion of a human-computer dialog in a manner that is most natural to them—in a mixed-initiative fashion—that resembles human-human interaction

Interaction objects in the MADE multimedia environment

In order to support flexible interaction in interactive multimedia systems, we have defined general interaction objects that allow dynamic and transparent coupling of user interaction to media objects and other types of objects. The interaction objects are programmable in that they may use scripting languages for defining their behavior. This mechanism can be used to glue existing application capabilities together to form new end-user functionality

Natural Language, Mixed-Initiative Personal Assistant Agents

The increasing popularity and use of personal voice assistant technologies, such as Siri and Google Now, is driving and expanding progress toward the long-term and lofty goal of using artificial intelligence to build human-computer dialog systems capable of understanding natural language. While dialog-based systems such as Siri support utterances communicated through natural language, they are limited in the flexibility they afford to the user in interacting with the system and, thus, support primarily action-requesting and information-seeking tasks. Mixed-initiative interaction, on the other hand, is a flexible interaction technique where the user and the system act as equal participants in an activity, and is often exhibited in human-human conversations. In this paper, we study user support for mixed-initiative interaction with dialog-based systems through natural language using a bag-of-words model and k-nearest-neighbor classifier. We study this problem in the context of a toolkit we developed for automated, mixed-initiative dialog system construction, involving a dialog authoring notation and management engine based on lambda calculus, for specifying and implementing task-based, mixed-initiative dialogs. We use ordering at Subway through natural language, human-computer dialogs as a case study. Our results demonstrate that the dialogs authored with our toolkit support the end user\u27s completion of a natural language, human-computer dialog in a mixed-initiative fashion. The use of natural language in the resulting mixed-initiative dialogs afford the user the ability to experience multiple self-directed paths through the dialog and makes the flexibility in communicating user utterances commensurate with that in dialog completion paths---an aspect missing from commercial assistants like Siri

Executable Semantics of Recursively Nestable Dialog Flow Specifications for Web Applications

Information systems for the support of complex business processes are often equipped with web-based front-ends to allow convenient user access. To produce executable specifications of the users’ interactions with such web-based applications, we use a visual language that enables developers to model their complex dialog structures. In this paper, we introduce the formal semantics of the core constructs of this Dialog Flow Notation: We define its syntax in terms of invariants about the permitted elements and their relations, and show how any words of the language (i.e. any syntactically correct dialog flow specifications) can be mapped to a deterministic pushdown automaton whose behavior defines the notation’s semantics. This gives us and other tool developers a formal basis for the design and implementation of tools and frameworks that mirror the precise meaning of all DFN constructs

Mobile Business Processes

Today’s global markets demand global processes. Increasingly, these processes are not only distributed, but also contain mobile aspects. We discuss two challenges brought about by these mobile business processes: Firstly, the need to specify the distribution of processes across several sites, and secondly, the need to specify the dialog flows of the applications implementing those processes on mobile devices. To remedy the first challenge, we give an overview of the Process Landscaping method with its support for refining processes across multiple abstraction layers and associating their activities and objects with distinguished locations. Next, we present a Dialog Flow Notation and Dialog Control Framework for the specification and management of complex hypertext-based dialog flows. These tools allow developers to build user interfaces for mobile client devices with different input/output capabilities, which all access the same application logic on a central server

DESIGNING INTERACTIVE USER INTERFACES: DIALOG CHARTS AND AN ASSESSMENT OF THEIR USE IN SPECIFYING CONCEPTUAL MODELS OF DIALOGS

The conceptual design of user interfaces focuses on arriving at a specification of the structure of the dialog, independent of any particular implementation approach. There is common agreement as to the importance of this activity to both IS professionals and end-users, but few -- if any -- modeling methods were developed to specifically support the process of conceptual design, and the usefulness of such methods has not been adequately addressed. This paper introduces the Dialog Charts (DCs), and documents a preliminary examination of their perceived usefulness by designers of user/system interaction who actually used them. The DCs yield high level dialog schemas that are abstract enough to support the conceptual design of dialog control structures. In a uniform diagramming framework they combine the concept of dialog independence, distinguish between the dialog parties, provide for hierarchical decomposition and enforce a structured control flow. The usefulness of the DCs has been studied empirically in a qualitative inquiry. Recalled experiences of designers were captured and analyzed to ascertain the concept of usability, as well as assess the usability of the DCs. Usability has emerged from this study as a set of 38 concerns that operationalizes the broader aspects of purpose of use, design stage, impact on product structure, impact on design process, and attitudinal patterns. In general, the Dialog Charts were found by these dialog designers to be a useful, exhibiting the essential attributes of tools for conceptual modeling.Information Systems Working Papers Serie