Intelligent user support in graphical user interfaces

1. This paper presents a frontend to an intelligent help system based on plans called InCome (Interaction Control Manager). It visualizes user actions previously executed in a specific application as a graph structure and enables the user to navigate through this structure. A higher level of abstraction on performed user actions shows the dialog history, the interaction context and reachable goals. Finally, the user is able to act on the application via InCome by performing undo mechanisms as well as specifying user goals inferred already by the help system. 2. This paper describes the system PLUS, a plan-based help system for applications offering an object-oriented user interface. Our plan recognition process is based on a predefined static hierarchical plan base, that is modelled using a goal plan language. This language is designed to especially cope with the problems arising when plan recognition is performed in a graphical user interface environment whose interaction is based on a user-directed dialog by means of direct manipulation -- so-called Direct Manipulation User Interfaces. The plan hierarchy is entered using the interactive graphics-oriented plan editor PlanEdit+. The plan recognition module PlanRecognizer+ builds a dynamic plan base by mapping user actions to plans stored in the static plan base. The dynamic plan base contains hypotheses about tasks the user is pursuing at the moment. These plan hypotheses serve as a basis to offer various kinds of assistance to the user. A central component of our graphical help is the module InCome+. InCome+ visualizes user actions previously executed in an application as a graph structure and enables the user to navigate through this structure. A higher level of abstraction on performed actions shows the dialog history, the interaction context, and reachable goals. InCome+ offers special features like task-oriented undo und redo facilities and a context-sensitive tutor. As a substantial extension of the graphical user assistance, we integrate the presentation of animated help within PLUS. Animation sequences are generated in the context of the tasks the user is currently working on

A Method of Context-sensitive Help Generation Using a Task Project

The article presents a new method to automatic generation of help in software. Help generation is realized in the framework of the tool for development and automatic generation of user interfaces based on ontologies. The principal features of the approach are: support for context-sensitive help, automatic generation of help using a task project and an expandable system of help generation

Intelligent user support in graphical user interfaces

1. This paper presents a frontend to an intelligent help system based on plans called InCome (Interaction Control Manager). It visualizes user actions previously executed in a specific application as a graph structure and enables the user to navigate through this structure. A higher level of abstraction on performed user actions shows the dialog history, the interaction context and reachable goals. Finally, the user is able to act on the application via InCome by performing undo mechanisms as well as specifying user goals inferred already by the help system. 2. This paper describes the system PLUS, a plan-based help system for applications offering an object-oriented user interface. Our plan recognition process is based on a predefined static hierarchical plan base, that is modelled using a goal plan language. This language is designed to especially cope with the problems arising when plan recognition is performed in a graphical user interface environment whose interaction is based on a user-directed dialog by means of direct manipulation -- so-called Direct Manipulation User Interfaces. The plan hierarchy is entered using the interactive graphics-oriented plan editor PlanEdit+. The plan recognition module PlanRecognizer+ builds a dynamic plan base by mapping user actions to plans stored in the static plan base. The dynamic plan base contains hypotheses about tasks the user is pursuing at the moment. These plan hypotheses serve as a basis to offer various kinds of assistance to the user. A central component of our graphical help is the module InCome+. InCome+ visualizes user actions previously executed in an application as a graph structure and enables the user to navigate through this structure. A higher level of abstraction on performed actions shows the dialog history, the interaction context, and reachable goals. InCome+ offers special features like task-oriented undo und redo facilities and a context-sensitive tutor. As a substantial extension of the graphical user assistance, we integrate the presentation of animated help within PLUS. Animation sequences are generated in the context of the tasks the user is currently working on

Planbasierte graphische Hilfe in objektorientierten Benutzungsoberflächen

Wir stellen das System PLUS vor, ein planbasiertes graphisches Hilfesystem für Applikationen mit einer objektorientierten Benutzerschnittstelle. Es werden die Hilfekomponente InCome+, die Animationskomponente und der graphik-orientierte Planeditor PlanEdit+ beschrieben. PlanEdit+ ermöglicht den interaktiven Aufbau der hierarchischen Planbasis, die die Grundlage für den Planerkennungsprozeß bildet.Eine zentrale Komponente der graphischen Hilfe in unserem System stellt das Modul InCome+ dar, das den Interaktionskontext des Benutzers visualisiert und darüberhinaus weitere Features wie semantische Undo- und Redo-Möglichkeiten und einen kontext-sensitiven Tutor zur Verfügung stellt. Als wesentliche Erweiterung der graphischen Benutzerunterstützung wird innerhalb von PLUS die Präsentation animierter Hilfe integriert. Es werden Benutzeraktionen simuliert, indem eine Animation über die aktuelle Benutzerschnittstelle gelagert wird. Die Animationssequenz wird im Kontext der aktuell vom Benutzer verfolgten Aufgabe generiert

Perspektiven zur Kombination von automatischem Animationsdesign und planbasierter Hilfe

Aktuelle Themen auf dem Gebiet der intelligenten Benutzerschnittstellen behandeln derzeit die automatische Planung multimodaler Präsentationen. Hierbei stand bisher im wesentlichen die koordinierte Generierung von Text und Graphik im Vordergrund. In Zukunft wird hier aufgrund der Komplexität der zu präsentierenden Information zunehmend auch die Einbeziehung realistischer animierter 3D-Graphiken gefordert sein. Einen anderen wichtigen Forschungsschwerpunkt bildet der Einsatz graphischer Ausgabekomponenten für planbasierte Hilfesysteme. Die vorliegende Arbeit hat zum Ziel zunächst einen Überblick über den derzeitigen Stand der Forschung in diesen beiden Bereichen zu geben, als auch neue Anforderungen an die automatische Animationsgenerierung und an Systeme zur planbasierten graphischen Hilfe zu formulieren. Anschließend wollen wir, basierend auf Ergebnissen und Erfahrungen aus WIP und PLUS, Perspektiven für eine mögliche Weiterentwicklung und Integration von Techniken der Animationsplanung und graphischen Hilfe präsentieren

Supporting GUI exploration through USS tool

Advances in usability and design techniques (e.g. user-centered design) try to facilitate the use of interactive systems. However, users still have to adapt to interactive systems, i.e. they have to learn the steps required to accomplish a task either by trial and error or by obtaining help. While advanced users are usually able to adapt without much effort this is far from being the case with beginners. Some interactive systems offer different interaction styles in an attempt to meet the needs of all types of user but this is not the case with all interactive systems. In this sense, we present an approach to support the use of any interactive system making use of enriched models and picture-driven computing to achieve tasks automation. The USS tool (User Support System) is the basis to the adaptation of interactive systems accordingly to the users' needs. The approach provides the foundation for the addition of help (based on demonstration) to any graphical user interfaces (GUI) facilitating learning and use. The work is illustrated by a case study and completed with a preliminary user evaluation which provides insights about the validity of the approach.info:eu-repo/semantics/publishedVersio

Sikuli: Using GUI screenshots for search and automation

We present Sikuli, a visual approach to search and automation of graphical user interfaces using screenshots. Sikuli allows users to take a screenshot of a GUI element (such as a toolbar button, icon, or dialog box) and query a help system using the screenshot instead of the element's name. Sikuli also provides a visual scripting API for automating GUI interactions, using screenshot patterns to direct mouse and keyboard events. We report a web-based user study showing that searching by screenshot is easy to learn and faster to specify than keywords. We also demonstrate several automation tasks suitable for visual scripting, such as map navigation and bus tracking, and show how visual scripting can improve interactive help systems previously proposed in the literature

Perspektiven zur Kombination von automatischem Animationsdesign und planbasierter Hilfe

Aktuelle Themen auf dem Gebiet der intelligenten Benutzerschnittstellen behandeln derzeit die automatische Planung multimodaler Präsentationen. Hierbei stand bisher im wesentlichen die koordinierte Generierung von Text und Graphik im Vordergrund. In Zukunft wird hier aufgrund der Komplexität der zu präsentierenden Information zunehmend auch die Einbeziehung realistischer animierter 3D-Graphiken gefordert sein. Einen anderen wichtigen Forschungsschwerpunkt bildet der Einsatz graphischer Ausgabekomponenten für planbasierte Hilfesysteme. Die vorliegende Arbeit hat zum Ziel zunächst einen Überblick über den derzeitigen Stand der Forschung in diesen beiden Bereichen zu geben, als auch neue Anforderungen an die automatische Animationsgenerierung und an Systeme zur planbasierten graphischen Hilfe zu formulieren. Anschließend wollen wir, basierend auf Ergebnissen und Erfahrungen aus WIP und PLUS, Perspektiven für eine mögliche Weiterentwicklung und Integration von Techniken der Animationsplanung und graphischen Hilfe präsentieren

Towards Low-cost Feature-rich Web User Interfaces

Web-based user interfaces are used widely. They are replacing conventional desktop-based user interfaces in many domains and are emerging as front-ends for online businesses. The technologies for web user interfaces have advanced considerably to support high-quality user interfaces. However, the usability of web interfaces continues to be an issue. We still encounter web forms where basic interactive features are missing or work unexpectedly. User interface is a costly and error-prone area of software construction. This is particularly true for web user interfaces. They are typically implemented with fewer reusable components on programmers' toolboxes than conventional user interfaces built using user interface frameworks such as Windows Forms, Cocoa, and Qt. Consequently, web interface programmers tend to struggle with low productivity, or low quality and high defect rates. This thesis focuses on property models, a declarative approach to programming user interfaces. In this approach, common user interface behaviors are automatically derived from the specifications of the data manipulated by user interfaces. The approach aims to reuse user interface algorithms that are common across interfaces and allow the programmers to focus on application-specific concerns. This thesis work is a part of project "hotdrink," a JavaScript implementation of the property model system, which has the goal of providing the benefits of property models for web interfaces. This thesis builds on previous work on property models, and adds to it three reusable help and convenience features, which can be especially useful for web forms. In particular, this thesis describes the generic mechanisms of the following user interface features: (1) validating data coming from a user and presenting useful messages that help the user to fix errors, (2) controlling the flow of data through "pinning," and (3) canceling the user's previous actions through undoing. The main contributions of the thesis are the mechanisms and the software architecture that enable implementing these behaviors in a reusable manner. This thesis also presents several examples to illustrate the benefits of the proposed mechanisms