A Framework for Specifying and Monitoring User Tasks

Knowledge about user task execution can help systems better reason about when to interrupt users. To enable recognition and forecasting of task execution, we develop a novel framework for specifying and monitoring user task sequences. For task specification, our framework provides an XML-based language with tags inspired by regular expressions. For task monitoring, our framework provides an event handler that manages events from any instrumented application and a monitor that observes a user's transitions within and among specified tasks. The monitor supports multiple active tasks and multiple instances of the same task. The use of our framework will enable systems to consider a user's position within a task model when reasoning about when to interrupt

Voice Controlled Music Sequencer

This project served as a proof-of-concept to determine the validity of a voice-controlled music sequencer. Working with Digital Performer and Dragon Dictate software, the project explored the history of voice-recognition technology, and the feasibility of this technology within a music sequencing environment

Ambient Gestures

We present Ambient Gestures, a novel gesture-based system designed to support ubiquitous ‘in the environment’ interactions with everyday computing technology. Hand gestures and audio feedback allow users to control computer applications without reliance on a graphical user interface, and without having to switch from the context of a non-computer task to the context of the computer. The Ambient Gestures system is composed of a vision recognition software application, a set of gestures to be processed by a scripting application and a navigation and selection application that is controlled by the gestures. This system allows us to explore gestures as the primary means of interaction within a multimodal, multimedia environment. In this paper we describe the Ambient Gestures system, define the gestures and the interactions that can be achieved in this environment and present a formative study of the system. We conclude with a discussion of our findings and future applications of Ambient Gestures in ubiquitous computing

Automating iterative tasks with programming by demonstration

Programming by demonstration is an end-user programming technique that allows people to create programs by showing the computer examples of what they want to do. Users do not need specialised programming skills. Instead, they instruct the computer by demonstrating examples, much as they might show another person how to do the task. Programming by demonstration empowers users to create programs that perform tedious and time-consuming computer chores. However, it is not in widespread use, and is instead confined to research applications that end users never see. This makes it difficult to evaluate programming by demonstration tools and techniques. This thesis claims that domain-independent programming by demonstration can be made available in existing applications and used to automate iterative tasks by end users. It is supported by Familiar, a domain-independent, AppleScript-based programming-by-demonstration tool embodying standard machine learning algorithms. Familiar is designed for end users, so works in the existing applications that they regularly use. The assertion that programming by demonstration can be made available in existing applications is validated by identifying the relevant platform requirements and a range of platforms that meet them. A detailed scrutiny of AppleScript highlights problems with the architecture and with many implementations, and yields a set of guidelines for designing applications that support programming-by-demonstration. An evaluation shows that end users are capable of using programming by demonstration to automate iterative tasks. However, the subjects tended to prefer other tools, choosing Familiar only when the alternatives were unsuitable or unavailable. Familiar's inferencing is evaluated on an extensive set of examples, highlighting the tasks it can perform and the functionality it requires

Language design for a personal learning environment design language

Approaching technology-enhanced learning from the perspective of a learner, we foster the idea of learning environment design, learner interactions, and tool interoperability. In this paper, we shortly summarize the motivation for our personal learning environment approach and describe the development of a domain-specific language for this purpose as well as its realization in practice. Consequently, we examine our learning environment design language according to its lexis and syntax, the semantics behind it, and pragmatical aspects within a first prototypic implementation. Finally, we discuss strengths, problematic aspects, and open issues of our approach

APEX: cross-platform analysis program for EXAFS

We have developed version 1.0 of a freely available (including source code) suite of basic X-Ray Absorption Fine Structure (XAFS) data analysis programs for data reduction and single scattering analysis. This package is based on the University of Washington (UW/NRL) Fortran 77 programs that are available on the International XAFS Society (IXS) database, complemented by a graphical TCL/TK scripting language based user interface which runs virtually unchanged between platforms, using the native look and feel of the corresponding platform. The package has been tested on MacOS 8.1, Linux, IRIX, Windows95 and NT. Particular emphasis is placed on simplicity, reliability, and (sup)portability. APEX 1.0 in its current form is suitable for routine data analysis and training, and systematic improvements and extensions to the underlying codes are planned

CLIPS, AppleEvents, and AppleScript: Integrating CLIPS with commercial software

Many of today's intelligent systems are comprised of several modules, perhaps written in different tools and languages, that together help solve the user's problem. These systems often employ a knowledge-based component that is not accessed directly by the user, but instead operates 'in the background' offering assistance to the user as necessary. In these types of modular systems, an efficient, flexible, and eady-to-use mechanism for sharing data between programs is crucial. To help permit transparent integration of CLIPS with other Macintosh applications, the AI Research Branch at NASA Ames Research Center has extended CLIPS to allow it to communicate transparently with other applications through two popular data-sharing mechanisms provided by the Macintosh operating system: Apple Events (a 'high-level' event mechanism for program-to-program communication), and AppleScript, a recently-released scripting language for the Macintosh. This capability permits other applications (running on either the same or a remote machine) to send a command to CLIPS, which then responds as if the command were typed into the CLIPS dialog window. Any result returned by the command is then automatically returned to the program that sent it. Likewise, CLIPS can send several types of Apple Events directly to other local or remote applications. This CLIPS system has been successfully integrated with a variety of commercial applications, including data collection programs, electronics forms packages, DBMS's, and email programs. These mechanisms can permit transparent user access to the knowledge base from within a commercial application, and allow a single copy of the knowledge base to service multiple users in a networked environment

Quasi-variances in Xlisp-Stat and on the web

The most common summary of a fitted statistical model, a list of parameter estimates and standard errors, does not give the precision of estimated combinations of the parameters, such as differences or ratios. For this, covariances also are needed; but space constraints typically mean that the full covariance matrix cannot routinely be reported. In the important case of parameters associated with the discrete levels of an experimental factor or with a categorical classifying variable, the identifiable parameter combinations are linear contrasts. The QV Calculator computes "quasi-variances" which may be used as an alternative summary of the precision of the estimated parameters. The summary based on quasi-variances is simple and permits good approximation of the standard error of any desired contrast. The idea of such a summary has been suggested by Ridout (1989) and, under the name "floating absolute risk", by Easton, Peto & Babiker (1991). It applies to a wide variety of statistical models, including linear and nonlinear regressions, generalized-linear and GEE models, Cox proportional-hazard models for survival data, generalized additive models, etc. The QV Calculator is written in Xlisp-Stat (Tierney,'90) and can be used either directly by users who have access to Xlisp-Stat or through a web interface by those who do not. The user either supplies the covariance matrix for the effect parameters of interest, or, if using Xlisp-Stat directly, can generate that matrix by interaction with a model object.

GALLAG Strip: A Mobile, Programming With Demonstration Environment for Sensor-Based Context-Aware Application Programming

abstract: The Game As Life - Life As Game (GALLAG) project investigates how people might change their lives if they think of and/or experience their life as a game. The GALLAG system aims to help people reach their personal goals through the use of context-aware computing, and tailored games and applications. To accomplish this, the GALLAG system uses a combination of sensing technologies, remote audio/video feedback, mobile devices and an application programming interface (API) to empower users to create their own context-aware applications. However, the API requires programming through source code, a task that is too complicated and abstract for many users. This thesis presents GALLAG Strip, a novel approach to programming sensor-based context-aware applications that combines the Programming With Demonstration technique and a mobile device to enable users to experience their applications as they program them. GALLAG Strip lets users create sensor-based context-aware applications in an intuitive and appealing way without the need of computer programming skills; instead, they program their applications by physically demonstrating their envisioned interactions within a space using the same interface that they will later use to interact with the system, that is, using GALLAG-compatible sensors and mobile devices. GALLAG Strip was evaluated through a study with end users in a real world setting, measuring their ability to program simple and complex applications accurately and in a timely manner. The evaluation also comprises a benchmark with expert GALLAG system programmers in creating the same applications. Data and feedback collected from the study show that GALLAG Strip successfully allows users to create sensor-based context-aware applications easily and accurately without the need of prior programming skills currently required by the GALLAG system and enables them to create almost all of their envisioned applications.Dissertation/ThesisM.S. Computer Science 201