ISML: an interface specification meta-language

In this paper we present an abstract metaphor model situated within a model-based user interface framework. The inclusion of metaphors in graphical user interfaces is a well established, but mostly craft-based strategy to design. A substantial body of notations and tools can be found within the model-based user interface design literature, however an explicit treatment of metaphor and its mappings to other design views has yet to be addressed. We introduce the Interface Specification Meta-Language (ISML) framework and demonstrate its use in comparing the semantic and syntactic features of an interactive system. Challenges facing this research are outlined and further work proposed

COMM Notation for Specifying Collaborative and MultiModal Interactive Systems

International audienceMulti-user multimodal interactive systems involve multiple users that can use multiple interaction modalities. Although multi-user multimodal systems are becoming more prevalent (especially multimodal systems involving multitouch surfaces), their design is still ad-hoc without properly keeping track of the design process. Addressing this issue of lack of design tools for multi-user multimodal systems, we present the COMM (Collaborative and MultiModal) notation and its on-line editor for specifying multi-user multimodal interactive systems. Extending the CTT notation, the salient features of the COMM notation include the concepts of interactive role and modal task as well as a refinement of the temporal operators applied to tasks using the Allen relationships. A multimodal military command post for the control of unmanned aerial vehicles (UAV) by two operators is used to illustrate the discussion

Formalizing graphical notations

The thesis describes research into graphical notations for software engineering, with a principal interest in ways of formalizing them. The research seeks to provide a theoretical basis that will help in designing both notations and the software tools that process them. The work starts from a survey of literature on notation, followed by a review of techniques for formal description and for computational handling of notations. The survey concentrates on collecting views of the benefits and the problems attending notation use in software development; the review covers picture description languages, grammars and tools such as generic editors and visual programming environments. The main problem of notation is found to be a lack of any coherent, rigorous description methods. The current approaches to this problem are analysed as lacking in consensus on syntax specification and also lacking a clear focus on a defined concept of notated expression. To address these deficiencies, the thesis embarks upon an exploration of serniotic, linguistic and logical theory; this culminates in a proposed formalization of serniosis in notations, using categorial model theory as a mathematical foundation. An argument about the structure of sign systems leads to an analysis of notation into a layered system of tractable theories, spanning the gap between expressive pictorial medium and subject domain. This notion of 'tectonic' theory aims to treat both diagrams and formulae together. The research gives details of how syntactic structure can be sketched in a mathematical sense, with examples applying to software development diagrams, offering a new solution to the problem of notation specification. Based on these methods, the thesis discusses directions for resolving the harder problems of supporting notation design, processing and computer-aided generic editing. A number of future research areas are thereby opened up. For practical trial of the ideas, the work proceeds to the development and partial implementation of a system to aid the design of notations and editors. Finally the thesis is evaluated as a contribution to theory in an area which has not attracted a standard approach

The development of an executable graphical notation for describing direct manipulation interfaces : a dissertation presented in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science at Massey University

The research reported in this thesis involves the development of an executable semi-formal graphical notation, Lean Cuisine+, for describing the underlying behaviour of event-based direct manipulation interfaces, and the application of the notation both in 'reverse engineering', and during the early design phase of the interface development life cycle. A motivation for the research stems from the need for tools and techniques to support high level interface design. The research supports and brings together a number of views concerning the requirements of notations at this level. These are that a notation should be semi-formal, graphical, executable, and object-oriented, and that to be most effective it should be targeted at a specific category of interaction. The Lean Cuisine+ notation meets all these criteria, the underlying meneme model matching closely with the selection-based nature of direct manipulation interfaces. Lean Cuisine+ is a multi-layered notation, and is a development of Lean Cuisine (Apperley & Spence, 1989). The base layer is a tree diagram which captures part of the behaviour of an interface in terms of constraints and dependencies between selectable dialogue primitives. Further constraints and dependencies associated with the dynamics of the interface are captured through overlays to the basic tree diagram. An orthogonal task layer captures any temporal relationships between primitive task actions, and provides a link with higher level functionality. Lean Cuisine+ is able to combine both static and dynamic modelling in a coherent manner, thus avoiding the necessity of employing separate and possibly disjoint models at the early design stage. A software support environment for the notation is also specified and partially prototyped. The research demonstrates the advantages of a notation which can be executed to provide limited but valid early simulation of the dynamic behaviour of the interface under design. A mapping from Lean Cuisine+ to a dialogue implementation language, DAL (Anderson, 1993), is also developed in support of the view that a multi-notational approach to interface development is required, and that it must be possible to move easily from initial specification to prototyping and implementation. The Lean Cuisine+ descriptions of aspects of the Apple Macintosh interface included in the thesis show the notation to be capable of handling a range of direct manipulation interaction styles and a variety of interface objects. A five stage methodology for the construction of Lean Cuisine+ specifications for new interfaces is also developed, and applied in two case studies

Freeform User Interfaces for Graphical Computing

報告番号: 甲15222 ; 学位授与年月日: 2000-03-29 ; 学位の種別: 課程博士 ; 学位の種類: 博士(工学) ; 学位記番号: 博工第4717号 ; 研究科・専攻: 工学系研究科情報工学専

Three Dimensional Software Modelling

Traditionally, diagrams used in software systems modelling have been two dimensional (2D). This is probably because graphical notations, such as those used in object-oriented and structured systems modelling, draw upon the topological graph metaphor, which, at its basic form, receives little benefit from three dimensional (3D) rendering. This paper presents a series of 3D graphical notations demonstrating effective use of the third dimension in modelling. This is done by e.g., connecting several graphs together, or in using the Z co-ordinate to show special kinds of edges. Each notation combines several familiar 2D diagrams, which can be reproduced from 2D projections of the 3D model. 3D models are useful even in the absence of a powerful graphical workstation: even 2D stereoscopic projections can expose more information than a plain planar diagram

Fifty years of the Psychology of Programming

This paper reflects on the evolution (past, present and future) of the ‘psychology of programming' over the 50 year period of this anniversary issue. The International Journal of Human-Computer Studies (IJHCS) has been a key venue for much seminal work in this field, including its first foundations, and we review the changing research concerns seen in publications over these five decades. We relate this thematic evolution to research taking place over the same period within more specialist communities, especially the Psychology of Programming Interest Group (PPIG), the Empirical Studies of Programming series (ESP), and the ongoing community in Visual Languages and Human-Centric Computing (VL/HCC). Many other communities have interacted with psychology of programming, both influenced by research published within the specialist groups, and in turn influencing research priorities. We end with an overview of the core theories that have been developed over this period, as an introductory resource for new researchers, and also with the authors’ own analysis of key priorities for future research

e-COMM, un éditeur pour spécifier l'interaction multimodale et multiutilisateur

National audienceCet article présente l'éditeur e-COMM pour la spécification de systèmes interactifs multimodaux et multiutilisateurs à l'aide de la notation COMM. La majorité des notations dédiées à la conception des collecticiels offrent des moyens limités pour décrire l'interaction multimodale. Ainsi la notation COMM comble ce manque en introduisant de nouveaux concepts (tâche modale et rôle interactif) pour lier ces deux aspects : l'interaction multimodale et multiutilisateur. L'éditeur e-COMM vise alors deux objectifs : founir un outil facilement accessible et aider le concepteur à se concentrer sur la tâche d'édition en reposant autant que possible sur la manipulation directe

The Design and evaluation of the specification framework for user interface design

This thesis presentsthe design and evaluation of an interface specification meta-language(ISML) that has been developed to explicitly support metaphor abstractions in a model-based, user interface design framework. The application of metaphor to user interface design is widely accepted within the HCI community, yet despite this, there exists relatively little formal support for user interface design practitioners. With the increasing range and power of user interface technologies made widely available comes the opportunity for the designof sophisticated, new forms of interactive environments. The inter-disciplinary nature of HCI offers many approaches to user interface design that include views on tasks, presentationand dialogue architectures and various domain models. Notations and tools that support these views vary equally, ranging from craft-based approachesthrough to computational or tool- based support and formal methods. Work in these areas depicts gradual cohesion of a number of these design views, but do not currently explicitly specify the application of metaphorical concepts in graphical user interface design. Towards addressing this omission, ISML was developed based on (and extending) some existing model- based user interface design concepts. Abstractions of metaphor and other interface design views are captured in the ISML framework using the extensible mark-up language(XML). A six-month case study, developing the `Urban Shout Cast' application is used to evaluate ISML. Two groups of four software engineers developed a networked, multi-user, virtual radio-broadcasting environment. A qualitative analysis examines both how each group developed metaphor designs within the ISML framework and also their perceptions of its utility and practicality. Subsequent analysis on the specification data from both groups reveals aspects of the project's design that ISML captured and those that were missed. Finally, the extent to which ISML can currently abstract the metaphors used in the case study is assessed through the development of a unified `meta-object' model. The results of the case study show that ISML is capable of expressing many of the features of each group's metaphor design, as well as highlighting important design considerations during development. Furthermore, it has been shown, in principle, how an underlying metaphor abstraction can be mapped to two different implementations. Evaluation of the case study also includes important design lessons: ISML metaphor models can be both very large and difficult to separate from other design views, some of which are either weakly expressed or unsupported. This suggests that the appropriate mappings between design abstractions cannot always be easily anticipated, and that understanding the use of model-based specifications in user interface design projects remains a challenge to the HCI community

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research