Denotation and connotation in the human-computer interface: The ‘Save as...’ command

This paper presents a semiotic technique as a means of exploring meaning and understanding in interface design and use. This is examined through a study of the interaction between the ‘file’ metaphor and ‘save as’ command metaphor. The behaviour of these (from a functional or computational basis) do not exactly match, or map onto, the meaning of the metaphor. We examine both the denotation of a term to the user, i.e. its literal meaning to that person, and the term’s connotations, i.e. any other meanings associated with the term. We suggest that the technique applied is useful in predicting future problems with understanding the use of metaphor at the interface and with designing appropriate signification for human-computer interaction. Variation in connotation was expected but a more fundamental difference in denotation was also uncovered. Moreover, the results clearly demonstrate that consistency in the denotation of a term is critical in achieving a good user understanding of the command

A semiotic approach to the use of metaphor in human-computer interfaces

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University, 20/9/1999.Although metaphors are common in computing, particularly in human-computer interfaces, opinion is divided on their usefulness to users and little evidence is available to help the designer in choosing or implementing them. Effective use of metaphors depends on understanding their role in the computer interface, which in tum means building a model of the metaphor process. This thesis examines some of the approaches which might be taken in constructing such a model before choosing one and testing its applicability to interface design. Earlier research into interface metaphors used experimental psychology techniques which proved useful in showing the benefits or drawbacks of specific metaphors, but did not give a general model of the metaphor process. A cognitive approach based on mental models has proved more successful in offering an overall model of the process, although this thesis questions whether the researchers tested it adequately. Other approaches which have examined the metaphor process (though not in the context of human-computer interaction) have come from linguistic fields, most notably semiotics, which extends linguistics to non-verbal communication and thus could cover graphical user interfaces (GUls). The main work described in this thesis was the construction of a semiotic model of human-computer interaction. The basic principle of this is that even the simplest element of the user interface will signify many simultaneous meanings to the user. Before building the model, a set of assertions and questions was developed to check the validity of the principles on which the model was based. Each of these was then tested by a technique appropriate to the type of issue raised. Rhetorical analysis was used to establish that metaphor is commonplace in command-line languages, in addition to its more obvious use in GUIs. A simple semiotic analysis, or deconstruction, of the Macintosh user interface was then used to establish the validity of viewing user interfaces as semiotic systems. Finally, an experiment was carried out to test a mental model approach proposed by previous researchers. By extending their original experiment to more realistically complex interfaces and tasks and using a more typical user population, it was shown that users do not always develop mental models of the type proposed in the original research. The experiment also provided evidence to support the existence of multiple layers of signification. Based on the results of the preliminary studies, a simple means of testing the semiotic model's relevance to interface design was developed, using an interview technique. The proposed interview technique was then used to question two groups of users about a simple interface element. Two independent researchers then carried out a content analysis of the responses. The mean number of significations in each interview, as categorised by the researchers, was 15. The levels of signification were rapidly revealed, with the mean time for each interview being under two minutes, providing effective evidence that interfaces signify many meanings to users, a substantial number of which are easily retrievable. It is proposed that the interview technique could provide a practical and valuable tool for systems analysis and interface designers. Finally, areas for further research are proposed, in particular to ascertain how the model and the interview technique could be integrated with other design methods

Information, knowledge and the context of interaction

Representing knowledge as information content alone is insufficient in providing us with an understanding of the world around us. A combination of context as well as reasoning of the information content is fundamental to representing knowledge, within information –based systems. The field of knowledge representation and knowledge management has thus far been concerned with providing structures and theories that can lead to some form of qualified intelligent reasoning, and contextualised information. By drawing upon previous research and applying and extending concepts of Semiotics and Symbiosis from the interaction design school of thought, this paper presents a conceptual framework for establishing the interplay between knowledge and users of knowledge via information systems constructs. Subsequently, by drawing upon notions of interfaces to knowledge, a conceptual framework which describes the relationship between the semiotic, symbiotic and interface to knowledge presented, along with a discussion of contemporary issues common to the field of knowledge management is developed

Screen-based musical instruments as semiotic machines

The ixi software project started in 2000 with the intention to explore new interactive patterns and virtual interfaces in computer music software. The aim of this paper is not to describe these programs, as they have been described elsewhere, but rather explicate the theoretical background that underlies the design of these screen-based instruments. After an analysis of the similarities and differences in the design of acoustic and screen-based instruments, the paper describes how the creation of an interface is essentially the creation of a semiotic system that affects and influences the musician and the composer. Finally the terminology of this semiotics is explained as an interaction model

Cognitively Sensitive User Interface for Command and Control Applications

While there are broad guidelines for display or user interface design, creating effective human-computer interfaces for complex, dynamic systems control is challenging. Ad hoc approaches which consider the human as an afterthought are limiting. This research proposed a systematic approach to human / computer interface design that focuses on both the semantic and syntactic aspects of display design in the context of human-in-the-loop supervisory control of intelligent, autonomous multi-agent simulated unmanned aerial vehicles (UAVs). A systematic way to understand what needs to be displayed, how it should be displayed, and how the integrated system needs to be assessed is outlined through a combination of concepts from naturalistic decision making, semiotic analysis, and situational awareness literature. A new sprocket-based design was designed and evaluated in this research. For the practical designer, this research developed a systematic, iterative design process: design using cognitive sensitive principles, test the new interface in a laboratory situation; bring in subject matter experts to examine the interface in isolation; and finally, incorporate the resulting feedback into a full-size simulation. At each one of these steps, the operator, the engineer and the designer reexamined the results

Knowledge Construction of 3D Geometry Concepts and Processes Within a Virtual Reality Learning Environment

A consensus has emerged within the mathematics education community about the limitations of traditional approaches for teaching and learning 3D geometry. Therefore, it has been suggested that new approaches based on the use of computers need to be adopted. One such new approach that has been proposed utilises Virtual Reality Learning Environment (VRLE). This paper reports on the initial phases of a research study whose major aim is to design and evaluate a VRLE to facilitate the construction of knowledge about 3D geometry concepts and processes. This research study investigates two primary school students’ construction of 3D geometry knowledge whilst engaged within a VRLE developed by the researcher. A design experiments research methodology was employed in this study. This is research that iterates through cycles of design and research with the objective of arriving at theoretical and design principles that will have application both within and beyond the immediate research study. Therefore, the results being reported in this paper will be used to inform the modification not only of the VRLE but also of theoretical frameworks underlying the design and implementation of VRLEs

Building a Socio-technical Perspective of Community Resilience with a Semiotic Approach

Situated in the diversity and adversity of real-life contexts facing crisis situations, this research aims at boosting the resilience process within communities supported by digital and social technology. In this paper, eight community leaders in different parts of the world are invited to express their issues and wishes regarding the support of technology to face social challenges. Methods and artefacts based on the Organisational Semiotics (OS) and the Socially-Aware computing have been applied to analyse and consolidate this data. By providing both a systemic view of the problem and also leading to the identification of requirements, the analysis evidences some benefits of the OS-based approach to consolidate perspectives from different real-life scenarios towards building a socio-technical solution

Toward a Semiotic Framework for Using Technology in Mathematics Education: The Case of Learning 3D Geometry

This paper proposes and examines a semiotic framework to inform the use of technology in mathematics education. Semiotics asserts that all cognition is irreducibly triadic, of the nature of a sign, fallible, and thoroughly immersed in a continuing process of interpretation (Halton, 1992). Mathematical meaning-making or meaningful knowledge construction is a continuing process of interpretation within multiple semiotic resources including typological, topological, and social-actional resources. Based on this semiotic framework, an application named VRMath has been developed to facilitate the learning of 3D geometry. VRMath utilises innovative virtual reality (VR) technology and integrates many semiotic resources to form a virtual reality learning environment (VRLE) as well as a mathematical microworld (Edwards, 1995) for learning 3D geometry. The semiotic framework and VRMath are both now being evaluated and will be re-examined continuously