Denotation and connotation in the human-computer interface: The ‘Save as...’ command

This paper presents a semiotic technique as a means of exploring meaning and understanding in interface design and use. This is examined through a study of the interaction between the ‘file’ metaphor and ‘save as’ command metaphor. The behaviour of these (from a functional or computational basis) do not exactly match, or map onto, the meaning of the metaphor. We examine both the denotation of a term to the user, i.e. its literal meaning to that person, and the term’s connotations, i.e. any other meanings associated with the term. We suggest that the technique applied is useful in predicting future problems with understanding the use of metaphor at the interface and with designing appropriate signification for human-computer interaction. Variation in connotation was expected but a more fundamental difference in denotation was also uncovered. Moreover, the results clearly demonstrate that consistency in the denotation of a term is critical in achieving a good user understanding of the command

Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students

Blind and visually impaired mathematics students must rely on accessible materials such as tactile diagrams to learn mathematics. However, these compensatory materials are frequently found to offer students inferior opportunities for engaging in mathematical practice and do not allow sensorily heterogenous students to collaborate. Such prevailing problems of access and interaction are central concerns of Universal Design for Learning (UDL), an engineering paradigm for inclusive participation in cultural praxis like mathematics. Rather than directly adapt existing artifacts for broader usage, UDL process begins by interrogating the praxis these artifacts serve and then radically re-imagining tools and ecologies to optimize usability for all learners. We argue for the utility of two additional frameworks to enhance UDL efforts: (a) enactivism, a cognitive-sciences view of learning, knowing, and reasoning as modal activity; and (b) ethnomethodological conversation analysis (EMCA), which investigates participants’ multimodal methods for coordinating action and meaning. Combined, these approaches help frame the design and evaluation of opportunities for heterogeneous students to learn mathematics collaboratively in inclusive classrooms by coordinating perceptuo-motor solutions to joint manipulation problems. We contextualize the thesis with a proposal for a pluralist design for proportions, in which a pair of students jointly operate an interactive technological device

The Emerging Nature of Participation in Multispecies Interaction Design

Interactive technology has become integral part of daily life for both humans and animals, with animals often interacting with technologized environments on behalf of humans. For some, animals' participation in the design process is essential to design technology that can adequately support their activities. For others, animals' inability to understand and control design activities inevitably stands in the way of multispecies participatory practices. Here, we consider the essential elements of participation within interspecies interactions and illustrate its emergence, in spite of contextual constraints and asymmetries. To move beyond anthropomorphic notions of participation, and consequent anthropocentric practices, we propose a broader participatory model based on indexical semiosis, volition and choice; and we highlight dimensions that could define inclusive participatory practices more resilient to the diversity of understandings and goals among part-taking agents, and better able to account for the contribution of diverse, multispecies agents in interaction design and beyond

Symbol Emergence in Robotics: A Survey

Humans can learn the use of language through physical interaction with their environment and semiotic communication with other people. It is very important to obtain a computational understanding of how humans can form a symbol system and obtain semiotic skills through their autonomous mental development. Recently, many studies have been conducted on the construction of robotic systems and machine-learning methods that can learn the use of language through embodied multimodal interaction with their environment and other systems. Understanding human social interactions and developing a robot that can smoothly communicate with human users in the long term, requires an understanding of the dynamics of symbol systems and is crucially important. The embodied cognition and social interaction of participants gradually change a symbol system in a constructive manner. In this paper, we introduce a field of research called symbol emergence in robotics (SER). SER is a constructive approach towards an emergent symbol system. The emergent symbol system is socially self-organized through both semiotic communications and physical interactions with autonomous cognitive developmental agents, i.e., humans and developmental robots. Specifically, we describe some state-of-art research topics concerning SER, e.g., multimodal categorization, word discovery, and a double articulation analysis, that enable a robot to obtain words and their embodied meanings from raw sensory--motor information, including visual information, haptic information, auditory information, and acoustic speech signals, in a totally unsupervised manner. Finally, we suggest future directions of research in SER.Comment: submitted to Advanced Robotic

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: NL

Proceedings ICPW'07: 2nd International Conference on the Pragmatic Web, 22-23 Oct. 2007, Tilburg: N

An aesthetics of touch: investigating the language of design relating to form

How well can designers communicate qualities of touch? This paper presents evidence that they have some capability to do so, much of which appears to have been learned, but at present make limited use of such language. Interviews with graduate designer-makers suggest that they are aware of and value the importance of touch and materiality in their work, but lack a vocabulary to fully relate to their detailed explanations of other aspects such as their intent or selection of materials. We believe that more attention should be paid to the verbal dialogue that happens in the design process, particularly as other researchers show that even making-based learning also has a strong verbal element to it. However, verbal language alone does not appear to be adequate for a comprehensive language of touch. Graduate designers-makers’ descriptive practices combined non-verbal manipulation within verbal accounts. We thus argue that haptic vocabularies do not simply describe material qualities, but rather are situated competences that physically demonstrate the presence of haptic qualities. Such competencies are more important than groups of verbal vocabularies in isolation. Design support for developing and extending haptic competences must take this wide range of considerations into account to comprehensively improve designers’ capabilities