The Spatial Nature of Thought: Understanding Systems Design Through Diagrams

Design entails the interaction of minds and the tools used to express the design, notably, diagrams. Systems designers use the affordances of the page when they generate structural diagrams of systems. Specifically, they use proximity to augment connectedness (path) information by grouping subsystems. They use horizontal position on the page to express sequence and vertical position to reflect actual spatial position. Finally, they use the permanence of diagrams to generate alternative designs. These conclusions were reached through the analysis of work by student designers, many of whom were practicing information technology professionals. The analysis of designs in topological and Euclidean space required the creation of computational tools that show promise as decision aids for designers, by separating the intertwined qualities of topological and Euclidean space, and by making visible the conceptual similarity of design alternatives

The Language of Architectural Diagrams

Complex buildings frequently present a challenge to users' understanding, which may affect wayfinding as well as appreciation of the building's structure. In this paper we focus on the building's diagram, a representation by the building's architect that captures its main 'idea'. Motivated by the intuition that a building may be easier to understand if its conceptual diagram can be clearly and easily described, we explored perceivers' descriptions of such diagrams' features. We asked students of Language and students of Architecture to write about the buildings represented in a variety of diagrams, and then repeated the task for photographs of the actual buildings. Using Cognitive Discourse Analysis, we aimed to create a first qualitative exploration of the linguistic and conceptual patterns that are associated with the perception of diagrams and images of complex buildings. Among other factors, results show how perception of the diagram's meaning is fundamentally affected by subject expertise. Linguistic patterns demonstrate the ways in which written descriptions reflect observers' understanding and concepts of building representations, providing a starting point for future studies which may address the possible relationship between the verbalisability of a diagram and the legibility of a building

Coordinating Gesture, Word, and Diagram: Explanations for Experts and Novices

abstract: Successful explanations are a symphony of gesture, language, and props. Here, we show how they are orchestrated in an experiment in which students explained complex systems to imagined novices and experts. Visual-spatial communication—diagram and gesture—was key; it represents thought more directly than language. The real or virtual diagrams created from gestures served as the stage for explanations, enriched by language and enlivened by deictic gestures to convey structure and iconic gestures to enact the behavior and functionality of the systems. Explanations to novices packed in more information than explanations to experts, emphasizing the information about action that is difficult for novices, and expressing information in multiple ways, using both virtual models created by gestures and visible ones.This is an Author's Accepted Manuscript of an article published as Kang, Seokmin, Tversky, Barbara, & Black, John B. (2015). Coordinating Gesture, Word, and Diagram: Explanations for Experts and Novices. SPATIAL COGNITION AND COMPUTATION, 15(1), 1-26. http://dx.doi.org/10.1080/13875868.2014.958837. Copyright Taylor & Francis, available online at: http://www.tandfonline.com.ezproxy1.lib.asu.edu/doi/abs/10.1080/13875868.2014.958837#.VRM1-PnF-K

Draw and Tell. Multimodal Descriptions Outperform Verbal- or Sketch-Only Descriptions in an Image Retrieval Task

Han T, Schlangen D. Draw and Tell. Multimodal Descriptions Outperform Verbal- or Sketch-Only Descriptions in an Image Retrieval Task. In: The 8th International Joint Conference on Natural Language Processing. Proceedings of the Conference. Vol. 2: Short Papers. Taipei; 2017: 361-365

The influence of visual feedback and gender dynamics on performance, perception and communication strategies in CSCW

The effects of gender in human communication and human-computer interaction are well-known, yet little is understood about how it influences performance in the complex, collaborative tasks in computer-mediated settings – referred to as Computer-Supported Collaborative Work (CSCW) – that are increasingly fundamental to the way in which people work. In such tasks, visual feedback about objects and events is particularly valuable because it facilitates joint reference and attention, and enables the monitoring of people’s actions and task progress. As such, software to support CSCW frequently provides shared visual workspace. While numerous studies describe and explain the impact of visual feedback in CSCW, research has not considered whether there are differences in how females and males use it, are aided by it, or are affected by its absence. To address these knowledge gaps, this study explores the effect of gender – and its interactions within pairs – in CSCW, with and without visual feedback. An experimental study is reported in which mixed-gender and same-gender pairs communicate to complete a collaborative navigation task, with one of the participants being under the impression that s/he is interacting with a robot (to avoid gender-related social preconceptions). The study analyses performance, perceptions and communication strategies. As predicted, there was a significant benefit associated with visual feedback in terms of language economy and efficiency. However, it was also found that visual feedback may be disruptive to task performance, because it relaxes the users’ precision criteria and inflates their assumptions of shared perspective. While no actual performance difference was found between males and females in the navigation task, females rated their own performance less positively than did males. In terms of communication strategies, males had a strong tendency to introduce novel vocabulary when communication problems occurred, while females exhibited more conservative behaviour. When visual feedback was removed, females adapted their strategies drastically and effectively, increasing the quality and specificity of the verbal interaction, repeating and re-using vocabulary, while the behaviour of males remained consistent. These results are used to produce design recommendations for CSCW systems that will suit users of both genders and enable effective collaboration

Laws, mechanisms, and models in Biology: the case of Mendelian genetics

La estructura lógico conceptual de la genética mendeliana (presentada en formato grafo-modelo) sirve de pretexto a la discusión de algunas peculiaridades de la relación entre los conceptos de teoría, ley, modelo y mecanismo. El ejercicio muestra que, bajo determinadas interpretaciones, estos conceptos pueden ser considerados dentro de un mismo sistema sin que exista contraposición en ello. Básicamente, una teoría consta de una familia de modelos, un modelo es una interpretación que satisface los postulados de la teoría, en particular las constricciones (leyes) que indican el funcionamiento de sus partes, y un mecanismo es un modelo que además permite la localización o la referenciación espaciotemporal de sus partes o componentes.The logical conceptual structure of Mendelian genetics, presented in a graph-model format, provides the context to discuss some peculiarities of relations among the concepts of law, model and mechanism are discussed. It is shown how such concepts can be considered within the same system without any contradictions. Basically, a theory is a family of models, a model is an interpretation that satisfies the postulates of the theory, that is the constraints (laws) that express the functioning of its parts, and a mechanism is a model that, in addition, permits the localization or the spatiotemporal referencing of its parts or components

The Effect of Pointing Gesture on Spatial Memory

Master'sMASTER OF SOCIAL SCIENCE

Phrasing Bimanual Interaction for Visual Design

Architects and other visual thinkers create external representations of their ideas to support early-stage design. They compose visual imagery with sketching to form abstract diagrams as representations. When working with digital media, they apply various visual operations to transform representations, often engaging in complex sequences. This research investigates how to build interactive capabilities to support designers in putting together, that is phrasing, sequences of operations using both hands. In particular, we examine how phrasing interactions with pen and multi-touch input can support modal switching among different visual operations that in many commercial design tools require using menus and tool palettes—techniques originally designed for the mouse, not pen and touch. We develop an interactive bimanual pen+touch diagramming environment and study its use in landscape architecture design studio education. We observe interesting forms of interaction that emerge, and how our bimanual interaction techniques support visual design processes. Based on the needs of architects, we develop LayerFish, a new bimanual technique for layering overlapping content. We conduct a controlled experiment to evaluate its efficacy. We explore the use of wearables to identify which user, and distinguish what hand, is touching to support phrasing together direct-touch interactions on large displays. From design and development of the environment and both field and controlled studies, we derive a set methods, based upon human bimanual specialization theory, for phrasing modal operations through bimanual interactions without menus or tool palettes

A study of stakeholders' experience of the architectural design process to stimulate an interactive form of communication

The architectural design process involves the input of many stakeholders. Communication between them is crucial as it ensures an effective design process. The practice of architecture has been transformed by advances in hardware and software technologies, to the point where both the workflow and the design process are changing. These technologies have also impacted on the methods of representing and communicating design work. At present, traditional communication uses 2D and 3D drawings, as well as digital media such as animation, computer gaming or graphic design, which have impacted on architectural representations. While these tools are very useful, problems in communication between stakeholders are revealed. For instance, differences in architectural background knowledge and requirements lead to misunderstanding the design, confusion caused by working on inconsistent information, and use of incompatible software which causes difficulties in accessing work. This research project attempts to identify and analyse issues relating to communication within the design process in order to improve it. The study is undertaken using a number of key questions to guide the development and progress of the research. The extent of communication via digital media in the design process, in contemporary architectural practice, is examined, along with the perceived value of digital technology by stakeholders. Architectural design work would benefit from exploiting digital media and the Internet to provide an effective form of communication for enabling a user/stakeholder-oriented involvement in the design process. The work presented revisits the conventional methods of communication in design work, between various interested parties in any given project (stakeholders: architects, engineers, planners and clients), with a view to formulating an outline for a potential system that facilitates communication as part of a participatory design process. This study puts forward suggestions to improve communication in the design process, through a storyboard represents users’ experiences in using an interactive communication system. The suggestions are being tested through a mock-up of the web application, which is then presented to participants to receive feedback. Three guiding principles inform the development of the final system: interaction (to allow fast input and feedback); accessibility (to ensure any particular design software is able to interact with the system); and inclusivity (to allow both specialists and lay people to use the system)