Interactive Technologies for the Public Sphere Toward a Theory of Critical Creative Technology

Digital media cultural practices continue to address the social, cultural and aesthetic contexts of the global information economy, perhaps better called ecology, by inventing new methods and genres that encourage interactive engagement, collaboration, exploration and learning. The theoretical framework for creative critical technology evolved from the confluence of the arts, human computer interaction, and critical theories of technology. Molding this nascent theoretical framework from these seemingly disparate disciplines was a reflexive process where the influence of each component on each other spiraled into the theory and practice as illustrated through the Constructed Narratives project. Research that evolves from an arts perspective encourages experimental processes of making as a method for defining research principles. The traditional reductionist approach to research requires that all confounding variables are eliminated or silenced using methods of statistics. However, that noise in the data, those confounding variables provide the rich context, media, and processes by which creative practices thrive. As research in the arts gains recognition for its contributions of new knowledge, the traditional reductive practice in search of general principles will be respectfully joined by methodologies for defining living principles that celebrate and build from the confounding variables, the data noise. The movement to develop research methodologies from the noisy edges of human interaction have been explored in the research and practices of ludic design and ambiguity (Gaver, 2003); affective gap (Sengers et al., 2005b; 2006); embodied interaction (Dourish, 2001); the felt life (McCarthy & Wright, 2004); and reflective HCI (Dourish, et al., 2004). The theory of critical creative technology examines the relationships between critical theories of technology, society and aesthetics, information technologies and contemporary practices in interaction design and creative digital media. The theory of critical creative technology is aligned with theories and practices in social navigation (Dourish, 1999) and community-based interactive systems (Stathis, 1999) in the development of smart appliances and network systems that support people in engaging in social activities, promoting communication and enhancing the potential for learning in a community-based environment. The theory of critical creative technology amends these community-based and collaborative design theories by emphasizing methods to facilitate face-to-face dialogical interaction when the exchange of ideas, observations, dreams, concerns, and celebrations may be silenced by societal norms about how to engage others in public spaces. The Constructed Narratives project is an experiment in the design of a critical creative technology that emphasizes the collaborative construction of new knowledge about one's lived world through computer-supported collaborative play (CSCP). To construct is to creatively invent one's world by engaging in creative decision-making, problem solving and acts of negotiation. The metaphor of construction is used to demonstrate how a simple artefact - a building block - can provide an interactive platform to support discourse between collaborating participants. The technical goal for this project was the development of a software and hardware platform for the design of critical creative technology applications that can process a dynamic flow of logistical and profile data from multiple users to be used in applications that facilitate dialogue between people in a real-time playful interactive experience

Skyler and Bliss

Hong Kong remains the backdrop to the science fiction movies of my youth. The city reminds me of my former training in the financial sector. It is a city in which I could have succeeded in finance, but as far as art goes it is a young city, and I am a young artist. A frustration emerges; much like the mould, the artist also had to develop new skills by killing off his former desires and manipulating technology. My new series entitled HONG KONG surface project shows a new direction in my artistic research in which my technique becomes ever simpler, reducing the traces of pixelation until objects appear almost as they were found and photographed. Skyler and Bliss presents tectonic plates based on satellite images of the Arctic. Working in a hot and humid Hong Kong where mushrooms grow ferociously, a city artificially refrigerated by climate control, this series provides a conceptual image of a imaginary typographic map for survival. (Laurent Segretier

L’auto-exploration des espaces sensorimoteurs chez les robots

Developmental robotics has begun in the last fifteen years to study robots that havea childhood—crawling before trying to run, playing before being useful—and that are basing their decisions upon a lifelong and embodied experience of the real-world. In this context, this thesis studies sensorimotor exploration—the discovery of a robot’s own body and proximal environment—during the early developmental stages, when no prior experience of the world is available. Specifically, we investigate how to generate a diversity of effects in an unknown environment. This approach distinguishes itself by its lack of user-defined reward or fitness function, making it especially suited for integration in self-sufficient platforms. In a first part, we motivate our approach, formalize the exploration problem, define quantitative measures to assess performance, and propose an architectural framework to devise algorithms. through the extensive examination of a multi-joint arm example, we explore some of the fundamental challenges that sensorimotor exploration faces, such as high-dimensionality and sensorimotor redundancy, in particular through a comparison between motor and goal babbling exploration strategies. We propose several algorithms and empirically study their behaviour, investigating the interactions with developmental constraints, external demonstrations and biologicallyinspired motor synergies. Furthermore, because even efficient algorithms can provide disastrous performance when their learning abilities do not align with the environment’s characteristics, we propose an architecture that can dynamically discriminate among a set of exploration strategies. Even with good algorithms, sensorimotor exploration is still an expensive proposition— a problem since robots inherently face constraints on the amount of data they are able to gather; each observation takes a non-negligible time to collect. [...] Throughout this thesis, our core contributions are algorithms description and empirical results. In order to allow unrestricted examination and reproduction of all our results, the entire code is made available. Sensorimotor exploration is a fundamental developmental mechanism of biological systems. By decoupling it from learning and studying it in its own right in this thesis, we engage in an approach that casts light on important problems facing robots developing on their own.La robotique développementale a entrepris, au courant des quinze dernières années,d’étudier les processus développementaux, similaires à ceux des systèmes biologiques,chez les robots. Le but est de créer des robots qui ont une enfance—qui rampent avant d’essayer de courir, qui jouent avant de travailler—et qui basent leurs décisions sur l’expérience de toute une vie, incarnés dans le monde réel.Dans ce contexte, cette thèse étudie l’exploration sensorimotrice—la découverte pour un robot de son propre corps et de son environnement proche—pendant les premiers stage du développement, lorsque qu’aucune expérience préalable du monde n’est disponible. Plus spécifiquement, cette thèse se penche sur comment générer une diversité d’effets dans un environnement inconnu. Cette approche se distingue par son absence de fonction de récompense ou de fitness définie par un expert, la rendant particulièrement apte à être intégrée sur des robots auto-suffisants.Dans une première partie, l’approche est motivée et le problème de l’exploration est formalisé, avec la définition de mesures quantitatives pour évaluer le comportement des algorithmes et d’un cadre architectural pour la création de ces derniers. Via l’examen détaillé de l’exemple d’un bras robot à multiple degrés de liberté, la thèse explore quelques unes des problématiques fondamentales que l’exploration sensorimotrice pose, comme la haute dimensionnalité et la redondance sensorimotrice. Cela est fait en particulier via la comparaison entre deux stratégies d’exploration: le babillage moteur et le babillage dirigé par les objectifs. Plusieurs algorithmes sont proposés tour à tour et leur comportement est évalué empiriquement, étudiant les interactions qui naissent avec les contraintes développementales, les démonstrations externes et les synergies motrices. De plus, parce que même des algorithmes efficaces peuvent se révéler terriblement inefficaces lorsque leurs capacités d’apprentissage ne sont pas adaptés aux caractéristiques de leur environnement, une architecture est proposée qui peut dynamiquement choisir la stratégie d’exploration la plus adaptée parmi un ensemble de stratégies. Mais même avec de bons algorithmes, l’exploration sensorimotrice reste une entreprise coûteuse—un problème important, étant donné que les robots font face à des contraintes fortes sur la quantité de données qu’ils peuvent extraire de leur environnement;chaque observation prenant un temps non-négligeable à récupérer. [...] À travers cette thèse, les contributions les plus importantes sont les descriptions algorithmiques et les résultats expérimentaux. De manière à permettre la reproduction et la réexamination sans contrainte de tous les résultats, l’ensemble du code est mis à disposition. L’exploration sensorimotrice est un mécanisme fondamental du développement des systèmes biologiques. La séparer délibérément des mécanismes d’apprentissage et l’étudier pour elle-même dans cette thèse permet d’éclairer des problèmes importants que les robots se développant seuls seront amenés à affronter

Using Tangible Interaction and Virtual Reality to Support Spatial Perspective Taking Ability

According to several large-scale and longitudinal studies, spatial ability, one of the primary mental abilities, has been shown as a significant predictor for STEM learning (Science, Technology, Engineering, and Mathematics) and career success. Frameworks in HCI (Human-Computer Interaction) and TEI (Tangible and Embodied Interaction) also indicated how the spatial-related aspects of interaction are a common design theme for interfaces using emerging technologies. However, currently only very few interactive systems (using TEI) are designed around a target spatial ability. TEI’s direct effects on spatial ability are also not well-investigated. Meanwhile, a growing body of research from cognitive sciences, such as embodied cognition and Common Coding Theory, shows that physical movements can enhance cognition in aspects that involve spatial thinking. Also, virtual reality (VR) affords better 3D perception for digital environments, and provides design opportunities to engage users with spatial tasks that may not be otherwise imagined or achieved in the real world. This research describes how we designed and built the system TASC (Tangibles for Augmenting Spatial Cognition), which combines body movement tracking and tangible objects with VR. We recap our design process and design rationales, along with how the finalized system was designed to enhance embodiment as a means to activate, support, engage, and hopefully augment spatial perspective taking ability. We conducted a user study with qualitative and quantitative evaluation methods. Respectively, the qualitative evaluation aimed to understand how the participants used the system; the quantitative evaluation was a multi-condition experiment with pre-tests and post-tests used to investigate if and how the system could improve spatial perspective taking ability. We built the digital pre/post-tests based on PTSOT (Perspective Taking/Spatial Orientation Test) (Hegarty, Kozhevnikov, & Waller, 2008). The study in total involved 52 participants: 6 participants (3M/3F) in the pilot study, 46 in the main study (3 conditions, around 15 per condition, each condition was overall gender-balanced). The qualitative analysis focused on the VR-TEI condition (the “main system”). Using thematic analysis with the video clips and written notes (both taken during the participants’ interaction), and audio clips (recorded during the post-interaction interview), we synthesized the qualitative results into 4 themes: (1) Spatial strategies: akin but unique; (2) The use of gestures & verbalization; (3) Positive experience with the system; (4) The potentials of the system. The quantitative statistical analysis, using ANOVA and t-test for the 3-condition experiment, showed that every condition yielded perspective taking improvement from taking the test twice. However, only the VR-TEI condition led to statistically significant improvement. We conclude the research with discussion and future possibilities in these themes of: (a) The role of embodiment; (2) Further explorations of intermediate conditions; (3) A deeper look at sample size and validity; (4) Designing & evaluating TEIs for other spatial abilities; (5) Integration with STEM curriculum. The main contribution of this dissertation is that it reports how a VR-TEI system can be designed, built, and evaluated for a target spatial ability. We hope this research also contributes to bridging some knowledge gaps between interaction design, cognitive science, and STEM learning

Data graphics and interactive information environments

The flow and exchange of information is characteristic of the digital age. Information is increasingly consumed and produced. It has to be stored, channelled and processed. It also has to be reproduced in new forms again and again. The exponential growth in the volume of digital data has led to new challenges to visualize this data, such that it makes a significant difference to ones understanding of this data. This gain is known as information, which ultimately transforms into knowledge. Information Visualization attempts to create tools and processes that are an aid to cognition. Although this is a relatively new but established discipline, its roots can be traced to developments in the early 17th century, to what are now termed as Data Graphics. As a precursor to modern, dynamic, computer-based visualizations, Data Graphics underpin the science of visualization. This thesis looks at the design principles that govern the construction of historic and contemporary data displays. Although the medium on which Data Graphics have historically been constructed and presented has changed from paper to the computer, design principles have remained the same regardless of the medium or the source of content. The thesis then applies these principles to construct two applications based on large, complex and multivariate data-sets. The first is a proposal for a three-dimensional radar display to visualize Air Traffic Control data. The second application is a visual navigation tool to search within a hypertext document. Both case studies apply design principles inherent in data graphics and utilise human perceptual and cognitive abilities to extract information inherent in data. Finally, both applications are tested by conducting user studies. The contributions of the thesis lie in the construction of the two novel information visualization applications stated above, and by demonstrating that data graphic design principles transcend the medium in which they are produced and presented and can greatly enhance the beauty, efficacy and effectiveness of data displays

Unfoldings

Thesis (M.S.V.S.)--Massachusetts Institute of Technology, Dept. of Architecture, 1992.Includes bibliographical references and index.Huizinga's analysis of play, described in his text Homo Ludens, is compared to the creative process in art-making and to the creative response of the viewer. The play process is examined through questionnaire responses and observations made during an evening of experimental play. Huizinga's assertion that play is not a factor in the plastic arts is challenged. Refutations and counterexamples drawn from the history of art since the Renaissance show that play is indeed a factor. The artistic movements cited are those which provide examples of works having either particularly playful or particularly mathematical content, or both, including Anamorphic painting; Dada; Bauhaus; Neo-Plasticism; Concrete Art; Op Art; Fluxus; and Kinetic Art. Special attention is given to the works of Alexander Calder, George Rickey, and Yaacov Agam. The author describes a personal iconography, and introduces the geometric foundation of her sculptural works, which derive from the geometry of R. Buckminster Fuller's "vector-equilibrium jitterbug." Descriptions, photographs, and drawings are included for the author's Thesis Project, comprising several kinetic, manipulatable jitterbug sculptures.Caryn L. Johnson.M.S.V.S

On Utterance Interpretation and Metalinguistic-Semantic Competence

This study explores the role of what I call metalinguistic-semantic competence (MSC) in the processes of utterance interpretation, and in some cases expression interpretation. MSC is so-called because it is grounded in a speaker's explicit knowledge of (or beliefs about) the lexically-encoded meanings of individual words. More specifically, MSC derives, in part, from having concepts of words--or conceptsW as I distinguish them--whose representational contents, I propose, are corresponding items in a speaker's mental lexicon. The leading idea is that once acquired speakers use their conceptsW to form explicit beliefs about the meanings of words in terms of which extralinguistic concepts those words can (and cannot) coherently be used to express in ordinary conversational situations as constrained by their linguistically-encoded meanings. Or to put the claim differently, I argue that a speaker's explicit conception of word-meanings is a direct conscious reflection of his/her tacit understanding of the various ways in which lexical meanings guide and constrain without fully determining what their host words can (and cannot) be used/uttered to talk about in ordinary discourse. Such metalinguistic knowledge, I contend, quite often plays crucial role in our ability to correctly interpret what other speakers say. The first part of this work details the cognitive mechanisms underlying MSC against the backdrop of a Chomskyan framework for natural language and a Fodorian theory of concepts and their representational contents. The second part explores three ways that MSC might contribute to what I call a speaker's core linguisticsemantic competence. Specifically, I argue that MSC can help explain (i) how competent speakers acquire conceptually underspecified words with their lexical meanings, (ii) the contextual disambiguation of inherently polysemous words, and (iii) the informativeness of true natural language identity statements involving coreferential proper names. The philosophically relevant conclusion is that if any of these proposals pan out then MSC constitutes a proper explanandum of semantic theory, and hence any complete/adequate theory of semantic competence