The acoustic, the digital and the body: a survey on musical instruments

This paper reports on a survey conducted in the autumn of 2006 with the objective to understand people's relationship to their musical tools. The survey focused on the question of embodiment and its different modalities in the fields of acoustic and digital instruments. The questions of control, instrumental entropy, limitations and creativity were addressed in relation to people's activities of playing, creating or modifying their instruments. The approach used in the survey was phenomenological, i.e. we were concerned with the experience of playing, composing for and designing digital or acoustic instruments. At the time of analysis, we had 209 replies from musicians, composers, engineers, designers, artists and others interested in this topic. The survey was mainly aimed at instrumentalists and people who create their own instruments or compositions in flexible audio programming environments such as SuperCollider, Pure Data, ChucK, Max/MSP, CSound, etc

Haptic Media Scenes

The aim of this thesis is to apply new media phenomenological and enactive embodied cognition approaches to explain the role of haptic sensitivity and communication in personal computer environments for productivity. Prior theory has given little attention to the role of haptic senses in influencing cognitive processes, and do not frame the richness of haptic communication in interaction design—as haptic interactivity in HCI has historically tended to be designed and analyzed from a perspective on communication as transmissions, sending and receiving haptic signals. The haptic sense may not only mediate contact confirmation and affirmation, but also rich semiotic and affective messages—yet this is a strong contrast between this inherent ability of haptic perception, and current day support for such haptic communication interfaces. I therefore ask: How do the haptic senses (touch and proprioception) impact our cognitive faculty when mediated through digital and sensor technologies? How may these insights be employed in interface design to facilitate rich haptic communication? To answer these questions, I use theoretical close readings that embrace two research fields, new media phenomenology and enactive embodied cognition. The theoretical discussion is supported by neuroscientific evidence, and tested empirically through case studies centered on digital art. I use these insights to develop the concept of the haptic figura, an analytical tool to frame the communicative qualities of haptic media. The concept gauges rich machine- mediated haptic interactivity and communication in systems with a material solution supporting active haptic perception, and the mediation of semiotic and affective messages that are understood and felt. As such the concept may function as a design tool for developers, but also for media critics evaluating haptic media. The tool is used to frame a discussion on opportunities and shortcomings of haptic interfaces for productivity, differentiating between media systems for the hand and the full body. The significance of this investigation is demonstrating that haptic communication is an underutilized element in personal computer environments for productivity and providing an analytical framework for a more nuanced understanding of haptic communication as enabling the mediation of a range of semiotic and affective messages, beyond notification and confirmation interactivity

Return of the man-machine interface: violent interactions

This paper presents the design and evaluation of “the man-machine interface” a punchable interface designed to criticise and react against the values inherent in modern systems that tacitly favour one type of user (linguistically and technically gifted) and alienate another (physically gifted). We report a user study, where participants used the device to express their opinions before engaging in a group discussion about the implications of strength-based interactions. We draw connections between our own work and that of evolutionary biologists whose recent findings indicate the shape of the human hand is likely to have been partly evolved for the purpose of punching, and conclude by examining violent force as an appropriate means for expressing thoughts and feelings

Interacting with an inferred world: The challenge of machine learning for humane computer interaction

<div class="page" title="Page 1"><div class="layoutArea"><div class="column"><p><span>Classic theories of user interaction have been framed in relation to symbolic models of planning and problem solving, responding in part to the cognitive theories associated with AI research. However, the behavior of modern machine-learning systems is determined by statistical models of the world rather than explicit symbolic descriptions. Users increasingly interact with the world and with others in ways that are mediated by such models. This paper explores the way in which this new generation of technology raises fresh challenges for the critical evaluation of interactive systems. It closes with some proposed measures for the design of inference-based systems that are more open to humane design and use. </span></p></div></div></div>This is the author accepted manuscript. The final version is available from the Association for Computing Machinery via http://dx.doi.org/10.7146/aahcc.v1i1.2119

Towards ai-based interactive game intervention to monitor concentration levels in children with attention deficit

—Preliminary results to a new approach for neurocognitive training on academic engagement and monitoring of attention levels in children with learning difficulties is presented. Machine Learning (ML) techniques and a Brain-Computer Interface (BCI) are used to develop an interactive AI-based game for educational therapy to monitor the progress of children’s concentration levels during specific cognitive tasks. Our approach resorts to data acquisition of brainwaves of children using electroencephalography (EEG) to classify concentration levels through model calibration. The real-time brainwave patterns are inputs to our game interface to monitor concentration levels. When the concentration drops, the educational game can personalize to the user by changing the challenge of the training or providing some new visual or auditory stimuli to the user in order to reduce the attention loss. To understand concentration level patterns, we collected brainwave data from children at various primary schools in Brazil who have intellectual disabilities e.g. autism spectrum disorder and attention deficit hyperactivity disorder. Preliminary results show that we successfully benchmarked (96%) the brainwave patterns acquired by using various classical ML techniques. The result obtained through the automatic classification of brainwaves will be fundamental to further develop our full approach. Positive feedback from questionnaires was obtained for both, the AI-based game and the engagement and motivation during the training sessions