Detecting Linguistic HCI Markers in an Online Aphasia Support Group

ABSTRACT Aphasia is an acquired language disorder resulting from trauma or injury to language areas of the brain. Despite extensive research on the impact of aphasia on traditional forms of communication, little is known about the impact of aphasia on computer-mediated communication (CMC). In this study we asked whether the welldocumented language deficits associated with aphasia can be detected in online writing of people with aphasia. We analyzed 150 messages (14,754 words) posted to an online aphasia support forum, by six people with aphasia and by four controls. Significant linguistic differences between people with aphasia and controls were detected, suggesting five putative linguistic HCI markers for aphasia. These findings suggest that interdisciplinary research on communication disorders and CMC has both applied and theoretical implications

Gesture and Speech in Interaction - 4th edition (GESPIN 4)

International audienceThe fourth edition of Gesture and Speech in Interaction (GESPIN) was held in Nantes, France. With more than 40 papers, these proceedings show just what a flourishing field of enquiry gesture studies continues to be. The keynote speeches of the conference addressed three different aspects of multimodal interaction:gesture and grammar, gesture acquisition, and gesture and social interaction. In a talk entitled Qualitiesof event construal in speech and gesture: Aspect and tense, Alan Cienki presented an ongoing researchproject on narratives in French, German and Russian, a project that focuses especially on the verbal andgestural expression of grammatical tense and aspect in narratives in the three languages. Jean-MarcColletta's talk, entitled Gesture and Language Development: towards a unified theoretical framework,described the joint acquisition and development of speech and early conventional and representationalgestures. In Grammar, deixis, and multimodality between code-manifestation and code-integration or whyKendon's Continuum should be transformed into a gestural circle, Ellen Fricke proposed a revisitedgrammar of noun phrases that integrates gestures as part of the semiotic and typological codes of individuallanguages. From a pragmatic and cognitive perspective, Judith Holler explored the use ofgaze and hand gestures as means of organizing turns at talk as well as establishing common ground in apresentation entitled On the pragmatics of multi-modal face-to-face communication: Gesture, speech andgaze in the coordination of mental states and social interaction.Among the talks and posters presented at the conference, the vast majority of topics related, quitenaturally, to gesture and speech in interaction - understood both in terms of mapping of units in differentsemiotic modes and of the use of gesture and speech in social interaction. Several presentations explored the effects of impairments(such as diseases or the natural ageing process) on gesture and speech. The communicative relevance ofgesture and speech and audience-design in natural interactions, as well as in more controlled settings liketelevision debates and reports, was another topic addressed during the conference. Some participantsalso presented research on first and second language learning, while others discussed the relationshipbetween gesture and intonation. While most participants presented research on gesture and speech froman observer's perspective, be it in semiotics or pragmatics, some nevertheless focused on another importantaspect: the cognitive processes involved in language production and perception. Last but not least,participants also presented talks and posters on the computational analysis of gestures, whether involvingexternal devices (e.g. mocap, kinect) or concerning the use of specially-designed computer software forthe post-treatment of gestural data. Importantly, new links were made between semiotics and mocap data

Practical, appropriate, empirically-validated guidelines for designing educational games

There has recently been a great deal of interest in the potential of computer games to function as innovative educational tools. However, there is very little evidence of games fulfilling that potential. Indeed, the process of merging the disparate goals of education and games design appears problematic, and there are currently no practical guidelines for how to do so in a coherent manner. In this paper, we describe the successful, empirically validated teaching methods developed by behavioural psychologists and point out how they are uniquely suited to take advantage of the benefits that games offer to education. We conclude by proposing some practical steps for designing educational games, based on the techniques of Applied Behaviour Analysis. It is intended that this paper can both focus educational games designers on the features of games that are genuinely useful for education, and also introduce a successful form of teaching that this audience may not yet be familiar with

Tracking Visible Features of Speech for Computer-Based Speech Therapy for Childhood Apraxia of Speech

At present, there are few, if any, effective computer-based speech therapy systems (CBSTs) that support the at-home component for clinical interventions for Childhood Apraxia of Speech (CAS). PROMPT, an established speech therapy intervention for CAS, has the potential to be supported via a CBST, which could increase engagement and provide valuable feedback to the child. However, the necessary computational techniques have not yet been developed and evaluated. In this thesis, I will describe the development of some of the key underlying computational components that are required for the development of such a system. These components concern camera-based tracking of visible features of speech which concern jaw kinematics. These components would also be necessary for the serious game that we have envisioned

Embodied Imagination: An Exploration of Participatory Performance and Interactive Technology to Support Stroke Recovery

Life after a stroke leads to the challenge of adjusting to new possibilities and fosters an increased risk of social isolation and depression. Re-building personal narratives and creating new community networks are key to conceiving an identity beyond the stroke. In this context, participatory performance provides methods for exploring physical and social identities, imagining new ways of being. Meanwhile, digital technology offers tools to help envision these possibilities. A participatory performance workshop supported by real-time motion capture technology has been redesigned in collaboration with the performance company Split Britches. The Green Screening workshop’s objective is to help stroke survivors imagine new physical and social possibilities by enacting fantasies of things they have always wanted to do. Participants construct storylines supported by a custom-built interactive scenography. Movement data is collected and transformed into real-time visualisations to progressively build fantasy narratives enacted with and for other participants. Three research studies conducted with stroke support groups around England analysed progressive iterations of the Green Screening workshop. The first study focused on the project's feasibility in aiding social support. The second study explored embodied imagination and social collaboration in enacted storylines. Finally, the third study analysed communication as a means of recovery and further potential to foster social collaboration. Findings are based on qualitative analysis of the participants' experience. Results reveal that the narrative process and visualisations encouraged a rich repertoire of improvised movements, and the communal aspect of the process was found especially important in achieving these results. This work argues that this framework can simultaneously bring a rich, prospective and political understanding of people’s lived experience to the design space in HCI and provide community stroke support

Design of a Controlled Language for Critical Infrastructures Protection

We describe a project for the construction of controlled language for critical infrastructures protection (CIP). This project originates from the need to coordinate and categorize the communications on CIP at the European level. These communications can be physically represented by official documents, reports on incidents, informal communications and plain e-mail. We explore the application of traditional library science tools for the construction of controlled languages in order to achieve our goal. Our starting point is an analogous work done during the sixties in the field of nuclear science known as the Euratom Thesaurus.JRC.G.6-Security technology assessmen

Image-Enabled Discourse: Investigating the Creation of Visual Information as Communicative Practice

Anyone who has clarified a thought or prompted a response during a conversation by drawing a picture has exploited the potential of image making as an interactive tool for conveying information. Images are increasingly ubiquitous in daily communication, in large part due to advances in visually enabled information and communication technologies (ICT), such as information visualization applications, image retrieval systems and visually enabled collaborative work tools. Human abilities to use images to communicate are however far more sophisticated and nuanced than these technologies currently support. In order to learn more about the practice of image making as a specialized form of information and communication behavior, this study examined face-to-face conversations involving the creation of ad hoc visualizations (i.e., napkin drawings ). A model of image-enabled discourse is introduced, which positions image making as a specialized form of communicative practice. Multimodal analysis of video-recorded conversations focused on identifying image-enabled communicative activities in terms of interactional sociolinguistic concepts of conversational involvement and coordination, specifically framing, footing and stance. The study shows that when drawing occurs in the context of an ongoing dialogue, the activity of visual representation performs key communicative tasks. Visualization is a form of social interaction that contributes to the maintenance of conversational involvement in ways that are not often evident in the image artifact. For example, drawing enables us to coordinate with each other, to introduce alternative perspectives into a conversation and even to temporarily suspend the primary thread of a discussion in order to explore a tangential thought. The study compares attributes of the image artifact with those of the activity of image making, described as a series of contrasting affordances. Visual information in complex systems is generally represented and managed based on the affordances of the artifact, neglecting to account for all that is communicated through the situated action of creating. These finding have heuristic and best-practice implications for a range of areas related to the design and evaluation of virtual collaboration environments, visual information extraction and retrieval systems, and data visualization tools

Cognitive and Neurophysiological Models of Brain Asymmetry

Asymmetry is an inherent characteristic of brain organization in both humans and other vertebrate species, and is evident at the behavioral, neurophysiological, and structural levels. Brain asymmetry underlies the organization of several cognitive systems, such as emotion, communication, and spatial processing. Despite this ubiquity of asymmetries in the vertebrate brain, we are only beginning to understand the complex neuronal mechanisms underlying the interaction between hemispheric asymmetries and cognitive systems. Unfortunately, despite the vast number of empirical studies on brain asymmetries, theoretical models that aim to provide mechanistic explanations of hemispheric asymmetries are sparse in the field. Therefore, this Special Issue aims to highlight empirically based mechanistic models of brain asymmetry. Overall, six theoretical and four empirical articles were published in the Special Issue, covering a wide range of topics, from human handedness to auditory laterality in bats. Two key challenges for theoretical models of brain asymmetry are the integration of increasingly complex molecular data into testable models, and the creation of theoretical models that are robust and testable across different species

Emotional recognition in computing

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University 8/4/2010.Emotions are fundamental to human lives and decision-making. Understanding and expression of emotional feeling between people forms an intricate web. This complex interactional phenomena, is a hot topic for research, as new techniques such as brain imaging give us insights about how emotions are tied to human functions. Communication of emotions is mixed with communication of other types of information (such as factual details) and emotions can be consciously or unconsciously displayed. Affective computer systems, using sensors for emotion recognition and able to make emotive responses are under development. The increased potential for emotional interaction with products and services, in many domains, is generating much interest. Emotionally enhanced systems have potential to improve human computer interaction and so to improve how systems are used and what they can deliver. They may also have adverse implications such as creating systems capable of emotional manipulation of users. Affective systems are in their infancy and lack human complexity and capability. This makes it difficult to assess whether human interaction with such systems will actually prove beneficial or desirable to users. By using experimental design, a Wizard of Oz methodology and a game that appeared to respond to the user’s emotional signals with human-like capability, I tested user experience and reactions to a system that appeared affective. To assess users’ behaviour, I developed a novel affective behaviour coding system called ‘affectemes’. I found significant gains in user satisfaction and performance when using an affective system. Those believing the system responded to emotional signals blinked more frequently. If the machine failed to respond to their emotional signals, they increased their efforts to convey emotion, which might be an attempt to ‘repair’ the interaction. This work highlights how very complex and difficult it is to design and evaluate affective systems. I identify many issues for future work, including the unconscious nature of emotions and how they are recognised and displayed with affective systems; issues about the power of emotionally interactive systems and their evaluation; and critical ethical issues. These are important considerations for future design of systems that use emotion recognition in computing.EPSRC project grant (R81374/01