Communication and Non-Speaking Children with Physical Disabilities: Opportunities and Reflections from Design-Oriented Research

This thesis presents a series of design-oriented studies for investigating and describing communication involving children with severe speech and physical impairments (SSPIs). The overarching goal is to inform how designers conceptualise communication that involves children with SSPIs beyond a widely cited view that communication centres around speech and happens at the level of the individual through the transmission of information. Instead, by positioning communication as co-constructed, situated and multimodal, the goal is to stimulate how one designs for digitally mediated communication by applying multiple, alternative frames that acknowledge these features. In order to achieve this goal, qualitative empirical fieldwork is undertaken that examines the everyday communication experiences of five children who have SSPIs. Drawing on theoretical influences from multimodal social semiotics and participatory design, study one and two investigate child centred accounts of communication involving children with SSPIs and their peers. The focus is on investigating communication, first in formal learning contexts involving existing augmentative and alternative communication (AAC) technologies, then in broader contexts beyond AAC use. Multi-layered perspectives are generated that consider: 1. a child’s view, by attending to children’s values and choices of modes; 2. an interactional view that attends to how communication is co-constructed in situ with other people and material objects, and; 3. a structural view, that examines the orderings of people, material objects and activities within an environment. These layered understandings produce research frames that are then utilised in study three. A design documentary is created and used to motivate design work for supporting face to face communication involving children with SSPIs and their peers with a team of designers who do not hold fixed orientations to designing assistive technologies. The findings of the three studies make three new contributions to the fields of HCI and AAC. First, the findings produce a theoretical perspective on communication, acknowledging multiple modes and displacing the taken for granted centrality of language. Second, the findings reveal design opportunities for new and existing technologies. Third, the studies contribute methodological insights for design work by considering ways of involving both children and designers when designing with and for children with SSPIs

Computer Entertainment Technologies for the Visually Impaired: An Overview

Over the last years, works related to accessible technologies have increased both in number and in quality. This work presents a series of articles which explore different trends in the field of accessible video games for the blind or visually impaired. Reviewed articles are distributed in four categories covering the following subjects: (1) video game design and architecture, (2) video game adaptations, (3) accessible games as learning tools or treatments and (4) navigation and interaction in virtual environments. Current trends in accessible game design are also analysed, and data is presented regarding keyword use and thematic evolution over time. As a conclusion, a relative stagnation in the field of human-computer interaction for the blind is detected. However, as the video game industry is becoming increasingly interested in accessibility, new research opportunities are starting to appear

A multilevel model for movement rehabilitation in Traumatic Brain Injury (TBI) using virtual environments

This paper presents a conceptual model for movement rehabilitation of traumatic brain injury (TBI) using virtual environments. This hybrid model integrates principles from ecological systems theory with recent advances in cognitive neuroscience, and supports a multilevel approach to both assessment and treatment. Performance outcomes at any stage of recovery are determined by the interplay of task, individual, and environmental/contextual factors. We argue that any system of rehabilitation should provide enough flexibility for task and context factors to be varied systematically, based on the current neuromotor and biomechanical capabilities of the performer or patient. Thus, in order to understand how treatment modalities are to be designed and implemented, there is a need to understand the function of brain systems that support learning at a given stage of recovery, and the inherent plasticity of the system. We know that virtual reality (VR) systems allow training environments to be presented in a highly automated, reliable, and scalable way. Presentation of these virtual environments (VEs) should permit movement analysis at three fundamental levels of behaviour: (i) neurocognitive bases of performance (we focus in particular on the development and use of internal models for action which support adaptive, on-line control); (ii) movement forms and patterns that describe the patients' movement signature at a given stage of recovery (i.e, kinetic and kinematic markers of movement proficiency), (iii) functional outcomes of the movement. Each level of analysis can also map quite seamlessly to different modes of treatment. At the neurocognitive level, for example, semi-immersive VEs can help retrain internal modeling processes by reinforcing the patients' sense of multimodal space (via augmented feedback), their position within it, and the ability to predict and control actions flexibly (via movement simulation and imagery training). More specifically, we derive four - key therapeutic environment concepts (or Elements) presented using VR technologies: Embodiment (simulation and imagery), Spatial Sense (augmenting position sense), Procedural (automaticity and dual-task control), and Participatory (self-initiated action). The use of tangible media/objects, force transduction, and vision-based tracking systems for the augmentation of gestures and physical presence will be discussed in this context

We can work it out: an enactive look at cooperation

The past years have seen an increasing debate on cooperation and its unique human character. Philosophers and psychologists have proposed that cooperative activities are characterized by shared goals to which participants are committed through the ability to understand each other’s intentions. Despite its popularity, some serious issues arise with this approach to cooperation. First, one may challenge the assumption that high-level mental processes are necessary for engaging in acting cooperatively. If they are, then how do agents that do not possess such ability (preverbal children, or children with autism who are often claimed to be mind-blind) engage in cooperative exchanges, as the evidence suggests? Secondly, to define cooperation as the result of two de-contextualized minds reading each other’s intentions may fail to fully acknowledge the complexity of situated, interactional dynamics and the interplay of variables such as the participants’ relational and personal history and experience. In this paper we challenge such accounts of cooperation, calling for an embodied approach that sees cooperation not only as an individual attitude toward the other, but also as a property of interaction processes. Taking an enactive perspective, we argue that cooperation is an intrinsic part of any interaction, and that there can be cooperative interaction before complex communicative abilities are achieved. The issue then is not whether one is able or not to read the other’s intentions, but what it takes to participate in joint action. From this basic account, it should be possible to build up more complex forms of cooperation as needed. Addressing the study of cooperation in these terms may enhance our understanding of human social development, and foster our knowledge of different ways of engaging with others, as in the case of autism

Proceedings of Designing Self-care for Everyday Life. Workshop in conjunction with NordiCHI 2014, 27th October.

Managing chronic conditions can be challenging. People in such conditions, and the people around them, have to, for example: deal with symptoms, adapt to the resulting disability, manage emotions, and change habits to keep the condition under control. Self-care technologies have the potential to support self-care, however they often disregard the complexity of the settings in which they are used and fail to become integrated in everyday life.The present collection of papers forms the Proceedings of the Workshop “Designing Selfcare for Everyday Life” conducted last October 27th, 2014 in Helsinki, where 14 participants from 7 different countries spent the day discussing how to design self-care technologies that are in harmony with people’s everyday life. During the morning, discussions were driven by poster presentations focused on the participants’ work. In the afternoon, we engaged in aparticipatory design exercise focused on the self-care of Parkinson’s disease. Our discussions were driven by the experience of two people living with Parkinson’s that participated in our workshop. At the end of the exercise, each group presented the different insights, concepts and problems that each patient experiences in their everyday life with the disease. Last, we all engaged in a broader discussion with a mapping exercise of issues and challenges in relation to self-care.The contributions featured in the proceedings have been peer-reviewed by the members of the Workshop Program Committee and selected on the basis of their quality, alignment with the workshop theme, and the extent (and diversity) of their backgrounds in design. They express points of view of researchers from both Academia and Industry and provide relevant insights in the design and development use of technologies for self-care.We want to thank all the participants and co-authors for contributing to the Workshop. We are particularly grateful to the two patients, members of the Finnish Parkinson’s Association, who accepted to participate in the workshop and enabled researchers to get aperspective on the challenges of their lives. We also want to thank all the Programme Committee members for all their work during the reviewing process as well as the organisers of NordiCHI 2014 for providing useful facilities

Targeting the expressive language of children with Down syndrome who are minimally verbal : bridging research and practice

textChildren with Down syndrome present with an array of physical and cognitive sequelae that can hinder speech and language development. These individuals can constitute a considerable portion of a speech-language pathologist’s caseload. Based on the principles of best evidence, clinicians are ethically responsible for providing the most effective treatment for their clients. The available literature focuses mainly on describing the linguistic characteristics in this population, while relatively less focus is placed on effective intervention programs. This paper investigates the available evidence regarding speech and language interventions for children with Down syndrome who are in the mild to moderate range of linguistic functioning, and provides an outlook for future research based on best evidence.Communication Sciences and Disorder

Taking into account sensory knowledge: the case of geo-techologies for children with visual impairments

This paper argues for designing geo-technologies supporting non-visual sensory knowledge. Sensory knowledge refers to the implicit and explicit knowledge guiding our uses of our senses to understand the world. To support our argument, we build on an 18 months field-study on geography classes for primary school children with visual impairments. Our findings show (1) a paradox in the use of non-visual sensory knowledge: described as fundamental to the geography curriculum, it is mostly kept out of school; (2) that accessible geo-technologies in the literature mainly focus on substituting vision with another modality, rather than enabling teachers to build on children's experiences; (3) the importance of the hearing sense in learning about space. We then introduce a probe, a wrist-worn device enabling children to record audio cues during field-trips. By giving importance to children's hearing skills, it modified existing practices and actors' opinions on non-visual sensory knowledge. We conclude by reflecting on design implications, and the role of technologies in valuing diverse ways of understanding the world

MapSense: Design and Field Study of Interactive Maps for Children Living with Visual Impairments

We report on the design process leading to the creation of MapSense, a multi-sensory interactive map for visually impaired children. We conducted a formative study in a specialized institute to understand children’s educational needs, their context of care and their preferences regarding interactive technologies. The findings (1) outline the needs for tools and methods to help children to acquire spatial skills and (2) provide four design guidelines for educational assistive technologies. Based on these findings and an iterative process, we designed and deployed MapSense in the institute during two days. It enables collaborations between children with a broad range of impairments, proposes reflective and ludic scenarios and allows caretakers to customize it as they wish. A field experiment reveals that both children and caretakers considered the system successful and empowering

Multisensory learning in adaptive interactive systems

The main purpose of my work is to investigate multisensory perceptual learning and sensory integration in the design and development of adaptive user interfaces for educational purposes. To this aim, starting from renewed understanding from neuroscience and cognitive science on multisensory perceptual learning and sensory integration, I developed a theoretical computational model for designing multimodal learning technologies that take into account these results. Main theoretical foundations of my research are multisensory perceptual learning theories and the research on sensory processing and integration, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models. Finally, a computational model was applied in two case studies, based on two EU ICT-H2020 Projects, "weDRAW" and "TELMI", on which I worked during the PhD