Discovery learning with tangible technologies: the case of children with intellectual disabilities

Intellectual disabilities cause significant sub--‐average achievement in learning, with difficulties in perception, attention, communication of ideas, language acquisition, abstraction and generalisation. From a socio--‐constructionist perspective, digital technologies can provide resources to help addressing these difficulties. Tangible technologies are considered particularly promising tools for children with intellectual disabilities, by enabling interaction through physical action and manipulation and facilitating representational concrete--‐ abstract links by integrating physical and digital worlds. However, hands--‐on learning activities remain a recommended but problematic approach for intellectually disabled students. This thesis investigates how and which characteristics of tangible interaction may support children with intellectual disabilities to productively engage in discovery learning. \ud Empirical studies were performed where children with intellectual disabilities used four tangible systems with distinct design characteristics. Four broad themes emerged from qualitative analysis which are central for identifying how to best support exploratory interaction: types of digital representations; physical affordances; representational mappings; and conceptual metaphors. Guidelines for the development of tangible artefacts and facilitation of discovery learning activities with tangibles were derived from these themes. A complementary quantitative analysis investigated the effects of external guidance in promoting episodes of discovery in tangible interaction. \ud This thesis argues that providing tangible interaction alone is not sufficient to bring significant benefits to the experience of intellectually disabled students in discovery learning. Visual digital representations, meaningful spatial configurations of physical representations, temporal and spatial contiguity between action and representations, simple causality and familiar conceptual metaphors are critical in providing informational intrinsic feedback to exploratory actions, which allied with external guidance that creates a minimal underlying structure for interaction, should establish an ideal environment for discovery. \u

NoiseBear: A Wireless Malleable Instrument Designed In Participation with Disabled Children

NoiseBear is a wireless malleable controller designed for, and in participation with, physically and cognitively dis- abled children. The aim of the project was to produce a musical controller that was robust, and flexible enough to be used in a wide range of interactive scenarios in partici- patory design workshops. NoiseBear demonstrates an open ended system for designing wireless malleable controllers in different shapes. It uses pressure sensitive material made from conductive thread and polyester cushion stuffing, to give the feel of a soft toy. The sensor networks with other devices using the Bluetooth Low Energy protocol, running on a BlueGiga BLE112 chip. This contains an embedded 8051 processor which manages the sensor. NoiseBear has undergone an initial formative evaluation in workshop ses- sions with four autistic children, and continues to evolve in series of participatory design workshops. The evaluation showed that controller could be engaging for the children to use, and highlighted some technical limitations of the design. Solutions to these limitations are discussed, along with plans for future design iterations

Tangibles: Technologies and interaction for learning

With recent developments in computing and networking new kinds of interfaces, such as tangible interfaces, and consequently new forms of interaction with technology, have emerged. ‘Tangibles’ generally refer to interfaces where computational power is embedded in everyday artefacts or customised objects, which can be wirelessly networked or linked to various forms of digital representation. The emergence of increasingly small microchips and digital sensing technologies means that embedding technology in both artefacts and the environment is becoming more commonplace. In the field of Human Computer Interaction (HCI) this group of technologies may be described as graspable interfaces (e.g. Fitzmaurice et al., 1995), tangible interaction (e.g. Ullmer and Ishii, 2001), and tangible bits (e.g. Ishii, 1997). Shaer and Hornecker’s (2010) definition offers a useful description for the purposes of this chapter: ‘‘Interfaces that are concerned with providing tangible representations to digital information and controls, allowing users to quite literally grasp data with their hands” and thus physically manipulate associated representations (p. 4). There are three key categories of systems that sit under this umbrella term: constructive assembly kits, token and constraint systems, and interactive surfaces

Tangible user interfaces and social interaction in children with autism

Tangible User Interfaces (TUIs) offer the potential for new modes of social interaction for children with Autism Spectrum Conditions (ASC). Familiar objects that are embedded with digital technology may help children with autism understand the actions of others by providing feedback that is logical and predictable. Objects that move, playback sound or create sound – thus repeating programmed effects – offer an exciting way for children to investigate objects and their effects. This thesis presents three studies of children with autism interacting with objects augmented with digital technology. Study one looked at Topobo, a construction toy augmented with kinetic memory. Children played with Topobo in groups of three of either Typically Developing (TD) or ASC children. The children were given a construction task, and were also allowed to play with the construction sets with no task. Topobo in the task condition showed an overall significant effect for more onlooker, cooperative, parallel, and less solitary behaviour. For ASC children significantly less solitary and more parallel behaviour was recorded than other play states. In study two, an Augmented Knights Castle (AKC) playset was presented to children with ASC. The task condition was extended to allow children to configure the playset with sound. A significant effect in a small sample was found for configuration of the AKC, leading to less solitary behaviour, and more cooperative behaviour. Compared to non-digital play, the AKC showed reduction of solitary behaviour because of augmentation. Qualitative analysis showed further differences in learning phase, user content, behaviour oriented to other children, and system responsiveness. Tangible musical blocks (‘d-touch’) in study three focused on the task. TD and ASC children were presented with a guided/non-guided task in pairs, to isolate effects of augmentation. Significant effects were found for an increase in cooperative symbolic play in the guided condition, and more solitary functional play was found in the unguided condition. Qualitative analysis highlighted differences in understanding blocks and block representation, exploratory and expressive play, understanding of shared space and understanding of the system. These studies suggest that the structure of the task conducted with TUIs may be an important factor for children’s use. When the task is undefined, play tends to lose structure and the benefits of TUIs decline. Tangible technology needs to be used in an appropriately structured manner with close coupling (the distance between digital housing and digital effect), and works best when objects are presented in familiar form

Proceedings of the 3rd IUI Workshop on Interacting with Smart Objects

These are the Proceedings of the 3rd IUI Workshop on Interacting with Smart Objects. Objects that we use in our everyday life are expanding their restricted interaction capabilities and provide functionalities that go far beyond their original functionality. They feature computing capabilities and are thus able to capture information, process and store it and interact with their environments, turning them into smart objects

A novel tangible interaction authoring tool for creating educational activities: analysis of its acceptance by educators

The creation of applications based on tangible interaction (TI applications), particularly on tabletops, is a developing area that requires the collaboration of professionals with expert knowledge in specific domains. Having an authoring tool that facilitates interdisciplinary intervention in the design and implementation of such applications is a current challenge to bring TI to different contexts. This article presents an authoring tool (named EDIT) and analyzes its acceptance by educators for creating educational activities. The novelty of the tool lies in the possibility of creating projects with a schedule of educational activities, sequenced as required for a group of students. In addition, it has specific characteristics for the educational scenario, such as the personalization of feedback and the meta-annotation of projects. Sessions were held with educators (n = 38) to analyze variables related to the Technology Acceptance Model (such as perceived usefulness and perceived ease of use) when creating TI educational activities on tabletops using the EDIT tool. The sessions were observed and recorded on video, and a Focus Group was held afterwards. During the sessions, educators gave a positive assessment in relation to using this type of tool. It was observed that, in general, they find tangible interaction valuable mostly for working with children. Finally, the results show a high acceptance obtained from the TAM and the novel features of EDIT were found to be useful

Musical Instruments for Novices: Comparing NIME, HCI and Crowdfunding Approaches

Designing musical instruments to make performance accessible to novice musicians is a goal which long predates digital technology. However, just in the space of the past 6 years, dozens of instrument designs have been introduced in various academic venues and in commercial crowdfunding campaigns. In this paper, we draw comparisons in design, evaluation and marketing across four domains: crowdfunding campaigns on Kickstarter and Indiegogo; the New Interfaces for Musical Expression (NIME) conference; conferences in human-computer interaction (HCI); and researchers creating accessible instruments for children and adults with disabilities. We observe striking differences in approach between commercial and academic projects, with less pronounced differences between each of the academic communities. The paper concludes with general reflections on the identity and purpose of instruments for novice musicians, with suggestions for future exploration

Characterising, assessing and responding to the needs of adults with intellectual disabilities and adults with mental health problems

This thesis outlines a body of research relating to the concept of need and needs-led health service responses for people with intellectual disabilities (ID). In summary, it describes the content, linkages, strengths and limitations of seven published research papers, each with a different, but related focus/question about need. The use of diagnoses in the field of mental health and ID is limited, leading some healthcare providers to adopt a more needs-based approach. Need, however, can be conceptualised in a variety of ways, each with its own benefits and drawbacks. Arguably the most objective of these are normative needs (i.e. a professional's assessment of an individual's need against a notional standard). This objectivity, combined with the limitations of diagnoses, has led some parts of the NHS to adopt normative needs assessment as the basis of a new payment system. Initially, the existing needs assessment tool was critiqued and recommendations for its improvement published. Outputs from the original tool's use (clusters of people with similar mental health needs) were also empirically confirmed to have logical relationships with the established diagnostic taxonomy. Subsequently the tool was adapted and validated for use in specialist ID healthcare settings. Analyses of data resulting from this new tool's use identified statistically robust groups/clusters of individuals with similar severities and combinations of objective needs. These clusters were clinically recognisable and differed according to key features such as severity of ID, challenging behaviours, Autism symptoms and physical health problems. Secondary analysis of these data then confirmed an existing clinical hypothesis i.e. that challenging behaviours in people with more severe ID may be viewed as behavioural manifestations of underlying mental health problems. Finally, these analyses suggested the normative needs assessment tool had utility in identifying users of community-based specialist ID services most at risk of admission to a specialist ID hospital

Ctrl Shift: How Crip Alt Ctrl Designers Change the Game and Reimagine Access

My journey as a disabled arts practitioner has been one of invention, hacking, and re-imagining what input systems could be. I have created my own modalities for creating work, rather than relying on commercially available options. This is a common practice within the disabled community, as individuals often modify and hack their surroundings to make them more usable. For example, ADAPT activists took sledgehammers to smash curb cuts and poured curb ramps with cement bags, ultimately leading to the widespread adoption of curb cuts as a standard architectural feature. As Yergeau notes, this type of "criptastic hacking" represents a creative resistance.(Yergeau, 2012) My interfaces and art projects are a combination of science fiction world-building, technology prototyping, and experimentation with novel ways of experiencing the world that work for my ability. I have been building interactive objects for over 20 years, and my bespoke controller games are both pieces I find comfortable to play and conceptual proposals that I share with the games community to spark consideration for alternative ways of interacting with games culture. This interdisciplinary design research herein crosses a range of disciplines, drawing inspiration from radical forms of cognitive science, games studies, feminist studies, HCI, crip technoscience, radical science fiction, disability studies, and making practices. What has emerged through studying my own practice and the practices of others during this research is a criptastic design framework for creating playful experiences. My research aims to gain a deeper understanding of the ways that hacking and remaking the world manifests as modifications to the design process itself. I created four versions of a physical alt ctrl game and conducted a design study with disabled artists and alt ctrl game creators. The game, Bot Party, was developed through a series of public exhibitions and explored my relationship between criptastic bespoke interface design and embodied experiences of group play. Bot Party involves physical interaction among players in groups to understand my own ways of designing, while the study looks three other disabled designers to understand the ways in which their process is similar or different to my own. By conducting this work, I aim to contribute to the larger conversation within the games studies community about the importance of accessibility and inclusivity in game design. The results highlight the need for continued exploration and development in this area, specifically in design methods. The study’s findings as they relate to my own practice revealed the importance of considering a set of values and design processes in relation to disability when creating games and playful experiences. With this perspective, I propose an initial framework that outlines possible key themes for disabled game designers. Using values as a starting point for creating deeply accessible games, this framework serves as a starting point for future research into accessible game design. This framework seeks to subvert the notion that accessibility is a list of UX best practices, audio descriptions, captions, and haptic additions and moves towards embedding within game design the values and practices used by disabled designers from the outset of the creative process. Access can be a creative framework. An important point to make is that my efforts to do a PhD resist the academic ableism limiting the participation of people who are not from a normative background. The act of creating this PhD has eaten at the edge of my ability, and the research here was often conducted in pain under extremely trying circumstances. This perspective is relevant because it often informed my design choices and thinking. Additionally, it was conducted at a university where I experienced active discrimination from members of staff who simply refused to believe in disabilities they could not see, and in one case writing down my disability was, “self-ascribed.” To work, I had to move outside the academy and seek out workshops which gave me accessible, ergonomic equipment as is discussed in the Bot Party section. This bears mentioning because it reflects on how threatening disabilities can be within academic settings and how even providing basic levels of accessibility remains a challenge for academic institutions. The above framework could benefit academia if used to redesign postgraduate academic research practices within the academy from a place of Crip-informed pedagogy. This is future work that this academic researcher hopes to explore in depth within their academic journey. It is important to note, much of the most relevant research to this thesis around disability studies and technology has emerged in recent years and as a result, was included iteratively in the literature review. It has informed the third study and my iterative design practice as part of the journey; however, I began this work before much of the writing in the literature review existed, including the creation of Bot Party’s first iterations. Finding this scholarship and these authors has been a kinning. Kinship, according to Gavin Van Horn, “can be considered a noun…shared and storied relations and memories that inhere in people and places; or more metaphorical imaginings that unite us to faith traditions, cultures, countries, or the planet…Perhaps this kinship-in-action should be called kinning.” (Horn et al., 2021) Kinning happened throughout this work and this thesis served me as a place for discovery, contemplation, and empowerment. It is my hope sections of it will serve this function for others within my community. I found kinship with other authors working in the field of disability studies and technology, particularly with Alison Kafer, who offers a critique of Donna Haraway's cyborg in her book "Feminist Queer Crip." (Kafer, 2013) Kafer's work highlights the limitations of Haraway's cyborg as a figure of empowerment for marginalized bodies and identities, and instead advocates for a crip-queer-feminist perspective on technology and embodiment. Additionally, the author has also found resonance in the work of Aimi Hamraie and Kelly Fritsh, whose work in disability studies and HCI has been instrumental in shaping this research. Specifically, their concept of "crip technoscience" has been a key framework for understanding technology creation by disabled technologists. (Hamraie and Fritsch, 2019) Overall, it is my hope that this thesis will serve as a generative resource for others within the community on this journey, particularly for those who are working towards a more inclusive and intersectional understanding of technology and embodiment