BendableSound: An Elastic Multisensory Surface Using Touch-based interactions to Assist Children with Severe Autism During Music Therapy

Neurological Music Therapy uses live music to improve the sensorimotor regulation of children with severe autism. However, they often lack musical training and their impairments limit their interactions with musical instruments. In this paper, we present our co-design work that led to the BendableSound prototype: an elastic multisensory surface encouraging users to practice coordination movements when touching a fabric to play sounds. We present the results of a formative study conducted with 18 teachers showing BendableSound was perceived as “usable” and “attractive”. Then, we present a deployment study with 24 children with severe autism showing BendableSound is “easy to use” and may potentially have therapeutic benefits regarding attention and motor development. We propose a set of design insights that could guide the design of natural user interfaces, particularly elastic multisensory surfaces. We close with a discussion and directions for future work

Assistive learning technologies for learners with ADHD and ASD: – a review 2006-2016

This literature review was a part of a research project investigating how technologies can be utilised to support learners with developmental and attention deficit to be included in mainstream learning contexts.  The review contains research within the field of assistive learning technologies for learners with Attention Deficit Hyperactivity Disorder and Autism Spectrum Disorder from 2006 to 2016. The review is conducted, by using systematic and qualitative academically acknowledged literature search methods. Seven categories of technology-based interventions are recognised from current research, where technologies have been used to support: 1) memory disabilities and/or brain training, 2) increasing focusing attention, 3) time and task management, 4) communication, 5) reading, writing, language and literacy skills, 6) changing behaviour and 7) group work and collaboration. The results indicate that the overall findings of the rendered studies present mixed conclusions. The review calls for more research in a mainstream learning context with a universal design for learning approach

Co-occurrence across time and space of drug- and cannabinoid- exposure and adverse mental health outcomes in the National Survey of Drug Use and Health: combined geotemporospatial and causal inference analysis

Background: Whilst many studies have linked increased drug and cannabis exposure to adverse mental health (MH) outcomes their effects on whole populations and geotemporospatial relationships are not well understood. Methods Ecological cohort study of National Survey of Drug Use and Health (NSDUH) geographically-linked substate-shapefiles 2010–2012 and 2014–2016 supplemented by five-year US American Community Survey. Drugs: cigarettes, alcohol abuse, last-month cannabis use and last-year cocaine use. MH: any mental illness, major depressive illness, serious mental illness and suicidal thinking. Data analysis: two-stage, geotemporospatial, robust generalized linear regression and causal inference methods in R. Results 410,138 NSDUH respondents. Average response rate 76.7%. When drug and sociodemographic variables were combined in geospatial models significant terms including tobacco, alcohol, cannabis exposure and various ethnicities remained in final models for all four major mental health outcomes. Interactive terms including cannabis were related to any mental illness (β-estimate = 1.97 (95%C.I. 1.56–2.37), P \u3c  2.2 × 10− 16), major depressive episode (β-estimate = 2.03 (1.54–2.52), P = 3.6 × 10− 16), serious mental illness (SMI, β-estimate = 2.04 (1.48–2.60), P = 1.0 × 10− 12), suicidal ideation (β-estimate = 1.99 (1.52–2.47), P \u3c  2.2 × 10− 16) and in each case cannabis alone was significantly associated (from β-estimate = − 3.43 (− 4.46 − −2.42), P = 3.4 × 10− 11) with adverse MH outcomes on complex interactive regression surfaces. Geospatial modelling showed a monotonic upward trajectory of SMI which doubled (3.62 to 7.06%) as cannabis use increased. Extrapolated to whole populations cannabis decriminalization (4.26%, (4.18, 4.34%)), Prevalence Ratio (PR) = 1.035(1.034–1.036), attributable fraction in the exposed (AFE) = 3.28%(3.18–3.37%), P \u3c 10− 300) and legalization (4.75% (4.65, 4.84%), PR = 1.155 (1.153–1.158), AFE = 12.91% (12.72–13.10%), P \u3c 10− 300) were associated with increased SMI vs. illegal status (4.26, (4.18–4.33%)). Conclusions Data show all four indices of mental ill-health track cannabis exposure across space and time and are robust to multivariable adjustment for ethnicity, socioeconomics and other drug use. MH deteriorated with cannabis legalization. Cannabis use-MH data are consistent with causal relationships in the forward direction and include dose-response and temporal-sequential relationships. Together with similar international reports and numerous mechanistic studies preventative action to reduce cannabis use is indicated

Serious games: design and development

With the growth of the video game industry, interest in video game research has increased, leading to the study of Serious Games. Serious Games are generally perceived as games that use the video games’ capabilities to emerge players, for other purposes besides entertainment. These purposes include education and training, among others. By using Serious Games for education, teachers could capture the students’ attention in the same way that video games often do, thus the learning process could be more efficient. Additionally, by exploiting the potential of these virtual worlds, it is possible to experience situations that would otherwise be very difficult to experience in the real world, mainly due to reasons of cost, safety and time. Serious Games research and development is still very scarse. However, nowadays there is a large number of available platforms and tools, which can be used to develop Serious Games and video games in general. For instance, web browsers can now provide easy access to realistic 3D virtual worlds. This grants video game developers the tools to create compelling and rich environments that can be accessed by anyone with an internet connection. Additionnaly, other development platforms can be used to achieve different goals. Desktop technologies provide greater processing power and achieve greater results in terms of visual quality, as well as in terms of creating more accurate simulations. This disseration describes the design and development of two Serious Games, one for PC, developed with XNA, and another for the web, developed with WebGL.O crescimento da indústria dos jogos de vídeo, despoletou um maior interesse no estudo deste fenómeno, o que consequentemente levou ao estudo de Jogos Sérios. Jogos Sérios são normalmente considerados jogos de vídeo que são desenvolvidos para outros fins para além do entretenimento. Estes fins incluem a educação e o treino, entre outros. Ao utilizar Jogos Sérios para a educação, os docentes poderiam conseguir captar a atenção dos alunos da mesma forma que os jogos de vídeo normalmente conseguem. Desta forma o processo de aprendizagem poderia ser mais eficiente. Adicionalmente, ao explorar o potencial destes mundos virtuais, é possível experienciar situações que de outra forma seriam difíceis de experienciar na vida real, devido ao seu custo, a razões de segurança e também ao tempo dispendido para as realizar. O estudo de Jogos Sérios é ainda bastante disperso. No entanto, hoje em dia existe já um grande número de plataformas e ferramentas disponíveis que podem ser usadas para desenvolver Jogos Sérios. Por exemplo, os web browsers podem agora fornecer acesso fácil a mundos virtuais 3D. Isto permite que os criadores de jogos de vídeo tenham acesso às ferramentas necessárias para criar ambientes ricos, que possam ser acedidos por qualquer pessoa através de uma ligacção à internet. Adicionalmente, existem outras plataformas de desenvolvimento que podem ser utilizadas para alcançar objetivos diferentes. Tecnologias desktop fornecem um maior poder de processamento e permitem alcançar melhores resultados em termos de qualidade visual, bem como em termos de criação de simulações mais precisas. Nesta dissertação descreve-se a criação e o desenvolvimento de dois Jogos Sérios, um para PC, desenvolvido em XNA e outro outro para a web, desenvolvido em WebGL

XR, music and neurodiversity: design and application of new mixed reality technologies that facilitate musical intervention for children with autism spectrum conditions

This thesis, accompanied by the practice outputs,investigates sensory integration, social interaction and creativity through a newly developed VR-musical interface designed exclusively for children with a high-functioning autism spectrum condition (ASC).The results aim to contribute to the limited expanse of literature and research surrounding Virtual Reality (VR) musical interventions and Immersive Virtual Environments (IVEs) designed to support individuals with neurodevelopmental conditions. The author has developed bespoke hardware, software and a new methodology to conduct field investigations. These outputs include a Virtual Immersive Musical Reality Intervention (ViMRI) protocol, a Supplemental Personalised, immersive Musical Experience(SPiME) programme, the Assisted Real-time Three-dimensional Immersive Musical Intervention System’ (ARTIMIS) and a bespoke (and fully configurable) ‘Creative immersive interactive Musical Software’ application (CiiMS). The outputs are each implemented within a series of institutional investigations of 18 autistic child participants. Four groups are evaluated using newly developed virtual assessment and scoring mechanisms devised exclusively from long-established rating scales. Key quantitative indicators from the datasets demonstrate consistent findings and significant improvements for individual preferences (likes), fear reduction efficacy, and social interaction. Six individual case studies present positive qualitative results demonstrating improved decision-making and sensorimotor processing. The preliminary research trials further indicate that using this virtual-reality music technology system and newly developed protocols produces notable improvements for participants with an ASC. More significantly, there is evidence that the supplemental technology facilitates a reduction in psychological anxiety and improvements in dexterity. The virtual music composition and improvisation system presented here require further extensive testing in different spheres for proof of concept

How a Diverse Research Ecosystem Has Generated New Rehabilitation Technologies: Review of NIDILRR’s Rehabilitation Engineering Research Centers

Over 50 million United States citizens (1 in 6 people in the US) have a developmental, acquired, or degenerative disability. The average US citizen can expect to live 20% of his or her life with a disability. Rehabilitation technologies play a major role in improving the quality of life for people with a disability, yet widespread and highly challenging needs remain. Within the US, a major effort aimed at the creation and evaluation of rehabilitation technology has been the Rehabilitation Engineering Research Centers (RERCs) sponsored by the National Institute on Disability, Independent Living, and Rehabilitation Research. As envisioned at their conception by a panel of the National Academy of Science in 1970, these centers were intended to take a “total approach to rehabilitation”, combining medicine, engineering, and related science, to improve the quality of life of individuals with a disability. Here, we review the scope, achievements, and ongoing projects of an unbiased sample of 19 currently active or recently terminated RERCs. Specifically, for each center, we briefly explain the needs it targets, summarize key historical advances, identify emerging innovations, and consider future directions. Our assessment from this review is that the RERC program indeed involves a multidisciplinary approach, with 36 professional fields involved, although 70% of research and development staff are in engineering fields, 23% in clinical fields, and only 7% in basic science fields; significantly, 11% of the professional staff have a disability related to their research. We observe that the RERC program has substantially diversified the scope of its work since the 1970’s, addressing more types of disabilities using more technologies, and, in particular, often now focusing on information technologies. RERC work also now often views users as integrated into an interdependent society through technologies that both people with and without disabilities co-use (such as the internet, wireless communication, and architecture). In addition, RERC research has evolved to view users as able at improving outcomes through learning, exercise, and plasticity (rather than being static), which can be optimally timed. We provide examples of rehabilitation technology innovation produced by the RERCs that illustrate this increasingly diversifying scope and evolving perspective. We conclude by discussing growth opportunities and possible future directions of the RERC program

Blending human and artificial intelligence to support autistic children’s social communication skills

This paper examines the educational efficacy of a learning environment in which children diagnosed with Autism Spectrum Conditions (ASC) engage in social interactions with an artificially intelligent (AI) virtual agent and where a human practitioner acts in support of the interactions. A multi-site intervention study in schools across the UK was conducted with 29 children with ASC and learning difficulties, aged 4-14 years old. For reasons related to data completeness and amount of exposure to the AI environment, data for 15 children was included in the analysis. The analysis revealed a significant increase in the proportion of social responses made by ASC children to human practitioners. The number of initiations made to human practitioners and to the virtual agent by the ASC children also increased numerically over the course of the sessions. However, due to large individual differences within the ASC group, this did not reach significance. Although no evidence of transfer to the real-world post-test was shown, anecdotal evidence of classroom transfer was reported. The work presented in this paper offers an important contribution to the growing body of research in the context of AI technology design and use for autism intervention in real school contexts. Specifically, the work highlights key methodological challenges and opportunities in this area by leveraging interdisciplinary insights in a way that (i) bridges between educational interventions and intelligent technology design practices, (ii) considers the design of technology as well as the design of its use (context and procedures) on par with one another, and (iii) includes design contributions from different stakeholders, including children with and without ASC diagnosis, educational practitioners and researchers

Ubiquitous Integration and Temporal Synchronisation (UbilTS) framework : a solution for building complex multimodal data capture and interactive systems

Contemporary Data Capture and Interactive Systems (DCIS) systems are tied in with various technical complexities such as multimodal data types, diverse hardware and software components, time synchronisation issues and distributed deployment configurations. Building these systems is inherently difficult and requires addressing of these complexities before the intended and purposeful functionalities can be attained. The technical issues are often common and similar among diverse applications. This thesis presents the Ubiquitous Integration and Temporal Synchronisation (UbiITS) framework, a generic solution to address the technical complexities in building DCISs. The proposed solution is an abstract software framework that can be extended and customised to any application requirements. UbiITS includes all fundamental software components, techniques, system level layer abstractions and reference architecture as a collection to enable the systematic construction of complex DCISs. This work details four case studies to showcase the versatility and extensibility of UbiITS framework’s functionalities and demonstrate how it was employed to successfully solve a range of technical requirements. In each case UbiITS operated as the core element of each application. Additionally, these case studies are novel systems by themselves in each of their domains. Longstanding technical issues such as flexibly integrating and interoperating multimodal tools, precise time synchronisation, etc., were resolved in each application by employing UbiITS. The framework enabled establishing a functional system infrastructure in these cases, essentially opening up new lines of research in each discipline where these research approaches would not have been possible without the infrastructure provided by the framework. The thesis further presents a sample implementation of the framework on a device firmware exhibiting its capability to be directly implemented on a hardware platform. Summary metrics are also produced to establish the complexity, reusability, extendibility, implementation and maintainability characteristics of the framework.Engineering and Physical Sciences Research Council (EPSRC) grants - EP/F02553X/1, 114433 and 11394

Beyond The ‘Untold’ Encountering Autism: Situated Insights On Ways To Enable Participation In Design

The number of people who experience autism is constantly increasing. In 2020, the Center for Disease Control and Prevention determined that, every year, approximately 1 out of 54 children is diagnosed with the autism spectrum disorder (ASD). In the last years, the number of research and studies dedicated to children with autism has increased; however, scarce attention was paid to research on adults with autism, particularly those on the severe end of the spectrum (low-functioning autism), who have limited verbal abilities and non-normative communication. Consequences of this include issues of social inclusion, lack of services, work opportunities, social and civil rights, to mention a few, which are not yet sufficiently explored nor implemented for this specific group of people. This thesis employs Participatory Design to explore novel ways to engage with people on the autistic spectrum, in the attempt to understand what processes and participatory dynamics become enacted in acknowledging them as equal, valuable presences in the constitution of our collective existence and society. In particular, the research looks at processes and activities that can lead to a collaboration between designers and adults with severe autism and non-normative communication abilities, and their active participation within the research process. By theoretically drawing from traditions of Feminist Theory and Participatory Design, this thesis considers one single case study, where collaboration with the participant is achieved by means of a situated approach to design research. My standpoint and philosophical positioning here are to acknowledge that there are multiple realities through which one can make sense of the world: this research envisions that each person constructs her or his own reality through their personal experiences. Such philosophical perspective aims at emphasising and valuing subjectivity, and the consequent, intrinsic partiality that constitute our human experience. Personalised and situated research objects are also used across the thesis as methodological elements, oriented to explore the potential of triggering participation, crossing the boundaries between different communication abilities and achieving mutual understanding between designer and participant. The contribution to knowledge provided from this research is three fold: first, it provides new insights about the creation of research objects that result from a situated and process-oriented approach to Participatory Design; second, it extends knowledge concerning design practices that involve the collaborative participation of people experiencing the severe 4 end of the autistic spectrum; third, it contributes to provide new narratives concerning peoplewith non-normative communication abilities and autism