Contributos para o desenvolvimento de aplicações para crianças com problemas de comunicação e integração

Communication, to express our needs and ideas, is a fundamental piece of our daily lives, enabling our interaction with others and contributing to establish social bonds. When this ability is affected, whether by contextual issues (e.g., noisy environment) or intrinsic conditions (e.g., speech articulation difficulties) it creates a challenging scenario for both the individuals and those surrounding them. Children diagnosed with Austism Spectrum Disorders (ASD) are a notable example of an audience facing these challenges given their frequent difficulties to communicate and socially interact with others. In this context, their challenges not only affect their ability to integrate with family and friends, but can also influence how they perform at school, an important environment for their development. In recent years, efforts have been made to propose technological solutions that contribute to support these children in their communication, but most of the work in the literature mainly focuses on the child entailing a support that does not truly addresses integration. Considering this context, we argue that these challenges need to be addressed considering the needs and motivations of children with ASD, but also keeping in mind the motivations of those around them, helping parents, teachers and friends to take part in the integration. Profiting from previous work on the characterization of the needs and motivations of children with ASD, their families, and teachers, the work carried out explores different communication-related scenarios, at school and at home, and proposes a set of requirements that should be considered to serve them. Considering these requirements, and adopting an iterative user-centered design approach, a first proof-of-concept application is proposed. AMICA, Adaptive Multimodal Integrated Communication for All, is a multi-platform, multiuser and multimodal application that aims to illustrate how some issues that affect children with ASD and those surrounding them can be addressed in order to their lives easier. At its current stage, AMICA already encompasses support for several communication features adapted to serve the child, family, friends, and teachers providing them a space for interaction. The modular multi-platform nature of the work carried out enables AMICA as a testbed for further research on different multimodal ways of communication for children with ASD (e.g., gaze or pictograms), but, also, by allowing the participation of different users, it fosters first studies on the dynamic nature of such communication.A comunicação, para expressar as nossas necessidades e ideias, é uma peça fundamental do nosso dia a dia, permitindo a nossa interação com os outros e contribuindo para estabelecer laços sociais. Quando essa capacidade é afetada, seja por questões contextuais (por exemplo, ambiente ruidoso) ou condições intrínsecas (por exemplo, dificuldades de articulação de fala), ela cria um cenário desafiante tanto para os indivíduos como para os que os rodeiam. As crianças diagnosticadas com Perturbação do Espetro do Autismo (PEA) são um exemplo notável de uma audiência que enfrenta esses desafios, devido às suas frequentes dificuldades para comunicar e interagir socialmente com os outros. Neste contexto, os seus desafios não afetam apenas a sua capacidade de integração com a família e os amigos, podendo, também, influenciar o seu desempenho na escola, um ambiente importante para o seu desenvolvimento. Nos últimos anos, têm sido feitos esforços para propor soluções tecnológicas que contribuam para apoiar essas crianças na sua comunicação, mas a maior parte do trabalho na literatura concentra-se principalmente no uso, pela criança, de apoios que não abordam verdadeiramente a integração. Considerando esse contexto, argumentamos que esses desafios precisam de ser abordados considerando as necessidades e motivações das crianças com PEA, mas também tendo em mente as motivações das pessoas ao seu redor, ajudando pais, professores e amigos a participarem na integração. Aproveitando os trabalhos anteriores sobre a caracterização das necessidades e motivações das crianças com PEA, as suas famílias e professores, o trabalho realizado explora diferentes cenários relacionados à comunicação, na escola e em casa, e propõe um conjunto de requisitos que devem ser considerados para os servir. Considerando esses requisitos e adotando uma abordagem de projeto iterativa, centrada no utilizador, uma primeira aplicação de prova de conceito é proposta. AMICA, Adaptive Multimodal Integrated Communication for All, é uma aplicação multi-plataforma, multi-utilizador e multimodal que visa ilustrar como algumas questões que afetam as crianças com PEA e aqueles que as rodeiam podem ser abordadas para tentar facilitar as suas vidas. No estado atual, a AMICA já engloba o suporte a vários recursos de comunicação adaptados para atender a criança, a família, os amigos e os professores, proporcionando-lhes um espaço de interação. A natureza modular multi-plataforma do trabalho realizado permite que AMICA seja uma base para mais pesquisas sobre diferentes formas multimodais de comunicação para crianças com PEA (por exemplo, olhar ou pictogramas), mas, também, permitindo a participação de diferentes utilizadores, promove primeiros estudos sobre a natureza dinâmica de tal comunicação.Mestrado em Engenharia de Computadores e Telemátic

Mobile devices as assistive technologies for ASD: experiences in the classroom

This is a post-peer-review, pre-copyedit version of an article published in Lecture Notes in Computer Science. The final authenticated version is available online at https://doi.org/10.1007/978-3-319-40355-7_18Information and Communication Technologies offer new opportunities to people with disabilities to develop their autonomy and independence in their daily life activities. However, more research should be done in order to comprehend how technology affects this collective of people. This paper presents two experiences where participants with cognitive disabilities and Autism Spectrum Disorder used AssisT-Task to perform job related activities and DEDOS to perform educational activities. Their performance is improved along the sessions using both tools. Combining visual and textual information help students with cognitive disabilities and ASD to focus on the contents presented, avoiding usability and accessibility issues, and therefore improving their learning process while they are having fun interacting with new technologiesThis work has been funded in part by the Spanish Economy and Competitiveness Ministry under project “e-Integra: e-Training y e-Coaching para la integración socio-laboral” (TIN2013-44586-R) and by the Region of Madrid under pro-ject “eMadrid – Investigación y Desarrollo de tecnologías para el e-learning en la Comunidad de Madrid (S2013/ICE-2715

Didactic Software for Autistic Children

In this paper we describe the aims and requirements of a project devoted to designing and developing Open Source didactic Software (SW) for children in the autism disorder spectrum, conforming to the Applied Behaviour Analysis (ABA) learning technique. In this context, participatory design with therapists and child?s parents is necessary to ensure a usable product that responds to these children?s special needs and respects education principles and constraints of the ABA methodology

Utilizing Mobile Technology to Improve Accessibility for Museum Visitors with Autism

What if museums could provide an app that makes interaction easier for visitors with autism and allows them to enjoy the museum any day of the week like their neurotypical counterparts? This research discusses how museums can utilize downloadable apps for personal devices to provide easily accessible resources for children on the autism spectrum and their families to use so they can have a more inclusive and sensory-friendly museum experience. To determine the feasibility of this, I worked in collaboration with an app developer, Peter Laurin, to create a prototype-app called SenseEase: Strong Museum for the Reading Adventure Land exhibit area at the Strong National Museum of Play in Rochester. The app caters to children ranging from four to twelve years of age, and aims to help prepare them and their families for their museum visit and help them cope with the overwhelming sensory stimulation that occurs in this particular museum environment. To evaluate the effectiveness of the app, I conducted two rounds of user testing, one with a general audience and one with my target audience for the app, and found that SenseEase: Strong Museum was a well-received and effective aid for children on the autism spectrum and their families navigating the unfamiliar and overstimulating environment at the museum

Examining the Usability of Touch Screen Gestures for Children With Down Syndrome

[EN] The use of multi-touch devices for all types of users (from children to the elderly) has grown considerably in the recent years. However, despite the huge interest in this technology there is a lack of research addressing usability studies on children with Down's Syndrome. This article evaluates the abilities of these children (aged from 5 to 10 years) when performing a basic set of multi-touch gestures (tap, double tap, long press, drag, scale up and down, rotation) in tablet devices. The results show that regardless of their more limited motor skills, DS children are able to perform most of the evaluated multi-touch gestures with success rates close to 100% and that this technology could be fully exploited for developing applications targeted specifically at this type of user.Spanish Ministry of Economy and Competitiveness and funded by the European Development Regional Fund (EDRF-FEDER) with the project TIN2014-60077-R (SUPEREMOS). This work was also supported by a pre-doctoral fellowship within the Formacion de Profesorado Universitario (FPU) program from the Spanish Ministry of Education, Culture and Sports to V. Nacher (FPU14/00136) and by a pre-doctoral scholarship given by the SENESCYT (Secretaria Nacional de Educacion Superior, Ciencia y Tecnologia e Innovacion) of the government of Ecuador (No. 381-2012) to Doris Caliz.Nácher-Soler, VE.; Cáliz, D.; Jaén Martínez, FJ.; Martínez, L. (2018). Examining the Usability of Touch Screen Gestures for Children With Down Syndrome. Interacting with Computers. 30(3):258-272. https://doi.org/10.1093/iwc/iwy011S25827230

Accessible Conversational User Interfaces: Considerations for design

Conversational user interfaces (CUIs), such as chatbots and voice assistants, are increasingly common in areas of day-to-day life, and can be expected to become ever more pervasive in the future. These interfaces are being designed for ever more complex interactions, and they appear to have potential to be beneficial to people with disabilities to interact through the web and with technologies embedded in the environment. However, to fulfil this promise they need to be designed to be accessible. This paper reviews a range of current guidance, reports, research and literature on accessible design for different disability groups, including users with mental health issues, autism, health conditions, cognitive disabilities, dyslexia or learning difficulties, and sensory, mobility or dexterity impairments. We collate the elements from this body of guidance that appear relevant to the design of accessible CUIs, and instances where guidance presents issues which are less conclusive, and require further exploration. Using this, we develop a set of questions which could be useful in the further research and development of accessible CUIs. We conclude by considering why CUIs could present opportunities for furthering accessibility, by introducing an example of this potential – a project to design an assistant to support students to disclose their disabilities and organise support, without the need to fill in forms

Design and evaluation of a graphical user interface for facilitating expert knowledge transfer: a teleoperation case study

Nowadays, teleoperation systems are increasingly used for the training of specific skills to carry out complex tasks in dangerous environments. One of the challenges of these systems is to ensure that the time it takes for users to acquire these skills is as short as possible. For this, the user interface must be intuitive and easy to use. This document describes the design and evaluation of a graphical user interface so that a non-expert user could use a teleoperated system intuitively and without excessive training time. To achieve our goal, we use a user-centered design process model. To evaluate the interface, we use our own methodology and the results allow improving its usability.Peer ReviewedPostprint (author's final draft

How a Diverse Research Ecosystem Has Generated New Rehabilitation Technologies: Review of NIDILRR’s Rehabilitation Engineering Research Centers

Over 50 million United States citizens (1 in 6 people in the US) have a developmental, acquired, or degenerative disability. The average US citizen can expect to live 20% of his or her life with a disability. Rehabilitation technologies play a major role in improving the quality of life for people with a disability, yet widespread and highly challenging needs remain. Within the US, a major effort aimed at the creation and evaluation of rehabilitation technology has been the Rehabilitation Engineering Research Centers (RERCs) sponsored by the National Institute on Disability, Independent Living, and Rehabilitation Research. As envisioned at their conception by a panel of the National Academy of Science in 1970, these centers were intended to take a “total approach to rehabilitation”, combining medicine, engineering, and related science, to improve the quality of life of individuals with a disability. Here, we review the scope, achievements, and ongoing projects of an unbiased sample of 19 currently active or recently terminated RERCs. Specifically, for each center, we briefly explain the needs it targets, summarize key historical advances, identify emerging innovations, and consider future directions. Our assessment from this review is that the RERC program indeed involves a multidisciplinary approach, with 36 professional fields involved, although 70% of research and development staff are in engineering fields, 23% in clinical fields, and only 7% in basic science fields; significantly, 11% of the professional staff have a disability related to their research. We observe that the RERC program has substantially diversified the scope of its work since the 1970’s, addressing more types of disabilities using more technologies, and, in particular, often now focusing on information technologies. RERC work also now often views users as integrated into an interdependent society through technologies that both people with and without disabilities co-use (such as the internet, wireless communication, and architecture). In addition, RERC research has evolved to view users as able at improving outcomes through learning, exercise, and plasticity (rather than being static), which can be optimally timed. We provide examples of rehabilitation technology innovation produced by the RERCs that illustrate this increasingly diversifying scope and evolving perspective. We conclude by discussing growth opportunities and possible future directions of the RERC program

An end-to-end review of gaze estimation and its interactive applications on handheld mobile devices

In recent years we have witnessed an increasing number of interactive systems on handheld mobile devices which utilise gaze as a single or complementary interaction modality. This trend is driven by the enhanced computational power of these devices, higher resolution and capacity of their cameras, and improved gaze estimation accuracy obtained from advanced machine learning techniques, especially in deep learning. As the literature is fast progressing, there is a pressing need to review the state of the art, delineate the boundary, and identify the key research challenges and opportunities in gaze estimation and interaction. This paper aims to serve this purpose by presenting an end-to-end holistic view in this area, from gaze capturing sensors, to gaze estimation workflows, to deep learning techniques, and to gaze interactive applications.PostprintPeer reviewe

Advanced displays and natural user interfaces to support learning

[EN] Advanced displays and Natural User Interfaces (NUI) are a very suitable combination for developing systems to provide an enhanced and richer user experience. This combination can be appropriate in several fields and has not been extensively exploited. One of the fields that this combination is especially suitable for is education. Nowadays, children are growing up playing with computer games, using mobile devices, and other technological devices. New learning methods that use these new technologies can help in the learning process. In this paper, two new methods that use advanced displays and NUI for learning about a period of history are presented. One of the methods is an autostereoscopic system that lets children see themselves as a background in the game and renders the elements in 3D without the need for special glasses; the second method is a frontal projection system that projects the image on a table in 2D and works similarly to a touch table. The Microsoft Kinect© is used in both systems for the interaction. A comparative study to check different aspects was carried out. A total of 128 children from 7 to 11 years old participated in the study. From the results, we observed that the different characteristics of the systems did not influence the children s acquired knowledge, engagement, or satisfaction. There were statistically significant differences for depth perception and presence in which the autostereoscopic system was scored higher. However, of the two systems, the children considered the frontal projection to be easier to use. We would like to highlight that the scores for the two systems and for all the questions were very high. These results suggest that games of this kind (advanced displays and NUI) could be appropriate educational games and that autostereoscopy is a technology to exploit in their development.This work was funded by the Spanish Ministry of Science and Innovation through the APRENDRA project (TIN2009-14319-C02-01).Martín San José, JF.; Juan, M.; Mollá Vayá, RP.; Vivó Hernando, RA. (2017). Advanced displays and natural user interfaces to support learning. Interactive Learning Environments. https://doi.org/10.1080/10494820.2015.1090455