HCI for the deaf community: developing human-like avatars for sign language synthesis

With ever increasing computing power and advances in 3D animation technologies it is no surprise that 3D avatars for sign language (SL) generation are advancing too. Traditionally these avatars have been driven by somewhat expensive and inflexible motion capture technologies and perhaps this is the reason avatars do not feature in all but a few user interfaces (UIs). SL synthesis is a competing technology that is less costly, more versatile and may prove to be the answer to the current lack of access for the Deaf in HCI. This paper outlines the current state of the art in SL synthesis for HCI and how we propose to advance this by improving avatar quality and realism with a view to ameliorating communication and computer interaction for the Deaf community as part of a wider localisation project

Low-cost natural interface based on head movements

Sometimes people look for freedom in the virtual world. However, not all have the possibility to interact with a computer in the same way. Nowadays, almost every job requires interaction with computerized systems, so people with physical impairments do not have the same freedom to control a mouse, a keyboard or a touchscreen. In the last years, some of the government programs to help people with reduced mobility suffered a lot with the global economic crisis and some of those programs were even cut down to reduce costs. This paper focuses on the development of a touchless human-computer interface, which allows anyone to control a computer without using a keyboard, mouse or touchscreen. By reusing Microsoft Kinect sensors from old videogames consoles, a cost-reduced, easy to use, and open-source interface was developed, allowing control of a computer using only the head, eyes or mouth movements, with the possibility of complementary sound commands. There are already available similar commercial solutions, but they are so expensive that their price tends to be a real obstacle in their purchase; on the other hand, free solutions usually do not offer the freedom that people with reduced mobility need. The present solution tries to address these drawbacks. (C) 2015 Published by Elsevier B.V

Spanish Sign Language synthesis system

This is the author’s version of a work that was accepted for publication in Journal of Visual Languages and Computing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of Visual Languages and Computing,23, 3, (2012) DOI: 10.1016/j.jvlc.2012.01.003This work presents a new approach to the synthesis of Spanish Sign Language (LSE). Its main contributions are the use of a centralized relational database for storing sign descriptions, the proposal of a new input notation and a new avatar design, the skeleton structure of which improves the synthesis process. The relational database facilitates a highly detailed phonologic description of the signs that include parameter synchronization and timing. The centralized database approach has been introduced to allow the representation of each sign to be validated by the LSE National Institution, FCNSE. The input notation, designated HLSML, presents multiple levels of abstraction compared with current input notations. Redesigned input notation is used to simplify the description and the manual definition of LSE messages. Synthetic messages obtained using our approach have been evaluated by deaf users; in this evaluation a maximum recognition rate of 98.5% was obtained for isolated signs and a recognition rate of 95% was achieved for signed sentences

Pre-Travel Training And Real-Time Guidance System For People With Disabilities In Indoor Environments

Public transportation provides people with access to education, employment, health and community activities. However, navigating inside public hubs for people with disabilities such as cognitive or mobility impairments can be very challenging and dangerous. With the rapid development of digital technology such as Smartphones and sensors, there are unprecedented opportunities to assist people with disabilities to conquer these challenges. In this research, we aim to create a two-step indoor navigation solution for users with different mobility and orientation abilities. In the first step, we developed a virtual reality-based pre-travel training module that enables users to familiarize themselves with the virtual environment which represents the physical environment. After users feel confident and familiar enough with the environment, they proceed to the second step in which they visit the physical environment and use our real-time navigation assistance module. The pre-travel training module is developed using a Unity-based 3D game and includes a virtual indoor environment that represents the physical environment. The game provides a navigation function that highlights the path between the user location and the chosen destination. Considering the unique needs of cognitive impaired users, we designed action training modules in the game environment which train the user to use the ticket machine, fare gate and call boxes. Such training modules help cognitive impaired users familiarize themselves with the environment as well as gain confidence to experience the physical environment. When the users are ready to visit the physical environment, they use our real-time navigation assistance module which includes the same 3D virtual environment developed for the pre-travel training module. This approach is particularly important for people with cognitive impairment since they cannot organize navigation cues effectively. Using the Bluetooth Low Energy (BLE) infrastructure in the environment, our localization algorithm can track the user location in real-time. Subsequently, the user’s location will be integrated into the game environment so that the navigation path between the user’s current location and the selected destination can be generated and visualized by the user on the fly

Data Analytics and Visualization for Virtual Simulation

Healthcare organizations attract a diversity of caregivers and patients by providing essential care. While interacting with people of various races, ethnicity, and economical background, caregivers need to be empathetic and compassionate. Proper training and exposure are needed to understand the patient’s background and handle different situations and provide the best care for the patient. With social determinants of health (SDOH) as the basis, the thesis focuses on providing exposure through “Wright LIFE (Lifelike Immersion for Equity) - A simulation-based training tool” to two such scenarios covering patients from the LGBTQIA+ community & autism spectrum disorder (ASD). This interactive tool helps to create mindfulness about the social and economic disparities faced by the patients through realistic and captivating gameplay. Though the primary focus of the “Wright LIFE” application is “Digital Learning”, it would help to understand how effective the application is in terms of improving the provider\u27s abilities. Through statistical evidence, the tool can be improved, which in turn will improve the user experience. For this analysis, during the simulation, we also focus on collecting the data gathered from the participants through surveys. The simulation includes different questionnaires where participants can provide feedback at various stages within the simulation. This then allows for a comparison between the participants’ responses to see the rate of improvement as a result of the simulation. To analyze the data from the participant\u27s responses, data analysis, and visualization tools help to represent the data using charts, infographics, animations, and many more to assist this in this analytic process. The analysis of the data can help to understand the trend of the participants’ responses to the questionnaire. The goal of the questionnaire is to collect participants’ responses to assess anxiety, frustration, and compassion levels pre- and post-simulation. A comparative analysis is then performed. This analysis shows that the provider’s anxiety and frustration decreased after the simulation whereas the compassion increased. This is an indication that the simulation can improve the provider’s experience while working with patients with biases. The data also helped to identify the users who actively participated in the survey based on demographic data like gender, profession, experience, and age. “A picture is worth a thousand words”. Through visualization, we can bring the data to life and provide a clear idea of what the data represents by giving visual context. Tableau is used for visualizing the survey data collected from the “SDOH” simulation consisting of responses from the providers before and after the interaction with the patients. The visualizations transform the raw data into simple and informative graphs to understand the behavioral trends and to check how the providers respond to the stories in the simulations. This allows us to determine the effectiveness of the simulation more efficiently

Physical and rehabilitation medicine (PRM) care pathways: Adults with severe traumatic brain injury

AbstractThis document is part of a series of guidelines documents designed by the French Physical and Rehabilitation Medicine Society (SOFMER) and the French Federation of PRM (FEDMER). These reference documents focus on a particular pathology (here patients with severe TBI). They describe for each given pathology patients’ clinical and social needs, PRM care objectives and necessary human and material resources of the pathology-dedicated pathway. ‘Care pathways in PRM’ is therefore a short document designed to enable readers (physician, decision-maker, administrator, lawyer, finance manager) to have a global understanding of available therapeutic care structures, organization and economic needs for patients’ optimal care and follow-up. After a severe traumatic brain injury, patients might be divided into three categories according to impairment's severity, to early outcomes in the intensive care unit and to functional prognosis. Each category is considered in line with six identical parameters used in the International Classification of Functioning, Disability and Health (World Health Organization), focusing thereafter on personal and environmental factors liable to affect the patients’ needs

Hybrid paradigm for Spanish Sign Language synthesis

The final publication is available at Springer via http://dx.doi.org/10.1007/s10209-011-0245-9This work presents a hybrid approach to sign language synthesis. This approach allows the hand-tuning of the phonetic description of the signs, which focuses on the time aspect of the sign. Therefore, the approach retains the capacity for the performing of morpho-phonological operations, like notation-based approaches, and improves the synthetic signing performance, such as the hand-tuned animations approach. The proposed approach simplifies the input message description using a new high-level notation and storage of sign phonetic descriptions in a relational database. Such relational database allows for more flexible sign phonetic descriptions; it also allows for a description of sign timing and the synchronization between sign phonemes. The new notation, named HLSML, is a gloss-based notation focusing on message description in it. HLSML introduces several tags that allow for the modification of the signs in the message that defines dialect and mood variations, both of which are defined in the relational database, and message timing, including transition durations and pauses. A new avatar design is also proposed that simplifies the development of the synthesizer and avoids any interference with the independence of the sign language phonemes during animation. The obtained results showed an increase of the sign recognition rate compared to other approaches. This improvement was based on the active role that the sign language experts had in the description of signs, which was the result of the flexibility of the sign storage approach. The approach will simplify the description of synthesizable signed messages, thus facilitating the creation of multimedia-signed contents

Virtual sculpting and 3D printing for young people with disabilities

In this paper, we present the SHIVA project which was designed to provide virtual sculpting tools for young people with complex disabilities, to allow them to engage with artistic and creative activities that they might otherwise never be able to access. Modern 3D printing then allows us to physically build their creations. To achieve this, we combined our expertise in education, accessible technology, user interfaces and geometric modelling. We built a generic accessible graphical user interface (GUI) and a suitable geometric modelling system and used these to produce two prototype modelling exercises. These tools were deployed in a school for students with complex disabilities and are now being used for a variety of educational and developmental purposes. In this paper, we present the project's motivations, approach and implementation details together with initial results, including 3D printed objects designed by young people who have disabilties

Virtual reality and its role in removing the barriers that turn cognitive impairments into intellectual disability

Early expectations of the contribution that virtual reality (VR) could make to education far exceeded actual applications. This was largely due to the initial immaturity of the technology and a lack of evidence base on which to base design and utilisation. While the early developments in computer based learning largely concentrated on mainstream education, leaving those with special needs behind, the potential of VR as an educational tool was exploited for those with intellectual disabilities right from the start. This paper describes the empirical evidence that has contributed to the development of educational virtual reality for those with intellectual disabilities: studies on transfer of learning from the virtual to the real world; how teachers might support those using VR; the design of virtual environments and what input/control devices best facilitate use of desktop VR. Future developments and ethical issues are also considered