Investigando Natural User Interfaces (NUIs) : tecnologias e interação em contexto de acessibilidade

Orientador: Maria Cecília Calani BaranauskasTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Natural User Interfaces (NUIs) representam um novo paradigma de interação, com a promessa de ser mais intuitivo e fácil de usar do que seu antecessor, que utiliza mouse e teclado. Em um contexto no qual as tecnologias estão cada vez mais invisíveis e pervasivas, não só a quantidade mas também a diversidade de pessoas que participam deste contexto é crescente. Nesse caso, é preciso estudar como esse novo paradigma de interação de fato consegue ser acessível a todas as pessoas que podem utilizá-lo no dia-a-dia. Ademais, é preciso também caracterizar o paradigma em si, para entender o que o torna, de fato, natural. Portanto, nesta tese apresentamos o caminho que percorremos em busca dessas duas respostas: como caracterizar NUIs, no atual contexto tecnológico, e como tornar NUIs acessíveis para todos. Para tanto, primeiro apresentamos uma revisão sistemática de literatura com o estado da arte. Depois, mostramos um conjunto de heurísticas para o design e a avaliação de NUIs, que foram aplicadas em estudos de caso práticos. Em seguida, estruturamos as ideias desta pesquisa dentro dos artefatos da Semiótica Organizacional, e obtivemos esclarecimentos sobre como fazer o design de NUIs com Acessibilidade, seja por meio de Design Universal, seja para propor Tecnologias Assistivas. Depois, apresentamos três estudos de caso com sistemas NUI cujo design foi feito por nós. A partir desses estudos de caso, expandimos nosso referencial teórico e conseguimos, por fim, encontrar três elementos que resumem a nossa caracterização de NUI: diferenças, affordances e enaçãoAbstract: Natural User Interfaces (NUIs) represent a new interaction paradigm, with the promise of being more intuitive and easy to use than its predecessor, that utilizes mouse and keyboard. In a context where technology is becoming each time more invisible and pervasive, not only the amount but also the diversity of people who participate in this context is increasing. In this case, it must be studied how this new interaction paradigm can, in fact, be accessible to all the people who may use it on their daily routine. Furthermore, it is also necessary to characterize the paradigm itself, to understand what makes it, in fact, natural. Therefore, in this thesis we present the path we took in search of these two answers: how to characterize NUIs in the current technological context, and how to make NUIs accessible to all. To do so, first we present a systematic literature review with the state of the art. Then, we show a set of heuristics for the design and evaluation of NUIs, which were applied in practical study cases. Afterwards, we structure the ideas of this research into the Organizational Semiotics artifacts, and we obtain insights into how to design NUIs with Accessibility, be it through Universal Design, be it to propose Assistive Technologies. Then, we present three case studies with NUI systems which we designed. From these case studies, we expanded our theoretical references were able to, finally, find three elements that sum up our characterization of NUI: differences, affordances and enactionDoutoradoCiência da ComputaçãoDoutora em Ciência da Computação160911/2015-0CAPESCNP

Digital inclusion of nursing home residents: a usability evaluation of the digital kiosk siosLIFE™

The fast demographic ageing and technological progress are leading to a greater demand to develop digital solutions that can foster communication, information, and socialization of the elderly population. In the past decade, kiosks have been used to prevent digital exclusion and to promote the quality of life of this age group. This paper analyses siosLIFE™, a digital kiosk that is gradually getting attention from the public. The methodology adopted consisted in a usability test with guided tasks, using a cognitive walkthrough and think aloud protocol, with participants being residents from a nursing home. The results show that siosLIFE™ complies with some usability recommendations, but there are several improvements regarding the interface, contents, integration of support systems, and assistive technologies that can be made.publishe

How can we enhance elderly health and well-being through various forms of game-based activities?

The health of an aging population is gradually turning into a topic of focus not only domestically but also internationally. The target group of this study is older Americans who are 65 and older. More specifically, elders who lack the motivation to exercise. Currently, related research has shown regular physical exercise is critical for elders to keep fit. However, participating in regular exercise can be challenging for older adults with physical limitations, and it’s often difficult to motivate oneself. There has been an expanded focus on the game design intended to motivate elders to contribute to their overall well-being.Through qualitative research, this project aims to understand which parts of the body require routine exercise and which game-based activities elders prefer. My goal is to design a motivational game that keeps the aging population physically fit, mentally sharp, and socially engaged. In conjunction with this research, I’ve designed a physical large-scale puzzle called “CubeX” as a method to motivate elders to incorporate exercise with a leisurely experience, and this activity might also serve to keep the brain active

Virtual sign : a real time bidirectional translator of portuguese sign language

Promoting equity, equal opportunities to all and social inclusion of people with disabilities is a concern of modern societies at large and a key topic in the agenda of European Higher Education. Despite all the progress, we cannot ignore the fact that the conditions provided by the society for the deaf are still far from being perfect. The communication with deaf by means of written text is not as efficient as it might seem at first. In fact, there is a very deep gap between sign language and spoken/written language. The vocabulary, the sentence construction and the grammatical rules are quite different among these two worlds. These facts bring significant difficulties in reading and understanding the meaning of text for deaf people and, on the other hand, make it quite difficult for people with no hearing disabilities to understand sign language. The deployment of tools to assist the daily communication, in schools, in public services, in museums and other, between deaf people and the rest may be a significant contribution to the social inclusion of the deaf community. The work described in this paper addresses the development of a bidirectional translator between Portuguese Sign Language and Portuguese text. The translator from sign language to text resorts to two devices, namely the Microsoft Kinect and 5DT Sensor Gloves in order to gather data about the motion and shape of the hands. The hands configurations are classified using Support Vector Machines. The classification of the movement and orientation of the hands are achieved through the use of Dynamic Time Warping algorithm. The translator exhibits a precision higher than 90%. In the other direction, the translation of Portuguese text to Portuguese Sign Language is supported by a 3D avatar which interprets the entered text and performs the corresponding animations

Machine Sensation

Emphasising the alien qualities of anthropomorphic technologies, Machine Sensation makes a conscious effort to increase rather than decrease the tension between nonhuman and human experience. In a series of rigorously executed cases studies, including natural user interfaces, artificial intelligence as well as sex robots, Leach shows how object-oriented ontology enables one to insist upon the unhuman nature of technology while acknowledging its immense power and significance in human life. Machine Sensation meticulously engages OOO, Actor Network Theory, the philosophy of technology, cybernetics and posthumanism in innovative and gripping ways

Adventures in software engineering : plugging HCI & acessibility gaps with open source solutions

There has been a great deal of research undertaken in the field of Human-Computer Interfaces (HCI), input devices, and output modalities in recent years. From touch-based and voice control input mechanisms such as those found on modern smart-devices to the use of touch-free input through video-stream/image analysis (including depth streams and skeletal mapping) and the inclusion of gaze tracking, head tracking, virtual reality and beyond - the availability and variety of these I/O (Input/Output) mechanisms has increased tremendously and progressed both into our living rooms and into our lives in general. With regard to modern desktop computers and videogame consoles, at present many of these technologies are at a relatively immature stage of development - their use often limited to simple adjuncts to the staple input mechanisms of mouse, keyboard, or joystick / joypad inputs. In effect, we have these new input devices - but we're not quite sure how best to use them yet; that is, where their various strengths and weaknesses lie, and how or if they can be used to conveniently and reliably drive or augment applications in our everyday lives. In addition, much of this technology is provided by proprietary hardware and software, providing limited options for customisation or adaptation to better meet the needs of specific users. Therefore, this project investigated the development of open source software solutions to address various aspects of innovative user I/O in a flexible manner. Towards this end, a number of original software applications have been developed which incorporate functionality aimed at enhancing the current state of the art in these areas and making that software freely available for use by any who may find it beneficial.Doctor of Philosoph

Designing movement-based play with young people using powered wheelchairs

Young people using powered wheelchairs have limited access to engaging leisure activities. We address this issue through a two-stage project; 1) the participatory development of a set of wheelchair-controlled, movement-based games (with 9 participants at a school that provides education for young people who have special needs) and 2) three case studies (4 participants) exploring player perspectives on a set of three wheelchair-controlled casual games. Our results show that movement-based playful experiences are engaging for young people using powered wheelchairs. However, the participatory design process and case studies also reveal challenges for game accessibility regarding the integration of movement in games, diversity of abilities among young people using powered wheelchairs, and the representation of disability in games. In our paper, we explore how to address those challenges in the development of accessible, empowering movement-based games, which is crucial to the wider participation of young people using powered wheelchairs in play

Study of emotion in videogames : understanding presence and behaviour

Only when videogames are released are we able to look at them and analyse them. Nowadays, platforms to share our thoughts and opinions about a videogame, or part of it, are everywhere, with both positive and negative commentaries being shared daily. However, what makes a game be seen as a positive experience and what components satisfy and engage players in it? In this Dissertation, we aim to comprehend how players perceive videogames and what motivates and triggers emotions one has during play. We will take a look at several different concepts that all work together when playing a videogame. We will start by understanding what Interaction is and how humans behave. Afterwards, we will better investigate the widely used topic of Immersion, and its unknown and unrecognized brother Presence. From there, we will divide involvement in game play in two parts, the technological side, which relates to natural interfaces and mastery of controls, and the side of design and implementation of content, more specifically the concept of Agency and how it plays a huge part in making players feel part of the game.Só quando um videojogo é lançado é que o podemos analisar e rever. Atualmente, encontramos plataformas para partilhar a nossa opinião acerca de um videojogo, ou parte dele, em qualquer lado, com comentários positivos e negativos a serem partilhados diariamente. No entanto, o que é que faz um jogo ser visto como uma experiência positiva e quais são os componentes que satisfazem e envolvem jogadores? Nesta Dissertação, pretendemos compreender como é que jogadores percecionam um videojogo e que emoções são despoletadas que os motiva a jogar. Iremos analisar diferentes conceitos que contribuem para o jogar de um videojogo. Começaremos por ver o que é a Interação e como é que o ser humano se comporta e age. Prosseguindo, iremos analisar o já bastante usado conceito de Imersão, e o seu desconhecido e menos reconhecido irmão, Presença. Daí iremos dividir o envolvimento com um videojogo em duas partes, no lado tecnológico, relacionado com interfaces naturais e mestria de controlos, e no lado de design e implementação de conteúdo, mais especificamente no conceito de Agência e a maneira como esta integra os jogadores no jogo

The Use of digital games to enhance the physical exercise activity of the elderly : a case of Finland

According to the World Health Organization (WHO), population ageing is a global phenomenon, which brings both challenges and opportunities for society. The current longer expected lifespan can create opportunities for the elderly to contribute in many ways to their families and communities. However, it greatly depends on their quality of life, which is affected by many factors, including physical and functional health, social well-being, and cognitive abilities. The WHO (2012) states that physical health is one of the indicators for the elderly’s quality of life, and it declines with increasing age. Participation in regular physical exercises can help the elderly improve their physical and mental health, and this has been aided by the use of modern technologies to promote the elderly’s physical and functional health. Of these latest technologies, digital games have shown promise to improve and enhance the elderly’s physical activities through fun and engaging gameplay. The literature highlights that some commercial games in the market (e.g. Microsoft Kinect- Sports and Nintendo Wii Sports games) have the potential to improve the elderly’s physical health such as gait, balance, and fall prevention. However, researchers argue that these commercial games are not designed specifically for the elderly and their physical exercise activities. They state that most commercial games are not user-friendly for the elderly whose functional and physical abilities are limited due to their advanced years. The literature points out that more studies need to be undertaken to understand the usability and usefulness of digital games for physical exercise activities so that game designers can create elderly-friendly digital games in the future. In Finland, the government has been focusing on promoting healthy ageing and increasing home care services for the elderly. In recent years, Finnish researchers have used digital games to promote older Finns’ healthy and active ageing. The existing literature, whilst showing the potential of digital games for elderly Finns’ physical health, also acknowledges further research is needed particularly in the context of Finland. Thus, in this study, we aimed at investigating digital games to specifically assess their applications for older Finns’ physical activities, focusing on the quality of users’ experiences, and their reported ease of use and perceived usefulness. We used the mixed methods approach, which applies both qualitative and quantitative research methods. The study design included four stages: requirements gathering, analysis and design, prototyping, and evaluation. Firstly, we conducted pre-studies to elicit users’ requirements. This was followed by the analysis of the resulting data to identify trends and patterns, which fuelled ideas in the brainstorming game design and development phases. The final product was a digital game-based physical exercise called the Skiing Game. We then evaluated the Skiing Game in Finland with 21 elderly Finns (M=7, F=14, Average Age =76). By using questionnaires, observation, and interviews, we investigated user experiences, focusing on the game’s usability, and usefulness for enhancing the physical activity and wellbeing of the elderly. We also conducted a comparative test of the Skiing Game in Japan with 24 elderly Japanese participants (M=12, F=12, Average Age = 72) to further understand non-Finnish elderly users’ experiences. The findings from the usability study of the Skiing Game in Finland demonstrated that elderly Finns had a positive experience in the gameplay, and their motivation was noticeably high. It also confirmed that elderly Finns have a genuine interest in digital game-based exercises and strong intentions to play digital games as a form of physical exercise in the future. Although prior to the study most of them had negative views and misconceptions about digital games, after the gameplay their attitudes were decidedly positive. They acknowledged that whilst playing digital games could be an alternative way of exercising for them their use would primarily be when they don’t have access to their usual non-digital physical exercise. The Japanese usability of the Skiing Game showed that the elderly Japanese people also had positive user experiences in playing digital games, and also intend to use them in the future. Similarly, after playing the game they reported that their attitudes towards digital games become positive, and indicated playing digital games could be an alternative way of exercising. Although the comparison of the two studies suggests that the elderly Finns had relatively more positive experiences whilst playing the Skiing Game, compared to their Japanese counterparts, in general, both groups had a positive experience in the gameplay and showed interest in digital games as an alternative exercise. Based on the usability lessons learned from these two studies, recommendations for practitioners and designers regarding improvements in game design and development are made in this report. Implementing these modifications into future designs and further development of digital games for the elderly will improve their commercial viability and user uptake. The findings from this study can provide valuable insights, particularly for Finnish policymakers and healthcare practitioners who are keen to introduce digital games into the aged-care sector in Finland. The studies have also provided valuable insights into the optimal methods for introducing Finnish digital games to international markets, in particular, digital games tailored specifically for the physical exercise needs and motivations of the elderly. By taking into consideration the limitations of the study, we provide our future studies and further improvements of the game to be conducted