Investigando Natural User Interfaces (NUIs) : tecnologias e interação em contexto de acessibilidade

Orientador: Maria Cecília Calani BaranauskasTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Natural User Interfaces (NUIs) representam um novo paradigma de interação, com a promessa de ser mais intuitivo e fácil de usar do que seu antecessor, que utiliza mouse e teclado. Em um contexto no qual as tecnologias estão cada vez mais invisíveis e pervasivas, não só a quantidade mas também a diversidade de pessoas que participam deste contexto é crescente. Nesse caso, é preciso estudar como esse novo paradigma de interação de fato consegue ser acessível a todas as pessoas que podem utilizá-lo no dia-a-dia. Ademais, é preciso também caracterizar o paradigma em si, para entender o que o torna, de fato, natural. Portanto, nesta tese apresentamos o caminho que percorremos em busca dessas duas respostas: como caracterizar NUIs, no atual contexto tecnológico, e como tornar NUIs acessíveis para todos. Para tanto, primeiro apresentamos uma revisão sistemática de literatura com o estado da arte. Depois, mostramos um conjunto de heurísticas para o design e a avaliação de NUIs, que foram aplicadas em estudos de caso práticos. Em seguida, estruturamos as ideias desta pesquisa dentro dos artefatos da Semiótica Organizacional, e obtivemos esclarecimentos sobre como fazer o design de NUIs com Acessibilidade, seja por meio de Design Universal, seja para propor Tecnologias Assistivas. Depois, apresentamos três estudos de caso com sistemas NUI cujo design foi feito por nós. A partir desses estudos de caso, expandimos nosso referencial teórico e conseguimos, por fim, encontrar três elementos que resumem a nossa caracterização de NUI: diferenças, affordances e enaçãoAbstract: Natural User Interfaces (NUIs) represent a new interaction paradigm, with the promise of being more intuitive and easy to use than its predecessor, that utilizes mouse and keyboard. In a context where technology is becoming each time more invisible and pervasive, not only the amount but also the diversity of people who participate in this context is increasing. In this case, it must be studied how this new interaction paradigm can, in fact, be accessible to all the people who may use it on their daily routine. Furthermore, it is also necessary to characterize the paradigm itself, to understand what makes it, in fact, natural. Therefore, in this thesis we present the path we took in search of these two answers: how to characterize NUIs in the current technological context, and how to make NUIs accessible to all. To do so, first we present a systematic literature review with the state of the art. Then, we show a set of heuristics for the design and evaluation of NUIs, which were applied in practical study cases. Afterwards, we structure the ideas of this research into the Organizational Semiotics artifacts, and we obtain insights into how to design NUIs with Accessibility, be it through Universal Design, be it to propose Assistive Technologies. Then, we present three case studies with NUI systems which we designed. From these case studies, we expanded our theoretical references were able to, finally, find three elements that sum up our characterization of NUI: differences, affordances and enactionDoutoradoCiência da ComputaçãoDoutora em Ciência da Computação160911/2015-0CAPESCNP

New Media, Multiliteracies, and the Globalized Classroom

New Media, Multiliteracies, and the Globalized Classroom contends that changing demographic diversity in American higher education and increasing globalization call for curricular and pedagogical transformation of writing courses. The writing curriculum for globalized classrooms should be responsive to the resources students bring with them to the classroom and cultivate in them the multiple literacies--both old and new--needed to successfully navigate an increasingly global workplace and a complexly interwoven world. The scaffolding of students\u27 natural meaning making capacities, and teaching of multiple literacies, such as visual, academic, critical, digital, multimodal, and intercultural, should take place simultaneously in the writing classes for new literacies are in a synergistic, reciprocal, and constantly evolving relationship with older literacies (Swenson et al. 357). The interplay of new and old literacies in literate activities in the academy and professions can lead to productive interactions and work. In fact, this study is an experiment with and investigation into how diverse students in a sophomore level writing class in a large research university in the U.S. North East responded to a curriculum and pedagogical approach framed around the idea of multiple literacies. The pilot course drew insights and resources from some closely aligned fields of new media, globalization, World Englishes, intercultural communication, literacy studies, and media studies, exploring and exploiting the potentials these intersecting fields have for improving the practice of teaching writing to a diverse body of students. It took a multiliterate approach to teaching writing in its expanded sense with assignments in multiple media and modes-- alphabetic and digital literacy narratives, rhetorical analysis of a digital artifact (music video, ad, cartoon/movie clip etc.), argument essay, remediation of argument essay into web forms for local and global audiences, collaborative documentary production, blogging, and small group presentations

Smart workplaces: a system proposal for stress management

Over the past last decades of contemporary society, workplaces have become the primary source of many health issues, leading to mental problems such as stress, depression, and anxiety. Among the others, environmental aspects have shown to be the causes of stress, illness, and lack of productivity. With the arrival of new technologies, especially in the smart workplaces field, most studies have focused on investigating the building energy efficiency models and human thermal comfort. However, little has been applied to occupants’ stress recognition and well-being overall. Due to this fact, this present study aims to propose a stress management solution for an interactive design system that allows the adapting of comfortable environmental conditions according to the user preferences by measuring in real-time the environmental and biological characteristics, thereby helping to prevent stress, as well as to enable users to cope stress when being stressed. The secondary objective will focus on evaluating one part of the system: the mobile application. The proposed system uses several usability methods to identify users’ needs, behavior, and expectations from the user-centered design approach. Applied methods, such as User Research, Card Sorting, and Expert Review, allowed us to evaluate the design system according to Heuristics Analysis, resulting in improved usability of interfaces and experience. The study presents the research results, the design interface, and usability tests. According to the User Research results, temperature and noise are the most common environmental stressors among the users causing stress and uncomfortable conditions to work in, and the preference for physical activities over the digital solutions for coping with stress. Additionally, the System Usability Scale (SUS) results identified that the system’s usability was measured as “excellent” and “acceptable” with a final score of 88 points out of the 100. It is expected that these conclusions can contribute to future investigations in the smart workplaces study field and their interaction with the people placed there.Nas últimas décadas da sociedade contemporânea, o local de trabalho tem se tornado principal fonte de muitos problemas de saúde mental, como o stress, depressão e ansiedade. Os aspetos ambientais têm se revelado como as causas de stress, doenças, falta de produtividade, entre outros. Atualmente, com a chegada de novas tecnologias, principalmente na área de locais de trabalho inteligentes, a maioria dos estudos tem se concentrado na investigação de modelos de eficiência energética de edifícios e conforto térmico humano. No entanto, pouco foi aplicado ao reconhecimento do stress dos ocupantes e ao bem-estar geral das pessoas. Diante disso, o objetivo principal é propor um sistema de design de gestão do stress para um sistema de design interativo que permita adaptar as condições ambientais de acordo com as preferências de utilizador, medindo em tempo real as características ambientais e biológicas, auxiliando assim na prevenção de stress, bem como ajuda os utilizadores a lidar com o stress quando estão sob o mesmo. O segundo objetivo é desenhar e avaliar uma parte do projeto — o protótipo da aplicação móvel através da realização de testes de usabilidade. O sistema proposto resulta da abordagem de design centrado no utilizador, utilizando diversos métodos de usabilidade para identificar as necessidades, comportamentos e as expectativas dos utilizadores. Métodos aplicados, como Pesquisa de Usuário, Card Sorting e Revisão de Especialistas, permitiram avaliar o sistema de design de acordo com a análise heurística, resultando numa melhoria na usabilidade das interfaces e experiência. O estudo apresenta os resultados da pesquisa, a interface do design e os testes de usabilidade. De acordo com os resultados de User Research, a temperatura e o ruído são os stressores ambientais mais comuns entre os utilizadores, causando stresse e condições menos favoráveis para trabalhar, igualmente existe uma preferência por atividades físicas sobre as soluções digitais na gestão do stresse. Adicionalmente, os resultados de System Usability Scale (SUS) identificaram a usabilidade do sistema de design como “excelente” e “aceitável” com pontuação final de 88 pontos em 100. É esperado que essas conclusões possam contribuir para futuras investigações no campo de estudo dos smart workplaces e sua interação com os utilizadores

Analyzing the Impact of Spatio-Temporal Sensor Resolution on Player Experience in Augmented Reality Games

Along with automating everyday tasks of human life, smartphones have become one of the most popular devices to play video games on due to their interactivity. Smartphones are embedded with various sensors which enhance their ability to adopt new new interaction techniques for video games. These integrated sen- sors, such as motion sensors or location sensors, make the device able to adopt new interaction techniques that enhance usability. However, despite their mobility and embedded sensor capacity, smartphones are limited in processing power and display area compared to desktop computer consoles. When it comes to evaluat- ing Player Experience (PX), players might not have as compelling an experience because the rich graphics environments that a desktop computer can provide are absent on a smartphone. A plausible alternative in this regard can be substituting the virtual game world with a real world game board, perceived through the device camera by rendering the digital artifacts over the camera view. This technology is widely known as Augmented Reality (AR). Smartphone sensors (e.g. GPS, accelerometer, gyro-meter, compass) have enhanced the capability for deploying Augmented Reality technology. AR has been applied to a large number of smartphone games including shooters, casual games, or puzzles. Because AR play environments are viewed through the camera, rendering the digital artifacts consistently and accurately is crucial because the digital characters need to move with respect to sensed orientation, then the accelerometer and gyroscope need to provide su ciently accurate and precise readings to make the game playable. In particular, determining the pose of the camera in space is vital as the appropriate angle to view the rendered digital characters are determined by the pose of the camera. This defines how well the players will be able interact with the digital game characters. Depending in the Quality of Service (QoS) of these sensors, the Player Experience (PX) may vary as the rendering of digital characters are affected by noisy sensors causing a loss of registration. Confronting such problem while developing AR games is di cult in general as it requires creating wide variety of game types, narratives, input modalities as well as user-testing. Moreover, current AR games developers do not have any specific guidelines for developing AR games, and concrete guidelines outlining the tradeoffs between QoS and PX for different genres and interaction techniques are required. My dissertation provides a complete view (a taxonomy) of the spatio-temporal sensor resolution depen- dency of the existing AR games. Four user experiments have been conducted and one experiment is proposed to validate the taxonomy and demonstrate the differential impact of sensor noise on gameplay of different genres of AR games in different aspect of PX. This analysis is performed in the context of a novel instru- mentation technology, which allows the controlled manipulation of QoS on position and orientation sensors. The experimental outcome demonstrated how the QoS of input sensor noise impacts the PX differently while playing AR game of different genre and the key elements creating this differential impact are - the input modality, narrative and game mechanics. Later, concrete guidelines are derived to regulate the sensor QoS as complete set of instructions to develop different genres or AR games

A Framework For Abstracting, Designing And Building Tangible Gesture Interactive Systems

This thesis discusses tangible gesture interaction, a novel paradigm for interacting with computer that blends concepts from the more popular fields of tangible interaction and gesture interaction. Taking advantage of the human innate abilities to manipulate physical objects and to communicate through gestures, tangible gesture interaction is particularly interesting for interacting in smart environments, bringing the interaction with computer beyond the screen, back to the real world. Since tangible gesture interaction is a relatively new field of research, this thesis presents a conceptual framework that aims at supporting future work in this field. The Tangible Gesture Interaction Framework provides support on three levels. First, it helps reflecting from a theoretical point of view on the different types of tangible gestures that can be designed, physically, through a taxonomy based on three components (move, hold and touch) and additional attributes, and semantically, through a taxonomy of the semantic constructs that can be used to associate meaning to tangible gestures. Second, it helps conceiving new tangible gesture interactive systems and designing new interactions based on gestures with objects, through dedicated guidelines for tangible gesture definition and common practices for different application domains. Third, it helps building new tangible gesture interactive systems supporting the choice between four different technological approaches (embedded and embodied, wearable, environmental or hybrid) and providing general guidance for the different approaches. As an application of this framework, this thesis presents also seven tangible gesture interactive systems for three different application domains, i.e., interacting with the In-Vehicle Infotainment System (IVIS) of the car, the emotional and interpersonal communication, and the interaction in a smart home. For the first application domain, four different systems that use gestures on the steering wheel as interaction means with the IVIS have been designed, developed and evaluated. For the second application domain, an anthropomorphic lamp able to recognize gestures that humans typically perform for interpersonal communication has been conceived and developed. A second system, based on smart t-shirts, recognizes when two people hug and reward the gesture with an exchange of digital information. Finally, a smart watch for recognizing gestures performed with objects held in the hand in the context of the smart home has been investigated. The analysis of existing systems found in literature and of the system developed during this thesis shows that the framework has a good descriptive and evaluative power. The applications developed during this thesis show that the proposed framework has also a good generative power.Questa tesi discute l’interazione gestuale tangibile, un nuovo paradigma per interagire con il computer che unisce i principi dei più comuni campi di studio dell’interazione tangibile e dell’interazione gestuale. Sfruttando le abilità innate dell’uomo di manipolare oggetti fisici e di comunicare con i gesti, l’interazione gestuale tangibile si rivela particolarmente interessante per interagire negli ambienti intelligenti, riportando l’attenzione sul nostro mondo reale, al di là dello schermo dei computer o degli smartphone. Poiché l’interazione gestuale tangibile è un campo di studio relativamente recente, questa tesi presenta un framework (quadro teorico) che ha lo scopo di assistere lavori futuri in questo campo. Il Framework per l’Interazione Gestuale Tangibile fornisce supporto su tre livelli. Per prima cosa, aiuta a riflettere da un punto di vista teorico sui diversi tipi di gesti tangibili che possono essere eseguiti fisicamente, grazie a una tassonomia basata su tre componenti (muovere, tenere, toccare) e attributi addizionali, e che possono essere concepiti semanticamente, grazie a una tassonomia di tutti i costrutti semantici che permettono di associare dei significati ai gesti tangibili. In secondo luogo, il framework proposto aiuta a concepire nuovi sistemi interattivi basati su gesti tangibili e a ideare nuove interazioni basate su gesti con gli oggetti, attraverso linee guida per la definizione di gesti tangibili e una selezione delle migliore pratiche per i differenti campi di applicazione. Infine, il framework aiuta a implementare nuovi sistemi interattivi basati su gesti tangibili, permettendo di scegliere tra quattro differenti approcci tecnologici (incarnato e integrato negli oggetti, indossabile, distribuito nell’ambiente, o ibrido) e fornendo una guida generale per la scelta tra questi differenti approcci. Come applicazione di questo framework, questa tesi presenta anche sette sistemi interattivi basati su gesti tangibili, realizzati per tre differenti campi di applicazione: l’interazione con i sistemi di infotainment degli autoveicoli, la comunicazione interpersonale delle emozioni, e l’interazione nella casa intelligente. Per il primo campo di applicazione, sono stati progettati, sviluppati e testati quattro differenti sistemi che usano gesti tangibili effettuati sul volante come modalità di interazione con il sistema di infotainment. Per il secondo campo di applicazione, è stata concepita e sviluppata una lampada antropomorfica in grado di riconoscere i gesti tipici dell’interazione interpersonale. Per lo stesso campo di applicazione, un secondo sistema, basato su una maglietta intelligente, riconosce quando due persone si abbracciano e ricompensa questo gesto con uno scambio di informazioni digitali. Infine, per l’interazione nella casa intelligente, è stata investigata la realizzazione di uno smart watch per il riconoscimento di gesti eseguiti con oggetti tenuti nella mano. L’analisi dei sistemi interattivi esistenti basati su gesti tangibili permette di dimostrare che il framework ha un buon potere descrittivo e valutativo. Le applicazioni sviluppate durante la tesi mostrano che il framework proposto ha anche un valido potere generativo

Design in the Age of Information: A Report to the National Science Foundation (NSF)

The Information Age is upon us - it has become a global force in our everyday lives. But the promise of significant benefits from this revolution, which has been driven largely by technologists, will not be realized without more careful planning and design of information systems that can be integral to the simultaneously emerging user-cultures. In cultural terms, information systems must be effective, reliable, affordable, intuitively meaningful, and available anytime and everywhere. In this phase of the information revolution, design will be essential

Information Technologies for Cognitive Decline

Information technology (IT) is used to establish a diagnosis and provide treatment for people with cognitive decline. The condition affects many before it becomes clear that more permanent changes, like dementia, could be noticed. Those who search for information are exposed to lots of information and different technologies which they need to make sense of and eventually use to help themselves. In this research literature and information available on the Internet were systematically analyzed to present methods used for diagnosis and treatment. Methods used for diagnosis are self-testing, sensors, Virtual Reality (VR), and brain imaging. Methods used for treatment are games, websites with information and media, Virtual Reality (VR), sensors, and robots. The resulting concept of knowledge was the basis of an artifact whose main goal was to present the facts to the broad public. This implied that a user-friendly artifact was developed through three iterations using the Design Science framework. A total of nine users and IT usability experts have evaluated the artifact returning the SUS score of 85,83 for users and 87,5 for IT usability experts. Nielsen´s heuristics were assessed by IT usability experts only, returning an average score of 4,28. The general response was positive regarding both the content and the attempt to present methods used in cognitive decline. It reminds to be seen how to bring this knowledge to those who are most affected by the decline.Masteroppgave i informasjonsvitenskapINFO390MASV-INF