Playful E-textile Sonic Interaction for Socially Engaged and Open-Ended Play Between Autistic Children

Research on the potential benefits of technology for autistic children is an emergent field in Human-Computer Interaction (HCI), especially within the Child-Computer Interaction Community. This thesis contributes a design approach grounded in theories of play, cognitive development, and autism to expand the discourse on methodological guidelines for performing empirical studies with non-verbal autistic children and to extend the design space to cater to the socio-emotional and sensory needs of this population. The thesis reveals how sonic e-textile Tangible User Interfaces (TUIs) can be used effectively to mediate children’s social participation in playful activities. This is demonstrated through developing three explorative field-studies conducted at a specialist school based in North-East London where two sonic e-textile playful TUIs, namely Mazi and Olly, have been created and tested with three groups of autistic children aged between 5-10. The three studies ran over the period of three years and were designed to investigate the potentials of TUIs as shareable toys during leisure and recreational activities to a) support social and playful interactions among peers and b) provide opportunities for self-regulation. The key contributions of this thesis are the designs of two tangible user interfaces, which offer a set of design approaches to guide researchers through creating shareable and playful tangibles for non-verbal autistic children; a framework for analysis and a thorough evaluation process that other researchers could use to assess the efficacy of playful TUI designs for nonverbal autistic children; and an in-depth discussion about the research process, which offers a new perspective about holistic designs and evaluation of technologies that aim to scaffold play in groups non-verbal autistic children

Tangible interaction with anthropomorphic smart objects in instrumented environments

A major technological trend is to augment everyday objects with sensing, computing and actuation power in order to provide new services beyond the objects' traditional purpose, indicating that such smart objects might become an integral part of our daily lives. To be able to interact with smart object systems, users will obviously need appropriate interfaces that regard their distinctive characteristics. Concepts of tangible and anthropomorphic user interfaces are combined in this dissertation to create a novel paradigm for smart object interaction. This work provides an exploration of the design space, introduces design guidelines, and provides a prototyping framework to support the realisation of the proposed interface paradigm. Furthermore, novel methods for expressing personality and emotion by auditory means are introduced and elaborated, constituting essential building blocks for anthropomorphised smart objects. Two experimental user studies are presented, confirming the endeavours to reflect personality attributes through prosody-modelled synthetic speech and to express emotional states through synthesised affect bursts. The dissertation concludes with three example applications, demonstrating the potentials of the concepts and methodologies elaborated in this thesis.Die Integration von Informationstechnologie in Gebrauchsgegenstände ist ein gegenwärtiger technologischer Trend, welcher es Alltagsgegenständen ermöglicht, durch den Einsatz von Sensorik, Aktorik und drahtloser Kommunikation neue Dienste anzubieten, die über den ursprünglichen Zweck des Objekts hinausgehen. Die Nutzung dieser sogenannten Smart Objects erfordert neuartige Benutzerschnittstellen, welche die speziellen Eigenschaften und Anwendungsbereiche solcher Systeme berücksichtigen. Konzepte aus den Bereichen Tangible Interaction und Anthropomorphe Benutzerschnittstellen werden in dieser Dissertation vereint, um ein neues Interaktionsparadigma für Smart Objects zu entwickeln. Die vorliegende Arbeit untersucht dafür die Gestaltungsmöglichkeiten und zeigt relevante Aspekte aus verwandten Disziplinen auf. Darauf aufbauend werden Richtlinien eingeführt, welche den Entwurf von Benutzerschnittstellen nach dem hier vorgestellten Ansatz begleiten und unterstützen sollen. Für eine prototypische Implementierung solcher Benutzerschnittstellen wird eine Architektur vorgestellt, welche die Anforderungen von Smart Object Systemen in instrumentierten Umgebungen berücksichtigt. Ein wichtiger Bestandteil stellt dabei die Sensorverarbeitung dar, welche unter anderem eine Interaktionserkennung am Objekt und damit auch eine physikalische Eingabe ermöglicht. Des Weiteren werden neuartige Methoden für den auditiven Ausdruck von Emotion und Persönlichkeit entwickelt, welche essentielle Bausteine für anthropomorphisierte Smart Objects darstellen und in Benutzerstudien untersucht wurden. Die Dissertation schliesst mit der Beschreibung von drei Applikationen, welche im Rahmen der Arbeit entwickelt wurden und das Potential der hier erarbeiteten Konzepte und Methoden widerspiegeln

Investigando Natural User Interfaces (NUIs) : tecnologias e interação em contexto de acessibilidade

Orientador: Maria Cecília Calani BaranauskasTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Natural User Interfaces (NUIs) representam um novo paradigma de interação, com a promessa de ser mais intuitivo e fácil de usar do que seu antecessor, que utiliza mouse e teclado. Em um contexto no qual as tecnologias estão cada vez mais invisíveis e pervasivas, não só a quantidade mas também a diversidade de pessoas que participam deste contexto é crescente. Nesse caso, é preciso estudar como esse novo paradigma de interação de fato consegue ser acessível a todas as pessoas que podem utilizá-lo no dia-a-dia. Ademais, é preciso também caracterizar o paradigma em si, para entender o que o torna, de fato, natural. Portanto, nesta tese apresentamos o caminho que percorremos em busca dessas duas respostas: como caracterizar NUIs, no atual contexto tecnológico, e como tornar NUIs acessíveis para todos. Para tanto, primeiro apresentamos uma revisão sistemática de literatura com o estado da arte. Depois, mostramos um conjunto de heurísticas para o design e a avaliação de NUIs, que foram aplicadas em estudos de caso práticos. Em seguida, estruturamos as ideias desta pesquisa dentro dos artefatos da Semiótica Organizacional, e obtivemos esclarecimentos sobre como fazer o design de NUIs com Acessibilidade, seja por meio de Design Universal, seja para propor Tecnologias Assistivas. Depois, apresentamos três estudos de caso com sistemas NUI cujo design foi feito por nós. A partir desses estudos de caso, expandimos nosso referencial teórico e conseguimos, por fim, encontrar três elementos que resumem a nossa caracterização de NUI: diferenças, affordances e enaçãoAbstract: Natural User Interfaces (NUIs) represent a new interaction paradigm, with the promise of being more intuitive and easy to use than its predecessor, that utilizes mouse and keyboard. In a context where technology is becoming each time more invisible and pervasive, not only the amount but also the diversity of people who participate in this context is increasing. In this case, it must be studied how this new interaction paradigm can, in fact, be accessible to all the people who may use it on their daily routine. Furthermore, it is also necessary to characterize the paradigm itself, to understand what makes it, in fact, natural. Therefore, in this thesis we present the path we took in search of these two answers: how to characterize NUIs in the current technological context, and how to make NUIs accessible to all. To do so, first we present a systematic literature review with the state of the art. Then, we show a set of heuristics for the design and evaluation of NUIs, which were applied in practical study cases. Afterwards, we structure the ideas of this research into the Organizational Semiotics artifacts, and we obtain insights into how to design NUIs with Accessibility, be it through Universal Design, be it to propose Assistive Technologies. Then, we present three case studies with NUI systems which we designed. From these case studies, we expanded our theoretical references were able to, finally, find three elements that sum up our characterization of NUI: differences, affordances and enactionDoutoradoCiência da ComputaçãoDoutora em Ciência da Computação160911/2015-0CAPESCNP

Designing Hybrid Interactions through an Understanding of the Affordances of Physical and Digital Technologies

Two recent technological advances have extended the diversity of domains and social contexts of Human-Computer Interaction: the embedding of computing capabilities into physical hand-held objects, and the emergence of large interactive surfaces, such as tabletops and wall boards. Both interactive surfaces and small computational devices usually allow for direct and space-multiplex input, i.e., for the spatial coincidence of physical action and digital output, in multiple points simultaneously. Such a powerful combination opens novel opportunities for the design of what are considered as hybrid interactions in this work. This thesis explores the affordances of physical interaction as resources for interface design of such hybrid interactions. The hybrid systems that are elaborated in this work are envisioned to support specific social and physical contexts, such as collaborative cooking in a domestic kitchen, or collaborative creativity in a design process. In particular, different aspects of physicality characteristic of those specific domains are explored, with the aim of promoting skill transfer across domains. irst, different approaches to the design of space-multiplex, function-specific interfaces are considered and investigated. Such design approaches build on related work on Graspable User Interfaces and extend the design space to direct touch interfaces such as touch-sensitive surfaces, in different sizes and orientations (i.e., tablets, interactive tabletops, and walls). These approaches are instantiated in the design of several experience prototypes: These are evaluated in different settings to assess the contextual implications of integrating aspects of physicality in the design of the interface. Such implications are observed both at the pragmatic level of interaction (i.e., patterns of users' behaviors on first contact with the interface), as well as on user' subjective response. The results indicate that the context of interaction affects the perception of the affordances of the system, and that some qualities of physicality such as the 3D space of manipulation and relative haptic feedback can affect the feeling of engagement and control. Building on these findings, two controlled studies are conducted to observe more systematically the implications of integrating some of the qualities of physical interaction into the design of hybrid ones. The results indicate that, despite the fact that several aspects of physical interaction are mimicked in the interface, the interaction with digital media is quite different and seems to reveal existing mental models and expectations resulting from previous experience with the WIMP paradigm on the desktop PC

On Feelings Of Comfort, Motivation And Joy That Gui And Tui Evoke

New ways to interact with technology are gaining ground over the familiar Graphical User Interfaces (GUI). The Tangible User Interfaces (TUI) are one example of this. However, while it may seem intuitive that such interfaces should evoke rather positive responses from users - e.g. feelings associated with pleasure - little has been studied in this sense. In this challenge of understanding the feelings that GUI and TUI have the potential to evoke, we present our findings from a research that involved more than a hundred people. The research question that guided our endeavors was: What are the relations between the feelings of joy, motivation and comfort when using TUI and GUI? We analyze the results and discuss some hypotheses to explain the behavior observed. © 2014 Springer International Publishing Switzerland.8520 LNCSPART 4273284Bernstein, D., (2013) Essentials of Psychology, , Cengage Learning BelmontBradley, M.M., Lang, P.J., Measuring emotion: The self-Assessment manikin and the semantic differential (1994) Journal of Behavior Therapy and Experimental Psychiatry, 25 (1), pp. 49-59Doering, T., Beckhaus, S., Schmidt, A., Towards a sensible integration of paper-based tangible user interfaces into creative work processes (2009) CHI 2009 Extended Abstracts on Human Factors in Computing Systems, pp. 4627-4632. , ACM, BostonFishkin, K.P., A taxonomy for and analysis of tangible interfaces (2004) Personal and Ubiquitous Computing, 8 (5), pp. 347-358Garzotto, F., Gonella, R., An open-ended tangible environment for disabled childrens learning (2011) Proceedings of the 10th International Conference on Interaction Design and Children, pp. 52-61. , ACM, MichiganHayashi, E.C., Baranauskas, M.C.C., The affectibility concept in systems for learning contexts (2011) International Journal for E-Learning Security (IJeLS), 1 (1-2), pp. 10-18Horn, M.S., Solovey, E.T., Crouser, R.J., Jacob, R.J., Comparing the use of tangible and graphical programming languages for informal science education (2009) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 975-984. , ACMIshii, H., Lakatos, D., Bonanni, L., Labrune, J.B., Radical atoms: Beyond tangible bits toward transformable materials (2012) Interactions ACM, 19 (1), pp. 38-51Ishii, H., Ullmer, B., Tangible bits: Towards seamless interfaces between people, bits and atoms (1997) Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 234-241. , ACMIshii, H., The tangible user interface and its evolution (2008) Communications of the ACM, 51 (6), pp. 32-36Kaltenbrunner, M., Reactivision and TUIO: A tangible tabletop toolkit (2009) Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, pp. 9-16. , ACM, BanffNorman, D.A., Cognitive engineering (1986) User Centered System Design, pp. 31-61. , Laurence Erlbaum Associates, Inc. Publisher, New JerseyNorman, D.A., Emotion &design: Attractive things work better (2002) Interactions ACM, 9, p. 4Oxford Dictionaries Language Matters, , http://www.oxforddictionaries.comPatten, J., Ishii, H., A comparison of spatial organization strategies in graphical and tangible user interfaces (2000) Proceedings of DARE 2000 on Designing Augmented Reality Environments, pp. 41-50. , ACM, New YorkRussell, J.A., Mehrabian, A., Evidence for a three-factor theory of emotions (1977) Journal of Research in Personality, 11, pp. 273-294Strawhacker, A., Sullivan, A., Bers, M.U., TUI, GUI, HUI: Is a bimodal interface truly worth the sum of its parts? (2013) Proceedings of the 12th International Conference on Interaction Design and Children, pp. 309-312. , ACM, New YorkSylla, C., Branco, P., Coutinho, C., Coquet, E., TUIs vs GUIs: Comparing the learning potential with preschoolers (2012) Personal and Ubiquitous Computing, 16 (4), pp. 421-432Xie, L., Antle, A.N., Motamedi, N., Are tangibles more fun?: Comparing childrens enjoyment and engagement using physical graphical and tangible user interfaces (2008) Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, pp. 191-198. , ACM, BonnXu, D., Design and evaluation of tangible interfaces for primary school children (2007) Proceedings of the 6th International Conference on Interaction Design and Children, , ACM, Aalbor

A corpus-based synchronic comparison and diachronic interpretation of lexicalized emotion metaphors in English and Chinese

This study is a corpus-based contrastive study of the cross-language diachronic changes and synchronic variations of lexicalized emotion metaphors (LEMS) in English and Chinese within the framework of cognitive linguistics. Since it is based on a series of basic assumptions of the Lakoffian Conceptual Metaphor Theory (CMT), it is also expected to prove or improve them by making this cross-cultural comparative study of LEMS in English and Chinese. Therefore this study aims at not only the diachronic changes and synchronic variations of LEMS but also the cultural factors underlying them. By applying CMT in the analysis of the corresponding data of LEMS in English and Chinese, and the method of comparative etymology to explore the cultural influences on the variations over the metaphor themes of LEMS in the two languages, this study has achieved the following findings: (1) Both embodied and non-embodied metaphors are possibly universal in different languages; (2) The cross-language variations of emotion metaphors are often characterized by the cultural variations of the prototypical source concept at the basic category level in different cultures; (3) The commonality and specificity of a metaphor theme in different languages are closely related to the levels of generality of the metaphor theme; (4) Although to a great extent our thinking and ideology are determined by our bodies and the metaphors that they give rise to, or vice versa, the em-minded cultural notions are the important ingredient producing the cross-language variations over the themes of emotion metaphors. It can thus be inferred that metaphor themes are cultural and ideological constructs to some extent;(5) Both the embodied physiological experiences and the em-minded cultural notions play an important role in the conceptualization of emotions; (6) The embodied conceptualization of emotion is sometimes subject to the em-minded cultural notions; (7) There exist three different types of metonyms underlying the conceptualization of emotions in English and Chinese; Based on these important findings, it proposes a three types of metonymy model which functions better in generalizing the different metonymies underlying the conceptualization of emotions in English and Chinese. In addition, this study opens the way for applying the semiotics and cognitive metaphor theory to the studies of metaphors in the etymological structures of LEMS in English and Chinese which might be of great importance for the future development of CMT