Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

Towards Intelligent Playful Environments for Animals based on Natural User Interfaces

Tesis por compendioEl estudio de la interacción de los animales con la tecnología y el desarrollo de sistemas tecnológicos centrados en el animal está ganando cada vez más atención desde la aparición del área de Animal Computer Interaction (ACI). ACI persigue mejorar el bienestar de los animales en diferentes entornos a través del desarrollo de tecnología adecuada para ellos siguiendo un enfoque centrado en el animal. Entre las líneas de investigación que ACI está explorando, ha habido bastante interés en la interacción de los animales con la tecnología basada en el juego. Las actividades de juego tecnológicas tienen el potencial de proveer estimulación mental y física a los animales en diferentes contextos, pudiendo ayudar a mejorar su bienestar. Mientras nos embarcamos en la era de la Internet de las Cosas, las actividades de juego tecnológicas actuales para animales todavía no han explorado el desarrollo de soluciones pervasivas que podrían proveerles de más adaptación a sus preferencias a la vez que ofrecer estímulos tecnológicos más variados. En su lugar, estas actividades están normalmente basadas en interacciones digitales en lugar de explorar dispositivos tangibles o aumentar las interacciones con otro tipo de estímulos. Además, estas actividades de juego están ya predefinidas y no cambian con el tiempo, y requieren que un humano provea el dispositivo o la tecnología al animal. Si los humanos pudiesen centrarse más en su participación como jugadores de un sistema interactivo para animales en lugar de estar pendientes de sujetar un dispositivo para el animal o de mantener el sistema ejecutándose, esto podría ayudar a crear lazos más fuertes entre especies y promover mejores relaciones con los animales. Asimismo, la estimulación mental y física de los animales son aspectos importantes que podrían fomentarse si los sistemas de juego diseñados para ellos pudieran ofrecer un variado rango de respuestas, adaptarse a los comportamientos del animal y evitar que se acostumbre al sistema y pierda el interés. Por tanto, esta tesis propone el diseño y desarrollo de entornos tecnológicos de juego basados en Interfaces Naturales de Usuario que puedan adaptarse y reaccionar a las interacciones naturales de los animales. Estos entornos pervasivos permitirían a los animales jugar por si mismos o con una persona, ofreciendo actividades de juego más dinámicas y atractivas capaces de adaptarse con el tiempo.L'estudi de la interacció dels animals amb la tecnologia i el desenvolupament de sistemes tecnològics centrats en l'animal està guanyant cada vegada més atenció des de l'aparició de l'àrea d'Animal Computer Interaction (ACI) . ACI persegueix millorar el benestar dels animals en diferents entorns a través del desenvolupament de tecnologia adequada per a ells amb un enfocament centrat en l'animal. Entre totes les línies d'investigació que ACI està explorant, hi ha hagut prou interès en la interacció dels animals amb la tecnologia basada en el joc. Les activitats de joc tecnològiques tenen el potencial de proveir estimulació mental i física als animals en diferents contextos, podent ajudar a millorar el seu benestar. Mentre ens embarquem en l'era de la Internet de les Coses, les activitats de joc tecnològiques actuals per a animals encara no han explorat el desenvolupament de solucions pervasives que podrien proveir-los de més adaptació a les seues preferències al mateix temps que oferir estímuls tecnològics més variats. En el seu lloc, estes activitats estan normalment basades en interaccions digitals en compte d'explorar dispositius tangibles o augmentar les interaccions amb estímuls de diferent tipus. A més, aquestes activitats de joc estan ja predefinides i no canvien amb el temps, mentre requereixen que un humà proveïsca el dispositiu o la tecnologia a l'animal. Si els humans pogueren centrar-se més en la seua participació com a jugadors actius d'un sistema interactiu per a animals en compte d'estar pendents de subjectar un dispositiu per a l'animal o de mantenir el sistema executant-se, açò podria ajudar a crear llaços més forts entre espècies i promoure millors relacions amb els animals. Així mateix, l'estimulació mental i física dels animals són aspectes importants que podrien fomentar-se si els sistemes de joc dissenyats per a ells pogueren oferir un rang variat de respostes, adaptar-se als comportaments de l'animal i evitar que aquest s'acostume al sistema i perda l'interès. Per tant, esta tesi proposa el disseny i desenvolupament d'entorns tecnològics de joc basats en Interfícies Naturals d'Usuari que puguen adaptar-se i reaccionar a les interaccions naturals dels animals. Aquestos escenaris pervasius podrien permetre als animals jugar per si mateixos o amb una persona, oferint activitats de joc més dinàmiques i atractives que siguen capaces d'adaptar-se amb el temps.The study of animals' interactions with technology and the development of animal-centered technological systems is gaining attention since the emergence of the research area of Animal Computer Interaction (ACI). ACI aims to improve animals' welfare and wellbeing in several scenarios by developing suitable technology for the animal following an animal-centered approach. Among all the research lines ACI is exploring, there has been significant interest in animals' playful interactions with technology. Technologically mediated playful activities have the potential to provide mental and physical stimulation for animals in different environmental contexts, which could in turn help to improve their wellbeing. As we embark in the era of the Internet of Things, current technological playful activities for animals have not yet explored the development of pervasive solutions that could provide animals with more adaptation to their preferences as well as offering varied technological stimuli. Instead, playful technology for animals is usually based on digital interactions rather than exploring tangible devices or augmenting the interactions with different stimuli. In addition, these playful activities are already predefined and do not change over time, while they require that a human has to be the one providing the device or technology to the animal. If humans could focus more on their participation as active players of an interactive system aimed for animals instead of being concerned about holding a device for the animal or keep the system running, this might help to create stronger bonds between species and foster better relationships with animals. Moreover, animals' mental and physical stimulation are important aspects that could be fostered if the playful systems designed for animals could offer a varied range of outputs, be tailored to the animal's behaviors and prevented the animal to get used to the system and lose interest. Therefore, this thesis proposes the design and development of technological playful environments based on Natural User Interfaces that could adapt and react to the animals' natural interactions. These pervasive scenarios would allow animals to play by themselves or with a human, providing more engaging and dynamic playful activities that are capable of adapting over time.Pons Tomás, P. (2018). Towards Intelligent Playful Environments for Animals based on Natural User Interfaces [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/113075TESISCompendi

Co-located Augmented Play-spaces:Past, Present, and Perspectives

In recent years, many different studies regarding Co-located Augmented Play-spaces (CAPs) have been published in a wide variety of conferences and journals. We present an overview. The work presented in these papers includes end user's perspectives as well as researcher's perspective. We place these within four aspects in this review: 1) Argumentation, the underlying reasons or the higher end goals to investigate interactive play from a user's perspective, 2) Systems, the kind of systems that are created, this includes their intended use which fits the end user's perspective, 3) Evaluation, the way in which the researchers evaluate the system, 4) Contribution, the goal of the studies from the researcher's perspective; what does the study contribute to the research community. CAPs are often multimodal in nature; this survey pays attention to the multimodal characteristics in relation to all four aspects. This overview contributes a clearer view on the current literature, points out where new opportunities lie, and hands us the tools for what we think is important: bringing the end-user and research perspective together in intervention based evaluations. In short, this paper discusses CAPs: their past, the present, and the perspectives

TACTOPI: a playful approach to promote computational thinking to visually impaired children

Tese de mestrado em Informática, Faculdade de Ciências, Universidade de Lisboa, 2021The use of playful activities is common in introductory programming settings. Visually, these activities tend to be stimulating enough. However, these are not accessible for visually impaired children. This work presents TACTOPI - a system that consists of a tangible environment that provides navigation skills training and enriches sensorial experiences using sound, visual and tactile elements; It allows the learning of introductory concepts of computational thinking embedded in playful activities with storytelling that promote environmental education for children with visual impairments from 4 to 7 years old. The map is modular, customizable and has a docking system to place the elements allowing a fun tactile interaction. Another essential element is the 3D printed helm containing a joystick and buttons for the child to control and pre-program the instructions to be played by the robot. A study was carried out using a qualitative questionnaire to evaluate the system. Suggestions were collected from respondents experienced with blind children about the suitability, relevance and accessibility of this system for these children. From the results, it is possible to conclude that this is a tool that, despite some limitations, is efficient to introduce computational thinking; interactive elements that support activities in other disciplines and contexts; a tool that ensures accessibility and supports task training for the development of blind children

Exploring Research through Design in Animal-Computer Interaction

This paper explores Research through Design (RtD) as a potential methodology for developing new interactive experiences for animals. We present an example study from an on-going project and examine whether RtD offers an appropriate framework for developing knowledge in the context of Animal-Computer Interaction, as well as considering how best to document such work. We discuss the design journey we undertook to develop interactive systems for captive elephants and the extent to which RtD has enabled us to explore concept development and documentation of research. As a result of our explorations, we propose that particular aspects of RtD can help ACI researchers gain fresh perspectives on the design of technology-enabled devices for non-human animals. We argue that these methods of working can support the investigation of particular and complex situations where no idiomatic interactions yet exist, where collaborative practice is desirable and where the designed objects themselves offer a conceptual window for future research and development

Playful interaction for learning collaboratively and individually

Playful interactions facilitate the development of engaging applications for different purposes. This aspect is very important for serious games, and especially when these games are for children. Another aspect to consider is the interaction among children, which could be a great reinforcement in learning environments. Children enjoy playing, and they like playing with other children. This relationship could encourage their motivation and their learning outcomes. In this paper, a playful interaction system for learning about a period of history is presented. The interaction of the system was achieved using natural gestures and the visuali-zation was autostereoscopic. A study was carried out to determine whether their learning outcomes were greater playing collabo-ratively or playing individually. Forty six children from 7 to 10 years old participated in the study. The analysis of the pre-tests and the post-tests indicate that the children increased their knowledge about historical periods after playing with the two modes. Therefore, the game could be used as an effective transmitter of knowledge both collaboratively and individually. When the post-knowledge scores for the two modes were compared, statistically significant differences were found in favor of the collaborative mode. Therefore, the collaborative mode facilitates learning to a greater extent than the individual mode. The rest of the questions indicated that the children had a lot of fun while playing the game; they found the game easy to play; they would recommend the game to their friends; and they scored the game as a mean of 9.57 over 10. Finally, we believe that the combination of playful interaction and autostereoscopy is an option that should be exploited not only for the development of computer-supported learning systems, but also for the development of systems for different purposesThis work was funded by the Spanish APRENDRA project (TIN2009-14319-C02-01).Martín San José, JF.; Juan Lizandra, MC.; Torres, E.; Vicent López, MJ. (2014). Playful interaction for learning collaboratively and individually. Journal of Ambient Intelligence and Smart Environments. 6(3):295-311. https://doi.org/10.3233/AIS-140257S2953116

Designing Modular Robotic Playware

In this paper, we explore the design of modular robotic objects that may enhance playful experiences. The approach builds upon the development of modular robotics to create a kind of playware, which is flexible in both set-up and activity building for the end-user to allow easy creation of games. Key features of this design approach are modularity, flexibility, and construction, immediate feedback to stimulate engagement, activity design by end-users, and creative exploration of play activities. These features permit the use of such modular playware by a vast array of users, including disabled children who often could be prevented from using and taking benefits from modern technologies. The objective is to get any children moving, exchanging, experimenting and having fun, regardless of their cognitive or physical ability levels. The paper describes two prototype systems developed as modular robotic tiles, and discusses the challenges and opportunities of this modular playware when used by children with different cognitive abilities

Envisioning Future Playful Interactive Environments for Animals

The final publication is available at Springer via http://dx.doi.org/10.1007/978-981-287-546-4_6Play stands as one of the most natural and inherent behavior among the majority of living species, specifically humans and animals. Human play has evolved significantly over the years, and so have done the artifacts which allow us to play: from children playing tag games without any tools other than their bodies, to modern video games using haptic and wearable devices to augment the playful experience. However, this ludic revolution has not been the same for the humans’ closest companions, our pets. Recently, a new discipline inside the human–computer interaction (HCI) community, called animal–computer interaction (ACI), has focused its attention on improving animals’ welfare using technology. Several works in the ACI field rely on playful interfaces to mediate this digital communication between animals and humans. Until now, the development of these interfaces only comprises a single goal or activity, and its adaptation to the animals’ needs requires the developers’ intervention. This work analyzes the existing approaches, proposing a more generic and autonomous system aimed at addressing several aspects of animal welfare at a time: Intelligent Playful Environments for Animals. The great potential of these systems is discussed, explaining how incorporating intelligent capabilities within playful environments could allow learning from the animals’ behavior and automatically adapt the game to the animals’ needs and preferences. The engaging playful activities created with these systems could serve different purposes and eventually improve animals’ quality of life.This work was partially funded by the Spanish Ministry of Science andInnovation under the National R&D&I Program within the projects Create Worlds (TIN2010-20488) and SUPEREMOS (TIN2014-60077-R), and from Universitat Politècnica de València under Project UPV-FE-2014-24. It also received support from a postdoctoral fellowship within theVALi+d Program of the Conselleria d’Educació, Cultura I Esport (Generalitat Valenciana) awarded to Alejandro Catalá (APOSTD/2013/013). The work of Patricia Pons has been supported by the Universitat Politècnica de València under the “Beca de Excelencia” program and currently by an FPU fellowship from the Spanish Ministry of Education, Culture, and Sports (FPU13/03831).Pons Tomás, P.; Jaén Martínez, FJ.; Catalá Bolós, A. (2015). Envisioning Future Playful Interactive Environments for Animals. En More Playful User Interfaces: Interfaces that Invite Social and Physical Interaction. Springer. 121-150. https://doi.org/10.1007/978-981-287-546-4_6S121150Alfrink, K., van Peer, I., Lagerweij H, et al.: Pig Chase. Playing with Pigs project. (2012) www.playingwithpigs.nlAmat, M., Camps, T., Le, Brech S., Manteca, X.: Separation anxiety in dogs: the implications of predictability and contextual fear for behavioural treatment. Anim. Welf. 23(3), 263–266 (2014). doi: 10.7120/09627286.23.3.263Barker, S.B., Dawson, K.S.: The effects of animal-assisted therapy on anxiety ratings of hospitalized psychiatric patients. Psychiatr. Serv. 49(6), 797–801 (1998)Bateson, P., Martin, P.: Play, Playfulness, Creativity and Innovation. Cambridge University Press, New York (2013)Bekoff, M., Allen, C.: Intentional communication and social play: how and why animals negotiate and agree to play. In: Bekoff, M., Byers, J.A. (eds.) Animal Play Evolutionary. Comparative and Ecological Perspectives, pp. 97–114. Cambridge University Press, New York (1997)Burghardt, G.M.: The Genesis of Animal Play. Testing the Limits. MIT Press, Cambridge (2006)Catalá, A., Pons, P., Jaén, J., et al.: A meta-model for dataflow-based rules in smart environments: evaluating user comprehension and performance. Sci. Comput. Prog. 78(10), 1930–1950 (2013). doi: 10.1016/j.scico.2012.06.010Cheok, A.D., Tan, R.T.K.C., Peiris, R.L., et al.: Metazoa ludens: mixed-reality interaction and play for small pets and humans. IEEE Trans. Syst. Man. Cybern.—Part A Syst. Hum. 41(5), 876–891 (2011). doi: 10.1109/TSMCA.2011.2108998Costello, B., Edmonds, E.: A study in play, pleasure and interaction design. In: Proceedings of the 2007 Conference on Designing Pleasurable Products and Interfaces, pp. 76–91 (2007)Csikszentmihalyi, M.: Beyond Boredom and Anxiety. The Experience of Play in Work and Games. Jossey-Bass Publishers, Hoboken (1975)Filan, S.L., Llewellyn-Jones, R.H.: Animal-assisted therapy for dementia: a review of the literature. Int. Psychogeriatr. 18(4), 597–611 (2006). doi: 10.1017/S1041610206003322García-Herranz, M., Haya, P.A., Alamán, X.: Towards a ubiquitous end-user programming system for smart spaces. J. Univ. Comput. Sci. 16(12), 1633–1649 (2010). doi: 10.3217/jucs-016-12-1633Hirskyj-Douglas, I., Read, J.C.: Who is really in the centre of dog computer interaction? In: Adjunct Proceedings of the 11th Conference on Advances in Computer Entertainment—Workshop on Animal Human Computer Interaction (2014)Hu, F., Silver, D., Trude, A.: LonelyDog@Home. In: International Conference Web Intelligence Intelligent Agent Technology—Workshops, 2007 IEEE/WIC/ACM IEEE, pp. 333–337, (2007)Huizinga, J.: Homo Ludens. Wolters-Noordhoff, Groningen (1985)Kamioka, H., Okada, S., Tsutani, K., et al.: Effectiveness of animal-assisted therapy: a systematic review of randomized controlled trials. Complement. Ther. Med. 22(2), 371–390 (2014). doi: 10.1016/j.ctim.2013.12.016Lee, S.P., Cheok, A.D., James, T.K.S., et al.: A mobile pet wearable computer and mixed reality system for human–poultry interaction through the internet. Pers. Ubiquit. Comput. 10(5), 301–317 (2006). doi: 10.1007/s00779-005-0051-6Leo, K., Tan, B.: User-tracking mobile floor projection virtual reality game system for paediatric gait and dynamic balance training. In: Proceedings of the 4th International Convention on Rehabilitation Engineering and Assistive Technology pp. 25:1–25:4 (2010)Mancini, C.: Animal-computer interaction: a manifesto. Mag. Interact. 18(4), 69–73 (2011). doi: 10.1145/1978822.1978836Mancini, C.: Animal-computer interaction (ACI): changing perspective on HCI, participation and sustainability. CHI ’13 Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, pp. 2227–2236 (2013)Mancini, C., van der Linden, J.: UbiComp for animal welfare: envisioning smart environments for kenneled dogs. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 117–128 (2014)Mancini, C., Harris, R., Aengenheister, B., Guest, C.: Re-centering multispecies practices: a canine interface for cancer detection dogs. In: Proceedings of the SIGCHI Conference on Human Factors in Computing System, pp. 2673–2682 (2015)Mancini, C., van der Linden, J., Bryan, J., Stuart, A.: Exploring interspecies sensemaking: dog tracking semiotics and multispecies ethnography. In: Proceedings of the 2012 ACM Conference on Ubiquitous Computing—UbiComp ’12. ACM Press, New York, pp. 143–152 (2012)Mankoff, D., Dey, A.K., Mankoff, J., Mankoff, K.: Supporting interspecies social awareness: using peripheral displays for distributed pack awareness. In: Proceedings of the 18th Annual ACM Symposium on User interface Software and Technology, pp. 253–258 (2005)Maternaghan, C., Turner, K.J.: A configurable telecare system. In: Proceedings of the 4th International Conference on Pervasive Technologies Related to Assistive Environments—PETRA ’11. ACM Press, New York, pp. 14:1–14:8 (2011)Matsuzawa, T.: The Ai project: historical and ecological contexts. Anim. Cogn. 6(4), 199–211 (2003). doi: 10.1007/s10071-003-0199-2McGrath, R.E.: Species-appropriate computer mediated interaction. CHI ‘09 Extended Abstracts on Human Factors in Computing Systems. ACM Press, New York, pp. 2529–2534 (2009)Mocholí, J.A., Jaén, J., Catalá, A.: A model of affective entities for effective learning environments. In: Innovations in Hybrid Intelligent Systems, pp. 337–344 (2007)Nijholt, A. (ed.): Playful User Interfaces. Springer, Singapore (2014)Norman, D.A.: The invisible computer. MIT Press, Cambridge (1998)Noz, F., An, J.: Cat cat revolution: an interspecies gaming experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2661–2664 (2011)Paldanius, M., Kärkkäinen, T., Väänänen-Vainio-Mattila, K., et al.: Communication technology for human-dog interaction: exploration of dog owners’ experiences and expectations. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press, New York, pp. 2641–2650 (2011)Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Pons, P., Jaén, J., Catalá, A.: Animal ludens: building intelligent playful environments for animals. In: Adjunct Proceedings of the 11th Conference on Advances in Computer Entertainment—Workshop on Animal Human Computer Interaction (2014)Resner, B.: Rover@Home: Computer Mediated Remote Interaction Between Humans and Dogs. M.Sc. thesis, Massachusetts Institute of Technology, Cambridge (2001)Ritvo, S.E., Allison, R.S.: Challenges related to nonhuman animal-computer interaction: usability and “liking”. In: Adjunct Proceedings of the 11th Conference on Advances in Computer Entertainment—Workshop on Animal Human Computer Interaction (2014)Robinson, C., Mncini, C., Van Der Linden, J., et al.: Canine-centered interface design: supporting the work of diabetes alert dogs. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3757–3766 (2014)Rumbaugh, D.M.: Language Learning by a Chimpanzee: The LANA Project. Academic Press, New York (1977)Rumbaugh, D.M.: Apes and their future in comparative psychology. Eye Psi Chi 18(1), 16–19 (2013)Rumbaugh, D.M., Gill, T.V., Brown, J.V., et al.: A computer-controlled language training system for investigating the language skills of young apes. Behav. Res. Methods Instrum. 5(5), 385–392 (1973)Schwartz, S.: Separation anxiety syndrome in cats: 136 cases (1991–2000). J. Am. Vet. Med. Assoc. 220(7), 1028–1033 (2002). doi: 10.2460/javma.2002.220.1028Schwartz, S.: Separation anxiety syndrome in dogs and cats. J. Am. Vet. Med. Assoc. 222(11), 1526–1532 (2003)Solomon, O.: What a dog can do: children with autism and therapy dogs in social interaction. Ethos J. Soc. Psychol. Anthropol. 38(1), 143–166 (2010). doi: 10.1111/j.1548-1352.2010.01085.xTeh, K.S., Lee, S.P., Cheok, A.D.: Poultry. Internet: a remote human-pet interaction system. In: CHI ’06 Extended Abstracts on Human Factors in Computing Systems, pp. 251–254 (2006)Väätäjä, H., Pesonen, E.: Ethical issues and guidelines when conducting HCI studies with animals. In: CHI ’13 Extended Abstracts on Human Factors in Computing Systems, pp. 2159–2168 (2013)Väätäjä, H.: Animal welfare as a design goal in technology mediated human-animal interaction—opportunities with haptics. In: Adjunct Proceedings of the 11th Conference on Advances in Computer Entertainment—Workshop on Animal Human Computer Interaction (2014)Weilenmann, A., Juhlin, O.: Understanding people and animals. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems—CHI ’11. ACM Press, New York, pp. 2631–2640 (2011)Weiser, M.: The computer for the 21st century. Sci. Am. 265(3), 94–104 (1991)Westerlaken, M., Gualeni, S., Geurtsen, A.: Grounded zoomorphism: an evaluation methodology for ACI design. In: Adjunct Proceedings of the 11th Conference on Advances in Computer Entertainment—Workshop on Animal Human Computer Interaction (2014)Westerlaken, M., Gualeni, S.: Felino: the philosophical practice of making an interspecies videogame. Philosophy of Computer Games Conference, pp. 1–12 (2014)Wingrave, C.A., Rose, J., Langston, T., LaViola, J.J.J.: Early explorations of CAT: canine amusement and training. In: CHI ’10 Extended Abstracts on Human Factors in Computing Systems, pp. 2661–2669 (2010