Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art

Tactile reliefs offer many benefits over the more classic raised line drawings or tactile diagrams, as depth, 3D shape, and surface textures are directly perceivable. Although often created for blind and visually impaired (BVI) people, a wider range of people may benefit from such multimodal material. However, some reliefs are still difficult to understand without proper guidance or accompanying verbal descriptions, hindering autonomous exploration. In this work, we present a gesture-controlled interactive audio guide (IAG) based on recent low-cost depth cameras that can be operated directly with the hands on relief surfaces during tactile exploration. The interactively explorable, location-dependent verbal and captioned descriptions promise rapid tactile accessibility to 2.5D spatial information in a home or education setting, to online resources, or as a kiosk installation at public places. We present a working prototype, discuss design decisions, and present the results of two evaluation studies: the first with 13 BVI test users and the second follow-up study with 14 test users across a wide range of people with differences and difficulties associated with perception, memory, cognition, and communication. The participant-led research method of this latter study prompted new, significant and innovative developments

Wiinteraction: a study on smart spaces interaction using the Wiimote

This paper describes a study on the use of the Wiimote as a generic interaction device for smart spaces. We have identified the range of interaction possibilities that can be explored when creating Wii-based interfaces for smart spaces and we have explored the application of those interactive features in the context of a typical museum guide. While there are many features that can be explored creatively to sustain the use of the Wiimote as a generic interaction device, we also found that there is at least one critical requirement that is not supported. More specifically, we have identified the need to include access to rich information, such as the one provided by digital displays. We thus propose a shared control mechanism for public displays that enables a user equipped with a Wiimote to first gain control and then browse information in a public display. The results of our study show no major limitation in the proposed approach, but identify device discovery as one major technical flaw that still needs to be overcome before the wiimotemay realistically become a generic interaction alternative for smart spaces

Human-Like Guide Robot that Proactively Explains Exhibits

We developed an autonomous human-like guide robot for a science museum. Its identifies individuals, estimates the exhibits at which visitors are looking, and proactively approaches them to provide explanations with gaze autonomously, using our new approach called speak-and-retreat interaction. The robot also performs such relation-building behaviors as greeting visitors by their names and expressing a friendlier attitude to repeat visitors. We conducted a field study in a science museum at which our system basically operated autonomously and the visitors responded quite positively. First-time visitors on average interacted with the robot for about 9 min, and 94.74% expressed a desire to interact with it again in the future. Repeat visitors noticed its relation-building capability and perceived a closer relationship with it

Explorations in engagement for humans and robots

This paper explores the concept of engagement, the process by which individuals in an interaction start, maintain and end their perceived connection to one another. The paper reports on one aspect of engagement among human interactors--the effect of tracking faces during an interaction. It also describes the architecture of a robot that can participate in conversational, collaborative interactions with engagement gestures. Finally, the paper reports on findings of experiments with human participants who interacted with a robot when it either performed or did not perform engagement gestures. Results of the human-robot studies indicate that people become engaged with robots: they direct their attention to the robot more often in interactions where engagement gestures are present, and they find interactions more appropriate when engagement gestures are present than when they are not.Comment: 31 pages, 5 figures, 3 table

Enhancing reflective learning experiences in museums through interactive installations

In this study we examine the effect that several technological affordances have upon the experiences of children while visiting a museum, focusing upon engagement, discussion and reflection. The museum is the Baba Nyonya Heritage Museum in Malacca, Malaysia. We created a number of public interactive installations designed to facilitate inter-cultural and inter-generational dialogues about cultural identity. The technology employed mixed and augmented reality techniques and gesture recognition to enable visitors to have a multisensory experience with the artefacts on display. Analysis of pre- to post-test knowledge based surveys showed significant learning gains as a result of interacting with the exhibits. Surveys of visitors’ attitudes showed that they felt they had benefitted from the physical interactivity. Qualitative analysis of observational and video data showed that the different interaction techniques provided both benefits and challenges for interaction, which we reflect upon in the paper

Tangible user interfaces : past, present and future directions

In the last two decades, Tangible User Interfaces (TUIs) have emerged as a new interface type that interlinks the digital and physical worlds. Drawing upon users' knowledge and skills of interaction with the real non-digital world, TUIs show a potential to enhance the way in which people interact with and leverage digital information. However, TUI research is still in its infancy and extensive research is required in or- der to fully understand the implications of tangible user interfaces, to develop technologies that further bridge the digital and the physical, and to guide TUI design with empirical knowledge. This paper examines the existing body of work on Tangible User In- terfaces. We start by sketching the history of tangible user interfaces, examining the intellectual origins of this field. We then present TUIs in a broader context, survey application domains, and review frame- works and taxonomies. We also discuss conceptual foundations of TUIs including perspectives from cognitive sciences, phycology, and philoso- phy. Methods and technologies for designing, building, and evaluating TUIs are also addressed. Finally, we discuss the strengths and limita- tions of TUIs and chart directions for future research

A Review of Verbal and Non-Verbal Human-Robot Interactive Communication

In this paper, an overview of human-robot interactive communication is presented, covering verbal as well as non-verbal aspects of human-robot interaction. Following a historical introduction, and motivation towards fluid human-robot communication, ten desiderata are proposed, which provide an organizational axis both of recent as well as of future research on human-robot communication. Then, the ten desiderata are examined in detail, culminating to a unifying discussion, and a forward-looking conclusion