Using sound in multi-touch interfaces to change materiality and touch behavior

Current development in multimodal interfaces allows us to interact with digitally represented objects. Sadly, these representations are often poor due to technical limitations in representing some of the sensorial properties. Here we explore the possibility of overcoming these limitations by exploiting multisensory integration processes and propose a sound-based interaction technique to alter the perceived materiality of a surface being touched and to shape users' touch behavior. The latter can be seen both as a cue of, and as a means to reinforce, the altered perception. We designed a prototype that dynamically alters the texture-related sound feedback based on touch behavior, as in natural surface touch interactions. A user study showed that the frequency of the sound feedback alters texture perception (coldness and material type) and touch behavior (velocity and pressure). We conclude by discussing lessons learnt from this work in terms of HCI applications and questions opened by this research. Copyright is held by the owner/author(s)

Decision Making Tasks in Time-Critical Medical Settings

ABSTRACT We examine decision-making tasks and information sources during fast-paced, high-risk medical events, such as trauma resuscitation. Interviews with surgical team leaders and ED physicians reveal several environmental aspects that make decision making difficult, including diagnostic tradeoffs, missing and unreliable information, and managing multiple patients simultaneously. We discuss the implications of these findings for the design of wall displays to support decision making in timecritical medical settings

Connecting in the Kitchen:An Empirical Study of Physical Interactions while Cooking Together at Home

Recent research has explored the role technology might play in future kitchens, including virtually dining together, recipe sharing, augmented kitchen furniture, reactive cooking utensils and gestural interaction. When people come together in a kitchen to cook it is about more than just production of sustenance – it is about being together, helping each other, exchanging stories, and contributing to the gradual emergence of a shared meal. In this paper we present a digital ethnography of how people coordinate and cooperate in their kitchens when cooking together for the purpose of inspiring the design of social natural user interactions for technologies in the kitchen. The study is based on 61 YouTube videos of people cooking together analyzed using the frameworks of proxemics and F-formations. Our findings unfold and illustrate relationships between people’s spatial organization, their cooking activities and physical kitchen layouts. Based on these we discuss the kitchen as a design space and particularly the opportunities for social natural user interaction design. Author Keywords F-formations; proxemics; natural user interaction; cooking together; digital ethnography; digital kitchens; the home ACM Classification Keywords H5.3 Computer-supported cooperative wor

Proxemic Interactions in Mobile Devices to Avoid the Spreading of Infections

International audienceCurrently, people's daily lives are affected by the pandemic produced by the COVID-19. One of the main problem is the quickly and easy spreading of the virus. Healthcare workers are affected by nosocomial infections (also called as hospital-acquired infections) that exist in workplaces and more specifically, from health care equipment. In practice, the use of technology is quite common in health care settings. However, due to the touchability of mobile digital devices, their use contributes to nosocomial infections, according to several studies. Some applications based on tracking people have been implemented in order to facilitate Human-Computer Interaction (HCI) and preventing contamination of surfaces by people's hands. Notwithstanding, their use still presents limitations related to implementing applications to be used in some hospital environments, such as care rooms, laboratories, clinical workrooms. To overcome these limitations, we propose the use of interpersonal distances and proxemic dimensions (i.e., Distance, Identity, Location, Movement, and Orientation-DILMO) for implementing HCI with mobile devices that reduces their touchability. The aim is to facilitate the development of mobile apps with proxemic HCI, supported in a proposed architecture, to stop spreading of nosocomial infection of COVID-19 and others. To show the usability and suitability of our proposal, we present two prototypes of apps for mobile devices as proof-of-concept, using several combination of proxemic DILMO dimensions to model proxemic HCI that allow flexibility in interpersonal and devicespeople interactions

Understanding interaction mechanics in touchless target selection

Indiana University-Purdue University Indianapolis (IUPUI)We use gestures frequently in daily life—to interact with people, pets, or objects. But interacting with computers using mid-air gestures continues to challenge the design of touchless systems. Traditional approaches to touchless interaction focus on exploring gesture inputs and evaluating user interfaces. I shift the focus from gesture elicitation and interface evaluation to touchless interaction mechanics. I argue for a novel approach to generate design guidelines for touchless systems: to use fundamental interaction principles, instead of a reactive adaptation to the sensing technology. In five sets of experiments, I explore visual and pseudo-haptic feedback, motor intuitiveness, handedness, and perceptual Gestalt effects. Particularly, I study the interaction mechanics in touchless target selection. To that end, I introduce two novel interaction techniques: touchless circular menus that allow command selection using directional strokes and interface topographies that use pseudo-haptic feedback to guide steering–targeting tasks. Results illuminate different facets of touchless interaction mechanics. For example, motor-intuitive touchless interactions explain how our sensorimotor abilities inform touchless interface affordances: we often make a holistic oblique gesture instead of several orthogonal hand gestures while reaching toward a distant display. Following the Gestalt theory of visual perception, we found similarity between user interface (UI) components decreased user accuracy while good continuity made users faster. Other findings include hemispheric asymmetry affecting transfer of training between dominant and nondominant hands and pseudo-haptic feedback improving touchless accuracy. The results of this dissertation contribute design guidelines for future touchless systems. Practical applications of this work include the use of touchless interaction techniques in various domains, such as entertainment, consumer appliances, surgery, patient-centric health settings, smart cities, interactive visualization, and collaboration