Why 2D layout in 3D images matters: evidence from visual search and eye-tracking

Precise perception of three-dimensional (3D) images is crucial for a rewarding experience when using novel displays. However, the capability of the human visual system to perceive binocular disparities varies across the visual field meaning that depth perception might be affected by the two-dimensional (2D) layout of items on the screen. Nevertheless, potential difficulties in perceiving 3D images during free viewing have received only a little attention so far, limiting opportunities to enhance visual effectiveness of information presentation. The aim of this study was to elucidate how the 2D layout of items in 3D images impacts visual search and distribution of maintaining attention based on the analysis of the viewer’s gaze. Participants were searching for a target which was projected one plane closer to the viewer compared to distractors on a multi-plane display. The 2D layout of items was manipulated by changing the item distance from the center of the display plane from 2° to 8°. As a result, the targets were identified correctly when the items were displayed close to the center of the display plane, however, the number of errors grew with an increase in distance. Moreover, correct responses were given more often when subjects paid more attention to targets compared to other items on the screen. However, a more balanced distribution of attention over time across all items was characteristic of the incorrectly completed trials. Thus, our results suggest that items should be displayed close to each other in a 2D layout to facilitate precise perception of 3D images and considering distribution of attention maintenance based on eye-tracking might be useful in the objective assessment of user experience for novel displays

Multimodality with Eye tracking and Haptics: A New Horizon for Serious Games?

The goal of this review is to illustrate the emerging use of multimodal virtual reality that can benefit learning-based games. The review begins with an introduction to multimodal virtual reality in serious games and we provide a brief discussion of why cognitive processes involved in learning and training are enhanced under immersive virtual environments. We initially outline studies that have used eye tracking and haptic feedback independently in serious games, and then review some innovative applications that have already combined eye tracking and haptic devices in order to provide applicable multimodal frameworks for learning-based games. Finally, some general conclusions are identified and clarified in order to advance current understanding in multimodal serious game production as well as exploring possible areas for new applications

User experience in cross-cultural contexts

This dissertation discusses how interdisciplinary UX teams can consider culturally sensitive design elements during the UX design process. It contributes a state-of-the-art meta review on UX evaluation methods, two software tool artifacts for cross-functional UX teams, and empirical insights in the differing usage behaviors of a website plug-in of French, German and Italian users, website design preferences of Vietnamese and German users, as well as learnings from a field trip that focused on studying privacy and personalization in Mumbai, India. Finally, based on these empirical insights, this work introduces the concept culturally sensitive design that goes beyond traditional cross-cultural design considerations in HCI that do not compare different approaches to consider culturally sensitive product aspects in user research

Dynamic horizontal image translation in stereo 3D

Im Bereich Stereo 3D (S3D) bezeichnet „Dynamic Horizontal Image Translation (DHIT)“ das Prinzip, die S3D-Ansichten einer Szene horizontal in entgegengesetzte Richtungen zu verschieben, wodurch die dargestellte Szene in der Tiefe verschoben wird. Dies wird vor allem im Kontext von „Active Depth Cuts“ eingesetzt. Hier werden die S3D-Ansichten vor und nach einem Szenenschnitt so verschoben, dass es nicht zu starken, störenden Tiefensprüngen kommt. Die menschliche Wahrnehmung der DHIT wurde experimentell untersucht. Eine der wichtigsten Erkenntnisse war, dass es starke individuelle Unterschiede in der Empfindlichkeit gegenüber der DHIT gibt. Daher wird empfohlen die Verschiebungsgeschwindigkeit einer S3D-Ansicht nicht höher als 0,10 °/s bis 0,12 °/s zu wählen, sodass Zuschauerinnen und Zuschauer nicht von der DHIT gestört werden. Bei der DHIT kommt es zu einer Verzerrung der dargestellten Szenentiefe. Dies wird bei dem vorgeschlagenen Ansatz „Distortion-Free Dynamic Horizontal Image Translation (DHIT+)“ kompensiert, indem der Abstand zwischen den S3D-Kameras durch Verfahren der Ansichtensynthese angepasst wird. Dieser Ansatz zeigte sich signifikant weniger störend im Vergleich zur DHIT. Die Ansichten konnten ohne Wahrnehmungsbeeinträchtigung etwa 50% schneller verschoben werden. Ein weiteres vorgeschlagenes Verfahren ist „Gaze Adaptive Convergence in Stereo 3D Applications (GACS3D)“. Unter Verwendung eines Eyetrackers wird die Disparität des geschätzten Blickpunkts langsam über die DHIT reduziert. Dies soll die Ermüdung des visuellen Systems mindern, da die Diskrepanz zwischen Akkommodation und Konvergenz reduziert wird. In einem Experiment mit emuliertem Eye-Tracking war GACS3D signifikant weniger störend als eine normale DHIT. Im Vergleich zwischen dem kompletten GACS3D-Prototypen und einer Bildsequenz ohne jegliche Verschiebungen konnte jedoch kein signifikanter Effekt auf den subjektiven Betrachterkomfort registriert werden. Eine Langzeituntersuchung der Ermüdung des visuellen Systems ist nötig, was über den Rahmen dieser Dissertation hinausgeht. Da für GACS3D eine hochgenaue Schätzung der Blickpunktdisparität benötigt wird, wurde die „Probabilistic Visual Focus Disparity Estimation“ entwickelt. Bei diesem Ansatz wird die 3D-Szenenstruktur in Echtzeit geschätzt und dazu verwendet, die Schätzung der Blickpunktdisparität deutlich zu verbessern.Dynamic horizontal image translation (DHIT) denotes the act of dynamically shifting the stereo 3D (S3D) views of a scene in opposite directions so that the portrayed scene is moved along the depth axis. This technique is predominantly used in the context of active depth cuts, where the shifting occurs just before and after a shot cut in order to mitigate depth discontinuities that would otherwise induce visual fatigue. The perception of the DHIT was investigated in an experiment. An important finding was that there are strong individual differences in the sensitivity towards DHIT. It is therefore recommended to keep the shift speed applied to each S3D view in the range of 0.10 °/s to 0.12 °/s so that nobody in the audience gets annoyed by this approach. When a DHIT is performed, the presented scene depth is distorted, i.e., compressed or stretched. A distortion-free dynamic horizontal image translation (DHIT+) is proposed that mitigates these distortions by adjusting the distance between the S3D cameras through depth-image-based rendering techniques. This approach proved to be significantly less annoying. The views could be shifted about 50% faster without perceptual side effects. Another proposed approach is called gaze adaptive convergence in stereo 3D applications (GACS3D). An eye tracker is used to estimate the visual focus whose disparity is then slowly reduced using the DHIT. This is supposed to lessen visual fatigue since the infamous accommodation vergence discrepancy is reduced. GACS3D with emulated eye tracking proved to be significantly less annoying than a regular DHIT. In a comparison between the complete prototype and a static horizontal image translation, no significant effect on subjective visual discomfort could be observed, however. A long-term evaluation of visual fatigue is necessary, which is beyond the scope of this work. In GACS3D, highly accurate visual focus disparity is required. Therefore, the probabilistic visual focus disparity estimation (PVFDE) was developed, which utilizes a real-time estimation of the 3D scene structure to improve the accuracy by orders of magnitude compared to commonly used approaches

Enriching mobile interaction with garment-based wearable computing devices

Wearable computing is on the brink of moving from research to mainstream. The first simple products, such as fitness wristbands and smart watches, hit the mass market and achieved considerable market penetration. However, the number and versatility of research prototypes in the field of wearable computing is far beyond the available devices on the market. Particularly, smart garments as a specific type of wearable computer, have high potential to change the way we interact with computing systems. Due to the proximity to the user`s body, smart garments allow to unobtrusively sense implicit and explicit user input. Smart garments are capable of sensing physiological information, detecting touch input, and recognizing the movement of the user. In this thesis, we explore how smart garments can enrich mobile interaction. Employing a user-centered design process, we demonstrate how different input and output modalities can enrich interaction capabilities of mobile devices such as mobile phones or smart watches. To understand the context of use, we chart the design space for mobile interaction through wearable devices. We focus on the device placement on the body as well as interaction modality. We use a probe-based research approach to systematically investigate the possible inputs and outputs for garment based wearable computing devices. We develop six different research probes showing how mobile interaction benefits from wearable computing devices and what requirements these devices pose for mobile operating systems. On the input side, we look at explicit input using touch and mid-air gestures as well as implicit input using physiological signals. Although touch input is well known from mobile devices, the limited screen real estate as well as the occlusion of the display by the input finger are challenges that can be overcome with touch-enabled garments. Additionally, mid-air gestures provide a more sophisticated and abstract form of input. We present a gesture elicitation study to address the special requirements of mobile interaction and present the resulting gesture set. As garments are worn, they allow different physiological signals to be sensed. We explore how we can leverage these physiological signals for implicit input. We conduct a study assessing physiological information by focusing on the workload of drivers in an automotive setting. We show that we can infer the driver´s workload using these physiological signals. Beside the input capabilities of garments, we explore how garments can be used as output. We present research probes covering the most important output modalities, namely visual, auditory, and haptic. We explore how low resolution displays can serve as a context display and how and where content should be placed on such a display. For auditory output, we investigate a novel authentication mechanism utilizing the closeness of wearable devices to the body. We show that by probing audio cues through the head of the user and re-recording them, user authentication is feasible. Last, we investigate EMS as a haptic feedback method. We show that by actuating the user`s body, an embodied form of haptic feedback can be achieved. From the aforementioned research probes, we distilled a set of design recommendations. These recommendations are grouped into interaction-based and technology-based recommendations and serve as a basis for designing novel ways of mobile interaction. We implement a system based on these recommendations. The system supports developers in integrating wearable sensors and actuators by providing an easy to use API for accessing these devices. In conclusion, this thesis broadens the understanding of how garment-based wearable computing devices can enrich mobile interaction. It outlines challenges and opportunities on an interaction and technological level. The unique characteristics of smart garments make them a promising technology for making the next step in mobile interaction