An Introduction to 3D User Interface Design

3D user interface design is a critical component of any virtual environment (VE) application. In this paper, we present a broad overview of three-dimensional (3D) interaction and user interfaces. We discuss the effect of common VE hardware devices on user interaction, as well as interaction techniques for generic 3D tasks and the use of traditional two-dimensional interaction styles in 3D environments. We divide most user interaction tasks into three categories: navigation, selection/manipulation, and system control. Throughout the paper, our focus is on presenting not only the available techniques, but also practical guidelines for 3D interaction design and widely held myths. Finally, we briefly discuss two approaches to 3D interaction design, and some example applications with complex 3D interaction requirements. We also present an annotated online bibliography as a reference companion to this article

From signal to substance and back: Insights from environmental sound research to auditory display design

Presented at the 15th International Conference on Auditory Display (ICAD2009), Copenhagen, Denmark, May 18-22, 2009A persistent concern in the field of auditory display design has been how to effectively use environmental sounds, which are naturally occurring familiar non-speech, non-musical sounds. Environmental sounds represent physical events in the everyday world, and thus they have a semantic content that enables learning and recognition. However, unless used appropriately, their functions in auditory displays may cause problems. One of the main considerations in using environmental sounds as auditory icons is how to ensure the identifiability of the sound sources. The identifiability of an auditory icon depends on both the intrinsic acoustic properties of the sound it represents, and on the semantic fit of the sound to its context, i.e., whether the context is one in which the sound naturally occurs or would be unlikely to occur. Relatively recent research has yielded some insights into both of these factors. A second major consideration is how to use the source properties to represent events in the auditory display. This entails parameterizing the environmental sounds so the acoustics will both relate to source properties familiar to the user and convey meaningful new information to the user. Finally, particular considerations come into play when designing auditory displays for special populations, such as hearing impaired listeners who may not have access to all the acoustic information available to a normal hearing listener, or to elderly or other individuals whose cognitive resources may be diminished. Some guidelines for designing displays for these populations will be outlined

An Investigation of non-verbal auditory cues within graphical user interfaces

As we approach the end of the century and enter into a digital era there is a need to concentrate on new methods of communication. The expansion of personal computers into the industrial sector and more recently into the home, has brought the ability to transmit various forms of data to the average user. This new ability is accompanied with a responsibility to communicate effectively and clearly. This study focuses on the ability of user interfaces to effectively communicate combinations of visual and auditory cues to produce a more comprehensive user experience. This work proposed that the addition of auditory cues to a graphical user interface allows the user to receive valuable feedback through multiple sensory channels. By combining the types of perceived stimuli, developers of user interfaces can effectively communicate with the user. Modification of Kodak\u27s Picture Easy v.2.0 software allowed for the study of varying amounts of auditory stimuli incorporated into a graphical user interface. The hypothesis states that by adding an appropriate level of non-verbal auditory cues, users will experience enhanced performance and understanding of a user interface. The augmentation of an existing application permitted the study of three prototypes varying in the level of auditory and visual stimuli. Users evaluated three combinations of the test application to determine the preferred level of nonverbal auditory feedback. The findings from this study may then be applied to the development of graphical user interfaces that permit the use of auditory cues. Although this research disproved the hypothesis and stated that participants preferred visual cues over auditory cues, I believe that they can be successfully combined to produce a more effective and pleasing interface. The reasons for failure of the hypothesis may lie embedded within incorrect combinations of the five characteristics of auditory queues. These queues are Type, Nature, Duration, Amplitude, and Placement. This study revealed that participants preferred tool tips when identifying features and functionality, it is my belief that auditory cues can and will play a major part in future interaction styles, not only for software development, but for all user interface interactions

Advancing performability in playable media : a simulation-based interface as a dynamic score

When designing playable media with non-game orientation, alternative play scenarios to gameplay scenarios must be accompanied by alternative mechanics to game mechanics. Problems of designing playable media with non-game orientation are stated as the problems of designing a platform for creative explorations and creative expressions. For such design problems, two requirements are articulated: 1) play state transitions must be dynamic in non-trivial ways in order to achieve a significant level of engagement, and 2) pathways for players’ experience from exploration to expression must be provided. The transformative pathway from creative exploration to creative expression is analogous to pathways for game players’ skill acquisition in gameplay. The paper first describes a concept of simulation-based interface, and then binds that concept with the concept of dynamic score. The former partially accounts for the first requirement, the latter the second requirement. The paper describes the prototype and realization of the two concepts’ binding. “Score” is here defined as a representation of cue organization through a transmodal abstraction. A simulation based interface is presented with swarm mechanics and its function as a dynamic score is demonstrated with an interactive musical composition and performance

RealPen: Providing Realism in Handwriting Tasks on Touch Surfaces using Auditory-Tactile Feedback

We present RealPen, an augmented stylus for capacitive tablet screens that recreates the physical sensation of writing on paper with a pencil, ball-point pen or marker pen. The aim is to create a more engaging experience when writing on touch surfaces, such as screens of tablet computers. This is achieved by regenerating the friction-induced oscillation and sound of a real writing tool in contact with paper. To generate realistic tactile feedback, our algorithm analyzes the frequency spectrum of the friction oscillation generated when writing with traditional tools, extracts principal frequencies, and uses the actuator's frequency response profile for an adjustment weighting function. We enhance the realism by providing the sound feedback aligned with the writing pressure and speed. Furthermore, we investigated the effects of superposition and fluctuation of several frequencies on human tactile perception, evaluated the performance of RealPen, and characterized users' perception and preference of each feedback type

An exploration of semiotics of new auditory displays: A comparative analysis with visual displays

Communicability is an important factor of user interfaces. To address communicability, extensive research has been done on visual displays, whereas relatively little research has been done on auditory displays. The present paper attempts to analyze semiotics of novel auditory displays (spearcon, spindex, and lyricon) using Peirce’s classification of signs: icon, symbol, and index. After the aesthetic developmental patterns of the visual counterparts are presented, semiotics of auditory cues is discussed with future design directions