Screen-based musical instruments as semiotic machines

The ixi software project started in 2000 with the intention to explore new interactive patterns and virtual interfaces in computer music software. The aim of this paper is not to describe these programs, as they have been described elsewhere, but rather explicate the theoretical background that underlies the design of these screen-based instruments. After an analysis of the similarities and differences in the design of acoustic and screen-based instruments, the paper describes how the creation of an interface is essentially the creation of a semiotic system that affects and influences the musician and the composer. Finally the terminology of this semiotics is explained as an interaction model

Animated virtual agents to cue user attention: comparison of static and dynamic deictic cues on gaze and touch responses

This paper describes an experiment developed to study the performance of virtual agent animated cues within digital interfaces. Increasingly, agents are used in virtual environments as part of the branding process and to guide user interaction. However, the level of agent detail required to establish and enhance efficient allocation of attention remains unclear. Although complex agent motion is now possible, it is costly to implement and so should only be routinely implemented if a clear benefit can be shown. Pevious methods of assessing the effect of gaze-cueing as a solution to scene complexity have relied principally on two-dimensional static scenes and manual peripheral inputs. Two experiments were run to address the question of agent cues on human-computer interfaces. Both experiments measured the efficiency of agent cues analyzing participant responses either by gaze or by touch respectively. In the first experiment, an eye-movement recorder was used to directly assess the immediate overt allocation of attention by capturing the participant’s eyefixations following presentation of a cueing stimulus. We found that a fully animated agent could speed up user interaction with the interface. When user attention was directed using a fully animated agent cue, users responded 35% faster when compared with stepped 2-image agent cues, and 42% faster when compared with a static 1-image cue. The second experiment recorded participant responses on a touch screen using same agent cues. Analysis of touch inputs confirmed the results of gaze-experiment, where fully animated agent made shortest time response with a slight decrease on the time difference comparisons. Responses to fully animated agent were 17% and 20% faster when compared with 2-image and 1-image cue severally. These results inform techniques aimed at engaging users’ attention in complex scenes such as computer games and digital transactions within public or social interaction contexts by demonstrating the benefits of dynamic gaze and head cueing directly on the users’ eye movements and touch responses

Control Mouse Cursor by Head Movement: Development and Implementation

L

Mouse for Computer Control from the Joystick of the Wheelchair

Becoming autonomous is one of the biggest challenges for many people with disabilities. Increasing their autonomy usually involves the use of a wheelchair and any kind of digital assistant, such as a computer or a tablet, to communicate, to work or even for leisure. In such a situation, those people are obliged to use two different human interfaces to move a pointer and to drive the wheelchair. A joystick is the most common option to control a wheelchair. On the other hand, there are many different adapted interfaces to emulate the use of a mouse. This paper presents a system, BJoy Ring mouse, which captures the motion of the joystick on a wheelchair. The captured signal is used to move the cursor or the pointer of any digital device including an USB port. This system avoids any mechanical or electronic change in the joystick to keep its original safety and warranty. Communication between the device and the computer (or any other digital assistant) uses the USB protocol, although it could be easily improved to a Bluetooth wireless connection. Validation tests with real users proved this system to be useful aid for people with motor disabilities