Mid-Air Haptics for Control Interfaces

Control interfaces and interactions based on touch-less gesture tracking devices have become a prevalent research topic in both industry and academia. Touch-less devices offer a unique interaction immediateness that makes them ideal for applications where direct contact with a physical controller is not desirable. On the other hand, these controllers inherently lack active or passive haptic feedback to inform users about the results of their interaction. Mid-air haptic interfaces, such as those using focused ultrasound waves, can close the feedback loop and provide new tools for the design of touch-less, un-instrumented control interactions. The goal of this workshop is to bring together the growing mid-air haptic research community to identify and discuss future challenges in control interfaces and their application in AR/VR, automotive, music, robotics and teleoperation

Force Dynamics as a Design Framework for Mid-Air Musical Instruments

In this paper we adopt the theory of force dynamics in human cognition as a fundamental design principle for the development of mid-air musical interfaces. We argue that this principle can provide more intuitive user experiences when the interface does not provide direct haptic feedback – such as interfaces made with various gesture-tracking technologies. Grounded in five concepts from the theoretical literature on force dynamics in musical cognition, the paper presents a set of principles for interaction design focused on five force schemas: Path restraint, Containment restraint, Counter-force, Attraction, and Compulsion. We describe an initial set of examples that implement these principles using a Leap Motion sensor for gesture tracking and SuperCollider for interactive audio design. Finally, the paper presents a pilot experiment that provides initial ratings of intuitiveness in the user experience

Virtual kompang: mapping in-air hand gestures for music interaction using gestural musical controller

The introduction of new gesture interfaces has been expanding the possibilities of creating new Digital Musical Instruments (DMI). However, the created interfaces are mainly focused on modern western musical instruments such as piano, drum and guitar. This paper presents a virtual musical instrument, namely Virtual Kompang, a traditional Malay percussion instrument. The interface design and its implementation are presented in this paper. The results of a guessability study are presented in the study to elicit end-user hand movement to map onto commands. The study demonstrated the existing of common hand gestures among the users on mapping with the selected commands. A consensus set of gestures is presented as the outcome of this study.Keywords: Digital Music Instrument, Virtual Environment, Gestural Control, Leap Motion, Virtual Instrumen

The Leap Motion Movement for 2D Pointing Tasks: Characterisation and Comparison to Other Devices

In this paper we present the results from an experiment designed to characterize the Leap Motion movement in 2D pointing tasks and compare it to a mouse and touchpad. We used the ISO 9241-9 multi-directional tapping test for comparing the devices, and we analyse the results using standard throughput and error rate measures as well as additional accuracy measures such as target re-entry, task axis crossing, movement direction change, orthogonal direction change, movement variability, movement offset, and movement error. We also present the results from the ISO9241-9 assessment of comfort questionnaire, and our observations from the participant’s postures when using the Leap Motion device. Results indicate that the Leap Motion performs poorly in these tasks when compared to a mouse or touchpad

Gamified Music Learning System with VR Force Feedback for Rehabilitation

Many conditions cause loss of coordination and motor capabilities in the extremities. One such condition is stroke, which affects approximately 15 million people worldwide each year. [1] Many robotic systems have been developed to assist in the physical and neurological rehabilitation of patients who have suffered a stroke. As a result of this project an actuator to be used for hand rehabilitation using visual processing and Bowden cables was designed. This project aims to use the design of the actuator combined with gamification elements to create an interface to be used in future robotic rehabilitation systems as well as address the compliance problem found in rehabilitation

MilliSonic: Pushing the Limits of Acoustic Motion Tracking

Recent years have seen interest in device tracking and localization using acoustic signals. State-of-the-art acoustic motion tracking systems however do not achieve millimeter accuracy and require large separation between microphones and speakers, and as a result, do not meet the requirements for many VR/AR applications. Further, tracking multiple concurrent acoustic transmissions from VR devices today requires sacrificing accuracy or frame rate. We present MilliSonic, a novel system that pushes the limits of acoustic based motion tracking. Our core contribution is a novel localization algorithm that can provably achieve sub-millimeter 1D tracking accuracy in the presence of multipath, while using only a single beacon with a small 4-microphone array.Further, MilliSonic enables concurrent tracking of up to four smartphones without reducing frame rate or accuracy. Our evaluation shows that MilliSonic achieves 0.7mm median 1D accuracy and a 2.6mm median 3D accuracy for smartphones, which is 5x more accurate than state-of-the-art systems. MilliSonic enables two previously infeasible interaction applications: a) 3D tracking of VR headsets using the smartphone as a beacon and b) fine-grained 3D tracking for the Google Cardboard VR system using a small microphone array

Implementations of the Leap Motion in Sound Synthesis, Effects Modulation and Assistive Performance Tools

(Abstract to follow