A haptic-enabled multimodal interface for the planning of hip arthroplasty

Multimodal environments help fuse a diverse range of sensory modalities, which is particularly important when integrating the complex data involved in surgical preoperative planning. The authors apply a multimodal interface for preoperative planning of hip arthroplasty with a user interface that integrates immersive stereo displays and haptic modalities. This article overviews this multimodal application framework and discusses the benefits of incorporating the haptic modality in this area

Model-based target sonification on mobile devices

We investigate the use of audio and haptic feedback to augment the display of a mobile device controlled by tilt input. We provide an example of this based on Doppler effects, which highlight the user's approach to a target, or a target's movement from the current state, in the same way we hear the pitch of a siren change as it passes us. Twelve participants practiced navigation/browsing a state-space that was displayed via audio and vibrotactile modalities. We implemented the experiment on a Pocket PC, with an accelerometer attached to the serial port and a headset attached to audio port. Users navigated through the environment by tilting the device. Feedback was provided via audio displayed via a headset, and by vibrotactile information displayed by a vibrotactile unit in the Pocket PC. Users selected targets placed randomly in the state-space, supported by combinations of audio, visual and vibrotactile cues. The speed of target acquisition and error rate were measured, and summary statistics on the acquisition trajectories were calculated. These data were used to compare different display combinations and configurations. The results in the paper quantified the changes brought by predictive or 'quickened' sonified displays in mobile, gestural interaction

Audio-tactile stimuli to improve health and well-being : a preliminary position paper

From literature and through common experience it is known that stimulation of the tactile (touch) sense or auditory (hearing) sense can be used to improve people's health and well-being. For example, to make people relax, feel better, sleep better or feel comforted. In this position paper we propose the concept of combined auditory-tactile stimulation and argue that it potentially has positive effects on human health and well-being through influencing a user's body and mental state. Such effects have, to date, not yet been fully explored in scientific research. The current relevant state of the art is briefly addressed and its limitations are indicated. Based on this, a vision is presented of how auditory-tactile stimulation could be used in healthcare and various other application domains. Three interesting research challenges in this field are identified: 1) identifying relevant mechanisms of human perception of combined auditory-tactile stimuli; 2) finding methods for automatic conversions between audio and tactile content; 3) using measurement and analysis of human bio-signals and behavior to adapt the stimulation in an optimal way to the user. Ideas and possible routes to address these challenges are presented

Overcoming barriers and increasing independence: service robots for elderly and disabled people

This paper discusses the potential for service robots to overcome barriers and increase independence of elderly and disabled people. It includes a brief overview of the existing uses of service robots by disabled and elderly people and advances in technology which will make new uses possible and provides suggestions for some of these new applications. The paper also considers the design and other conditions to be met for user acceptance. It also discusses the complementarity of assistive service robots and personal assistance and considers the types of applications and users for which service robots are and are not suitable

From presence to consciousness through virtual reality

Immersive virtual environments can break the deep, everyday connection between where our senses tell us we are and where we are actually located and whom we are with. The concept of 'presence' refers to the phenomenon of behaving and feeling as if we are in the virtual world created by computer displays. In this article, we argue that presence is worthy of study by neuroscientists, and that it might aid the study of perception and consciousness

Assistive technologies for severe and profound hearing loss: beyond hearing aids and implants

Assistive technologies offer capabilities that were previously inaccessible to individuals with severe and profound hearing loss who have no or limited access to hearing aids and implants. This literature review aims to explore existing assistive technologies and identify what still needs to be done. It is found that there is a lack of focus on the overall objectives of assistive technologies. In addition, several other issues are identified i.e. only a very small number of assistive technologies developed within a research context have led to commercial devices, there is a predisposition to use the latest expensive technologies and a tendency to avoid designing products universally. Finally, the further development of plug-ins that translate the text content of a website to various sign languages is needed to make information on the internet more accessible

Multimodality in {VR}: {A} Survey

Virtual reality has the potential to change the way we create and consume content in our everyday life. Entertainment, training, design and manufacturing, communication, or advertising are all applications that already benefit from this new medium reaching consumer level. VR is inherently different from traditional media: it offers a more immersive experience, and has the ability to elicit a sense of presence through the place and plausibility illusions. It also gives the user unprecedented capabilities to explore their environment, in contrast with traditional media. In VR, like in the real world, users integrate the multimodal sensory information they receive to create a unified perception of the virtual world. Therefore, the sensory cues that are available in a virtual environment can be leveraged to enhance the final experience. This may include increasing realism, or the sense of presence; predicting or guiding the attention of the user through the experience; or increasing their performance if the experience involves the completion of certain tasks. In this state-of-the-art report, we survey the body of work addressing multimodality in virtual reality, its role and benefits in the final user experience. The works here reviewed thus encompass several fields of research, including computer graphics, human computer interaction, or psychology and perception. Additionally, we give an overview of different applications that leverage multimodal input in areas such as medicine, training and education, or entertainment; we include works in which the integration of multiple sensory information yields significant improvements, demonstrating how multimodality can play a fundamental role in the way VR systems are designed, and VR experiences created and consumed

Robotic and clinical evaluation of upper limb motor performance in patients with Friedreich's Ataxia: an observational study

Background: Friedreich’s ataxia (FRDA) is the most common hereditary autosomal recessive form of ataxia. In this disease there is early manifestation of gait ataxia, and dysmetria of the arms and legs which causes impairment in daily activities that require fine manual dexterity. To date there is no cure for this disease. Some novel therapeutic approaches are ongoing in different steps of clinical trial. Development of sensitive outcome measures is crucial to prove therapeutic effectiveness. The aim of the study was to assess the reliability and sensitivity of quantitative and objective assessment of upper limb performance computed by means of the robotic device and to evaluate the correlation with clinical and functional markers of the disease severity. Methods: Here we assess upper limb performances by means of the InMotion Arm Robot, a robot designed for clinical neurological applications, in a cohort of 14 children and young adults affected by FRDA, matched for age and gender with 18 healthy subjects. We focused on the analysis of kinematics, accuracy, smoothness, and submovements of the upper limb while reaching movements were performed. The robotic evaluation of upper limb performance consisted of planar reaching movements performed with the robotic system. The motors of the robot were turned off, so that the device worked as a measurement tool. The status of the disease was scored using the Scale for the Assessment and Rating of Ataxia (SARA). Relationships between robotic indices and a range of clinical and disease characteristics were examined. Results: All our robotic indices were significantly different between the two cohorts except for two, and were highly and reliably discriminative between healthy and subjects with FRDA. In particular, subjects with FRDA exhibited slower movements as well as loss of accuracy and smoothness, which are typical of the disease. Duration of Movement, Normalized Jerk, and Number of Submovements were the best discriminative indices, as they were directly and easily measurable and correlated with the status of the disease, as measured by SARA. Conclusions: Our results suggest that outcome measures obtained by means of robotic devices can improve the sensitivity of clinical evaluations of patients’ dexterity and can accurately and efficiently quantify changes over time in clinical trials, particularly when functional scales appear to be no longer sensitive