Multimodal “sensory illusions” for improving spatial awareness in virtual environments

Inaccurate judgement of distances in virtual environments (VEs) restricts their usefulness for engineering development, in which engineers must have a good understanding of the spaces they are designing. Multimodal feedback can improve depth perception in VEs, but this has yet to be implemented and tested in engineering applications with systems which provide haptic feedback to the body. The project reported in this paper will develop a multimodal VE to improve engineers’ understanding of 3D spaces. It will test the concept of “sensory illusions” where the point of collision in the VE differs to the point of haptic feedback on the body. This will permit the use of fewer vibrotactile devices and therefore the development of a more wearable system. This paper describes related work in multisensory and tactile stimulation which suggests that our perception of a stimulus is not fixed to the point of contact

Multimodal "Sensory Illusions" for Improving Spatial Awareness in Virtual Environments

Inaccurate judgement of distances in virtual environments (VEs) restricts their usefulness for engineering development, in which engineers must have a good understanding of the spaces they are designing. Multimodal feedback can improve depth perception in VEs, but this has yet to be implemented and tested in engineering applications with systems which provide haptic feedback to the body. The project reported in this paper will develop a multimodal VE to improve engineers’ understanding of 3D spaces. It will test the concept of “sensory illusions” where the point of collision in the VE differs to the point of haptic feedback on the body. This will permit the use of fewer vibrotactile devices and therefore the development of a more wearable system. This paper describes related work in multisensory and tactile stimulation which suggests that our perception of a stimulus is not fixed to the point of contact

The Heat is On: Exploring User Behaviour in a Multisensory Virtual Environment for Fire Evacuation

Understanding validity of user behaviour in Virtual Environments (VEs) is critical as they are increasingly being used for serious Health and Safety applications such as predicting human behaviour and training in hazardous situations. This paper presents a comparative study exploring user behaviour in VE-based fire evacuation and investigates whether this is affected by the addition of thermal and olfactory simulation. Participants (N=43) were exposed to a virtual fire in an office building. Quantitative and qualitative analyses of participant attitudes and behaviours found deviations from those we would expect in real life (e.g. pre-evacuation actions), but also valid behaviours like fire avoidance. Potentially important differences were found between multisensory and audiovisual-only conditions (e.g. perceived urgency). We conclude VEs have significant potential in safety-related applications, and that multimodality may afford additional uses in this context, but the identified limitations of behavioural validity must be carefully considered to avoid misapplication of the technology.Comment: CHI Conference on Human Factors in Computing System

Multimodal virtual environments: an opportunity to improve fire safety training?

Fires and fire-related fatalities remain a tragic and frequent occurrence. Evidence has shown that humans adopt sub-optimal behaviours during fire incidents and, therefore, training is one possible means to improve occupant survival rates. We present the potential benefits of using Virtual Environment Training (VET) for fire evacuation. These include experiential and active learning, the ability to interact with contexts which would be dangerous to experience in real life, the ability to customise training and scenarios to the learner, and analytics on learner performance. While several studies have investigated fire safety in VET, generally with positive outcomes, challenges related to cybersickness, interaction and content creation remain. Moreover, issues such as lack of behavioural realism have been attributed to the lack realistic sensory feedback. We argue for multimodal (visual, audio, olfactory, heat) virtual fire safety training to address limitations with existing simulators, and ultimately improve the outcomes of fire incidents. © 2020, Institution of Occupational Safety and Health

Ativação de um braço robótico através de sinais mioelétricos e redes neurais artificiais como protótipo para um estudo preliminar de sensibilidade háptica

As próteses de membro superior representam uma alternativa para melhorar a qualidade de vida de pessoas que passaram pelo processo de amputação. Apesar dos benefícios potencialmente propostos, a taxa de rejeição desse tipo de tecnologia é alta. Dentre os motivos, destaca-se a ausência de uma resposta referente à sensibilidade háptica, relacionada à sensação do toque e à propriocepção, o que torna o acionamento da prótese estressante e os períodos de treinamento prolongados. Com o objetivo de avaliar o impacto do retorno háptico no acionamento de próteses mioelétricas de membro superior, foi implementada a ativação de um braço robótico utilizando sinais mioelétricos e um classificador baseado em Redes Neurais Artificiais (RNA), capaz de distinguir entre movimentos e intensidades a partir de características dos sinais de Eletromigrafia de Superfície (sEMG). O sistema é realimentado, fornecendo uma resposta vibrotátil referente à intensidade de movimento realizada pelo dispositivo. Foram realizados testes com voluntários utilizando apenas resposta visual, apenas resposta háptica e as duas respostas em conjunto. Para os testes com resposta apenas visual, a taxa de estabilidade da ativação foi de 81.1% para a intensidade Fraca, 55.6% para a intensidade Média e 78.4% para a intensidade Forte. Utilizando apenas a resposta háptica, as taxas de estabilidade foram 69.3%, 60.0% e 77.9%, respectivamente. Por fim, as taxas de estabilidade utilizando os dois tipos de reposta foi 81.7%, 65.5% e 84.2%, demonstrando preliminarmente que a performance dos usuários foi superior nessa configuração. As respostas dos questionátios preenchidos ao final dos testes endossaram os resultados obtidos. Porém, é importante notar que os estudos na área devem ser continuados para sua correspondente validação.Upper limb prostheses represent an alternative to improve the life quality of people who have undergone an amputation process. Despite the potential benefits proposed, the rejection rate of this type of technology is high. Among the reasons, the absence of a response regarding haptic sensitivity, related to touch sensation and proprioception, stands out, which makes the activation of the prosthesis stressful and the training periods prolonged. With the objective of evaluating the impact of the haptic feedback on the activation of myeloelectric upper limb prostheses, the activation of a myoelectric robotic arm was implemented using a classifier based on Artificial Neural Networks (ANN), capable of distinguishing between movements and intensities using characteristics of Surface Electrographic signs (sEMG). The system loop is closed, providing a vibratory response relative to the movement intensity performed by the device. Tests with volunteers were performed using only visual response, only haptic response and the two responses together. For the visual-only response tests, the activation stability rate was 81, 1% for the Weak intensity, 55, 6% for the Medium intensity, and 78.4% for the intensity Strong. Using only the haptic response, stability rates were 69, 3%, 60, 0% and 77, 9%, respectively. Finally, stability rates using the two types of response were 81, 7%, 65, 5% and 84, 2%, showing preliminarily that the users performance was superior in this configuration. The questionnaire completed at the end of the tests corroborates the preliminary statement. However, it is important to note that continued studies should be carried out for their corresponding validation