PhysioVR: a novel mobile virtual reality framework for physiological computing

Virtual Reality (VR) is morphing into a ubiquitous technology by leveraging of smartphones and screenless cases in order to provide highly immersive experiences at a low price point. The result of this shift in paradigm is now known as mobile VR (mVR). Although mVR offers numerous advantages over conventional immersive VR methods, one of the biggest limitations is related with the interaction pathways available for the mVR experiences. Using physiological computing principles, we created the PhysioVR framework, an Open-Source software tool developed to facilitate the integration of physiological signals measured through wearable devices in mVR applications. PhysioVR includes heart rate (HR) signals from Android wearables, electroencephalography (EEG) signals from a low cost brain computer interface and electromyography (EMG) signals from a wireless armband. The physiological sensors are connected with a smartphone via Bluetooth and the PhysioVR facilitates the streaming of the data using UDP communication protocol, thus allowing a multicast transmission for a third party application such as the Unity3D game engine. Furthermore, the framework provides a bidirectional communication with the VR content allowing an external event triggering using a real-time control as well as data recording options. We developed a demo game project called EmoCat Rescue which encourage players to modulate HR levels in order to successfully complete the in-game mission. EmoCat Rescue is included in the PhysioVR project which can be freely downloaded. This framework simplifies the acquisition, streaming and recording of multiple physiological signals and parameters from wearable consumer devices providing a single and efficient interface to create novel physiologically-responsive mVR applications.info:eu-repo/semantics/publishedVersio

Development and pilot testing of HEXORR: Hand EXOskeleton Rehabilitation Robot

<p>Abstract</p> <p>Background</p> <p>Following acute therapeutic interventions, the majority of stroke survivors are left with a poorly functioning hemiparetic hand. Rehabilitation robotics has shown promise in providing patients with intensive therapy leading to functional gains. Because of the hand's crucial role in performing activities of daily living, attention to hand therapy has recently increased.</p> <p>Methods</p> <p>This paper introduces a newly developed Hand Exoskeleton Rehabilitation Robot (HEXORR). This device has been designed to provide full range of motion (ROM) for all of the hand's digits. The thumb actuator allows for variable thumb plane of motion to incorporate different degrees of extension/flexion and abduction/adduction. Compensation algorithms have been developed to improve the exoskeleton's backdrivability by counteracting gravity, stiction and kinetic friction. We have also designed a force assistance mode that provides extension assistance based on each individual's needs. A pilot study was conducted on 9 unimpaired and 5 chronic stroke subjects to investigate the device's ability to allow physiologically accurate hand movements throughout the full ROM. The study also tested the efficacy of the force assistance mode with the goal of increasing stroke subjects' active ROM while still requiring active extension torque on the part of the subject.</p> <p>Results</p> <p>For 12 of the hand digits'15 joints in neurologically normal subjects, there were no significant ROM differences (P > 0.05) between active movements performed inside and outside of HEXORR. Interjoint coordination was examined in the 1^stand 3^rddigits, and no differences were found between inside and outside of the device (P > 0.05). Stroke subjects were capable of performing free hand movements inside of the exoskeleton and the force assistance mode was successful in increasing active ROM by 43 ± 5% (P < 0.001) and 24 ± 6% (P = 0.041) for the fingers and thumb, respectively.</p> <p>Conclusions</p> <p>Our pilot study shows that this device is capable of moving the hand's digits through nearly the entire ROM with physiologically accurate trajectories. Stroke subjects received the device intervention well and device impedance was minimized so that subjects could freely extend and flex their digits inside of HEXORR. Our active force-assisted condition was successful in increasing the subjects' ROM while promoting active participation.</p

Data-driven control design for neuroprotheses: a virtual reference feedback tuning (VRFT) approach

This paper deals with design of feedback controllers for knee joint movement of paraplegics using functional electrical stimulation (FES) of the paralyzed quadriceps muscle group. The controller design approach, virtual reference feedback tuning (VRFT), is directly based on open loop measured data and fits the controller in such a way that the closed-loop meets a model reference objective. The use of this strategy, avoiding the modeling step, significantly reduces the time required for controller design and considerably simplifies the rehabilitation protocols. Linear and nonlinear controllers have been designed and experimentally tested, preliminarily on a healthy subject and finally on a paraplegic patient. Linear controller is effective when applied on small range of knee joint angle. The design of a nonlinear controller allows better performances. It is also shown that the control design is effective in tracking assigned knee angle trajectories and rejecting disturbances

An Overview of Wearable Haptic Technologies and Their Performance in Virtual Object Exploration.

We often interact with our environment through manual handling of objects and exploration of their properties. Object properties (OP), such as texture, stiffness, size, shape, temperature, weight, and orientation provide necessary information to successfully perform interactions. The human haptic perception system plays a key role in this. As virtual reality (VR) has been a growing field of interest with many applications, adding haptic feedback to virtual experiences is another step towards more realistic virtual interactions. However, integrating haptics in a realistic manner, requires complex technological solutions and actual user-testing in virtual environments (VEs) for verification. This review provides a comprehensive overview of recent wearable haptic devices (HDs) categorized by the OP exploration for which they have been verified in a VE. We found 13 studies which specifically addressed user-testing of wearable HDs in healthy subjects. We map and discuss the different technological solutions for different OP exploration which are useful for the design of future haptic object interactions in VR, and provide future recommendations

A Biomimetic Approach to Controlling Restorative Robotics

Movement is the only way a person can interact with the world around them. When trauma to the neuromuscular systems disrupts the control of movement, quality of life suffers. To restore limb functionality, active robotic interventions and/or rehabilitation are required. Unfortunately, the primary obstacle in a person’s recovery is the limited robustness of the human-machine interfaces. Current systems rely on control approaches that rely on the person to learn how the system works instead of the system being more intuitive and working with the person naturally. My research goal is to design intuitive control mechanisms based on biological processes termed the biomimetic approach. I have applied this control scheme to problems with restorative robotics focused on the upper and lower limb control. Operating an advanced active prosthetic hand is a two-pronged problem of actuating a high-dimensional mechanism and controlling it with an intuitive interface. Our approach attempts to solve these problems by going from muscle activity, electromyography (EMG), to limb kinematics calculated through dynamic simulation of a musculoskeletal model. This control is more intuitive to the user because they attempt to move their hand naturally, and the prosthetic hand performs that movement. The key to this approach was validating simulated muscle paths using both experimental measurements and anatomical constraints where data is missing. After the validation, simulated muscle paths and forces are used to decipher the intended movement. After we have calculated the intended movement, we can move a prosthetic hand to match. This approach required minimal training to give an amputee the ability to control prosthetic hand movements, such as grasping. A more intuitive controller has the potential to improve how people interact and use their prosthetic hands. Similarly, the rehabilitation of the locomotor system in people with damaged motor pathways or missing limbs require appropriate interventions. The problem of decoding human motor intent in a treadmill walking task can be solved with a biomimetic approach. Estimated limb speed is essential for this approach according to the theoretical input-output computation performed by spinal central pattern generators (CPGs), which represents neural circuitry responsible for autonomous control of stepping. The system used the locomotor phases, swing and stance, to estimate leg speeds and enable self-paced walking as well as steering in virtual reality with congruent visual flow. The unique advantage of this system over the previous state-of-art is the independent leg speed control, which is required for multidirectional movement in VR. This system has the potential to contribute to VR gait rehab techniques. Creating biologically-inspired controllers has the potential to improve restorative robotics and allow people a better opportunity to recover lost functionality post-injury. Movement is the only way a person can interact with the world around them. When trauma to the neuromuscular systems disrupts the control of movement, quality of life suffers. To restore limb functionality, active robotic interventions and/or rehabilitation are required. Unfortunately, the primary obstacle in a person’s recovery is the limited robustness of the human-machine interfaces. Current systems rely on control approaches that rely on the person to learn how the system works instead of the system being more intuitive and working with the person naturally. My research goal is to design intuitive control mechanisms based on biological processes termed the biomimetic approach. I have applied this control scheme to problems with restorative robotics focused on the upper and lower limb control.Operating an advanced active prosthetic hand is a two-pronged problem of actuating a high-dimensional mechanism and controlling it with an intuitive interface. Our approach attempts to solve these problems by going from muscle activity, electromyography (EMG), to limb kinematics calculated through dynamic simulation of a musculoskeletal model. This control is more intuitive to the user because they attempt to move their hand naturally, and the prosthetic hand performs that movement. The key to this approach was validating simulated muscle paths using both experimental measurements and anatomical constraints where data is missing. After the validation, simulated muscle paths and forces are used to decipher the intended movement. After we have calculated the intended movement, we can move a prosthetic hand to match. This approach required minimal training to give an amputee the ability to control prosthetic hand movements, such as grasping. A more intuitive controller has the potential to improve how people interact and use their prosthetic hands.Similarly, the rehabilitation of the locomotor system in people with damaged motor pathways or missing limbs require appropriate interventions. The problem of decoding human motor intent in a treadmill walking task can be solved with a biomimetic approach. Estimated limb speed is essential for this approach according to the theoretical input-output computation performed by spinal central pattern generators (CPGs), which represents neural circuitry responsible for autonomous control of stepping. The system used the locomotor phases, swing and stance, to estimate leg speeds and enable self-paced walking as well as steering in virtual reality with congruent visual flow. The unique advantage of this system over the previous state-of-art is the independent leg speed control, which is required for multidirectional movement in VR. This system has the potential to contribute to VR gait rehab techniques.Creating biologically-inspired controllers has the potential to improve restorative robotics and allow people a better opportunity to recover lost functionality post-injury

Model-based control of individual finger movements for prosthetic hand function

The authors gratefully acknowledge the support of the Engineering and Physical Sciences Research Council (EP/M025977/1) and the National Institutes of Health (NIH5R01EB011615) in this research.Peer reviewedPostprin

Controlling patient participation during robot-assisted gait training

Background The overall goal of this paper was to investigate approaches to controlling active participation in stroke patients during robot-assisted gait therapy. Although active physical participation during gait rehabilitation after stroke was shown to improve therapy outcome, some patients can behave passively during rehabilitation, not maximally benefiting from the gait training. Up to now, there has not been an effective method for forcing patient activity to the desired level that would most benefit stroke patients with a broad variety of cognitive and biomechanical impairments. Methods Patient activity was quantified in two ways: by heart rate (HR), a physiological parameter that reflected physical effort during body weight supported treadmill training, and by a weighted sum of the interaction torques (WIT) between robot and patient, recorded from hip and knee joints of both legs. We recorded data in three experiments, each with five stroke patients, and controlled HR and WIT to a desired temporal profile. Depending on the patient's cognitive capabilities, two different approaches were taken: either by allowing voluntary patient effort via visual instructions or by forcing the patient to vary physical effort by adapting the treadmill speed. Results We successfully controlled patient activity quantified by WIT and by HR to a desired level. The setup was thereby individually adaptable to the specific cognitive and biomechanical needs of each patient. Conclusion Based on the three successful approaches to controlling patient participation, we propose a metric which enables clinicians to select the best strategy for each patient, according to the patient's physical and cognitive capabilities. Our framework will enable therapists to challenge the patient to more activity by automatically controlling the patient effort to a desired level. We expect that the increase in activity will lead to improved rehabilitation outcome

From presence to consciousness through virtual reality

Immersive virtual environments can break the deep, everyday connection between where our senses tell us we are and where we are actually located and whom we are with. The concept of 'presence' refers to the phenomenon of behaving and feeling as if we are in the virtual world created by computer displays. In this article, we argue that presence is worthy of study by neuroscientists, and that it might aid the study of perception and consciousness