Memristor-Based Hierarchical Attention Network for Multimodal Affective Computing in Mental Health Monitoring

10.13039/501100001809-National Natural Science Foundation of China (Grant Number: 62001149 and U1909201); 10.13039/501100004731-Natural Science Foundation of Zhejiang Province (Grant Number: LQ21F010009)

Calming Effects of Touch in Human, Animal, and Robotic Interaction—Scientific State-of-the-Art and Technical Advances

Small everyday gestures such as a tap on the shoulder can affect the way humans feel and act. Touch can have a calming effect and alter the way stress is handled, thereby promoting mental and physical health. Due to current technical advances and the growing role of intelligent robots in households and healthcare, recent research also addressed the potential of robotic touch for stress reduction. In addition, touch by non-human agents such as animals or inanimate objects may have a calming effect. This conceptual article will review a selection of the most relevant studies reporting the physiological, hormonal, neural, and subjective effects of touch on stress, arousal, and negative affect. Robotic systems capable of non-social touch will be assessed together with control strategies and sensor technologies. Parallels and differences of human-to-human touch and human-to-non-human touch will be discussed. We propose that, under appropriate conditions, touch can act as (social) signal for safety, even when the interaction partner is an animal or a machine. We will also outline potential directions for future research and clinical relevance. Thereby, this review can provide a foundation for further investigations into the beneficial contribution of touch by different agents to regulate negative affect and arousal in humans

Augmenting communication technologies with non-primary sensory modalities

Humans combine their senses to enhance the world around them. While computers have evolved to reflect these sensory demands, only the primary senses of vision and audition (and to an extent, touch) are used in modern communication. This thesis investigated how additional information, such as emotion and navigational assistance, might be communicated using technology-based implementations of sensory displays that output the non-primary modalities of smell, vibrotactile touch, and thermo-touch. This thesis explored using a portable atomiser sprayer to deliver emotional information via smell to mobile phone users, a ring-shaped device worn on the finger to display emotional information using vibration and colours, and an array of thermoelectric coolers worn on the arm to create temperature sensations. Additionally, this thesis explored two methods of signalling temperature using the thermal implementation, and finally, used it in a controlled study to augment the perceived emotion of text messages using temperature. There were challenges with using some of these implementations to display information. Smells produced with the scent technology were ambiguous and highly cognitive, and poor delivery to the user produced undesirable cross-adaption effects when smells lingered and mixed in the environment. The device used to communicate vibrotactile and colour lighting cues neutralized emotions in text messages. Furthermore, temperature pattern discrimination using the thermal implementation was difficult due to non-linear interaction effects that occurred on the skin’s surface, as well as latency resulting from the thermal neurological pathway and the technology used to heat and cool the skin. However, the thermal implementation enabled more accurate user discrimination between thermal signals than what a single stimulator design provided. Furthermore, the utility of continuous thermal feedback, in the context of spatial navigation, was demonstrated, which improved user performance compared to when the user was not presented with any thermal information. Finally, temperature was demonstrated to elicit arousal reactions across subjects using the thermal implementation, and could augment the arousal of text messages, especially when the content of the message was strongly neutral. However, no similar statistical significance was observed with valence, demonstrating the complex implications of using thermal cues to convey emotional information

Expectations and Beliefs in Immersive Virtual Reality Environments: Managing of Body Perception

Real and Perceived Feet Orientation Under Fatiguing and Non-Fatiguing Conditions in an Immersive Virtual Reality Environment ABSTRACT Lower limbs position sense is a complex yet poorly understood mechanism, influenced by many factors. Hence, we investigated the position sense of lower limbs through feet orientation with the use of Immersive Virtual Reality (IVR). Participants had to indicate how they perceived the real 1050 orientation of their feet by orientating a virtual representation of the feet that was shown in an IVR 1051 scenario. We calculated the angle between the two virtual feet (α-VR) after a high-knee step-in-1052 place task. Simultaneously, we recorded the real angle between the two feet (α-R) (T1). Hence, we 1053 assessed if the acute fatigue impacted the position sense. The same procedure was repeated after 1054 inducing muscle fatigue (T2) and after 10 minutes from T2 (T3). Finally, we also recorded the time 1055 needed to confirm the perceived position before and after the acute fatigue protocol. Thirty healthy 1056 adults (27.5 ± 3.8: 57% female, 43% male) were immersed in an IVR scenario with a representation 1057 of two feet. We found a mean difference between α-VR and α-R of 20.89° [95% CI: 14.67°, 27.10°] 1058 in T1, 16.76° [9.57°, 23.94°] in T2, and 16.34° [10.00°, 22.68°] in T3. Participants spent 12.59, 17.50 1059 and 17.95 seconds confirming the perceived position of their feet at T1, T2, T3, respectively. 1060 Participants indicated their feet as forwarding parallel though divergent, showing a mismatch in the 1061 perceived position of feet. Fatigue seemed not to have an impact on position sense but delayed the 1062 time to accomplish this task.The Effect of Context on Eye-Height Estimation in Immersive Virtual Reality: a Cross-Sectional Study ABSTRACT Eye-height spatial perception provides a reference to scale the surrounding environment. It is the result of the integration of visual and postural information. When these stimuli are discordant, the perceived spatial parameters are distorted. Previous studies in immersive virtual reality (IVR) showed that spatial perception is influenced by the visual context of the environment. Hence, this study explored how manipulating the context in IVR affects individuals’ eye-height estimation. Two groups of twenty participants each were immersed in two different IVR environments, represented by a closed room (Wall - W) and an open field (No Wall - NW). Under these two different conditions, participants had to adjust their virtual perspective, estimating their eye height. We calculated the perceived visual offset as the difference between virtual and real eye height, to assess whether the scenarios and the presence of virtual shoes (Feet, No Feet) influenced participants’ estimates at three initial offsets (+100 cm, +0 cm, -100 cm). We found a mean difference between the visual 1679 offsets registered in those trials that started with 100 cm and 0 cm offsets (17.24 cm [8.78; 25.69]) 1680 and between 100 cm and -100 cm offsets (22.35 cm [15.65; 29.05]). Furthermore, a noticeable mean difference was found between the visual offsets recorded in group W, depending on the presence or absence of the virtual shoes (Feet VS No Feet: -6.12 [-10.29, -1.95]). These findings describe that different contexts influenced eye-height perception.Positive Expectations led to Motor Improvement: an Immersive Virtual Reality Pilot Study ABSTRACT This pilot study tested the feasibility of an experimental protocol that evaluated the effect of different positive expectations (verbal and visual-haptic) on anterior trunk flexion. Thirty-six participants were assigned to 3 groups (G0, G+ and G++) that received a sham manoeuvre while immersed in Immersive Virtual Reality (IVR). In G0, the manouvre was paired with by neutral verbal statement. In G+ and G++ the manouvre was paired with a positive verbal statement, but only G++ received a visual-haptic illusion. The illusion consisted of lifting a movable tile placed in front of the participants, using its height to raise the floor level in virtual reality. In this way, participants experienced the perception of touching the floor, through the tactile and the virtual visual afference. The distance between fingertips and the floor was measured before, immediately after, and after 5 minutes from the different manouvres. A major difference in anterior trunk flexion was found for G++ compared to the other groups, although it was only significant compared to G0. This result highlighted the feasibility of the present study for future research on people with mobility limitations (e.g., low back pain or kinesiophobia) and the potential role of a visual-haptic illusion in modifying the performance of trunk flexion

Virtual reality and body rotation: 2 flight experiences in comparison

Embodied interfaces, represented by devices that incorporate bodily motion and proprioceptive stimulation, are promising for Virtual Reality (VR) because they can improve immersion and user experience while at the same time reducing simulator sickness compared to more traditional handheld interfaces (e.g.,gamepads). The aim of the study is to evaluate a novel embodied interface called VitruvianVR. The machine is composed of two separate rings that allow its users to bodily rotate onto three different axes. The suitability of the VitruvianVR was tested in a Virtual Reality flight scenario. In order to reach the goal we compared the VitruvianVR to a gamepad using perfomance measures (i.e., accuracy, fails), head movements and position of the body. Furthermore, a series of data coming from questionnaires about sense of presence, user experience, cognitive load, usability and cybersickness was retrieved.Embodied interfaces, represented by devices that incorporate bodily motion and proprioceptive stimulation, are promising for Virtual Reality (VR) because they can improve immersion and user experience while at the same time reducing simulator sickness compared to more traditional handheld interfaces (e.g.,gamepads). The aim of the study is to evaluate a novel embodied interface called VitruvianVR. The machine is composed of two separate rings that allow its users to bodily rotate onto three different axes. The suitability of the VitruvianVR was tested in a Virtual Reality flight scenario. In order to reach the goal we compared the VitruvianVR to a gamepad using perfomance measures (i.e., accuracy, fails), head movements and position of the body. Furthermore, a series of data coming from questionnaires about sense of presence, user experience, cognitive load, usability and cybersickness was retrieved

Biomechatronics: Harmonizing Mechatronic Systems with Human Beings

This eBook provides a comprehensive treatise on modern biomechatronic systems centred around human applications. A particular emphasis is given to exoskeleton designs for assistance and training with advanced interfaces in human-machine interaction. Some of these designs are validated with experimental results which the reader will find very informative as building-blocks for designing such systems. This eBook will be ideally suited to those researching in biomechatronic area with bio-feedback applications or those who are involved in high-end research on manmachine interfaces. This may also serve as a textbook for biomechatronic design at post-graduate level

Human Health Engineering Volume II

In this Special Issue on “Human Health Engineering Volume II”, we invited submissions exploring recent contributions to the field of human health engineering, i.e., technology for monitoring the physical or mental health status of individuals in a variety of applications. Contributions could focus on sensors, wearable hardware, algorithms, or integrated monitoring systems. We organized the different papers according to their contributions to the main parts of the monitoring and control engineering scheme applied to human health applications, namely papers focusing on measuring/sensing physiological variables, papers highlighting health-monitoring applications, and examples of control and process management applications for human health. In comparison to biomedical engineering, we envision that the field of human health engineering will also cover applications for healthy humans (e.g., sports, sleep, and stress), and thus not only contribute to the development of technology for curing patients or supporting chronically ill people, but also to more general disease prevention and optimization of human well-being

Experimental Study on Human Arm Reaching with and without a Reduced Mobility for Applications in Medical Human-Interactive Robotics

Along with increasing advances in robotic technologies, there are now significant efforts under way to improve the quality of life especially those with physical disabilities or impairments. Control of such medical human-interactive robotics (HIR) involves complications in its design and control due to uncertain human factors. This dissertation makes its efforts to resolve three main challenges of an advanced HIR controller development: 1) detecting the operator’s motion intent, 2) understanding human motor behavior from the robotic perspective, and 3) generating reference motion for the HIR. Our interests in such challenges are limited to the point-to-point reaching of the human arm for applications of their solutions in the control of rehabilitation exoskeletons, therapeutic haptic devices, and prosthetic arms. In the context of human motion intent detection, a mobile motion capture system (MCS) enhanced with myoprocessors is developed to capture kinematics and dynamics of human arm in reaching movements. The developed MCS adopts wireless IMU (inertial measurement unit) sensors to capture ADL (activities of daily life) motions in the real-life environment. In addition, measured muscle activation patterns from selected muscle groups are converted into muscular force values by myoprocessors. This allows a reliable motion intent detection by quantify one of the most frequently used driving signal of the HIR, EMG (electromyography), in a standardized way. In order to understand the human motor behavior from the robotic viewpoint, a computational model on reaching is required. Since such model can be constituted by experimental observations, this dissertation look into invariant motion features of reaching with and without elbow constraint condition to establish a foundation of the computational model. The HIR should generate its reference motions by reflecting motor behavior of the natural human reaching. Though the accurate approximation of such behavior is critical, we also need to take into account the computational cost, especially for real-time applications such as the HIR control. In this manner, a higher order kinematic synthesis of mechanical linkage systems is adopted to approximate natural human hand profiles. Finally, a novel control concept of a myo-prosthetic arm is proposed as an application of all findings and efforts made in this dissertation