Safe and efficient mobile robot teleoperation via a network with communication delay

International audienceThis paper presents the remote control of a mobile robot via the Internet. The delay time is one of the biggest obstacles for many real-time applications, especially for Internet-based control systems. This article presents three architectures to control a robot in an Internet network with-out guaranteed quality of service. These architectures allow us to control the robot movements accurately and safely. This work presents the use of virtual reality in the context of the remote control. Virtual reality can be used in a conventional manner to simulate the behavior of a system, but also in parallel with the real system to improve the quality of control. To validate our work, we conducted teleoperation experiments in various places, and the experimental results show the advantages and drawbacks of each proposed architecture

Real-Time Prediction of RTT Based on Holt-Winters Method for Internet-Based Teleoperation

International audienceThe teleoperation of complex systems became possible and cheaper by the adoption of Internet network as a communication channel. This induces specific problems due to the constraints of communications such as round trip time delay (RTT), bandwidth limitation, channel congestion, availability and transmitted information losses or errors. All these parameters are attributes of the Quality of Service (QoS) offered by the communication network. Predicting the RTT plays a major role in the dynamic enhancements for many applications such as telemedicine or mobile telerobotics. So the ideal approach concerning this important parameter in internet must be strictly studied. This paper proposes to use the Holt-Winters method to explore real-time predicting of RTT. This technique is largely used for the analysis and treatment of time series data. Real measurements of the RTT were taken between 4 geographically distant nodes. The obtained results confirm the efficiency and precision of the proposed method

Towards an adaptive Virtual Reality Serious Game System for Motor Rehabilitation based on Facial Emotion Recognition

International audienceIn this paper, an adaptive serious game-based functional rehabilitation system was proposed. This system provides each patient with the appropriate game during his rehabilitation sessions. The game's complexity, speed, and level vary according to patients’ facial emotions. Moreover, two modules were presented namely: Convolutional Neural Networks-based motion recognition module which analyzes the patient's facial emotions and the Virtual Reality-based serious game module that retrieves the updated emotion's data and provides the patient's suitable game so that he/she can perform rehabilitation gestures in good conditions. When a patient expresses emotions of stress or tiredness, the first module analyzes this situation and sends a notification to use the same game but with less complexity, or another game. Preliminary tests of the system were carried out on three patients and the feedback is very encouraging

VR-PEER: A Personalized Exer-Game Platform Based on Emotion Recognition

International audienceMotor rehabilitation exercises require recurrent repetitions to enhance patients’ gestures. However, these repetitive gestures usually decrease the patients’ motivation and stress them. Virtual Reality (VR) exer-games (serious games in general) could be an alternative solution to address the problem. This innovative technology encourages patients to train different gestures with less effort since they are totally immersed in an easy to play exer-game. Despite this evolution, patients, with available exer-games, still suffer in performing their gestures correctly without pain. The developed applications do not consider the patients psychological states when playing an exer-game. Therefore, we believe that is necessary to develop personalized and adaptive exer-games that take into consideration the patients’ emotions during rehabilitation exercises. This paper proposed a VR-PEER adaptive exer-game system based on emotion recognition. The platform contain three main modules: (1) computing and interpretation module, (2) emotion recognition module, (3) adaptation module. Furthermore, a virtual reality-based serious game is developed as a case study, that uses updated facial expression data and provides dynamically the patient’s appropriate game to play during rehabilitation exercises. An experimental study has been conducted on fifteen subjects who expressed the usefulness of the proposed system in motor rehabilitation process

COVI³D: Automatic COVID-19 CT Image-Based Classification and Visualization Platform Utilizing Virtual and Augmented Reality Technologies

Recently many studies have shown the effectiveness of using augmented reality (AR) and virtual reality (VR) in biomedical image analysis. However, they are not automating the COVID level classification process. Additionally, even with the high potential of CT scan imagery to contribute to research and clinical use of COVID-19 (including two common tasks in lung image analysis: segmentation and classification of infection regions), publicly available data-sets are still a missing part in the system care for Algerian patients. This article proposes designing an automatic VR and AR platform for the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) pandemic data analysis, classification, and visualization to address the above-mentioned challenges including (1) utilizing a novel automatic CT image segmentation and localization system to deliver critical information about the shapes and volumes of infected lungs, (2) elaborating volume measurements and lung voxel-based classification procedure, and (3) developing an AR and VR user-friendly three-dimensional interface. It also centered on developing patient questionings and medical staff qualitative feedback, which led to advances in scalability and higher levels of engagement/evaluations. The extensive computer simulations on CT image classification show a better efficiency against the state-of-the-art methods using a COVID-19 dataset of 500 Algerian patients. The developed system has been used by medical professionals for better and faster diagnosis of the disease and providing an effective treatment plan more accurately by using real-time data and patient information

Tumor Lung Visualization and Localization through Virtual Reality and Thermal Feedback Interface

The World Health Organization estimates that there were around 10 million deaths due to cancer in 2020, and lung cancer was the most common type of cancer, with over 2.2 million new cases and 1.8 million deaths. While there have been advances in the diagnosis and prediction of lung cancer, there is still a need for new, intelligent methods or diagnostic tools to help medical professionals detect the disease. Since it is currently unable to detect at an early stage, speedy detection and identification are crucial because they can increase a patient’s chances of survival. This article focuses on developing a new tool for diagnosing lung tumors and providing thermal touch feedback using virtual reality visualization and thermal technology. This tool is intended to help identify and locate tumors and measure the size and temperature of the tumor surface. The tool uses data from CT scans to create a virtual reality visualization of the lung tissue and includes a thermal display incorporated into a haptic device. The tool is also tested by touching virtual tumors in a virtual reality application. On the other hand, thermal feedback could be used as a sensory substitute or adjunct for visual or tactile feedback. The experimental results are evaluated with the performance comparison of different algorithms and demonstrate that the proposed thermal model is effective. The results also show that the tool can estimate the characteristics of tumors accurately and that it has the potential to be used in a virtual reality application to “touch” virtual tumors. In other words, the results support the use of the tool for diagnosing lung tumors and providing thermal touch feedback using virtual reality visualization, force, and thermal technology

Avatar-Facilitated Therapy and Virtual Reality: Next-Generation of Functional Rehabilitation Methods

The control of a synthetic human body, or Avatar, is an essential step before the animation process. These two important stages (control and animation) are applied in diverse areas, including video games and 3D animation films. Avatars deserve particular consideration in virtual reality (VR) applications as they represent a fundamental mechanism for interacting with a virtual environment. Avatar’s animation require a precise tracking of the user’s movements, that must be faithfully replicated, in the 3D scenes, by avatars. The control of the ability to animate the upper or lower limbs articulation of the human body, in 3D scenes, is a rather complex task, due to the movements to be reproduced with exact closure, by the avatar, while tracking them by using a camera sensor. These problems are mainly associated with the 3D design of the human body joints and the exact reproduction of anthropomorphic properties in virtual scenes. This article presents an original framework capable of capturing and reproducing, in real time, the same movements provided by humans, using a Microsoft Kinect camera in virtual landscapes. The proposal concentrates on human’s posture failures monitoring by avatars, to improve the posture of people, in rehabilitation stages (victims of stroke or mobility impairment). These scenes contain real motion capture data, which helps to solve postural problems of persons with physical disabilities. Experiments prove the flexibility of the proposed model. During the evaluation phase, we set different scenarios, for an extended analysis of patients’ movements through those of avatars