Recent Advancements in Augmented Reality for Robotic Applications: A Survey

Robots are expanding from industrial applications to daily life, in areas such as medical robotics, rehabilitative robotics, social robotics, and mobile/aerial robotics systems. In recent years, augmented reality (AR) has been integrated into many robotic applications, including medical, industrial, human–robot interactions, and collaboration scenarios. In this work, AR for both medical and industrial robot applications is reviewed and summarized. For medical robot applications, we investigated the integration of AR in (1) preoperative and surgical task planning; (2) image-guided robotic surgery; (3) surgical training and simulation; and (4) telesurgery. AR for industrial scenarios is reviewed in (1) human–robot interactions and collaborations; (2) path planning and task allocation; (3) training and simulation; and (4) teleoperation control/assistance. In addition, the limitations and challenges are discussed. Overall, this article serves as a valuable resource for working in the field of AR and robotic research, offering insights into the recent state of the art and prospects for improvement

A Deep Segmentation Network of Stent Structs Based on IoT for Interventional Cardiovascular Diagnosis

[EN] The Internet of Things (IoT) technology has been widely introduced to the existing medical system. An eHealth system based on IoT devices has gained widespread popularity. In this article, we propose an IoT eHealth framework to provide an autonomous solution for patients with interventional cardiovascular diseases. In this framework, wearable sensors are used to collect a patient's health data, which is daily monitored by a remote doctor. When the monitoring data is abnormal, the remote doctor will ask for image acquisition of the patient's cardiovascular internal conditions. We leverage edge computing to classify these training images by the local base classifier; thereafter, pseudo-labels are generated according to its output. Moreover, a deep segmentation network is leveraged for the segmentation of stent structs in intravascular optical coherence tomography and intravenous ultrasound images of patients. The experimental results demonstrate that remote and local doctors perform real-time visual communication to complete telesurgery. In the experiments, we adopt the U-net backbone with a pretrained SeResNet34 as the encoder to segment the stent structs. Meanwhile, a series of comparative experiments have been conducted to demonstrate the effectiveness of our method based on accuracy, sensitivity, Jaccard, and dice.This work was supported by the National Key Research and Development Program of China (Grant no. 2020YFB1313703), the National Natural Science Foundation of China (Grant no. 62002304), and the Natural Science Foundation of Fujian Province of China (Grant no. 2020J05002).Huang, C.; Zong, Y.; Chen, J.; Liu, W.; Lloret, J.; Mukherjee, M. (2021). A Deep Segmentation Network of Stent Structs Based on IoT for Interventional Cardiovascular Diagnosis. IEEE Wireless Communications. 28(3):36-43. https://doi.org/10.1109/MWC.001.2000407S364328

End-Effector Contact and Force Detection for Miniature Autonomous Robots Performing Lunar and Expeditionary Surgery

Introduction: The U.S. Space Force was stood up on December 20, 2019 as an independent branch under the Air Force consisting of about 16,000 active duty and civilian personnel focused singularly on space. In addition to the Space Force, the plans by NASA and private industry for exploration-class long-duration missions to the moon, near-earth asteroids, and Mars makes semi-independent medical capability in space a priority. Current practice for space-based medicine is limited and relies on a “life-raft” scenario for emergencies. Discussions by working groups on military space-based medicine include placing a Role III equivalent facility in a lunar surface station. Surgical capability is a key requirement for that facility. Materials and Methods: To prepare for the eventuality of surgery in space, it is necessary to develop low-mass, low power, mini-surgical robots, which could serve as a celestial replacement for existing terrestrial robots. The current study focused on developing semi-autonomous capability in surgical robotics, specifically related to task automation. Two categories for end-effector tissue interaction were developed: Visual feedback from the robot to detect tissue contact, and motor current waveform measurements to detect contact force. Results: Using a pixel-to-pixel deep neural network to train, we were able to achieve an accuracy of nearly 90% for contact/nocontact detection. Large torques were predicted well by a trained long short-term memory recursive network, but the technique did not predict small torques well. Conclusion: Surgical capability on long-duration missions will require human/machine teaming with semi-autonomous surgical robots. Our existing small, lightweight, low-power miniature robots perform multiple essential tasks in one design including hemostasis, fluid management, suturing for traumatic wounds, and are fully insertable for internal surgical procedures. To prepare for the inevitable eventuality of an emergency surgery in space, it is essential that automated surgical robot capabilities be developed

Robot Autonomy for Surgery

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots

Application and Prospect of Telesurgery: The Role of Artificial Intelligence

Remote surgery refers to a new surgical mode in which doctors operate on patients with the help of surgical robots, network technology, and virtual reality technology. These robots are located far away from patients. The remote surgical robot system integrates key technologies such as robot, communication technology, remote control technology, space mapping algorithm, and fault tolerance analysis. Apply a variety of emerging networking modes such as 5G, optical fiber private network, fusion network technology, and deterministic network to realize the motion of the subordinate surgical robot and the vision of the main knife, and ensure stable signal transmission and safe remote operation. The development and application of remote surgical robots has become a new trend, which helps to break the barriers of unbalanced regional medical resource allocation, promote the rational allocation of high-quality medical resources, and solve the telemedicine problems in special areas and special circumstances. The development prospect is broad. In the future, relying on the 5G network technology with high speed, low power consumption, and low latency, remote surgery can operate more efficiently and stably, and the surgical robot will also develop toward a more portable and flexible direction, so as to better serve patients

VIRTUAL REALITY IN BIOMEDICAL

Virtual reality (VR) encompasses a number of surgical research topics, including computer graphics, imaging, visualization, simulation, data fusion and telemedicine. In this paper we define virtual reality and each of these areas, focusing on a state-of-the-art review of the research and recent accomplishments. .Virtual reality is a powerful technology for solving today’s real-world problems. Virtual reality refers to computer-generated, interactive, three-dimensional (3-D) environments into which patients are immersed. It provides a way for surgeon to visualize, manipulate and interact with simulated environments through the use of computers and extremely complex data

Telerehabilitation: State-of-the-Art from an Informatics Perspective

Abstract to follow --please check bac