Complementary Situational Awareness for an Intelligent Telerobotic Surgical Assistant System

Robotic surgical systems have contributed greatly to the advancement of Minimally Invasive Surgeries (MIS). More specifically, telesurgical robots have provided enhanced dexterity to surgeons performing MIS procedures. However, current robotic teleoperated systems have only limited situational awareness of the patient anatomy and surgical environment that would typically be available to a surgeon in an open surgery. Although the endoscopic view enhances the visualization of the anatomy, perceptual understanding of the environment and anatomy is still lacking due to the absence of sensory feedback. In this work, these limitations are addressed by developing a computational framework to provide Complementary Situational Awareness (CSA) in a surgical assistant. This framework aims at improving the human-robot relationship by providing elaborate guidance and sensory feedback capabilities for the surgeon in complex MIS procedures. Unlike traditional teleoperation, this framework enables the user to telemanipulate the situational model in a virtual environment and uses that information to command the slave robot with appropriate admittance gains and environmental constraints. Simultaneously, the situational model is updated based on interaction of the slave robot with the task space environment. However, developing such a system to provide real-time situational awareness requires that many technical challenges be met. To estimate intraoperative organ information continuous palpation primitives are required. Intraoperative surface information needs to be estimated in real-time while the organ is being palpated/scanned. The model of the task environment needs to be updated in near real-time using the estimated organ geometry so that the force-feedback applied on the surgeon's hand would correspond to the actual location of the model. This work presents a real-time framework that meets these requirements/challenges to provide situational awareness of the environment in the task space. Further, visual feedback is also provided for the surgeon/developer to view the near video frame rate updates of the task model. All these functions are executed in parallel and need to have a synchronized data exchange. The system is very portable and can be incorporated to any existing telerobotic platforms with minimal overhead

A Passive Variable Impedance Control Strategy with Viscoelastic Parameters Estimation of Soft Tissues for Safe Ultrasonography

In the context of telehealth, robotic approaches have proven a valuable solution to in-person visits in remote areas, with decreased costs for patients and infection risks. In particular, in ultrasonography, robots have the potential to reproduce the skills required to acquire high-quality images while reducing the sonographer's physical efforts. In this paper, we address the control of the interaction of the probe with the patient's body, a critical aspect of ensuring safe and effective ultrasonography. We introduce a novel approach based on variable impedance control, allowing real-time optimisation of a compliant controller parameters during ultrasound procedures. This optimisation is formulated as a quadratic programming problem and incorporates physical constraints derived from viscoelastic parameter estimations. Safety and passivity constraints, including an energy tank, are also integrated to minimise potential risks during human-robot interaction. The proposed method's efficacy is demonstrated through experiments on a patient dummy torso, highlighting its potential for achieving safe behaviour and accurate force control during ultrasound procedures, even in cases of contact loss.Comment: 7 pages, 7 figures, submitted to ICRA 202

A Broad Range Triboelectric Stiffness Sensor for Variable Inclusions Recognition.

With the development of artificial intelligence, stiffness sensors are extensively utilized in various fields, and their integration with robots for automated palpation has gained significant attention. This study presents a broad range self-powered stiffness sensor based on the triboelectric nanogenerator (Stiff-TENG) for variable inclusions in soft objects detection. The Stiff-TENG employs a stacked structure comprising an indium tin oxide film, an elastic sponge, a fluorinated ethylene propylene film with a conductive ink electrode, and two acrylic pieces with a shielding layer. Through the decoupling method, the Stiff-TENG achieves stiffness detection of objects within 1.0 s. The output performance and characteristics of the TENG for different stiffness objects under 4 mm displacement are analyzed. The Stiff-TENG is successfully used to detect the heterogeneous stiffness structures, enabling effective recognition of variable inclusions in soft object, reaching a recognition accuracy of 99.7%. Furthermore, its adaptability makes it well-suited for the detection of pathological conditions within the human body, as pathological tissues often exhibit changes in the stiffness of internal organs. This research highlights the innovative applications of TENG and thereby showcases its immense potential in healthcare applications such as palpation which assesses pathological conditions based on organ stiffness

Tactile Ergodic Control Using Diffusion and Geometric Algebra

Continuous physical interaction between robots and their environment is a requirement in many industrial and household tasks, such as sanding and cleaning. Due to the complex tactile information, these tasks are notoriously difficult to model and to sense. In this article, we introduce a closed-loop control method that is constrained to surfaces. The applications that we target have in common that they can be represented by probability distributions on the surface that correlate to the time the robot should spend in a region. These surfaces can easily be captured jointly with the target distributions using coloured point clouds. We present the extension of an ergodic control approach that can be used with point clouds, based on heat equation-driven area coverage (HEDAC). Our method enables closed-loop exploration by measuring the actual coverage using vision. Unlike existing approaches, we approximate the potential field from non-stationary diffusion using spectral acceleration, which does not require complex preprocessing steps and achieves real-time closed-loop control frequencies. We exploit geometric algebra to stay in contact with the target surface by tracking a line while simultaneously exerting a desired force along that line. Our approach is suitable for fully autonomous and human-robot interaction settings where the robot can either directly measure the coverage of the target with its sensors or by being guided online by markings or annotations of a human expert. We tested the performance of the approach in kinematic simulation using point clouds, ranging from the Stanford bunny to a variety of kitchen utensils. Our real-world experiments demonstrate that the proposed approach can successfully be used to wash kitchenware with curved surfaces, by cleaning the dirt detected by vision in an online manner. Website: https://geometric-algebra.tobiloew.ch/tactile_ergodic_controlComment: Submitted to the special issue for IEEE Transactions on Robotics (T-RO) on Tactile Robotic

Snake-Like Robots for Minimally Invasive, Single Port, and Intraluminal Surgeries

The surgical paradigm of Minimally Invasive Surgery (MIS) has been a key driver to the adoption of robotic surgical assistance. Progress in the last three decades has led to a gradual transition from manual laparoscopic surgery with rigid instruments to robot-assisted surgery. In the last decade, the increasing demand for new surgical paradigms to enable access into the anatomy without skin incision (intraluminal surgery) or with a single skin incision (Single Port Access surgery - SPA) has led researchers to investigate snake-like flexible surgical devices. In this chapter, we first present an overview of the background, motivation, and taxonomy of MIS and its newer derivatives. Challenges of MIS and its newer derivatives (SPA and intraluminal surgery) are outlined along with the architectures of new snake-like robots meeting these challenges. We also examine the commercial and research surgical platforms developed over the years, to address the specific functional requirements and constraints imposed by operations in confined spaces. The chapter concludes with an evaluation of open problems in surgical robotics for intraluminal and SPA, and a look at future trends in surgical robot design that could potentially address these unmet needs.Comment: 41 pages, 18 figures. Preprint of article published in the Encyclopedia of Medical Robotics 2018, World Scientific Publishing Company www.worldscientific.com/doi/abs/10.1142/9789813232266_000

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility