Human-Machine Collaborative Telerobotics: Computer Assistance for Manually Controlled Telesurgery and Teleoperation

Thesis (Ph.D.)--University of Washington, 2014Teleoperated robots are controlled by human operators and allow us to act in environments that are inaccessible for reasons of safety, scale, or remoteness. Unfortunately, due to lack of tactile sense and dexterity and proprioception, task performance can suffer compared to direct object manipulation with the hands. Still, use of human controlled robots is the only viable option for these dangerous or hard-to-reach applications, since autonomous robots do not yet have the capability to autonomously perform most unstructured tasks. This thesis presents new contributions in telerobotics that show the way to improved access and control. The target application domain is in telerobotic surgery. A novel surgical robotic system is presented, the Raven II, which forms the basis of a collaborative telerobotic research network with open open source software and tool interface. The robot is now in use at more than a dozen institutions and is a shared platform for developing and sharing new hardware, software, and methods. Collaborating on telerobotics research requires agreement on a teleoperation protocol. To this end a new data interface was developed and tested for compatibility among fourteen master-slave teleoperation robots. Results of this study suggest avenues for a common internet standard for teleoperation robots. To study performance with this system, beyond purely manual control, two types of computer assistance are studied: virtual fixtures and mixed autonomy. Virtual fixtures are spatially varying environment features that provide motion control assistance. Mixed autonomy combines autonomous robotic motion with human control or commands. In this work, a new mixed autonomy method is presented based on a division of the workspace into regions that are either safe for computer assisted control, or require pure human control. This new method is evaluated experimentally and compared with novel virtual fixture implementations and pure manual control. Experimental evaluation uses uni- and bi-manual peg-in-hole object grasping and manipulation tasks, and several metrics are reported. The aim of these experiments is to determine when and how computer assistance functions will help improve human control of robots. Results of this experiment show that performance on some sub-components of object manipulation is improved. However a performance detriment is also noticed due to a mis-match of state information when control is exchanged from computer to human. Results of this research demonstrate ways to improve many high-value tasks being carried out today under telerobotic operation; successful application of this research may literally save lives in the operating room, disaster relief situations, nuclear environments and other critical applications

Establishing Multimodal Telepresence Sessions using the Session Initiation Protocol (SIP) and Advanced Haptic Codecs

In telepresence and telemanipulation systems, multimodal data is exchanged over a network allowing humans to experience and to operate in remote or inaccessible environments. To operate over the global Internet and connect to multiple telepresence systems, a flexible framework for initiating, handling and terminating Internet-based telerobotic sessions becomes necessary. In this work, we explore the use of standard Internet session and transport protocols in the context of telerobotic applications. Session Initiation Protocol (SIP) is widely used to handle multimedia teleconference sessions with audio, video or text, and provides many services advantageous for establishing connections between heterogeneous haptic interfaces and telerobotic systems. We apply the session paradigm to the creation and negotiation of haptic telepresence sessions and propose to extend this framework to work with the haptic modality. The notion of a “haptic codec ” is introduced for transforming haptic data into a common format, applying data reduction or compression techniques and implementing teleoperation control architectures. The use of the Real-Time Transport Protocol (RTP) is explored for transport of teleoperation data. Finally, a prototype and demonstrator system is presented for evaluation of the proposed framework. Keywords: Telerobotics, telepresence, teleoperation, SIP, haptic codec