Implementation and design of a teleoperation system based on a VMEBUS/68020 pipelined architecture

A pipelined control design and architecture for a force-feedback teleoperation system that is being implemented at the Jet Propulsion Laboratory and which will be integrated with the autonomous portion of the testbed to achieve share control is described. At the local site, the operator sees real-time force/torque displays and moves two 6-degree of freedom (dof) force-reflecting hand-controllers as his hands feel the contact force/torques generated at the remote site where the robots interact with the environment. He also uses a graphical user menu to monitor robot states and specify system options. The teleoperation software is written in the C language and runs on MC68020-based processor boards in the VME chassis, which utilizes a real-time operating system; the hardware is configured to realize a four-stage pipeline configuration. The environment is very flexible, such that the system can easily be configured as a stand-alone facility for performing independent research in human factors, force control, and time-delayed systems

Kinesthetic Haptics Sensing and Discovery with Bilateral Teleoperation Systems

In the mechanical engineering field of robotics, bilateral teleoperation is a classic but still increasing research topic. In bilateral teleoperation, a human operator moves the master manipulator, and a slave manipulator is controlled to follow the motion of the master in a remote, potentially hostile environment. This dissertation focuses on kinesthetic perception analysis in teleoperation systems. Design of the controllers of the systems is studied as the influential factor of this issue. The controllers that can provide different force tracking capability are compared using the same experimental protocol. A 6 DOF teleoperation system is configured as the system testbed. An innovative master manipulator is developed and a 7 DOF redundant manipulator is used as the slave robot. A singularity avoidance inverse kinematics algorithm is developed to resolve the redundancy of the slave manipulator. An experimental protocol is addressed and three dynamics attributes related to kineshtetic feedback are investigated: weight, center of gravity and inertia. The results support our hypothesis: the controller that can bring a better force feedback can improve the performance in the experiments

Design of a six degree-of-freedom haptic hybrid platform manipultor

Thesis (Master)--Izmir Institute of Technology, Mechanical Engineering, Izmir, 2010Includes bibliographical references (leaves: 97-103)Text in English; Abstract: Turkish and Englishxv, 115 leavesThe word Haptic, based on an ancient Greek word called haptios, means related with touch. As an area of robotics, haptics technology provides the sense of touch for robotic applications that involve interaction with human operator and the environment. The sense of touch accompanied with the visual feedback is enough to gather most of the information about a certain environment. It increases the precision of teleoperation and sensation levels of the virtual reality (VR) applications by exerting physical properties of the environment such as forces, motions, textures. Currently, haptic devices find use in many VR and teleoperation applications. The objective of this thesis is to design a novel Six Degree-of-Freedom (DOF) haptic desktop device with a new structure that has the potential to increase the precision in the haptics technology. First, previously developed haptic devices and manipulator structures are reviewed. Following this, the conceptual designs are formed and a hybrid structured haptic device is designed manufactured and tested. Developed haptic device.s control algorithm and VR application is developed in Matlab© Simulink. Integration of the mechanism with mechanical, electromechanical and electronic components and the initial tests of the system are executed and the results are presented. According to the results, performance of the developed device is discussed and future works are addressed

Integration of the hybrid-structure haptic interface: HIPHAD v1.0

Design, manufacturing, integration and initial test results of the 6-DoF haptic interface, HIPHAD v1.0, are presented in this paper. The hybrid haptic robot mechanism is composed of a 3-DoF parallel platform manipulator, R-Cube, for translational motions and a 3-DoF serial wrist mechanism for monitoring the rotational motions of the handle. The device is capable of displaying point-type of contact since only the R-Cube mechanism is actuated. The dimensions and the orientation of the R-Cube mechanism are reconfigured to comply with the requirements of the haptic system design criteria. The system has several advantages such as relatively trivial kinematical analysis, compactness and high stiffness. The integration of the system along with its mechanism, data acquisition card (DAQ), motor drivers, motors, position sensors, and computer control interface are outlined.Marie Curie International Reintegration Grant within the 7th European Community Framework Programm

Force reflecting joystick control for applications to bilateral teleoperation in construction machinery

This paper presents a simple and effective force reflecting joystick controller for applications to bilateral teleoperation in construction machinery. First, this controller is a combination of an advanced force reflecting gain tuner and two local adaptive controllers, master and slave. Second, the force reflecting gain tuner is effectively designed using recursive least square method and fuzzy logics to estimate directly and accurately the environmental characteristics and, consequently, to produce properly a force reflection. Third, the local adaptive controllers are simply designed using fuzzy technique and optimized using a smart leaning mechanism to ensure that the slave follows well any given trajectory while the operator is able to achieve truly physical perception of interactions at the remote site. An experimental master-slave manipulator is setup and real-time control tests are carried out under various environmental conditions to evaluate the effectiveness of the proposed controller

Architectural study of the design and operation of advanced force feedback manual controllers

A teleoperator system consists of a manual controller, control hardware/software, and a remote manipulator. It was employed in either hazardous or unstructured, and/or remote environments. In teleoperation, the main-in-the-loop is the central concept that brings human intelligence to the teleoperator system. When teleoperation involves contact with an uncertain environment, providing the feeling of telepresence to the human operator is one of desired characteristics of the teleoperator system. Unfortunately, most available manual controllers in bilateral or force-reflecting teleoperator systems can be characterized by their bulky size, high costs, or lack of smoothness and transparency, and elementary architectures. To investigate other alternatives, a force-reflecting, 3 degree of freedom (dof) spherical manual controller is designed, analyzed, and implemented as a test bed demonstration in this research effort. To achieve an improved level of design to meet criteria such as compactness, portability, and a somewhat enhanced force-reflecting capability, the demonstration manual controller employs high gear-ratio reducers. To reduce the effects of the inertia and friction on the system, various force control strategies are applied and their performance investigated. The spherical manual controller uses a parallel geometry to minimize inertial and gravitational effects on its primary task of transparent information transfer. As an alternative to the spherical 3-dof manual controller, a new conceptual (or parallel) spherical 3-dof module is introduced with a full kinematic analysis. Also, the resulting kinematic properties are compared to those of other typical spherical 3-dof systems. The conceptual design of a parallel 6-dof manual controller and its kinematic analysis is presented. This 6-dof manual controller is similar to the Stewart Platform with the actuators located on the base to minimize the dynamic effects. Finally, a combination of the new 3-dof and 6-dof concepts is presented as a feasible test-bed for enhanced performance in a 9-dof system

Development of a twisted-string actuator for a cable-driven haptic interface

Il seguente lavoro di tesi ha avuto lo scopo di analizzare dal punto di vista della progettazione software le procedure ed i protocolli di scambio di informazioni tra i vari componenti di una nuova concezione di Interfaccia Aptica guidata da 4 tendini che muovono un braccialetto centrale collegato al braccio di un operatore così da fornirgli un feedback di forza. In particolare ci si è focalizzati sullo sviluppo di un firmware applicabile ai 4 motori che muovono la struttura centrale della interfaccia. Il firmware deve essere in grado di ricevere da una piattaforma Ros, usata da un operatore, pacchetti di dati contenenti i set point per i vari motori e il tipo di controllo , posizione o forza, che gli attuatori devono effettuare grazie ad uno schema PID. Inoltre l'invio di feedback all'operatore è stato previsto in modo da permettere una maggiore supervisione dell'intero funzionamento. La realizzazione di un Ros Bridge tra l'utente e il sistema da comandare è stato implementato con la formula della programmazione ad oggetti in cui varie classi sono dedicate a compiti differenti come l'impacchettamento di dati da mandare ai motori e la contemporanea ricezione dei feedback. Per completare tutta l'architettura si è anche sviluppato un sistema di trasformazione dei set point provenienti dall'operatore espressi nello spazio di lavoro Cartesiano in riferimenti per i singoli motori e ciò è stato possibile sfruttando la matrice Jacobiana. Una particolare attenzione è stata data all'aspetto di comunicazione dei dati e per fare ciò si è dovuta usare una architettura di codice a multithread e un protocollo UDP

Multi-robot cooperative platform : a task-oriented teleoperation paradigm

This thesis proposes the study and development of a teleoperation system based on multi-robot cooperation under the task oriented teleoperation paradigm: Multi-Robot Cooperative Paradigm, MRCP. In standard teleoperation, the operator uses the master devices to control the remote slave robot arms. These arms reproduce the desired movements and perform the task. With the developed work, the operator can virtually manipulate an object. MRCP automatically generates the arms orders to perform the task. The operator does not have to solve situations arising from possible restrictions that the slave arms may have. The research carried out is therefore aimed at improving the accuracy teleoperation tasks in complex environments, particularly in the field of robot assisted minimally invasive surgery. This field requires patient safety and the workspace entails many restrictions to teleoperation. MRCP can be defined as a platform composed of several robots that cooperate automatically to perform a teleoperated task, creating a robotic system with increased capacity (workspace volume, accessibility, dexterity ...). The cooperation is based on transferring the task between robots when necessary to enable a smooth task execution. The MRCP control evaluates the suitability of each robot to continue with the ongoing task and the optimal time to execute a task transfer between the current selected robot and the best candidate to continue with the task. From the operator¿s point of view, MRCP provides an interface that enables the teleoperation though the task-oriented paradigm: operator orders are translated into task actions instead of robot orders. This thesis is structured as follows: The first part is dedicated to review the current solutions in the teleoperation of complex tasks and compare them with those proposed in this research. The second part of the thesis presents and reviews in depth the different evaluation criteria to determine the suitability of each robot to continue with the execution of a task, considering the configuration of the robots and emphasizing the criterion of dexterity and manipulability. The study reviews the different required control algorithms to enable the task oriented telemanipulation. This proposed teleoperation paradigm is transparent to the operator. Then, the Thesis presents and analyses several experimental results using MRCP in the field of minimally invasive surgery. These experiments study the effectiveness of MRCP in various tasks requiring the cooperation of two hands. A type task is used: a suture using minimally invasive surgery technique. The analysis is done in terms of execution time, economy of movement, quality and patient safety (potential damage produced by undesired interaction between the tools and the vital tissues of the patient). The final part of the thesis proposes the implementation of different virtual aids and restrictions (guided teleoperation based on haptic visual and audio feedback, protection of restricted workspace regions, etc.) using the task oriented teleoperation paradigm. A framework is defined for implementing and applying a basic set of virtual aids and constraints within the framework of a virtual simulator for laparoscopic abdominal surgery. The set of experiments have allowed to validate the developed work. The study revealed the influence of virtual aids in the learning process of laparoscopic techniques. It has also demonstrated the improvement of learning curves, which paves the way for its implementation as a methodology for training new surgeons.Aquesta tesi doctoral proposa l'estudi i desenvolupament d'un sistema de teleoperació basat en la cooperació multi-robot sota el paradigma de la teleoperació orientada a tasca: Multi-Robot Cooperative Paradigm, MRCP. En la teleoperació clàssica, l'operador utilitza els telecomandaments perquè els braços robots reprodueixin els seus moviments i es realitzi la tasca desitjada. Amb el treball realitzat, l'operador pot manipular virtualment un objecte i és mitjançant el MRCP que s'adjudica a cada braç les ordres necessàries per realitzar la tasca, sense que l'operador hagi de resoldre les situacions derivades de possibles restriccions que puguin tenir els braços executors. La recerca desenvolupada està doncs orientada a millorar la teleoperació en tasques de precisió en entorns complexos i, en particular, en el camp de la cirurgia mínimament invasiva assistida per robots. Aquest camp imposa condicions de seguretat del pacient i l'espai de treball comporta moltes restriccions a la teleoperació. MRCP es pot definir com a una plataforma formada per diversos robots que cooperen de forma automàtica per dur a terme una tasca teleoperada, generant un sistema robòtic amb capacitats augmentades (volums de treball, accessibilitat, destresa,...). La cooperació es basa en transferir la tasca entre robots a partir de determinar quin és aquell que és més adequat per continuar amb la seva execució i el moment òptim per realitzar la transferència de la tasca entre el robot actiu i el millor candidat a continuar-la. Des del punt de vista de l'operari, MRCP ofereix una interfície de teleoperació que permet la realització de la teleoperació mitjançant el paradigma d'ordres orientades a la tasca: les ordres es tradueixen en accions sobre la tasca en comptes d'estar dirigides als robots. Aquesta tesi està estructurada de la següent manera: Primerament es fa una revisió de l'estat actual de les diverses solucions desenvolupades actualment en el camp de la teleoperació de tasques complexes, comparant-les amb les proposades en aquest treball de recerca. En el segon bloc de la tesi es presenten i s'analitzen a fons els diversos criteris per determinar la capacitat de cada robot per continuar l'execució d'una tasca, segons la configuració del conjunt de robots i fent especial èmfasi en el criteri de destresa i manipulabilitat. Seguint aquest estudi, es presenten els diferents processos de control emprats per tal d'assolir la telemanipulació orientada a tasca de forma transparent a l'operari. Seguidament es presenten diversos resultats experimentals aplicant MRCP al camp de la cirurgia mínimament invasiva. En aquests experiments s'estudia l'eficàcia de MRCP en diverses tasques que requereixen de la cooperació de dues mans. S'ha escollit una tasca tipus: sutura amb tècnica de cirurgia mínimament invasiva. L'anàlisi es fa en termes de temps d'execució, economia de moviment, qualitat i seguretat del pacient (potencials danys causats per la interacció no desitjada entre les eines i els teixits vitals del pacient). Finalment s'ha estudiat l'ús de diferents ajudes i restriccions virtuals (guiat de la teleoperació via retorn hàptic, visual o auditiu, protecció de regions de l'espai de treball, etc) dins el paradigma de teleoperació orientada a tasca. S'ha definint un marc d'aplicació base i implementant un conjunt de restriccions virtuals dins el marc d'un simulador de cirurgia laparoscòpia abdominal. El conjunt d'experiments realitzats han permès validar el treball realitzat. Aquest estudi ha permès determinar la influencia de les ajudes virtuals en el procés d'aprenentatge de les tècniques laparoscòpiques. S'ha evidenciat una millora en les corbes d'aprenentatge i obre el camí a la seva implantació com a metodologia d'entrenament de nous cirurgians.Postprint (published version

Haptics-Enabled Teleoperation for Robotics-Assisted Minimally Invasive Surgery

The lack of force feedback (haptics) in robotic surgery can be considered to be a safety risk leading to accidental tissue damage and puncturing of blood vessels due to excessive forces being applied to tissue and vessels or causing inefficient control over the instruments because of insufficient applied force. This project focuses on providing a satisfactory solution for introducing haptic feedback in robotics-assisted minimally invasive surgical (RAMIS) systems. The research addresses several key issues associated with the incorporation of haptics in a master-slave (teleoperated) robotic environment for minimally invasive surgery (MIS). In this project, we designed a haptics-enabled dual-arm (two masters - two slaves) robotic MIS testbed to investigate and validate various single-arm as well as dual-arm teleoperation scenarios. The most important feature of this setup is the capability of providing haptic feedback in all 7 degrees of freedom (DOF) required for RAMIS (3 translations, 3 rotations and pinch motion of the laparoscopic tool). The setup also enables the evaluation of the effect of replacing haptic feedback by other sensory cues such as visual representation of haptic information (sensory substitution) and the hypothesis that surgical outcomes may be improved by substituting or augmenting haptic feedback by such sensory cues

Model Based Teleoperation to Eliminate Feedback Delay NSF Grant BCS89-01352 Second Report

We are conducting research in the area of teleoperation with feedback delay. Delay occurs with earth-based teleoperation in space and with surface-based teleoperation with untethered submersibles when acoustic communication links are involved. The delay in obtaining position and force feedback from remote slave arms makes teleoperation extremely difficult leading to very low productivity. We have combined computer graphics with manipulator programming to provide a solution to the problem. A teleoperator master arm is interfaced to a graphics based simulator of the remote environment. The system is then coupled with a robot manipulator at the remote, delayed site. The operator\u27s actions are monitored to provide both kinesthetic and visual feedback and to generate symbolic motion commands to the remote slave. The slave robot then executes these symbolic commands delayed in time. While much of a task proceeds error free, when an error does occur, the slave system transmits data back to the master environment which is then reset to the error state from which the operator continues the task