A Novel Head-mounted Display based Control in Robotic Surgery

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 협동과정 바이오엔지니어링전공, 2020. 8. Sungwan Kim.현존하는 복강경 로봇 수술은 다양한 이점을 제공하지만 수술 중 집도의는 목, 어깨 그리고 등의 통증을 유발하는 불편한 자세를 유지해야 한다. 본 연구에서는 이러한 단점을 개선하기 위하여 머리 착용형 디스플레이(Head-mounted display, HMD)에 기반한 제어 시스템을 제안한다. 본 연구에서 활용되는 전체 시스템은 da Vinci research kit (dVRK)와 4 자유도의 내시경 제어 시스템 (Endoscope control system, ECS), 내시경 모듈, Attitude and heading reference system (AHRS)이 내장된 HMD로 구성된다. 4 자유도 ECS에 결합되어 사용되는 내시경 모듈은 HMD에 내장된 AHRS에 의해 제어되며, 이 HMD은 dVRK에 있는 Stereo viewer를 대체하여 수술 로봇 시스템의 크기를 축소할 수 있다. 수술 로봇 플랫폼에서 HMD 기반 제어의 적용 가능성을 평가하기 위해 해당 시스템을 다루어 본 경험이 없는 4명의 지원자들을 모집하여 Peg transfer task를 수행하게 하였다. 또한, HMD 기반 제어의 사용성을 평가하기 위해 Line tracking test를 진행하였다. 지원자들은 본 시스템에 빠른 속도로 학습하는 경향을 보여주었으며, 사용자 간 변산도 (Inter-user variability) 또한 매우 작았다. 내시경과 수술 도구를 동시에 제어하는 경우, 내시경 하드웨어와 수술 도구 사이의 충돌에 취약할 수 있다. 이에, HMD을 기반으로 ECS을 조종할 때에 적용되는 충돌 방지 전략을 개발하였다. 수술 기구와 내시경을 둘러싸는 Oriented bounding boxes (OBBs)를 정의하고 박스들 간의 충돌 가능성을 계산함으로써 수술 기구와 내시경 간의 충돌 여부를 추정하였으며, 사용자의 의도와 충돌 방지를 동시에 제어 신호에 반영하였다. dVRK에서는 End-effector의 실시간 위치 추적 데이터를 제공하지 않기 때문에 충돌 방지 전략을 검증하기 위해 MATLAB®을 활용한 컴퓨터 시뮬레이션을 수행하였다. 그 결과, 충돌 방지 전략이 수술 환경의 안전성을 보장함을 확인할 수 있었고, 안전성과 사용자의 의도 간 Trade-off를 적절히 고려한 Blending 파라미터 범위를 제안하였다. 본 연구를 통해 제안하는 HMD 기반 내시경 제어는 집도의의 목, 어깨 그리고 등에 발생하는 통증을 감소시킬 수 있어, 결과적으로 보다 효율적인 수술이 가능할 것이다. 또한, 기존의 Stereo viewer와 비교하여 공간적 효율성 또한 크게 개선할 수 있어, 차세대 수술 로봇의 제어 인터페이스로서 활용이 가능할 것으로 사료된다.Robotic laparoscopic surgery has provided various benefits, but during the surgery, the surgeons are experiencing uncomfortable positioning issue which leads to neck, shoulder, and back pain. For improving this issue, a novel head-mounted display (HMD) based endoscope control system (ECS) considering an ergonomic aspect is proposed in this research. The overall system is composed of a da Vinci research kit (dVRK), 4-degree-of-freedom ECS, endoscope module, and HMD with a built-in attitude and heading reference system (AHRS). The endoscope module is controlled by a built-in AHRS in the HMD. The stereo viewer in dVRK could be replaced by the HMD, so it would reduce the size of surgical robot system. Applicability of the proposed system to surgical robot platform was verified by peg-transfer task with four novice volunteers. Also, line tracking test was conducted to assess usability of the HMD based control. They showed rapid learning to the system and small value of inter-user variability. In the case of simultaneous control of HMD and surgical instruments, the collision issue between them could be raised. Thus, a collision avoidance strategy for HMD based ECS control was developed. Oriented bounding boxes (OBBs) containing the surgical instruments and endoscope were defined. And then, it is estimated whether the surgical instruments and endoscope collide through calculating the possibility between the OBBs. The control signal to endoscope includes both the user intention and collision avoidance strategy. dVRK does not provide real-time position data of its end-effectors, so computer-based simulations through MATLAB® were performed to verify the collision avoidance strategy. As a result, the strategy was assured of safety of surgery, and the range of blending parameter considering a trade-off between the user intention and safety was proposed. The HMD based ECS proposed in this research could reduce surgeons pains in neck, shoulder, and back, so it would lead to more efficient surgery. Additionally, space efficiency could be improved compared with the existing stereo viewer, so it is considered that the proposed system could be used as the control interface of the next-generation surgical robot.1. 서론 1 1.1. 수술 로봇 개요 1 1.1.1. 로봇을 활용한 최소 침습 수술 1 1.1.2. 현존하는 수술 로봇의 문제점 3 1.1.3. 선행 연구들의 문제점 3 1.2. 연구의 목적 4 1.2.1. HMD 기반의 내시경 제어 시스템 4 1.2.3. 충돌 방지 전략 5 1.3. 기대 효과 5 2. 방법 7 2.1. Hardware 구현 7 2.1.1. ECS를 활용한 수술 로봇의 구성 7 2.1.2. ECS 제어를 위한 HMD 9 2.2. HMD 기반의 제어 알고리즘 9 2.3. HMD 기반 제어의 검증 12 2.3.1. Peg transfer task 12 2.3.2. Line tracking test 15 2.4. 충돌 방지 12 2.4.1. 충돌 방지 알고리즘 17 2.4.2. 이론적 고찰 18 2.5. OBB 12 2.5.1. OBB의 정의 19 2.5.2. OBB 간의 거리 20 2.6. 보상 벡터의 계산 23 2.7. 충돌 방지의 검증 24 3. 결과 및 분석 27 3.1. HMD 기반 제어의 평가 27 3.2. 충돌 방지의 평가 29 4. 고찰 40 4.1. Peg transfer task 40 4.2. Line tracking test 41 4.3. 충돌 방지 알고리즘 42 5. 결론 43 5.1. 결론 43 5.2. 향후 연구 44 참고 문헌 45 Abstract 48 감사의 글 51Maste

Parallel Manipulators

In recent years, parallel kinematics mechanisms have attracted a lot of attention from the academic and industrial communities due to potential applications not only as robot manipulators but also as machine tools. Generally, the criteria used to compare the performance of traditional serial robots and parallel robots are the workspace, the ratio between the payload and the robot mass, accuracy, and dynamic behaviour. In addition to the reduced coupling effect between joints, parallel robots bring the benefits of much higher payload-robot mass ratios, superior accuracy and greater stiffness; qualities which lead to better dynamic performance. The main drawback with parallel robots is the relatively small workspace. A great deal of research on parallel robots has been carried out worldwide, and a large number of parallel mechanism systems have been built for various applications, such as remote handling, machine tools, medical robots, simulators, micro-robots, and humanoid robots. This book opens a window to exceptional research and development work on parallel mechanisms contributed by authors from around the world. Through this window the reader can get a good view of current parallel robot research and applications

Advanced Strategies for Robot Manipulators

Amongst the robotic systems, robot manipulators have proven themselves to be of increasing importance and are widely adopted to substitute for human in repetitive and/or hazardous tasks. Modern manipulators are designed complicatedly and need to do more precise, crucial and critical tasks. So, the simple traditional control methods cannot be efficient, and advanced control strategies with considering special constraints are needed to establish. In spite of the fact that groundbreaking researches have been carried out in this realm until now, there are still many novel aspects which have to be explored

Adaptive and reconfigurable robotic gripper hands with a meso-scale gripping range

Grippers and robotic hands are essential and important end-effectors of robotic manipulators. Developing a gripper hand that can grasp a large variety of objects precisely and stably is still an aspiration even though research in this area has been carried out for several decades. This thesis provides a development approach and a series of gripper hands which can bridge the gap between micro-gripper and macro-gripper by extending the gripping range to the mesoscopic scale (meso-scale). Reconfigurable topology and variable mobility of the design offer versatility and adaptability for the changing environment and demands. By investigating human grasping behaviours and the unique structures of human hand, a CFB-based finger joint for anthropomorphic finger is developed to mimic a human finger with a large grasping range. The centrodes of CFB mechanism are explored and a contact-aided CFB mechanism is developed to increase stiffness of finger joints. An integrated gripper structure comprising cross four-bar (CFB) and remote-centre-of-motion (RCM) mechanisms is developed to mimic key functionalities of human hand. Kinematics and kinetostatic analyses of the CFB mechanism for multimode gripping are conducted to achieve passive-adjusting motion. A novel RCM-based finger with angular, parallel and underactuated motion is invented. Kinematics and stable gripping analyses of the RCM-based multi-motion finger are also investigated. The integrated design with CFB and RCM mechanisms provides a novel concept of a multi-mode gripper that aims to tackle the challenge of changing over for various sizes of objects gripping in mesoscopic scale range. Based on the novel designed mechanisms and design philosophy, a class of gripper hands in terms of adaptive meso-grippers, power-precision grippers and reconfigurable hands are developed. The novel features of the gripper hands are one degree of freedom (DoF), self-adaptive, reconfigurable and multi-mode. Prototypes are manufactured by 3D printing and the grasping abilities are tested to verify the design approach.EPSR