Posture Operating Method by Foot Posture Change and Characteristics of Foot Motion

The lower limbs of the human body actually can perform the multiple-degree-of-freedom motion, just like the upper limbs. This suggests the possibility for the lower limbs to be used in the operation of multiple-degree-of-freedom devices, such as a robot arm. With that point in mind, the present paper focuses on the foot motion and examines its characteristics under the situation in which the posture of the object is manipulated by the posture change of the foot. First, we investigated how well the foot of the operator moved in accordance with the intention of the operator in order to clarify the motion characteristics of the foot experimentally by measuring the foot motion with a motion capture system under the assumption that the operator manipulates an object in virtual space. The results showed that there are differences between the intended and actual foot motions, especially when the tilt angle change was accompanied by a rotation angle change, which might be because of the joints whose axes of motion are nonparallel to the foot coordinate system, such as the talocalcaneal joint or Chopart joint. Next, an operating system considering the motion characteristics of the foot was proposed, and an experiment to verify its effectiveness was conducted. When the proposed conversion formula was used to calculate the intended foot motion based on the actual foot motion, the operability improved with respect to the required time and path-following accuracy while manipulating an object to the target posture and with respect to subjective operability

A Novel Head-mounted Display based Control in Robotic Surgery

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 협동과정 바이오엔지니어링전공, 2020. 8. Sungwan Kim.현존하는 복강경 로봇 수술은 다양한 이점을 제공하지만 수술 중 집도의는 목, 어깨 그리고 등의 통증을 유발하는 불편한 자세를 유지해야 한다. 본 연구에서는 이러한 단점을 개선하기 위하여 머리 착용형 디스플레이(Head-mounted display, HMD)에 기반한 제어 시스템을 제안한다. 본 연구에서 활용되는 전체 시스템은 da Vinci research kit (dVRK)와 4 자유도의 내시경 제어 시스템 (Endoscope control system, ECS), 내시경 모듈, Attitude and heading reference system (AHRS)이 내장된 HMD로 구성된다. 4 자유도 ECS에 결합되어 사용되는 내시경 모듈은 HMD에 내장된 AHRS에 의해 제어되며, 이 HMD은 dVRK에 있는 Stereo viewer를 대체하여 수술 로봇 시스템의 크기를 축소할 수 있다. 수술 로봇 플랫폼에서 HMD 기반 제어의 적용 가능성을 평가하기 위해 해당 시스템을 다루어 본 경험이 없는 4명의 지원자들을 모집하여 Peg transfer task를 수행하게 하였다. 또한, HMD 기반 제어의 사용성을 평가하기 위해 Line tracking test를 진행하였다. 지원자들은 본 시스템에 빠른 속도로 학습하는 경향을 보여주었으며, 사용자 간 변산도 (Inter-user variability) 또한 매우 작았다. 내시경과 수술 도구를 동시에 제어하는 경우, 내시경 하드웨어와 수술 도구 사이의 충돌에 취약할 수 있다. 이에, HMD을 기반으로 ECS을 조종할 때에 적용되는 충돌 방지 전략을 개발하였다. 수술 기구와 내시경을 둘러싸는 Oriented bounding boxes (OBBs)를 정의하고 박스들 간의 충돌 가능성을 계산함으로써 수술 기구와 내시경 간의 충돌 여부를 추정하였으며, 사용자의 의도와 충돌 방지를 동시에 제어 신호에 반영하였다. dVRK에서는 End-effector의 실시간 위치 추적 데이터를 제공하지 않기 때문에 충돌 방지 전략을 검증하기 위해 MATLAB®을 활용한 컴퓨터 시뮬레이션을 수행하였다. 그 결과, 충돌 방지 전략이 수술 환경의 안전성을 보장함을 확인할 수 있었고, 안전성과 사용자의 의도 간 Trade-off를 적절히 고려한 Blending 파라미터 범위를 제안하였다. 본 연구를 통해 제안하는 HMD 기반 내시경 제어는 집도의의 목, 어깨 그리고 등에 발생하는 통증을 감소시킬 수 있어, 결과적으로 보다 효율적인 수술이 가능할 것이다. 또한, 기존의 Stereo viewer와 비교하여 공간적 효율성 또한 크게 개선할 수 있어, 차세대 수술 로봇의 제어 인터페이스로서 활용이 가능할 것으로 사료된다.Robotic laparoscopic surgery has provided various benefits, but during the surgery, the surgeons are experiencing uncomfortable positioning issue which leads to neck, shoulder, and back pain. For improving this issue, a novel head-mounted display (HMD) based endoscope control system (ECS) considering an ergonomic aspect is proposed in this research. The overall system is composed of a da Vinci research kit (dVRK), 4-degree-of-freedom ECS, endoscope module, and HMD with a built-in attitude and heading reference system (AHRS). The endoscope module is controlled by a built-in AHRS in the HMD. The stereo viewer in dVRK could be replaced by the HMD, so it would reduce the size of surgical robot system. Applicability of the proposed system to surgical robot platform was verified by peg-transfer task with four novice volunteers. Also, line tracking test was conducted to assess usability of the HMD based control. They showed rapid learning to the system and small value of inter-user variability. In the case of simultaneous control of HMD and surgical instruments, the collision issue between them could be raised. Thus, a collision avoidance strategy for HMD based ECS control was developed. Oriented bounding boxes (OBBs) containing the surgical instruments and endoscope were defined. And then, it is estimated whether the surgical instruments and endoscope collide through calculating the possibility between the OBBs. The control signal to endoscope includes both the user intention and collision avoidance strategy. dVRK does not provide real-time position data of its end-effectors, so computer-based simulations through MATLAB® were performed to verify the collision avoidance strategy. As a result, the strategy was assured of safety of surgery, and the range of blending parameter considering a trade-off between the user intention and safety was proposed. The HMD based ECS proposed in this research could reduce surgeons pains in neck, shoulder, and back, so it would lead to more efficient surgery. Additionally, space efficiency could be improved compared with the existing stereo viewer, so it is considered that the proposed system could be used as the control interface of the next-generation surgical robot.1. 서론 1 1.1. 수술 로봇 개요 1 1.1.1. 로봇을 활용한 최소 침습 수술 1 1.1.2. 현존하는 수술 로봇의 문제점 3 1.1.3. 선행 연구들의 문제점 3 1.2. 연구의 목적 4 1.2.1. HMD 기반의 내시경 제어 시스템 4 1.2.3. 충돌 방지 전략 5 1.3. 기대 효과 5 2. 방법 7 2.1. Hardware 구현 7 2.1.1. ECS를 활용한 수술 로봇의 구성 7 2.1.2. ECS 제어를 위한 HMD 9 2.2. HMD 기반의 제어 알고리즘 9 2.3. HMD 기반 제어의 검증 12 2.3.1. Peg transfer task 12 2.3.2. Line tracking test 15 2.4. 충돌 방지 12 2.4.1. 충돌 방지 알고리즘 17 2.4.2. 이론적 고찰 18 2.5. OBB 12 2.5.1. OBB의 정의 19 2.5.2. OBB 간의 거리 20 2.6. 보상 벡터의 계산 23 2.7. 충돌 방지의 검증 24 3. 결과 및 분석 27 3.1. HMD 기반 제어의 평가 27 3.2. 충돌 방지의 평가 29 4. 고찰 40 4.1. Peg transfer task 40 4.2. Line tracking test 41 4.3. 충돌 방지 알고리즘 42 5. 결론 43 5.1. 결론 43 5.2. 향후 연구 44 참고 문헌 45 Abstract 48 감사의 글 51Maste

복강경 수술 로봇 시스템의 활용도 향상을 위한 추가적인 마스터 인터페이스 개발과 이를 이용한 응용 시스템 개발 연구

학위논문 (박사)-- 서울대학교 대학원 공과대학 협동과정 바이오엔지니어링전공, 2017. 8. Sungwan Kim.Robot-assisted laparoscopic surgery offers several advantages compared to open surgery and conventional minimally invasive surgery. However, important issues which need to be resolved are the complexity of current operation room environment for laparoscopic robotic surgery and demand for a larger operation room. To overcome these issues, additional interfaces based on Hands-On-Throttle-And-Stick (HOTAS) concept which can be simply attached and integrated with master interface of da Vinci surgical robot system were proposed. HOTAS controller is widely used for flight control in the aerospace field which can manipulate hundreds of functions and provide feedback to the pilot on flight conditions. The implementation of HOTAS controller significantly reduced the complexity of flights and reduced the number of pilots required in a cockpit from two to one. In this study, to provide above benefits to the operation room for robotic laparoscopic surgery, two types of additional interfaces are proposed. Proposed additional interfaces can be easily manipulated by the surgeons index finger, which is currently operated only by finger clutch buttons, and therefore enable the surgeon to use multiple functions. Initially, a novel master interface (NMI) was developed. The NMI mainly consists of a 9-way switch and a microprocessor with a wireless communication module. Thus, the NMI can be also regarded as a 9-way compact HOTAS. The performance test, latency, and power consumption of the developed NMI were verified by repeated experiments. Then, an improved novel master interface (iNMI) was developed to provide more intuitive and convenient manipulation. The iNMI was developed based on a capacitive touch sensor array and a wireless microprocessor to intuitively reflect the surgeons decision. Multiple experiments were performed to evaluate the iNMI performance in terms of performance test, latency, and power consumption. In addition, two application systems based on Surgical-Operation-By-Wire (SOBW) concept are proposed in this research to enhance the function of laparoscopic surgical robot system based on clinical needs that are stated below. The size of the additional interface is small enough to be easily installed to the master tool manipulators (MTMs) of da Vinci research kit (dVRK), which was used as an operation robot arm system, to maximize convenience to the surgeon when using the additional interfaces to simultaneously manipulate the application systems with the MTMs. Firstly, a robotic assistant that can be simultaneously manipulated via a wireless controller is proposed to allow the surgeon to control the assistant instrument. This approach not only decreases surgeon fatigue by eliminating communication process with assistants, but also resolves collision between the operation robot arms and the assistant instruments that can be caused by an inexperienced assistant or miscommunication and misaligned intent between the surgeon and the assistant. The system comprises two additional interfaces, a surgical instrument with a gripper actuated by a micromotor and a 6-axis robot arm. The gripping force of the surgical instrument was comparable to that of conventional systems and was consistent even after 1,000 times of gripping motion. The workspace was calculated to be 8,397.4 cm3. Recruited volunteers were able to execute the simple peg task within the cut-off time and successfully performed the in vitro test. Secondly, a wirelessly controllable stereo endoscope system which enables simultaneous control with the operating robot arm system is proposed. This is able to remove any discontinuous surgical flow that occurs when the control is swapped between the endoscope system and the operating robot arm system, and therefore prevent problems such as increased operation time, collision among surgical instruments, and injury to patients. The proposed system consists of two additional interfaces, a four-degrees of freedom (4-DOFs) endoscope control system (ECS) and a simple three-dimensional (3D) endoscope. The 4-DOFs ECS consists of four servo motors and employs a two-parallel link structure to provide translational and fulcrum point motions to the simple 3D endoscope. The workspace was calculated to be 20,378.3 cm3, which exceeds the reference workspace. The novice volunteers were able to successfully execute the modified peg transfer task. Throughout the various verifications, it has been confirmed that the proposed interfaces could make the surgical robot system more efficiently by overcoming its several limitations.1. Introduction 1 1.1. Robotic Laparoscopic Surgery 1 1.2. Objectives and Scope 8 1.2.1. Additional Master Interfaces 14 1.2.2. Application Systems 15 2. Materials and Methods 20 2.1. Additional Master Interfaces 20 2.1.1. Novel Master Interface: 9-way Compact Hands-On-Throttle-And-Stick 20 2.1.2. improved Novel Master Interface: Capacitive Touch Type Compact Hands-On-Throttle-And-Stick 26 2.2. Application Systems 34 2.2.1. Robotic Assistant 34 2.2.2. Stereo Endoscope System 49 3. Results 57 3.1. Novel Master Interface with Application Systems 57 3.1.1. Novel Master Interface 57 3.1.2. Robotic Assistant 59 3.1.3. Novel Master Interface with Robotic Assistant 67 3.1.4. Stereo Endoscope System 76 3.1.5. Novel Master Interface with Stereo Endoscope System 82 3.2. improved Novel Master Interface with Application Systems 87 3.2.1. improved Novel Master Interface 87 3.2.2. improved Novel Master Interface with Stereo Endoscope System 90 4. Discussion 91 5. Conclusion 102 References 105 Abstract in Korean 117Docto