Virtual reality training and assessment in laparoscopic rectum surgery

Background: Virtual-reality (VR) based simulation techniques offer an efficient and low cost alternative to conventional surgery training. This article describes a VR training and assessment system in laparoscopic rectum surgery. Methods: To give a realistic visual performance of interaction between membrane tissue and surgery tools, a generalized cylinder based collision detection and a multi-layer mass-spring model are presented. A dynamic assessment model is also designed for hierarchy training evaluation. Results: With this simulator, trainees can operate on the virtual rectum with both visual and haptic sensation feedback simultaneously. The system also offers surgeons instructions in real time when improper manipulation happens. The simulator has been tested and evaluated by ten subjects. Conclusions: This prototype system has been verified by colorectal surgeons through a pilot study. They believe the visual performance and the tactile feedback are realistic. It exhibits the potential to effectively improve the surgical skills of trainee surgeons and significantly shorten their learning curve. © 2014 John Wiley & Sons, Ltd

Coronary Artery Segmentation and Motion Modelling

Conventional coronary artery bypass surgery requires invasive sternotomy and the use of a cardiopulmonary bypass, which leads to long recovery period and has high infectious potential. Totally endoscopic coronary artery bypass (TECAB) surgery based on image guided robotic surgical approaches have been developed to allow the clinicians to conduct the bypass surgery off-pump with only three pin holes incisions in the chest cavity, through which two robotic arms and one stereo endoscopic camera are inserted. However, the restricted field of view of the stereo endoscopic images leads to possible vessel misidentification and coronary artery mis-localization. This results in 20-30% conversion rates from TECAB surgery to the conventional approach. We have constructed patient-specific 3D + time coronary artery and left ventricle motion models from preoperative 4D Computed Tomography Angiography (CTA) scans. Through temporally and spatially aligning this model with the intraoperative endoscopic views of the patient's beating heart, this work assists the surgeon to identify and locate the correct coronaries during the TECAB precedures. Thus this work has the prospect of reducing the conversion rate from TECAB to conventional coronary bypass procedures. This thesis mainly focus on designing segmentation and motion tracking methods of the coronary arteries in order to build pre-operative patient-specific motion models. Various vessel centreline extraction and lumen segmentation algorithms are presented, including intensity based approaches, geometric model matching method and morphology-based method. A probabilistic atlas of the coronary arteries is formed from a group of subjects to facilitate the vascular segmentation and registration procedures. Non-rigid registration framework based on a free-form deformation model and multi-level multi-channel large deformation diffeomorphic metric mapping are proposed to track the coronary motion. The methods are applied to 4D CTA images acquired from various groups of patients and quantitatively evaluated

Beam-inside-beam contact: Mechanical simulations of slender medical instruments inside the human body

Background and Objective This contribution presents a rapid computational framework to mechanically simulate the insertion of a slender medical instrument in a tubular structure such as an artery, the cochlea or another slender instrument. Methods Beams are employed to rapidly simulate the mechanical behaviour of the medical instrument and the tubular structure. However, the framework’s novelty is its capability to handle the mechanical contact between an inner beam (representing the medical instrument) embedded in a hollow outer beam (representing the tubular structure). This “beam-inside-beam” contact framework, which forces two beams to remain embedded, is the first of its kind since existing contact frameworks for beams are “beam-to-beam” approaches, i.e. they repel beams from each other. Furthermore, we propose contact kinematics such that not only instruments and tubes with circular cross-sections can be considered, but also those with elliptical cross-sections. This provides flexibility for the optimization of patient-specific instruments. Results The results demonstrate that the framework’s robustness is substantial, because only a few increments per simulation and a few iterations per increment are required, even though large deformations, large rotations and large curvature changes of both the instrument and tubular structure occur. The stability of the framework remains high even if the modulus of the inner tube is thousand times larger than that of the outer tube. A mesh convergence study furthermore exposes that a relatively small number of elements is required to accurately approach the reference solution. Conclusions The framework’s high simulation speed originates from the exploitation of the rigidity of the beams’ cross-sections to quantify the exclusion between the inner and the hollow outer beam. This rigidity limits the accuracy of the framework at the same time, but this is unavoidable since simulation accuracy and simulation speed are two competing interests. Hence, the framework is particularly attractive if simulation speed is preferred over accuracy

A model-based approach to intraoperative guidance of flexible endoscopy

Endoscopy is a medical procedure, where a physician uses an optical instrument called an "endoscope" to obtain a magnified view of the inner surface of hollow organs and to access the tissue through surgical tools. An endoscope is a flexible tube, which is inserted into the body through either natural body openings or small incisions. Flexible endoscopy often results in a better outcome for the patient as opposed to open surgery. However, this technique also presents increased challenges to the physician and therefore often necessitates intraoperative guidance. This dissertation presents a new approach to the intraoperative guidance of flexible endoscopy. It proposes to calculate a patient specific "protocol" prior to the intervention to achieve a sensor­less guidance during the procedure. This protocol prescribes in detail how to handle the endoscope and tools in order to successfully perform an endoscopic procedure. During the intervention, the physician executes the protocol, by setting endoscope and tools to the prescribed configuration. The calculation of the protocol is based on three components: (1) A 3D model of the target anatomy derived from a CT/MRI scan of the patient, (2) a deformable model representing the endoscope and (3) a virtual endoscopy system. These components are combined to simulate an endoscopic procedures and to estimate a set of endoscope parameters. This general approach is validated for an endoscopic procedure called "Transbronchial Needle Aspiration" (TBNA), which involves the "blind" placement of a needle into a target. Based on a set of candidate shapes for the real endoscope obtained from the endoscope model, an "optimal" needle placement strategy is presented that maximizes the probability of success for TBNAs

Middeck Active Control Experiment (MACE), phase A

A rationale to determine which structural experiments are sufficient to verify the design of structures employing Controlled Structures Technology was derived. A survey of proposed NASA missions was undertaken to identify candidate test articles for use in the Middeck Active Control Experiment (MACE). The survey revealed that potential test articles could be classified into one of three roles: development, demonstration, and qualification, depending on the maturity of the technology and the mission the structure must fulfill. A set of criteria was derived that allowed determination of which role a potential test article must fulfill. A review of the capabilities and limitations of the STS middeck was conducted. A reference design for the MACE test article was presented. Computing requirements for running typical closed-loop controllers was determined, and various computer configurations were studied. The various components required to manufacture the structure were identified. A management plan was established for the remainder of the program experiment development, flight and ground systems development, and integration to the carrier. Procedures for configuration control, fiscal control, and safety, reliabilty, and quality assurance were developed

Advanced Endoscopic Navigation:Surgical Big Data,Methodology,and Applications

随着科学技术的飞速发展，健康与环境问题日益成为人类面临的最重大问题之一。信息科学、计算机技术、电子工程与生物医学工程等学科的综合应用交叉前沿课题，研究现代工程技术方法，探索肿瘤癌症等疾病早期诊断、治疗和康复手段。本论文综述了计算机辅助微创外科手术导航、多模态医疗大数据、方法论及其临床应用：从引入微创外科手术导航概念出发，介绍了医疗大数据的术前与术中多模态医学成像方法、阐述了先进微创外科手术导航的核心流程包括计算解剖模型、术中实时导航方案、三维可视化方法及交互式软件技术，归纳了各类微创外科手术方法的临床应用。同时，重点讨论了全球各种手术导航技术在临床应用中的优缺点，分析了目前手术导航领域内的最新技术方法。在此基础上，提出了微创外科手术方法正向数字化、个性化、精准化、诊疗一体化、机器人化以及高度智能化的发展趋势。【Abstract】Interventional endoscopy (e.g., bronchoscopy, colonoscopy, laparoscopy, cystoscopy) is a widely performed procedure that involves either diagnosis of suspicious lesions or guidance for minimally invasive surgery in a variety of organs within the body cavity. Endoscopy may also be used to guide the introduction of certain items (e.g., stents) into the body. Endoscopic navigation systems seek to integrate big data with multimodal information (e.g., computed tomography, magnetic resonance images, endoscopic video sequences, ultrasound images, external trackers) relative to the patient's anatomy, control the movement of medical endoscopes and surgical tools, and guide the surgeon's actions during endoscopic interventions. Nevertheless, it remains challenging to realize the next generation of context-aware navigated endoscopy. This review presents a broad survey of various aspects of endoscopic navigation, particularly with respect to the development of endoscopic navigation techniques. First, we investigate big data with multimodal information involved in endoscopic navigation. Next, we focus on numerous methodologies used for endoscopic navigation. We then review different endoscopic procedures in clinical applications. Finally, we discuss novel techniques and promising directions for the development of endoscopic navigation.X.L. acknowledges funding from the Fundamental Research Funds for the Central Universities. T.M.P. acknowledges funding from the Canadian Foundation for Innovation, the Canadian Institutes for Health Research, the National Sciences and Engineering Research Council of Canada, and a grant from Intuitive Surgical Inc

Surgical Subtask Automation for Intraluminal Procedures using Deep Reinforcement Learning

Intraluminal procedures have opened up a new sub-field of minimally invasive surgery that use flexible instruments to navigate through complex luminal structures of the body, resulting in reduced invasiveness and improved patient benefits. One of the major challenges in this field is the accurate and precise control of the instrument inside the human body. Robotics has emerged as a promising solution to this problem. However, to achieve successful robotic intraluminal interventions, the control of the instrument needs to be automated to a large extent. The thesis first examines the state-of-the-art in intraluminal surgical robotics and identifies the key challenges in this field, which include the need for safe and effective tool manipulation, and the ability to adapt to unexpected changes in the luminal environment. To address these challenges, the thesis proposes several levels of autonomy that enable the robotic system to perform individual subtasks autonomously, while still allowing the surgeon to retain overall control of the procedure. The approach facilitates the development of specialized algorithms such as Deep Reinforcement Learning (DRL) for subtasks like navigation and tissue manipulation to produce robust surgical gestures. Additionally, the thesis proposes a safety framework that provides formal guarantees to prevent risky actions. The presented approaches are evaluated through a series of experiments using simulation and robotic platforms. The experiments demonstrate that subtask automation can improve the accuracy and efficiency of tool positioning and tissue manipulation, while also reducing the cognitive load on the surgeon. The results of this research have the potential to improve the reliability and safety of intraluminal surgical interventions, ultimately leading to better outcomes for patients and surgeons

Tracking and Mapping in Medical Computer Vision: A Review

As computer vision algorithms are becoming more capable, their applications in clinical systems will become more pervasive. These applications include diagnostics such as colonoscopy and bronchoscopy, guiding biopsies and minimally invasive interventions and surgery, automating instrument motion and providing image guidance using pre-operative scans. Many of these applications depend on the specific visual nature of medical scenes and require designing and applying algorithms to perform in this environment. In this review, we provide an update to the field of camera-based tracking and scene mapping in surgery and diagnostics in medical computer vision. We begin with describing our review process, which results in a final list of 515 papers that we cover. We then give a high-level summary of the state of the art and provide relevant background for those who need tracking and mapping for their clinical applications. We then review datasets provided in the field and the clinical needs therein. Then, we delve in depth into the algorithmic side, and summarize recent developments, which should be especially useful for algorithm designers and to those looking to understand the capability of off-the-shelf methods. We focus on algorithms for deformable environments while also reviewing the essential building blocks in rigid tracking and mapping since there is a large amount of crossover in methods. Finally, we discuss the current state of the tracking and mapping methods along with needs for future algorithms, needs for quantification, and the viability of clinical applications in the field. We conclude that new methods need to be designed or combined to support clinical applications in deformable environments, and more focus needs to be put into collecting datasets for training and evaluation.Comment: 31 pages, 17 figure

Development of A Kinetic Model For Loop-Free Colonoscopy Technology

The colonoscope is an important tool in diagnosis and management of diseases of the colon. One of the ongoing challenges with this device is that the colonoscope may form a loop together with the colon during the procedure. The result of the loop is that further insertion of the scope in the colon may not be possible. The loop may also cause risks of perforation of the colon and pain in the patient. There are currently several existing devices to overcome loop formation in colonoscopy, some of which have been introduced in clinical work. However, empirical assessment shows that these devices do not work very well. This is the motivation for the research presented in this thesis. In this thesis, a new paradigm of thinking, “doctor-assisted colonoscopy,” is proposed to overcome loop formation. In this new approach, the physician’s role is enhanced with new information that is acquired by sensors outside the human body and inferred from the mathematical model. It is referred to as a kinetic model due to the fact that this model describes the kinetic behaviour of the scope. This thesis is devoted to development of this kinetic model. In this study, the model of the colonoscope and the model of the colon are developed based on the Timoshenko beam theory, and parameters in both models are determined by the experiments. The following conclusions then are made: (1) self-locking of the colonoscope is the most basic cause for a loop to occur, while structural instability of the colonsocope is dependent on the self-locking; (2) both the scope and the colon can be well represented with the Timoshenko beam elements and the Linear Complementary Problem (LCP) formulation derived from Signorini’s law, and Coulom’s law for representation of interactions between the colon and scope is adequate; (3) there are effects from the location, looping, and tip deflection of the scope on flexural rigidity of the scope. Approximately, the flexural rigidity of the CF-Q160L colonoscope ranges from 300 to 650 N•cm2, and its accuracy is proven by a good agreement between the model predicted result and experimental result; (4) Rayleigh damping for the CF-Q160L colonoscope depends more on the mass matrix [M] of the colonoscope than the stiffness matrix [K], which is evident by the large coefficient value of “alpha” (0.3864) and the small coefficient value of “beta” (0.0164). The contributions of this thesis are: (1) the finding that the main cause of the loop is not structural instability of the colonoscope but rather self-locking of the colonoscope, which could lead to design of a “new-generation” colonoscope to avoid the loop; (2) a systematic evaluation of the existing colonoscopy technologies based on the well-proven Axiomatic Design Theory (ADT), which will serve as a guideline for the development of future new colonoscopes in future; (3) an approach to developing a kinetic model of the colonoscope useful to modeling similar objects such as a catheter guide-wire; (4) a novel ex-vivo colonoscopy test-bed with the kinetic and kinematic measurements useful for validation of new designs in colonoscopy technology and also useful for training physicians who perform the colonoscopy procedure; and (5) a new paradigm of thinking for colonoscopy called “doctor-assisted colonoscopy,” which has potential applications to other medical procedures such as catheter-based procedures