2,025 research outputs found
μμμ κΈ° ν₯μμ μν λ₯λ¬λ κΈ°λ² μ°κ΅¬: λμ₯λ΄μκ²½ μ§λ¨ λ° λ‘λ΄μμ μ κΈ° νκ°μ μ μ©
νμλ
Όλ¬Έ (λ°μ¬) -- μμΈλνκ΅ λνμ : 곡과λν νλκ³Όμ μμ©μ체곡νμ 곡, 2020. 8. κΉν¬μ°¬.This paper presents deep learning-based methods for improving performance of clinicians. Novel methods were applied to the following two clinical cases and the results were evaluated.
In the first study, a deep learning-based polyp classification algorithm for improving clinical performance of endoscopist during colonoscopy diagnosis was developed. Colonoscopy is the main method for diagnosing adenomatous polyp, which can multiply into a colorectal cancer and hyperplastic polyps. The classification algorithm was developed using convolutional neural network (CNN), trained with colorectal polyp images taken by a narrow-band imaging colonoscopy. The proposed method is built around an automatic machine learning (AutoML) which searches for the optimal architecture of CNN for colorectal polyp image classification and trains the weights of the architecture. In addition, gradient-weighted class activation mapping technique was used to overlay the probabilistic basis of the prediction result on the polyp location to aid the endoscopists visually. To verify the improvement in diagnostic performance, the efficacy of endoscopists with varying proficiency levels were compared with or without the aid of the proposed polyp classification algorithm. The results confirmed that, on average, diagnostic accuracy was improved and diagnosis time was shortened in all proficiency groups significantly.
In the second study, a surgical instruments tracking algorithm for robotic surgery video was developed, and a model for quantitatively evaluating the surgeons surgical skill based on the acquired motion information of the surgical instruments was proposed. The movement of surgical instruments is the main component of evaluation for surgical skill. Therefore, the focus of this study was develop an automatic surgical instruments tracking algorithm, and to overcome the limitations presented by previous methods. The instance segmentation framework was developed to solve the instrument occlusion issue, and a tracking framework composed of a tracker and a re-identification algorithm was developed to maintain the type of surgical instruments being tracked in the video. In addition, algorithms for detecting the tip position of instruments and arm-indicator were developed to acquire the movement of devices specialized for the robotic surgery video. The performance of the proposed method was evaluated by measuring the difference between the predicted tip position and the ground truth position of the instruments using root mean square error, area under the curve, and Pearsons correlation analysis. Furthermore, motion metrics were calculated from the movement of surgical instruments, and a machine learning-based robotic surgical skill evaluation model was developed based on these metrics. These models were used to evaluate clinicians, and results were similar in the developed evaluation models, the Objective Structured Assessment of Technical Skill (OSATS), and the Global Evaluative Assessment of Robotic Surgery (GEARS) evaluation methods.
In this study, deep learning technology was applied to colorectal polyp images for a polyp classification, and to robotic surgery videos for surgical instruments tracking. The improvement in clinical performance with the aid of these methods were evaluated and verified.λ³Έ λ
Όλ¬Έμ μλ£μ§μ μμμ κΈ° λ₯λ ₯μ ν₯μμν€κΈ° μνμ¬ μλ‘μ΄ λ₯λ¬λ κΈ°λ²λ€μ μ μνκ³ λ€μ λ κ°μ§ μ€λ‘μ λν΄ μ μ©νμ¬ κ·Έ κ²°κ³Όλ₯Ό νκ°νμλ€.
첫 λ²μ§Έ μ°κ΅¬μμλ λμ₯λ΄μκ²½μΌλ‘ κ΄ν μ§λ¨ μ, λ΄μκ²½ μ λ¬Έμμ μ§λ¨ λ₯λ ₯μ ν₯μμν€κΈ° μνμ¬ λ₯λ¬λ κΈ°λ°μ μ©μ’
λΆλ₯ μκ³ λ¦¬μ¦μ κ°λ°νκ³ , λ΄μκ²½ μ λ¬Έμμ μ§λ¨ λ₯λ ₯ ν₯μ μ¬λΆλ₯Ό κ²μ¦νκ³ μ νμλ€. λμ₯λ΄μκ²½ κ²μ¬λ‘ μμ’
μΌλ‘ μ¦μν μ μλ μ μ’
κ³Ό κ³Όμ¦μμ± μ©μ’
μ μ§λ¨νλ κ²μ μ€μνλ€. λ³Έ μ°κ΅¬μμλ νλμ μμ λ΄μκ²½μΌλ‘ 촬μν λμ₯ μ©μ’
μμμΌλ‘ ν©μ±κ³± μ κ²½λ§μ νμ΅νμ¬ λΆλ₯ μκ³ λ¦¬μ¦μ κ°λ°νμλ€. μ μνλ μκ³ λ¦¬μ¦μ μλ κΈ°κ³νμ΅ (AutoML) λ°©λ²μΌλ‘, λμ₯ μ©μ’
μμμ μ΅μ νλ ν©μ±κ³± μ κ²½λ§ κ΅¬μ‘°λ₯Ό μ°Ύκ³ μ κ²½λ§μ κ°μ€μΉλ₯Ό νμ΅νμλ€. λν κΈ°μΈκΈ°-κ°μ€μΉ ν΄λμ€ νμ±ν 맡ν κΈ°λ²μ μ΄μ©νμ¬ κ°λ°ν ν©μ±κ³± μ κ²½λ§ κ²°κ³Όμ νλ₯ μ κ·Όκ±°λ₯Ό μ©μ’
μμΉμ μκ°μ μΌλ‘ λνλλλ‘ ν¨μΌλ‘ λ΄μκ²½ μ λ¬Έμμ μ§λ¨μ λλλ‘ νμλ€. λ§μ§λ§μΌλ‘, μλ ¨λ κ·Έλ£Ήλ³λ‘ λ΄μκ²½ μ λ¬Έμκ° μ©μ’
λΆλ₯ μκ³ λ¦¬μ¦μ κ²°κ³Όλ₯Ό μ°Έκ³ νμμ λ μ§λ¨ λ₯λ ₯μ΄ ν₯μλμλμ§ λΉκ΅ μ€νμ μ§ννμκ³ , λͺ¨λ κ·Έλ£Ήμμ μ μλ―Ένκ² μ§λ¨ μ νλκ° ν₯μλκ³ μ§λ¨ μκ°μ΄ λ¨μΆλμμμ νμΈνμλ€.
λ λ²μ§Έ μ°κ΅¬μμλ λ‘λ΄μμ λμμμμ μμ λꡬ μμΉ μΆμ μκ³ λ¦¬μ¦μ κ°λ°νκ³ , νλν μμ λꡬμ μμ§μ μ 보λ₯Ό λ°νμΌλ‘ μμ μμ μλ ¨λλ₯Ό μ λμ μΌλ‘ νκ°νλ λͺ¨λΈμ μ μνμλ€. μμ λꡬμ μμ§μμ μμ μμ λ‘λ΄μμ μλ ¨λλ₯Ό νκ°νκΈ° μν μ£Όμν μ 보μ΄λ€. λ°λΌμ λ³Έ μ°κ΅¬λ λ₯λ¬λ κΈ°λ°μ μλ μμ λꡬ μΆμ μκ³ λ¦¬μ¦μ κ°λ°νμμΌλ©°, λ€μ λκ°μ§ μ νμ°κ΅¬μ νκ³μ μ 극볡νμλ€. μΈμ€ν΄μ€ λΆν (Instance Segmentation) νλ μμμ κ°λ°νμ¬ νμ (Occlusion) λ¬Έμ λ₯Ό ν΄κ²°νμκ³ , μΆμ κΈ° (Tracker)μ μ¬μλ³ν (Re-Identification) μκ³ λ¦¬μ¦μΌλ‘ ꡬμ±λ μΆμ νλ μμμ κ°λ°νμ¬ λμμμμ μΆμ νλ μμ λꡬμ μ’
λ₯κ° μ μ§λλλ‘ νμλ€. λν λ‘λ΄μμ λμμμ νΉμμ±μ κ³ λ €νμ¬ μμ λꡬμ μμ§μμ νλνκΈ°μν΄ μμ λꡬ λ μμΉμ λ‘λ΄ ν-μΈλμΌμ΄ν° (Arm-Indicator) μΈμ μκ³ λ¦¬μ¦μ κ°λ°νμλ€. μ μνλ μκ³ λ¦¬μ¦μ μ±λ₯μ μμΈ‘ν μμ λꡬ λ μμΉμ μ λ΅ μμΉ κ°μ νκ· μ κ³±κ·Ό μ€μ°¨, 곑μ μλ λ©΄μ , νΌμ΄μ¨ μκ΄λΆμμΌλ‘ νκ°νμλ€. λ§μ§λ§μΌλ‘, μμ λꡬμ μμ§μμΌλ‘λΆν° μμ§μ μ§νλ₯Ό κ³μ°νκ³ μ΄λ₯Ό λ°νμΌλ‘ κΈ°κ³νμ΅ κΈ°λ°μ λ‘λ΄μμ μλ ¨λ νκ° λͺ¨λΈμ κ°λ°νμλ€. κ°λ°ν νκ° λͺ¨λΈμ κΈ°μ‘΄μ Objective Structured Assessment of Technical Skill (OSATS), Global Evaluative Assessment of Robotic Surgery (GEARS) νκ° λ°©λ²κ³Ό μ μ¬ν μ±λ₯μ 보μμ νμΈνμλ€.
λ³Έ λ
Όλ¬Έμ μλ£μ§μ μμμ κΈ° λ₯λ ₯μ ν₯μμν€κΈ° μνμ¬ λμ₯ μ©μ’
μμκ³Ό λ‘λ΄μμ λμμμ λ₯λ¬λ κΈ°μ μ μ μ©νκ³ κ·Έ μ ν¨μ±μ νμΈνμμΌλ©°, ν₯νμ μ μνλ λ°©λ²μ΄ μμμμ μ¬μ©λκ³ μλ μ§λ¨ λ° νκ° λ°©λ²μ λμμ΄ λ κ²μΌλ‘ κΈ°λνλ€.Chapter 1 General Introduction 1
1.1 Deep Learning for Medical Image Analysis 1
1.2 Deep Learning for Colonoscipic Diagnosis 2
1.3 Deep Learning for Robotic Surgical Skill Assessment 3
1.4 Thesis Objectives 5
Chapter 2 Optical Diagnosis of Colorectal Polyps using Deep Learning with Visual Explanations 7
2.1 Introduction 7
2.1.1 Background 7
2.1.2 Needs 8
2.1.3 Related Work 9
2.2 Methods 11
2.2.1 Study Design 11
2.2.2 Dataset 14
2.2.3 Preprocessing 17
2.2.4 Convolutional Neural Networks (CNN) 21
2.2.4.1 Standard CNN 21
2.2.4.2 Search for CNN Architecture 22
2.2.4.3 Searched CNN Training 23
2.2.4.4 Visual Explanation 24
2.2.5 Evaluation of CNN and Endoscopist Performances 25
2.3 Experiments and Results 27
2.3.1 CNN Performance 27
2.3.2 Results of Visual Explanation 31
2.3.3 Endoscopist with CNN Performance 33
2.4 Discussion 45
2.4.1 Research Significance 45
2.4.2 Limitations 47
2.5 Conclusion 49
Chapter 3 Surgical Skill Assessment during Robotic Surgery by Deep Learning-based Surgical Instrument Tracking 50
3.1 Introduction 50
3.1.1 Background 50
3.1.2 Needs 51
3.1.3 Related Work 52
3.2 Methods 56
3.2.1 Study Design 56
3.2.2 Dataset 59
3.2.3 Instance Segmentation Framework 63
3.2.4 Tracking Framework 66
3.2.4.1 Tracker 66
3.2.4.2 Re-identification 68
3.2.5 Surgical Instrument Tip Detection 69
3.2.6 Arm-Indicator Recognition 71
3.2.7 Surgical Skill Prediction Model 71
3.3 Experiments and Results 78
3.3.1 Performance of Instance Segmentation Framework 78
3.3.2 Performance of Tracking Framework 82
3.3.3 Evaluation of Surgical Instruments Trajectory 83
3.3.4 Evaluation of Surgical Skill Prediction Model 86
3.4 Discussion 90
3.4.1 Research Significance 90
3.4.2 Limitations 92
3.5 Conclusion 96
Chapter 4 Summary and Future Works 97
4.1 Thesis Summary 97
4.2 Limitations and Future Works 98
Bibliography 100
Abstract in Korean 116
Acknowledgement 119Docto
Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions
Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future
Augmentative and alternative communication (AAC) advances: A review of configurations for individuals with a speech disability
High-tech augmentative and alternative communication (AAC) methods are on a constant rise; however, the interaction between the user and the assistive technology is still challenged for an optimal user experience centered around the desired activity. This review presents a range of signal sensing and acquisition methods utilized in conjunction with the existing high-tech AAC platforms for individuals with a speech disability, including imaging methods, touch-enabled systems, mechanical and electro-mechanical access, breath-activated methods, and brainβcomputer interfaces (BCI). The listed AAC sensing modalities are compared in terms of ease of access, affordability, complexity, portability, and typical conversational speeds. A revelation of the associated AAC signal processing, encoding, and retrieval highlights the roles of machine learning (ML) and deep learning (DL) in the development of intelligent AAC solutions. The demands and the affordability of most systems hinder the scale of usage of high-tech AAC. Further research is indeed needed for the development of intelligent AAC applications reducing the associated costs and enhancing the portability of the solutions for a real userβs environment. The consolidation of natural language processing with current solutions also needs to be further explored for the amelioration of the conversational speeds. The recommendations for prospective advances in coming high-tech AAC are addressed in terms of developments to support mobile health communicative applications
Estimation of Absolute States of Human Skeletal Muscle via Standard B-Mode Ultrasound Imaging and Deep Convolutional Neural Networks
Objective: To test automated in vivo estimation of active and passive skeletal muscle states using ultrasonic imaging. Background: Current technology (electromyography, dynamometry, shear wave imaging) provides no general, non-invasive method for online estimation of skeletal muscle states. Ultrasound (US) allows non-invasive imaging of muscle, yet current computational approaches have never achieved simultaneous extraction nor generalisation of independently varying, active and passive states. We use deep learning to investigate the generalizable content of 2D US muscle images. Method: US data synchronized with electromyography of the calf muscles, with measures of joint moment/angle were recorded from 32 healthy participants (7 female, ages: 27.5, 19-65). We extracted a region of interest of medial gastrocnemius and soleus using our prior developed accurate segmentation algorithm. From the segmented images, a deep convolutional neural network was trained to predict three absolute, driftfree, components of the neurobiomechanical state (activity, joint angle, joint moment) during experimentally designed, simultaneous, independent variation of passive (joint angle) and active (electromyography) inputs. Results: For all 32 held-out participants (16-fold cross-validation) the ankle joint angle, electromyography, and joint moment were estimated to accuracy 55Β±8%, 57Β±11%, and 46Β±9% respectively. Significance: With 2D US imaging, deep neural networks can encode in generalizable form, the activitylength-tension state relationship of these muscles. Observation only, low power, 2D US imaging can provide a new category of technology for non-invasive estimation of neural output, length and tension in skeletal muscle. This proof of principle has value for personalised muscle assessment in pain, injury, neurological conditions, neuropathies, myopathies and ageing
Haptic Enhancement of Sensorimotor Learning for Clinical Training Applications
Modern surgical training requires radical change with the advent of increasingly complex procedures, restricted working hours, and reduced βhands-onβ training in the operating theatre. Moreover, an increased focus on patient safety means there is a greater need to objectively measure proficiency in trainee surgeons. Indeed, the existing evidence suggests that surgical sensorimotor skill training is not adequate for modern surgery. This calls for new training methodologies which can increase the acquisition rate of sensorimotor skill. Haptic interventions offer one exciting possible avenue for enhancing surgical skills in a safe environment. Nevertheless, the best approach for implementing novel training methodologies involving haptic intervention within existing clinical training curricula has yet to be determined. This thesis set out to address this issue. In Chapter 2, the development of two novel tools which enable the implementation of bespoke visuohaptic environments within robust experimental protocols is described. Chapters 3 and 4 report the effects of intensive, long-term training on the acquisition of a compliance discrimination skill. The results indicate that active behaviour is intrinsically linked to compliance perception, and that long-term training can help to improve the ability of detecting compliance differences. Chapter 5 explores the effects of error augmentation and parameter space exploration on the learning of a complex novel task. The results indicate that error augmentation can help improve learning rate, and that physical workspace exploration may be a driver for motor learning. This research is a first step towards the design of objective haptic intervention strategies to help support the rapid acquisition of sensorimotor skill. The work has applications in clinical settings such as surgical training, dentistry and physical rehabilitation, as well as other areas such as sport
Development and Validation of a Hybrid Virtual/Physical Nuss Procedure Surgical Trainer
With continuous advancements and adoption of minimally invasive surgery, proficiency with nontrivial surgical skills involved is becoming a greater concern. Consequently, the use of surgical simulation has been increasingly embraced by many for training and skill transfer purposes. Some systems utilize haptic feedback within a high-fidelity anatomically-correct virtual environment whereas others use manikins, synthetic components, or box trainers to mimic primary components of a corresponding procedure.
Surgical simulation development for some minimally invasive procedures is still, however, suboptimal or otherwise embryonic. This is true for the Nuss procedure, which is a minimally invasive surgery for correcting pectus excavatum (PE) β a congenital chest wall deformity. This work aims to address this gap by exploring the challenges of developing both a purely virtual and a purely physical simulation platform of the Nuss procedure and their implications in a training context. This work then describes the development of a hybrid mixed-reality system that integrates virtual and physical constituents as well as an augmentation of the haptic interface, to carry out a reproduction of the primary steps of the Nuss procedure and satisfy clinically relevant prerequisites for its training platform.
Furthermore, this work carries out a user study to investigate the systemβs face, content, and construct validity to establish its faithfulness as a training platform
Medical Robotics
The first generation of surgical robots are already being installed in a number of operating rooms around the world. Robotics is being introduced to medicine because it allows for unprecedented control and precision of surgical instruments in minimally invasive procedures. So far, robots have been used to position an endoscope, perform gallbladder surgery and correct gastroesophogeal reflux and heartburn. The ultimate goal of the robotic surgery field is to design a robot that can be used to perform closed-chest, beating-heart surgery. The use of robotics in surgery will expand over the next decades without any doubt. Minimally Invasive Surgery (MIS) is a revolutionary approach in surgery. In MIS, the operation is performed with instruments and viewing equipment inserted into the body through small incisions created by the surgeon, in contrast to open surgery with large incisions. This minimizes surgical trauma and damage to healthy tissue, resulting in shorter patient recovery time. The aim of this book is to provide an overview of the state-of-art, to present new ideas, original results and practical experiences in this expanding area. Nevertheless, many chapters in the book concern advanced research on this growing area. The book provides critical analysis of clinical trials, assessment of the benefits and risks of the application of these technologies. This book is certainly a small sample of the research activity on Medical Robotics going on around the globe as you read it, but it surely covers a good deal of what has been done in the field recently, and as such it works as a valuable source for researchers interested in the involved subjects, whether they are currently βmedical roboticistsβ or not
Shear-promoted drug encapsulation into red blood cells: a CFD model and ΞΌ-PIV analysis
The present work focuses on the main parameters that influence shear-promoted encapsulation of drugs into erythrocytes. A CFD model was built to investigate the fluid dynamics of a suspension of particles flowing in a commercial micro channel. Micro Particle Image Velocimetry (ΞΌ-PIV) allowed to take into account for the real properties of the red blood cell (RBC), thus having a deeper understanding of the process. Coupling these results with an analytical diffusion model, suitable working conditions were defined for different values of haematocrit
- β¦