21 research outputs found

    Virtual and Augmented Reality Techniques for Minimally Invasive Cardiac Interventions: Concept, Design, Evaluation and Pre-clinical Implementation

    Get PDF
    While less invasive techniques have been employed for some procedures, most intracardiac interventions are still performed under cardiopulmonary bypass, on the drained, arrested heart. The progress toward off-pump intracardiac interventions has been hampered by the lack of adequate visualization inside the beating heart. This thesis describes the development, assessment, and pre-clinical implementation of a mixed reality environment that integrates pre-operative imaging and modeling with surgical tracking technologies and real-time ultrasound imaging. The intra-operative echo images are augmented with pre-operative representations of the cardiac anatomy and virtual models of the delivery instruments tracked in real time using magnetic tracking technologies. As a result, the otherwise context-less images can now be interpreted within the anatomical context provided by the anatomical models. The virtual models assist the user with the tool-to-target navigation, while real-time ultrasound ensures accurate positioning of the tool on target, providing the surgeon with sufficient information to ``see\u27\u27 and manipulate instruments in absence of direct vision. Several pre-clinical acute evaluation studies have been conducted in vivo on swine models to assess the feasibility of the proposed environment in a clinical context. Following direct access inside the beating heart using the UCI, the proposed mixed reality environment was used to provide the necessary visualization and navigation to position a prosthetic mitral valve on the the native annulus, or to place a repair patch on a created septal defect in vivo in porcine models. Following further development and seamless integration into the clinical workflow, we hope that the proposed mixed reality guidance environment may become a significant milestone toward enabling minimally invasive therapy on the beating heart

    Computer-Assisted Planning and Robotics in Epilepsy Surgery

    Get PDF
    Epilepsy is a severe and devastating condition that affects ~1% of the population. Around 30% of these patients are drug-refractory. Epilepsy surgery may provide a cure in selected individuals with drug-resistant focal epilepsy if the epileptogenic zone can be identified and safely resected or ablated. Stereoelectroencephalography (SEEG) is a diagnostic procedure that is performed to aid in the delineation of the seizure onset zone when non-invasive investigations are not sufficiently informative or discordant. Utilizing a multi-modal imaging platform, a novel computer-assisted planning (CAP) algorithm was adapted, applied and clinically validated for optimizing safe SEEG trajectory planning. In an initial retrospective validation study, 13 patients with 116 electrodes were enrolled and safety parameters between automated CAP trajectories and expert manual plans were compared. The automated CAP trajectories returned statistically significant improvements in all of the compared clinical metrics including overall risk score (CAP 0.57 +/- 0.39 (mean +/- SD) and manual 1.00 +/- 0.60, p < 0.001). Assessment of the inter-rater variability revealed there was no difference in external expert surgeon ratings. Both manual and CAP electrodes were rated as feasible in 42.8% (42/98) of cases. CAP was able to provide feasible electrodes in 19.4% (19/98), whereas manual planning was able to generate a feasible electrode in 26.5% (26/98) when the alternative generation method was not feasible. Based on the encouraging results from the retrospective analysis a prospective validation study including an additional 125 electrodes in 13 patients was then undertaken to compare CAP to expert manual plans from two neurosurgeons. The manual plans were performed separately and blindly from the CAP. Computer-generated trajectories were found to carry lower risks scores (absolute difference of 0.04 mm (95% CI = -0.42-0.01), p = 0.04) and were subsequently implanted in all cases without complication. The pipeline has been fully integrated into the clinical service and has now replaced manual SEEG planning at our institution. Further efforts were then focused on the distillation of optimal entry and target points for common SEEG trajectories and applying machine learning methods to develop an active learning algorithm to adapt to individual surgeon preferences. Thirty-two patients were prospectively enrolled in the study. The first 12 patients underwent prospective CAP planning and implantation following the pipeline outlined in the previous study. These patients were used as a training set and all of the 108 electrodes after successful implantation were normalized to atlas space to generate โ€˜spatial priorsโ€™, using a K-Nearest Neighbour (K-NN) classifier. A subsequent test set of 20 patients (210 electrodes) were then used to prospectively validate the spatial priors. From the test set, 78% (123/157) of the implanted trajectories passed through both the entry and target spatial priors defined from the training set. To improve the generalizability of the spatial priors to other neurosurgical centres undertaking SEEG and to take into account the potential for changing institutional practices, an active learning algorithm was implemented. The K-NN classifier was shown to dynamically learn and refine the spatial priors. The progressive refinement of CAP SEEG planning outlined in this and previous studies has culminated in an algorithm that not only optimizes the surgical heuristics and risk scores related to SEEG planning but can also learn from previous experience. Overall, safe and feasible trajectory schema were returning in 30% of the time required for manual SEEG planning. Computer-assisted planning was then applied to optimize laser interstitial thermal therapy (LITT) trajectory planning, which is a minimally invasive alternative to open mesial temporal resections, focal lesion ablation and anterior 2/3 corpus callosotomy. We describe and validate the first CAP algorithm for mesial temporal LITT ablations for epilepsy treatment. Twenty-five patients that had previously undergone LITT ablations at a single institution and with a median follow up of 2 years were included. Trajectory parameters for the CAP algorithm were derived from expert consensus to maximize distance from vasculature and ablation of the amygdalohippocampal complex, minimize collateral damage to adjacent brain structures whilst avoiding transgression of the ventricles and sulci. Trajectory parameters were also optimized to reduce the drilling angle to the skull and overall catheter length. Simulated cavities attributable to the CAP trajectories were calculated using a 5-15 mm ablation diameter. In comparison to manually planned and implemented LITT trajectories,CAP resulted in a significant increase in the percentage ablation of the amygdalohippocampal complex (manual 57.82 +/- 15.05% (mean +/- S.D.) and unablated medial hippocampal head depth (manual 4.45 +/- 1.58 mm (mean +/- S.D.), CAP 1.19 +/- 1.37 (mean +/- S.D.), p = 0.0001). As LITT ablation of the mesial temporal structures is a novel procedure there are no established standards for trajectory planning. A data-driven machine learning approach was, therefore, applied to identify hitherto unknown CAP trajectory parameter combinations. All possible combinations of planning parameters were calculated culminating in 720 unique combinations per patient. Linear regression and random forest machine learning algorithms were trained on half of the data set (3800 trajectories) and tested on the remaining unseen trajectories (3800 trajectories). The linear regression and random forest methods returned good predictive accuracies with both returning Pearson correlations of ฯ = 0.7 and root mean squared errors of 0.13 and 0.12 respectively. The machine learning algorithm revealed that the optimal entry points were centred over the junction of the inferior occipital, middle temporal and middle occipital gyri. The optimal target points were anterior and medial translations of the centre of the amygdala. A large multicenter external validation study of 95 patients was then undertaken comparing the manually planned and implemented trajectories, CAP trajectories targeting the centre of the amygdala, the CAP parameters derived from expert consensus and the CAP trajectories utilizing the machine learning derived parameters. Three external blinded expert surgeons were then selected to undertake feasibility ratings and preference rankings of the trajectories. CAP generated trajectories result in a significant improvement in many of the planning metrics, notably the risk score (manual 1.3 +/- 0.1 (mean +/- S.D.), CAP 1.1 +/- 0.2 (mean +/- S.D.), p<0.000) and overall ablation of the amygdala (manual 45.3 +/- 22.2 % (mean +/- S.D.), CAP 64.2 +/- 20 % (mean +/- S.D.), p<0.000). Blinded external feasibility ratings revealed that manual trajectories were less preferable than CAP planned trajectories with an estimated probability of being ranked 4th (lowest) of 0.62. Traditional open corpus callosotomy requires a midline craniotomy, interhemispheric dissection and disconnection of the rostrum, genu and body of the corpus callosum. In cases where drop attacks persist a completion corpus callosotomy to disrupt the remaining fibres in the splenium is then performed. The emergence of LITT technology has raised the possibility of being able to undertake this procedure in a minimally invasive fashion and without the need for a craniotomy using two or three individual trajectories. Early case series have shown LITT anterior two-thirds corpus callosotomy to be safe and efficacious. Whole-brain probabilistic tractography connectomes were generated utilizing 3-Tesla multi-shell imaging data and constrained spherical deconvolution (CSD). Two independent blinded expert neurosurgeons with experience of performing the procedure using LITT then planned the trajectories in each patient following their current clinical practice. Automated trajectories returned a significant reduction in the risk score (manual 1.3 +/- 0.1 (mean +/- S.D.), CAP 1.1 +/- 0.1 (mean +/- S.D.), p<0.000). Finally, we investigate the different methods of surgical implantation for SEEG electrodes. As an initial study, a systematic review and meta-analysis of the literature to date were performed. This revealed a wide variety of implantation methods including traditional frame-based, frameless, robotic and custom-3D printed jigs were being used in clinical practice. Of concern, all comparative reports from institutions that had changed from one implantation method to another, such as following the introduction of robotic systems, did not undertake parallel-group comparisons. This suggests that patients may have been exposed to risks associated with learning curves and potential harms related to the new device until the efficacy was known. A pragmatic randomized control trial of a novel non-CE marked robotic trajectory guidance system (iSYS1) was then devised. Before clinical implantations began a series of pre-clinical investigations utilizing 3D printed phantom heads from previously implanted patients was performed to provide pilot data and also assess the surgical learning curve. The surgeons had comparatively little clinical experience with the new robotic device which replicates the introduction of such novel technologies to clinical practice. The study confirmed that the learning curve with the iSYS1 devices was minimal and the accuracies and workflow were similar to the conventional manual method. The randomized control trial represents the first of its kind for stereotactic neurosurgical procedures. Thirty-two patients were enrolled with 16 patients randomized to the iSYS1 intervention arm and 16 patients to the manual implantation arm. The intervention allocation was concealed from the patients. The surgical and research team could be not blinded. Trial management, independent data monitoring and trial steering committees were convened at four points doing the trial (after every 8 patients implanted). Based on the high level of accuracy required for both methods, the main distinguishing factor would be the time to achieve the alignment to the prespecified trajectory. The primary outcome for comparison, therefore, was the time for individual SEEG electrode implantation. Secondary outcomes included the implantation accuracy derived from the post-operative CT scan, infection, intracranial haemorrhage and neurological deficit rates. Overall, 32 patients (328 electrodes) completed the trial (16 in each intervention arm) and the baseline demographics were broadly similar between the two groups. The time for individual electrode implantation was significantly less with the iSYS1 device (median of 3.36 (95% CI 5.72 to 7.07) than for the PAD group (median of 9.06 minutes (95% CI 8.16 to 10.06), p=0.0001). Target point accuracy was significantly greater with the PAD (median of 1.58 mm (95% CI 1.38 to 1.82) compared to the iSYS1 (median of 1.16 mm (95% CI 1.01 to 1.33), p=0.004). The difference between the target point accuracies are not clinically significant for SEEG but may have implications for procedures such as deep brain stimulation that require higher placement accuracy. All of the electrodes achieved their respective intended anatomical targets. In 12 of 16 patients following robotic implantations, and 10 of 16 following manual PAD implantations a seizure onset zone was identified and resection recommended. The aforementioned systematic review and meta-analysis were updated to include additional studies published during the trial duration. In this context, the iSYS1 device entry and target point accuracies were similar to those reported in other published studies of robotic devices including the ROSA, Neuromate and iSYS1. The PAD accuracies, however, outperformed the previously published results for other frameless stereotaxy methods. In conclusion, the presented studies report the integration and validation of a complex clinical decision support software into the clinical neurosurgical workflow for SEEG planning. The stereotactic planning platform was further refined by integrating machine learning techniques and also extended towards optimisation of LITT trajectories for ablation of mesial temporal structures and corpus callosotomy. The platform was then used to seamlessly integrate with a novel trajectory planning software to effectively and safely guide the implantation of the SEEG electrodes. Through a single-blinded randomised control trial, the ISYS1 device was shown to reduce the time taken for individual electrode insertion. Taken together, this work presents and validates the first fully integrated stereotactic trajectory planning platform that can be used for both SEEG and LITT trajectory planning followed by surgical implantation through the use of a novel trajectory guidance system

    A minimally invasive surgical system for 3D ultrasound guided robotic retrieval of foreign bodies from a beating heart

    Get PDF
    The result of various medical conditions and trauma, foreign bodies in the heart pose a serious health risk as they may interfere with cardiovascular function. Particles such as thrombi, bullet fragments, and shrapnel can become trapped in a person's heart after migrating through the venous system, or by direct penetration. The severity of disruption can range from benign to fatal, with associated symptoms including anxiety, fever, cardiac tamponade, hemorrhage, infection, embolism, arrhythmia, and valve dysfunction. Injuries of this nature are common in both civilian and military populations. For symptomatic cases, conventional treatment is removal of the foreign body through open surgery via a median sternotomy, the use of cardiopulmonary bypass, and a wide incision in the heart muscle; these methods incur pronounced perioperative risks and long recovery periods. In order to improve upon the standard of care, we propose an image guided robotic system and a corresponding minimally invasive surgical approach. The system employs a dexterous robotic capture device that can maneuver inside the heart through a small incision. Visualization and guidance within the otherwise occluded internal regions are provided by 3D transesophageal echocardiography (TEE), an emerging form of intraoperative medical imaging used in interventions such as mitral valve repair and device implantation. A robotic approach, as opposed to a manual procedure using rigid instruments, is motivated by the various challenges inherent in minimally invasive surgery, which arise from attempts to perform skilled surgical tasks through small incisions without direct vision. Challenges include reduced dexterity, constrained workspace, limited visualization, and difficult hand-eye coordination, which ultimately lead to poor manipulability. A dexterous robotic end effector with real-time image guidance can help overcome these challenges and potentially improve surgical performance. However promising, such a system and approach require that several technical hurdles be resolved. The foreign body must be automatically tracked as it travels about the dynamic environment of the heart. The erratically moving particle must then be captured using a dexterous robot that moves much more slowly in comparison. Furthermore, retrieval must be performed under 3D ultrasound guidance, amidst the uncertainties presented by both the turbulent flow and by the imaging modality itself. In addressing such barriers, this thesis explores the development of a prototype system capable of retrieving a foreign body from a beating heart, culminating in a set of demonstrative in vitro experiments

    ๋”ฅ๋Ÿฌ๋‹์— ๊ธฐ์ดˆํ•œ ํšจ๊ณผ์ ์ธ Visual Odometry ๊ฐœ์„  ๋ฐฉ๋ฒ•

    Get PDF
    ํ•™์œ„๋…ผ๋ฌธ (๋ฐ•์‚ฌ) -- ์„œ์šธ๋Œ€ํ•™๊ต ๋Œ€ํ•™์› : ๊ณต๊ณผ๋Œ€ํ•™ ์ „๊ธฐยท์ •๋ณด๊ณตํ•™๋ถ€, 2020. 8. ์ด๋ฒ”ํฌ.Understanding the three-dimensional environment is one of the most important issues in robotics and computer vision. For this purpose, sensors such as a lidar, a ultrasound, infrared devices, an inertial measurement unit (IMU) and cameras are used, individually or simultaneously, through sensor fusion. Among these sensors, in recent years, researches for use of visual sensors, which can obtain a lot of information at a low price, have been actively underway. Understanding of the 3D environment using cameras includes depth restoration, optical/scene flow estimation, and visual odometry (VO). Among them, VO estimates location of a camera and maps the surrounding environment, while a camera-equipped robot or person travels. This technology must be preceded by other tasks such as path planning and collision avoidance. Also, it can be applied to practical applications such as autonomous driving, augmented reality (AR), unmanned aerial vehicle (UAV) control, and 3D modeling. So far, researches on various VO algorithms have been proposed. Initial VO researches were conducted by filtering poses of robot and map features. Because of the disadvantage of the amount of computation being too large and errors are accumulated, a method using a keyframe was studied. Traditional VO can be divided into a feature-based method and a direct method. Methods using features obtain pose transformation between two images through feature extraction and matching. Direct methods directly compare the intensity of image pixels to obtain poses that minimize the sum of photometric errors. Recently, due to the development of deep learning skills, many studies have been conducted to apply deep learning to VO. Deep learning-based VO, like other fields using deep learning with images, first extracts convolutional neural network (CNN) features and calculates pose transformation between images. Deep learning-based VO can be divided into supervised learning-based and unsupervised learning-based. For VO, using supervised learning, a neural network is trained using ground truth poses, and the unsupervised learning-based method learns poses using only image sequences without given ground truth values. While existing research papers show decent performance, the image datasets used in these studies are all composed of high quality and clear images obtained using expensive cameras. There are also algorithms that can be operated only if non-image information such as exposure time, nonlinear response functions, and camera parameters is provided. In order for VO to be more widely applied to real-world application problems, odometry estimation should be performed even if the datasets are incomplete. Therefore, in this dissertation, two methods are proposed to improve VO performance using deep learning. First, I adopt a super-resolution (SR) technique to improve the performance of VO using images with low-resolution and noises. The existing SR techniques have mainly focused on increasing image resolution rather than execution time. However, a real-time property is very important for VO. Therefore, the SR network should be designed considering the execution time, resolution increment, and noise reduction in this case. Conducting a VO after passing through this SR network, a higher performance VO can be carried out, than using original images. Experimental results using the TUM dataset show that the proposed method outperforms the conventional VO and other SR methods. Second, I propose a fully unsupervised learning-based VO that performs odometry estimation, single-view depth estimation, and camera intrinsic parameter estimation simultaneously using a dataset consisting only of image sequences. In the existing unsupervised learning-based VO, algorithms were performed using the images and intrinsic parameters of the camera. Based on existing the technique, I propose a method for additionally estimating camera parameters from the deep intrinsic network. Intrinsic parameters are estimated by two assumptions using the properties of camera parameters in an intrinsic network. Experiments using the KITTI dataset show that the results are comparable to those of the conventional method.3์ฐจ์› ํ™˜๊ฒฝ์— ๋Œ€ํ•œ ์ดํ•ด๋Š” ๋กœ๋ณดํ‹ฑ์Šค์™€ ์ปดํ“จํ„ฐ ๋น„์ „ ๋ถ„์•ผ์—์„œ ๊ต‰์žฅํžˆ ์ค‘์š”ํ•œ ๋ฌธ์ œ ์ค‘ ํ•˜๋‚˜์ด๋‹ค. ์ด๋ฅผ ์œ„ํ•ด ๋ผ์ด๋‹ค, ์ดˆ์ŒํŒŒ, ์ ์™ธ์„ , inertial measurement unit (IMU), ์นด๋ฉ”๋ผ ๋“ฑ์˜ ์„ผ์„œ๊ฐ€ ๊ฐœ๋ณ„์ ์œผ๋กœ ๋˜๋Š” ์„ผ์„œ ์œตํ•ฉ์„ ํ†ตํ•ด ์—ฌ๋Ÿฌ ์„ผ์„œ๊ฐ€ ๋™์‹œ์— ์‚ฌ์šฉ๋˜๊ธฐ๋„ ํ•œ๋‹ค. ์ด ์ค‘์—์„œ๋„ ์ตœ๊ทผ์—๋Š” ์ƒ๋Œ€์ ์œผ๋กœ ์ €๋ ดํ•œ ๊ฐ€๊ฒฉ์— ๋งŽ์€ ์ •๋ณด๋ฅผ ์–ป์„ ์ˆ˜ ์žˆ๋Š” ์นด๋ฉ”๋ผ๋ฅผ ์ด์šฉํ•œ ์—ฐ๊ตฌ๊ฐ€ ํ™œ๋ฐœํžˆ ์ง„ํ–‰๋˜๊ณ  ์žˆ๋‹ค. ์นด๋ฉ”๋ผ๋ฅผ ์ด์šฉํ•œ 3์ฐจ์› ํ™˜๊ฒฝ ์ธ์ง€๋Š” ๊นŠ์ด ๋ณต์›, optical/scene flow ์ถ”์ •, visual odometry (VO) ๋“ฑ์ด ์žˆ๋‹ค. ์ด ์ค‘ VO๋Š” ์นด๋ฉ”๋ผ๋ฅผ ์žฅ์ฐฉํ•œ ๋กœ๋ด‡ ํ˜น์€ ์‚ฌ๋žŒ์ด ์ด๋™ํ•˜๋ฉฐ ์ž์‹ ์˜ ์œ„์น˜๋ฅผ ํŒŒ์•…ํ•˜๊ณ  ์ฃผ๋ณ€ ํ™˜๊ฒฝ์˜ ์ง€๋„๋ฅผ ์ž‘์„ฑํ•˜๋Š” ๊ธฐ์ˆ ์ด๋‹ค. ์ด ๊ธฐ์ˆ ์€ ๊ฒฝ๋กœ ์„ค์ •, ์ถฉ๋Œ ํšŒํ”ผ ๋“ฑ ๋‹ค๋ฅธ ์ž„๋ฌด๋ฅผ ์ˆ˜ํ–‰ํ•˜๊ธฐ ์ „์— ํ•„์ˆ˜์ ์œผ๋กœ ์„ ํ–‰๋˜์–ด์•ผ ํ•˜๋ฉฐ ์ž์œจ ์ฃผํ–‰, AR, UAV contron, 3D modelling ๋“ฑ ์‹ค์ œ ์‘์šฉ ๋ฌธ์ œ์— ์ ์šฉ๋  ์ˆ˜ ์žˆ๋‹ค. ํ˜„์žฌ ๋‹ค์–‘ํ•œ VO ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๋Œ€ํ•œ ๋…ผ๋ฌธ์ด ์ œ์•ˆ๋˜์—ˆ๋‹ค. ์ดˆ๊ธฐ VO ์—ฐ๊ตฌ๋Š” feature๋ฅผ ์ด์šฉํ•˜์—ฌ feature์™€ ๋กœ๋ด‡์˜ pose๋ฅผ ํ•„ํ„ฐ๋ง ํ•˜๋Š” ๋ฐฉ์‹์œผ๋กœ ์ง„ํ–‰๋˜์—ˆ๋‹ค. ํ•„ํ„ฐ๋ฅผ ์ด์šฉํ•œ ๋ฐฉ๋ฒ•์€ ๊ณ„์‚ฐ๋Ÿ‰์ด ๋„ˆ๋ฌด ๋งŽ๊ณ  ์˜ค์ฐจ๊ฐ€ ๋ˆ„์ ๋œ๋‹ค๋Š” ๋‹จ์  ๋•Œ๋ฌธ์— keyframe์„ ์ด์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•์ด ์—ฐ๊ตฌ๋˜์—ˆ๋‹ค. ์ด ๋ฐฉ์‹์œผ๋กœ feature๋ฅผ ์ด์šฉํ•˜๋Š” ๋ฐฉ์‹๊ณผ ํ”ฝ์…€์˜ intensity๋ฅผ ์ง์ ‘ ์‚ฌ์šฉํ•˜๋Š” direct ๋ฐฉ์‹์ด ์—ฐ๊ตฌ๋˜์—ˆ๋‹ค. feature๋ฅผ ์ด์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•๋“ค์€ feature์˜ ์ถ”์ถœ๊ณผ ๋งค์นญ์„ ์ด์šฉํ•˜์—ฌ ๋‘ ์ด๋ฏธ์ง€ ์‚ฌ์ด์˜ pose ๋ณ€ํ™”๋ฅผ ๊ตฌํ•˜๋ฉฐ direct ๋ฐฉ๋ฒ•๋“ค์€ ์ด๋ฏธ์ง€ ํ”ฝ์…€์˜ intensity๋ฅผ ์ง์ ‘ ๋น„๊ตํ•˜์—ฌ photometric error๋ฅผ ์ตœ์†Œํ™” ์‹œํ‚ค๋Š” pose๋ฅผ ๊ตฌํ•˜๋Š” ๋ฐฉ์‹์ด๋‹ค. ์ตœ๊ทผ์—๋Š” deep learning ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ๋ฐœ๋‹ฌ๋กœ ์ธํ•ด VO์—๋„ deep learning์„ ์ ์šฉ์‹œํ‚ค๋Š” ์—ฐ๊ตฌ๊ฐ€ ๋งŽ์ด ์ง„ํ–‰๋˜๊ณ  ์žˆ๋‹ค. Deep learning-based VO๋Š” ์ด๋ฏธ์ง€๋ฅผ ์ด์šฉํ•œ ๋‹ค๋ฅธ ๋ถ„์•ผ์™€ ๊ฐ™์ด ๊ธฐ๋ณธ์ ์œผ๋กœ CNN์„ ์ด์šฉํ•˜์—ฌ feature๋ฅผ ์ถ”์ถœํ•œ ๋’ค ์ด๋ฏธ์ง€ ์‚ฌ์ด์˜ pose ๋ณ€ํ™”๋ฅผ ๊ณ„์‚ฐํ•œ๋‹ค. ์ด๋Š” ๋‹ค์‹œ supervised learning์„ ์ด์šฉํ•œ ๋ฐฉ์‹๊ณผ unsupervised learning์„ ์ด์šฉํ•œ ๋ฐฉ๋ฒ•์œผ๋กœ ๋‚˜๋ˆŒ ์ˆ˜ ์žˆ๋‹ค. supervised learning์„ ์ด์šฉํ•œ VO๋Š” pose์˜ ์ฐธ๊ฐ’์„ ์‚ฌ์šฉํ•˜์—ฌ ํ•™์Šต์„ ์‹œํ‚ค๋ฉฐ, unsupervised learning์„ ์ด์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ์ฃผ์–ด์ง€๋Š” ์ฐธ๊ฐ’ ์—†์ด ์ด๋ฏธ์ง€์˜ ์ •๋ณด๋งŒ์„ ์ด์šฉํ•˜์—ฌ pose๋ฅผ ํ•™์Šต์‹œํ‚ค๋Š” ๋ฐฉ์‹์ด๋‹ค. ๊ธฐ์กด VO ๋…ผ๋ฌธ๋“ค์€ ์ข‹์€ ์„ฑ๋Šฅ์„ ๋ณด์˜€์ง€๋งŒ ์—ฐ๊ตฌ์— ์‚ฌ์šฉ๋œ ์ด๋ฏธ์ง€ dataset๋“ค์€ ๋ชจ๋‘ ๊ณ ๊ฐ€์˜ ์นด๋ฉ”๋ผ๋ฅผ ์ด์šฉํ•˜์—ฌ ์–ป์–ด์ง„ ๊ณ ํ™”์งˆ์˜ ์„ ๋ช…ํ•œ ์ด๋ฏธ์ง€๋“ค๋กœ ๊ตฌ์„ฑ๋˜์–ด ์žˆ๋‹ค. ๋˜ํ•œ ๋…ธ์ถœ ์‹œ๊ฐ„, ๋น„์„ ํ˜• ๋ฐ˜์‘ ํ•จ์ˆ˜, ์นด๋ฉ”๋ผ ํŒŒ๋ผ๋ฏธํ„ฐ ๋“ฑ์˜ ์ด๋ฏธ์ง€ ์™ธ์ ์ธ ์ •๋ณด๋ฅผ ์ด์šฉํ•ด์•ผ๋งŒ ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ๋™์ž‘์ด ๊ฐ€๋Šฅํ•˜๋‹ค. VO๊ฐ€ ์‹ค์ œ ์‘์šฉ ๋ฌธ์ œ์— ๋” ๋„๋ฆฌ ์ ์šฉ๋˜๊ธฐ ์œ„ํ•ด์„œ๋Š” dataset์ด ๋ถˆ์™„์ „ํ•  ๊ฒฝ์šฐ์—๋„ odometry ์ถ”์ •์ด ์ž˜ ์ด๋ฃจ์–ด์ ธ์•ผ ํ•œ๋‹ค. ์ด์— ๋ณธ ๋…ผ๋ฌธ์—์„œ๋Š” deep learning์„ ์ด์šฉํ•˜์—ฌ VO์˜ ์„ฑ๋Šฅ์„ ๋†’์ด๋Š” ๋‘ ๊ฐ€์ง€ ๋ฐฉ๋ฒ•์„ ์ œ์•ˆํ•˜์˜€๋‹ค. ์ฒซ ๋ฒˆ์งธ๋กœ๋Š” super-resolution (SR) ๊ธฐ๋ฒ•์œผ๋กœ ์ €ํ•ด์ƒ๋„, ๋…ธ์ด์ฆˆ๊ฐ€ ํฌํ•จ๋œ ์ด๋ฏธ์ง€๋ฅผ ์ด์šฉํ•œ VO์˜ ์„ฑ๋Šฅ์„ ๋†’์ด๋Š” ๋ฐฉ๋ฒ•์„ ์ œ์•ˆํ•œ๋‹ค. ๊ธฐ์กด์˜ SR ๊ธฐ๋ฒ•์€ ์ˆ˜ํ–‰ ์‹œ๊ฐ„๋ณด๋‹ค๋Š” ์ด๋ฏธ์ง€์˜ ํ•ด์ƒ๋„๋ฅผ ํ–ฅ์ƒ์‹œํ‚ค๋Š” ๋ฐฉ๋ฒ•์— ์ฃผ๋กœ ์ง‘์ค‘ํ•˜์˜€๋‹ค. ํ•˜์ง€๋งŒ VO ์ˆ˜ํ–‰์— ์žˆ์–ด์„œ๋Š” ์‹ค์‹œ๊ฐ„์„ฑ์ด ๊ต‰์žฅํžˆ ์ค‘์š”ํ•˜๋‹ค. ๋”ฐ๋ผ์„œ ์ˆ˜ํ–‰ ์‹œ๊ฐ„์„ ๊ณ ๋ คํ•œ SR ๋„คํŠธ์›Œํฌ์˜ ์„ค๊ณ„ํ•˜์—ฌ ์ด๋ฏธ์ง€์˜ ํ•ด์ƒ๋„๋ฅผ ๋†’์ด๊ณ  ๋…ธ์ด์ฆˆ๋ฅผ ์ค„์˜€๋‹ค. ์ด SR ๋„คํŠธ์›Œํฌ๋ฅผ ํ†ต๊ณผ์‹œํ‚จ ๋’ค VO๋ฅผ ์ˆ˜ํ–‰ํ•˜๋ฉด ๊ธฐ์กด์˜ ์ด๋ฏธ์ง€๋ฅผ ์‚ฌ์šฉํ•  ๋•Œ๋ณด๋‹ค ๋†’์€ ์„ฑ๋Šฅ์˜ VO๋ฅผ ์‹ค์‹œ๊ฐ„์œผ๋กœ ์ˆ˜ํ–‰ํ•  ์ˆ˜ ์žˆ๋‹ค. TUM dataset์„ ์ด์šฉํ•œ ์‹คํ—˜ ๊ฒฐ๊ณผ ๊ธฐ์กด์˜ VO ๊ธฐ๋ฒ•๊ณผ ๋‹ค๋ฅธ SR ๊ธฐ๋ฒ•์„ ์ ์šฉํ•˜์˜€์„ ๋•Œ ๋ณด๋‹ค ์ œ์•ˆํ•˜๋Š” ๋ฐฉ๋ฒ•์˜ ์„ฑ๋Šฅ์ด ๋” ๋†’์€ ๊ฒƒ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์—ˆ๋‹ค. ๋‘ ๋ฒˆ์งธ๋กœ๋Š” ์—ฐ์†๋œ ์ด๋ฏธ์ง€๋งŒ์œผ๋กœ ๊ตฌ์„ฑ๋œ dataset์„ ์ด์šฉํ•˜์—ฌ VO, ๋‹จ์ผ ์ด๋ฏธ์ง€ ๊นŠ์ด ์ถ”์ •, ์นด๋ฉ”๋ผ ๋‚ด๋ถ€ ํŒŒ๋ผ๋ฏธํ„ฐ ์ถ”์ •์„ ์ˆ˜ํ–‰ํ•˜๋Š” fully unsupervised learning-based VO๋ฅผ ์ œ์•ˆํ•œ๋‹ค. ๊ธฐ์กด unsupervised learning์„ ์ด์šฉํ•œ VO์—์„œ๋Š” ์ด๋ฏธ์ง€๋“ค๊ณผ ์ด๋ฏธ์ง€๋ฅผ ์ดฌ์˜ํ•œ ์นด๋ฉ”๋ผ์˜ ๋‚ด๋ถ€ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์ด์šฉํ•˜์—ฌ VO๋ฅผ ์ˆ˜ํ–‰ํ•˜์˜€๋‹ค. ์ด ๊ธฐ์ˆ ์„ ๊ธฐ๋ฐ˜์œผ๋กœ ๋ณธ ๋…ผ๋ฌธ์—์„œ๋Š” deep intrinsic ๋„คํŠธ์›Œํฌ๋ฅผ ์ถ”๊ฐ€ํ•˜์—ฌ ์นด๋ฉ”๋ผ ํŒŒ๋ผ๋ฏธํ„ฐ๊นŒ์ง€ ๋„คํŠธ์›Œํฌ์—์„œ ์ถ”์ •ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์ œ์•ˆํ•œ๋‹ค. 0์œผ๋กœ ์ˆ˜๋ ดํ•˜๊ฑฐ๋‚˜ ์‰ฝ๊ฒŒ ๋ฐœ์‚ฐํ•˜๋Š” intrinsic ๋„คํŠธ์›Œํฌ์— ์นด๋ฉ”๋ผ ํŒŒ๋ผ๋ฏธํ„ฐ์˜ ์„ฑ์งˆ์„ ์ด์šฉํ•œ ๋‘ ๊ฐ€์ง€ ๊ฐ€์ •์„ ํ†ตํ•ด ๋‚ด๋ถ€ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ์ถ”์ •ํ•  ์ˆ˜ ์žˆ์—ˆ๋‹ค. KITTI dataset์„ ์ด์šฉํ•œ ์‹คํ—˜์„ ํ†ตํ•ด intrinsic parameter ์ •๋ณด๋ฅผ ์ œ๊ณต๋ฐ›์•„ ์ง„ํ–‰๋œ ๊ธฐ์กด์˜ ๋ฐฉ๋ฒ•๊ณผ ์œ ์‚ฌํ•œ ์„ฑ๋Šฅ์„ ํ™•์ธํ•  ์ˆ˜ ์žˆ์—ˆ๋‹ค.1 INTRODUCTION 1 1.1 Background and Motivation 1 1.2 Literature Review 3 1.3 Contributions 10 1.4 Thesis Structure 11 2 Mathematical Preliminaries of Visual Odometry 13 2.1 Feature-based VO 13 2.2 Direct VO 17 2.3 Learning-based VO 21 2.3.1 Supervised learning-based VO 22 2.3.2 Unsupervised learning-based VO 25 3 Error Improvement in Visual Odometry Using Super-resolution 29 3.1 Introduction 29 3.2 Related Work 31 3.2.1 Visual Odometry 31 3.2.2 Super-resolution 33 3.3 SR-VO 34 3.3.1 VO performance analysis according to changing resolution 34 3.3.2 Super-Resolution Network 37 3.4 Experiments 40 3.4.1 Super-Resolution Procedure 40 3.4.2 VO with SR images 42 3.5 Summary 54 4 Visual Odometry Enhancement Method Using Fully Unsupervised Learning 55 4.1 Introduction 55 4.2 Related Work 57 4.2.1 Traditional Visual Odometry 57 4.2.2 Single-view Depth Recovery 58 4.2.3 Supervised Learning-based Visual Odometry 59 4.2.4 Unsupervised Learning-based Visual Odometry 60 4.2.5 Architecture Overview 62 4.3 Methods 62 4.3.1 Predicting the Target Image using Source Images 62 4.3.2 Intrinsic Parameters Regressor 63 4.4 Experiments 66 4.4.1 Monocular Depth Estimation 66 4.4.2 Visual Odometry 67 4.4.3 Intrinsic Parameters Estimation 77 5 Conclusion and Future Work 82 5.1 Conclusion 82 5.2 Future Work 85 Bibliography 86 Abstract (In Korean) 101Docto

    Characterization of alar ligament on 3.0T MRI: a cross-sectional study in IIUM Medical Centre, Kuantan

    Get PDF
    INTRODUCTION: The main purpose of the study is to compare the normal anatomy of alar ligament on MRI between male and female. The specific objectives are to assess the prevalence of alar ligament visualized on MRI, to describe its characteristics in term of its course, shape and signal homogeneity and to find differences in alar ligament signal intensity between male and female. This study also aims to determine the association between the heights of respondents with alar ligament signal intensity and dimensions. MATERIALS & METHODS: 50 healthy volunteers were studied on 3.0T MR scanner Siemens Magnetom Spectra using 2-mm proton density, T2 and fat-suppression sequences. Alar ligament is depicted in 3 planes and the visualization and variability of the ligament courses, shapes and signal intensity characteristics were determined. The alar ligament dimensions were also measured. RESULTS: Alar ligament was best depicted in coronal plane, followed by sagittal and axial planes. The orientations were laterally ascending in most of the subjects (60%), predominantly oval in shaped (54%) and 67% showed inhomogenous signal. No significant difference of alar ligament signal intensity between male and female respondents. No significant association was found between the heights of the respondents with alar ligament signal intensity and dimensions. CONCLUSION: Employing a 3.0T MR scanner, the alar ligament is best portrayed on coronal plane, followed by sagittal and axial planes. However, tremendous variability of alar ligament as depicted in our data shows that caution needs to be exercised when evaluating alar ligament, especially during circumstances of injury

    Case series of breast fillers and how things may go wrong: radiology point of view

    Get PDF
    INTRODUCTION: Breast augmentation is a procedure opted by women to overcome sagging breast due to breastfeeding or aging as well as small breast size. Recent years have shown the emergence of a variety of injectable materials on market as breast fillers. These injectable breast fillers have swiftly gained popularity among women, considering the minimal invasiveness of the procedure, nullifying the need for terrifying surgery. Little do they know that the procedure may pose detrimental complications, while visualization of breast parenchyma infiltrated by these fillers is also deemed substandard; posing diagnostic challenges. We present a case series of three patients with prior history of hyaluronic acid and collagen breast injections. REPORT: The first patient is a 37-year-old lady who presented to casualty with worsening shortness of breath, non-productive cough, central chest pain; associated with fever and chills for 2-weeks duration. The second patient is a 34-year-old lady who complained of cough, fever and haemoptysis; associated with shortness of breath for 1-week duration. CT in these cases revealed non thrombotic wedge-shaped peripheral air-space densities. The third patient is a 37โ€yearโ€old female with right breast pain, swelling and redness for 2- weeks duration. Previous collagen breast injection performed 1 year ago had impeded sonographic visualization of the breast parenchyma. MRI breasts showed multiple non- enhancing round and oval shaped lesions exhibiting fat intensity. CONCLUSION: Radiologists should be familiar with the potential risks and hazards as well as limitations of imaging posed by breast fillers such that MRI is required as problem-solving tool

    Imaging Sensors and Applications

    Get PDF
    In past decades, various sensor technologies have been used in all areas of our lives, thus improving our quality of life. In particular, imaging sensors have been widely applied in the development of various imaging approaches such as optical imaging, ultrasound imaging, X-ray imaging, and nuclear imaging, and contributed to achieve high sensitivity, miniaturization, and real-time imaging. These advanced image sensing technologies play an important role not only in the medical field but also in the industrial field. This Special Issue covers broad topics on imaging sensors and applications. The scope range of imaging sensors can be extended to novel imaging sensors and diverse imaging systems, including hardware and software advancements. Additionally, biomedical and nondestructive sensing applications are welcome

    Determination and quantitative evaluation of image-based registration accuracy for robotic neurosurgery

    Get PDF
    Stereotactic neurosurgical robots allow quick, accurate location of small targets within the brain, relying on accurate registration of preoperative MRI/CT images with patient and robot coordinate systems. Fiducial markers or a stereotactic frame are used as registration landmarks and the patientโ€™s head is fixed in position. An image-based system could be quick, non-invasive and allow the head to be moved during surgery giving greater ease of access. Submillimetre surgical precision at the target point is required. An octant representation is utilized to investigate full region of interest (ROI) head registration using parts only, with registration performed using the Iterative Closest Point (ICP) algorithm. Use of two octants sequentially obtained a mean RMS distance of 0.813ยฑ0.026 mm; adding subsequent octants did not significantly improve performance. An RMS distance of 0.812ยฑ0.025 mm was obtained for three octants used simultaneously. ICP was compared with Coherent Point Drift, and 3D Normal Distribution Transform, with and without added or smoothed noise, and was least affected by starting position or noise added; a mean accuracy of 0.884ยฑ0.050 mm across ten noise levels and four starting positions was achieved, which was shown to translate to submillimetre accuracy at points within the head
    corecore