628 research outputs found

    Teleoperation Methods for High-Risk, High-Latency Environments

    Get PDF
    In-Space Servicing, Assembly, and Manufacturing (ISAM) can enable larger-scale and longer-lived infrastructure projects in space, with interest ranging from commercial entities to the US government. Servicing, in particular, has the potential to vastly increase the usable lifetimes of satellites. However, the vast majority of spacecraft on low Earth orbit today were not designed to be serviced on-orbit. As such, several of the manipulations during servicing cannot easily be automated and instead require ground-based teleoperation. Ground-based teleoperation of on-orbit robots brings its own challenges of high latency communications, with telemetry delays of several seconds, and difficulties in visualizing the remote environment due to limited camera views. We explore teleoperation methods to alleviate these difficulties, increase task success, and reduce operator load. First, we investigate a model-based teleoperation interface intended to provide the benefits of direct teleoperation even in the presence of time delay. We evaluate the model-based teleoperation method using professional robot operators, then use feedback from that study to inform the design of a visual planning tool for this task, Interactive Planning and Supervised Execution (IPSE). We describe and evaluate the IPSE system and two interfaces, one 2D using a traditional mouse and keyboard and one 3D using an Intuitive Surgical da Vinci master console. We then describe and evaluate an alternative 3D interface using a Meta Quest head-mounted display. Finally, we describe an extension of IPSE to allow human-in-the-loop planning for a redundant robot. Overall, we find that IPSE improves task success rate and decreases operator workload compared to a conventional teleoperation interface

    Innovative Tool for Automatic Detection of Arterial Stenosis on Cone Beam Computed Tomography

    Get PDF
    Arterial stenosis is one of the main vascular diseases that are treated with minimally invasive surgery approaches. The aim of this study was to provide a tool to support the medical doctor in planning endovascular surgery, allowing the rapid detection of stenotic vessels and the quantification of the stenosis. Skeletonization was used to improve vessels’ visualization. The distance transform was used to obtain a linear representation of the diameter of critical vessels selected by the user. The system also provides an estimate of the exact distance between landmarks on the vascular tree and the occlusion, important information that can be used in the planning of the surgery. The advantage of the proposed tool is to lead the examination on the linear representation of the chosen vessels that are free from tortuous vascular courses and from vessel crossings

    Current and Future Advances in Surgical Therapy for Pituitary Adenoma

    Get PDF
    The vital physiological role of the pituitary gland, alongside its proximal critical neurovascular structures means pituitary adenomas cause significant morbidity or mortality. Whilst enormous advancements have been made in the surgical care of pituitary adenomas, treatment failure and recurrence remain challenges. To meet these clinical challenges, there has been an enormous expansion of novel medical technologies (e.g. endoscopy, advanced imaging, artificial intelligence). These innovations have the potential to benefit each step of the patient journey, and ultimately, drive improved outcomes. Earlier and more accurate diagnosis addresses this in part. Analysis of novel patient data sets, such as automated facial analysis or natural language processing of medical records holds potential in achieving an earlier diagnosis. After diagnosis, treatment decision-making and planning will benefit from radiomics and multimodal machine learning models. Surgical safety and effectiveness will be transformed by smart simulation methods for trainees. Next-generation imaging techniques and augmented reality will enhance surgical planning and intraoperative navigation. Similarly, the future armamentarium of pituitary surgeons, including advanced optical devices, smart instruments and surgical robotics, will augment the surgeon's abilities. Intraoperative support to team members will benefit from a surgical data science approach, utilising machine learning analysis of operative videos to improve patient safety and orientate team members to a common workflow. Postoperatively, early detection of individuals at risk of complications and prediction of treatment failure through neural networks of multimodal datasets will support earlier intervention, safer hospital discharge, guide follow-up and adjuvant treatment decisions. Whilst advancements in pituitary surgery hold promise to enhance the quality of care, clinicians must be the gatekeepers of technological translation, ensuring systematic assessment of risk and benefit. In doing so, the synergy between these innovations can be leveraged to drive improved outcomes for patients of the future

    Application and challenges of a metaverse in medicine

    Get PDF
    Metaverse has been confirmed as a relatively amorphous concept of innovation, which refers to technological advancement. Metaverse, i.e., a coalition between reality world and virtual world, has created significant significance and convenience in education, communication, economy, etc. The COVID-19 outbreak has stimulated the growth of metaverse applications in medicine. The above-mentioned technology has broad applications while comprising online remote medical treatment, online conferences, medical education, preparation of surgical plans, etc. Moreover, technical, security, and financial challenges should be tackled down by the future widespread use of metaverse. Metaverse is limitlessly promising, and it will exert a certain effect on future scientific and technological advancements in the medical industry. The review article primarily aims to summarize the application of the metaverse in medicine and their challenge in the future of medicine

    Towards Robot Autonomy in Medical Procedures Via Visual Localization and Motion Planning

    Get PDF
    Robots performing medical procedures with autonomous capabilities have the potential to positively effect patient care and healthcare system efficiency. These benefits can be realized by autonomous robots facilitating novel procedures, increasing operative efficiency, standardizing intra- and inter-physician performance, democratizing specialized care, and focusing the physician’s time on subtasks that best leverage their expertise. However, enabling medical robots to act autonomously in a procedural environment is extremely challenging. The deforming and unstructured nature of the environment, the lack of features in the anatomy, and sensor size constraints coupled with the millimeter level accuracy required for safe medical procedures introduce a host of challenges not faced by robots operating in structured environments such as factories or warehouses. Robot motion planning and localization are two fundamental abilities for enabling robot autonomy. Motion planning methods compute a sequence of safe and feasible motions for a robot to accomplish a specified task, where safe and feasible are defined by constraints with respect to the robot and its environment. Localization methods estimate the position and orientation of a robot in its environment. Developing such methods for medical robots that overcome the unique challenges in procedural environments is critical for enabling medical robot autonomy. In this dissertation, I developed and evaluated motion planning and localization algorithms towards robot autonomy in medical procedures. A majority of my work was done in the context of an autonomous medical robot built for enhanced lung nodule biopsy. First, I developed a dataset of medical environments spanning various organs and procedures to foster future research into medical robots and automation. I used this data in my own work described throughout this dissertation. Next, I used motion planning to characterize the capabilities of the lung nodule biopsy robot compared to existing clinical tools and I highlighted trade-offs in robot design considerations. Then, I conducted a study to experimentally demonstrate the benefits of the autonomous lung robot in accessing otherwise hard-to-reach lung nodules. I showed that the robot enables access to lung regions beyond the reach of existing clinical tools with millimeter-level accuracy sufficient for accessing the smallest clinically operable nodules. Next, I developed a localization method to estimate the bronchoscope’s position and orientation in the airways with respect to a preoperatively planned needle insertion pose. The method can be used by robotic bronchoscopy systems and by traditional manually navigated bronchoscopes. The method is designed to overcome challenges with tissue motion and visual homogeneity in the airways. I demonstrated the success of this method in simulated lungs undergoing respiratory motion and showed the method’s ability to generalize across patients.Doctor of Philosoph

    Image-guidance in endoscopic pituitary surgery: an in-silico study of errors involved in tracker-based techniques

    Get PDF
    Background: Endoscopic endonasal surgery is an established minimally invasive technique for resecting pituitary adenomas. However, understanding orientation and identifying critical neurovascular structures in this anatomically dense region can be challenging. In clinical practice, commercial navigation systems use a tracked pointer for guidance. Augmented Reality (AR) is an emerging technology used for surgical guidance. It can be tracker based or vision based, but neither is widely used in pituitary surgery. Methods: This pre-clinical study aims to assess the accuracy of tracker-based navigation systems, including those that allow for AR. Two setups were used to conduct simulations: (1) the standard pointer setup, tracked by an infrared camera; and (2) the endoscope setup that allows for AR, using reflective markers on the end of the endoscope, tracked by infrared cameras. The error sources were estimated by calculating the Euclidean distance between a point’s true location and the point’s location after passing it through the noisy system. A phantom study was then conducted to verify the in-silico simulation results and show a working example of image-based navigation errors in current methodologies. Results: The errors of the tracked pointer and tracked endoscope simulations were 1.7 and 2.5 mm respectively. The phantom study showed errors of 2.14 and 3.21 mm for the tracked pointer and tracked endoscope setups respectively. Discussion: In pituitary surgery, precise neighboring structure identification is crucial for success. However, our simulations reveal that the errors of tracked approaches were too large to meet the fine error margins required for pituitary surgery. In order to achieve the required accuracy, we would need much more accurate tracking, better calibration and improved registration techniques

    Bronchoscopy using a head-mounted mixed reality device—a phantom study and a first in-patient user experience

    Get PDF
    Background: Bronchoscopy for peripheral lung lesions may involve image sources such as computed tomography (CT), fluoroscopy, radial endobronchial ultrasound (R-EBUS), and virtual/electromagnetic navigation bronchoscopy. Our objective was to evaluate the feasibility of replacing these multiple monitors with a head-mounted display (HMD), always providing relevant image data in the line of sight of the bronchoscopist.Methods: A total of 17 pulmonologists wearing a HMD (Microsoft® HoloLens 2) performed bronchoscopy with electromagnetic navigation in a lung phantom. The bronchoscopists first conducted an endobronchial inspection and navigation to the target, followed by an endobronchial ultrasound bronchoscopy. The HMD experience was evaluated using a questionnaire. Finally, the HMD was used in bronchoscopy inspection and electromagnetic navigation of two patients presenting with hemoptysis.Results: In the phantom study, the perceived quality of video and ultrasound images was assessed using a visual analog scale, with 100% representing optimal image quality. The score for video quality was 58% (95% confidence interval [CI] 48%–68%) and for ultrasound image quality, the score was 43% (95% CI 30%–56%). Contrast, color rendering, and resolution were all considered suboptimal. Despite adjusting the brightness settings, video image rendering was considered too dark. Navigation to the target for biopsy sampling was accomplished by all participants, with no significant difference in procedure time between experienced and less experienced bronchoscopists. The overall system latency for the image stream was 0.33–0.35 s. Fifteen of the pulmonologists would consider using HoloLens for navigation in the periphery, and two would not consider using HoloLens in bronchoscopy at all. In the human study, bronchoscopy inspection was feasible for both patients.Conclusion: Bronchoscopy using an HMD was feasible in a lung phantom and in two patients. Video and ultrasound image quality was considered inferior to that of video monitors. HoloLens 2 was suboptimal for airway and mucosa inspection but may be adequate for virtual bronchoscopy navigation

    SURGICAL NAVIGATION AND AUGMENTED REALITY FOR MARGINS CONTROL IN HEAD AND NECK CANCER

    Get PDF
    I tumori maligni del distretto testa-collo rappresentano un insieme di lesioni dalle diverse caratteristiche patologiche, epidemiologiche e prognostiche. Per una porzione considerevole di tali patologie, l’intervento chirurgico finalizzato all’asportazione completa del tumore rappresenta l’elemento chiave del trattamento, quand’anche esso includa altre modalità quali la radioterapia e la terapia sistemica. La qualità dell’atto chirurgico ablativo è pertanto essenziale al fine di garantire le massime chance di cura al paziente. Nell’ambito della chirurgia oncologica, la qualità delle ablazioni viene misurata attraverso l’analisi dello stato dei margini di resezione. Oltre a rappresentare un surrogato della qualità della resezione chirurgica, lo stato dei margini di resezione ha notevoli implicazioni da un punto di vista clinico e prognostico. Infatti, il coinvolgimento dei margini di resezione da parte della neoplasia rappresenta invariabilmente un fattore prognostico sfavorevole, oltre che implicare la necessità di intensificare i trattamenti postchirurgici (e.g., ponendo indicazione alla chemioradioterapia adiuvante), comportando una maggiore tossicità per il paziente. La proporzione di resezioni con margini positivi (i.e., coinvolti dalla neoplasia) nel distretto testa-collo è tra le più elevate in ambito di chirurgia oncologica. In tale contesto si pone l’obiettivo del dottorato di cui questa tesi riporta i risultati. Le due tecnologie di cui si è analizzata l’utilità in termini di ottimizzazione dello stato dei margini di resezione sono la navigazione chirurgica con rendering tridimensionale e la realtà aumentata basata sulla videoproiezione di immagini. Le sperimentazioni sono state svolte parzialmente presso l’Università degli Studi di Brescia, parzialmente presso l’Azienda Ospedale Università di Padova e parzialmente presso l’University Health Network (Toronto, Ontario, Canada). I risultati delle sperimentazioni incluse in questo elaborato dimostrano che l'impiego della navigazione chirurgica con rendering tridimensionale nel contesto di procedure oncologiche ablative cervico-cefaliche risulta associata ad un vantaggio significativo in termini di riduzione della frequenza di margini positivi. Al contrario, le tecniche di realtà aumentata basata sulla videoproiezione, nell'ambito della sperimentazione preclinica effettuata, non sono risultate associate a vantaggi sufficienti per poter considerare tale tecnologia per la traslazione clinica.Head and neck malignancies are an heterogeneous group of tumors. Surgery represents the mainstay of treatment for the large majority of head and neck cancers, with ablation being aimed at removing completely the tumor. Radiotherapy and systemic therapy have also a substantial role in the multidisciplinary management of head and neck cancers. The quality of surgical ablation is intimately related to margin status evaluated at a microscopic level. Indeed, margin involvement has a remarkably negative effect on prognosis of patients and mandates the escalation of postoperative treatment by adding concomitant chemotherapy to radiotherapy and accordingly increasing the toxicity of overall treatment. The rate of margin involvement in the head and neck is among the highest in the entire field of surgical oncology. In this context, the present PhD project was aimed at testing the utility of 2 technologies, namely surgical navigation with 3-dimensional rendering and pico projector-based augmented reality, in decreasing the rate of involved margins during oncologic surgical ablations in the craniofacial area. Experiments were performed in the University of Brescia, University of Padua, and University Health Network (Toronto, Ontario, Canada). The research activities completed in the context of this PhD course demonstrated that surgical navigation with 3-dimensional rendering confers a higher quality to oncologic ablations in the head and neck, irrespective of the open or endoscopic surgical technique. The benefits deriving from this implementation come with no relevant drawbacks from a logistical and practical standpoint, nor were major adverse events observed. Thus, implementation of this technology into the standard care is the logical proposed step forward. However, the genuine presence of a prognostic advantage needs longer and larger study to be formally addressed. On the other hand, pico projector-based augmented reality showed no sufficient advantages to encourage translation into the clinical setting. Although observing a clear practical advantage deriving from the projection of osteotomy lines onto the surgical field, no substantial benefits were measured when comparing this technology with surgical navigation with 3-dimensional rendering. Yet recognizing a potential value of this technology from an educational standpoint, the performance displayed in the preclinical setting in terms of surgical margins optimization is not in favor of a clinical translation with this specific aim
    • …
    corecore