Augmented Reality and Artificial Intelligence in Image-Guided and Robot-Assisted Interventions

In minimally invasive orthopedic procedures, the surgeon places wires, screws, and surgical implants through the muscles and bony structures under image guidance. These interventions require alignment of the pre- and intra-operative patient data, the intra-operative scanner, surgical instruments, and the patient. Suboptimal interaction with patient data and challenges in mastering 3D anatomy based on ill-posed 2D interventional images are essential concerns in image-guided therapies. State of the art approaches often support the surgeon by using external navigation systems or ill-conditioned image-based registration methods that both have certain drawbacks. Augmented reality (AR) has been introduced in the operating rooms in the last decade; however, in image-guided interventions, it has often only been considered as a visualization device improving traditional workflows. Consequently, the technology is gaining minimum maturity that it requires to redefine new procedures, user interfaces, and interactions. This dissertation investigates the applications of AR, artificial intelligence, and robotics in interventional medicine. Our solutions were applied in a broad spectrum of problems for various tasks, namely improving imaging and acquisition, image computing and analytics for registration and image understanding, and enhancing the interventional visualization. The benefits of these approaches were also discovered in robot-assisted interventions. We revealed how exemplary workflows are redefined via AR by taking full advantage of head-mounted displays when entirely co-registered with the imaging systems and the environment at all times. The proposed AR landscape is enabled by co-localizing the users and the imaging devices via the operating room environment and exploiting all involved frustums to move spatial information between different bodies. The system's awareness of the geometric and physical characteristics of X-ray imaging allows the exploration of different human-machine interfaces. We also leveraged the principles governing image formation and combined it with deep learning and RGBD sensing to fuse images and reconstruct interventional data. We hope that our holistic approaches towards improving the interface of surgery and enhancing the usability of interventional imaging, not only augments the surgeon's capabilities but also augments the surgical team's experience in carrying out an effective intervention with reduced complications

The CAMP Lab Computer Aided Medical Procedures and Augmented Reality

Abstract-The CAMP lab is integrated within the Department of Informatics at Technical University of Munich and is considered one of the leading groups concerned with medical augmented reality, computer assisted interventions, as well as non-medical related computer vision. In this short paper, we give an outline of the history of the lab and present a summary of some of our past and current activities relevant to augmented and virtual reality in computer assisted interventions and surgeries. References to published work in major journals and conferences allow the reader to get access to more detailed information on each subject. It was not possible to cover all aspects of our research within this paper, but we hope to provide an overview on some of these within this short paper. The readers are also invited to visit our web-site at http://campar.in.tum.de to get more information on aspects of our work. Applications for PhD and PostDoc positions can be made through the form a

Development, Implementation and Pre-clinical Evaluation of Medical Image Computing Tools in Support of Computer-aided Diagnosis: Respiratory, Orthopedic and Cardiac Applications

Over the last decade, image processing tools have become crucial components of all clinical and research efforts involving medical imaging and associated applications. The imaging data available to the radiologists continue to increase their workload, raising the need for efficient identification and visualization of the required image data necessary for clinical assessment. Computer-aided diagnosis (CAD) in medical imaging has evolved in response to the need for techniques that can assist the radiologists to increase throughput while reducing human error and bias without compromising the outcome of the screening, diagnosis or disease assessment. More intelligent, but simple, consistent and less time-consuming methods will become more widespread, reducing user variability, while also revealing information in a more clear, visual way. Several routine image processing approaches, including localization, segmentation, registration, and fusion, are critical for enhancing and enabling the development of CAD techniques. However, changes in clinical workflow require significant adjustments and re-training and, despite the efforts of the academic research community to develop state-of-the-art algorithms and high-performance techniques, their footprint often hampers their clinical use. Currently, the main challenge seems to not be the lack of tools and techniques for medical image processing, analysis, and computing, but rather the lack of clinically feasible solutions that leverage the already developed and existing tools and techniques, as well as a demonstration of the potential clinical impact of such tools. Recently, more and more efforts have been dedicated to devising new algorithms for localization, segmentation or registration, while their potential and much intended clinical use and their actual utility is dwarfed by the scientific, algorithmic and developmental novelty that only result in incremental improvements over already algorithms. In this thesis, we propose and demonstrate the implementation and evaluation of several different methodological guidelines that ensure the development of image processing tools --- localization, segmentation and registration --- and illustrate their use across several medical imaging modalities --- X-ray, computed tomography, ultrasound and magnetic resonance imaging --- and several clinical applications: Lung CT image registration in support for assessment of pulmonary nodule growth rate and disease progression from thoracic CT images. Automated reconstruction of standing X-ray panoramas from multi-sector X-ray images for assessment of long limb mechanical axis and knee misalignment. Left and right ventricle localization, segmentation, reconstruction, ejection fraction measurement from cine cardiac MRI or multi-plane trans-esophageal ultrasound images for cardiac function assessment. When devising and evaluating our developed tools, we use clinical patient data to illustrate the inherent clinical challenges associated with highly variable imaging data that need to be addressed before potential pre-clinical validation and implementation. In an effort to provide plausible solutions to the selected applications, the proposed methodological guidelines ensure the development of image processing tools that help achieve sufficiently reliable solutions that not only have the potential to address the clinical needs, but are sufficiently streamlined to be potentially translated into eventual clinical tools provided proper implementation. G1: Reducing the number of degrees of freedom (DOF) of the designed tool, with a plausible example being avoiding the use of inefficient non-rigid image registration methods. This guideline addresses the risk of artificial deformation during registration and it clearly aims at reducing complexity and the number of degrees of freedom. G2: The use of shape-based features to most efficiently represent the image content, either by using edges instead of or in addition to intensities and motion, where useful. Edges capture the most useful information in the image and can be used to identify the most important image features. As a result, this guideline ensures a more robust performance when key image information is missing. G3: Efficient method of implementation. This guideline focuses on efficiency in terms of the minimum number of steps required and avoiding the recalculation of terms that only need to be calculated once in an iterative process. An efficient implementation leads to reduced computational effort and improved performance. G4: Commence the workflow by establishing an optimized initialization and gradually converge toward the final acceptable result. This guideline aims to ensure reasonable outcomes in consistent ways and it avoids convergence to local minima, while gradually ensuring convergence to the global minimum solution. These guidelines lead to the development of interactive, semi-automated or fully-automated approaches that still enable the clinicians to perform final refinements, while they reduce the overall inter- and intra-observer variability, reduce ambiguity, increase accuracy and precision, and have the potential to yield mechanisms that will aid with providing an overall more consistent diagnosis in a timely fashion

Assessment of plastics in the National Trust: a case study at Mr Straw's House

The National Trust is a charity that cares for over 300 publically accessible historic buildings and their contents across England, Wales and Northern Ireland. There have been few previous studies on preservation of plastics within National Trust collections, which form a significant part of the more modern collections of objects. This paper describes the design of an assessment system which was successfully trialled at Mr Straws House, a National Trust property in Worksop, UK. This system can now be used for future plastic surveys at other National Trust properties. In addition, the survey gave valuable information about the state of the collection, demonstrating that the plastics that are deteriorating are those that are known to be vulnerable, namely cellulose nitrate/acetate, PVC and rubber. Verifying this knowledge of the most vulnerable plastics enables us to recommend to properties across National Trust that these types should be seen as a priority for correct storage and in-depth recording

Radiologia diagnostica per immagini estese e soluzioni di stitching con pannelli DR

In radiologia diagnostica, talvolta, grandi aree del corpo umano vengono esaminate tramite l’utilizzo dei raggi x, per esempio vengono eseguiti esami alla colonna vertebrale oppure agli arti inferiori. Con le cassette digitali (CR), per via della loro estesa area superficiale, è possibile eseguire questo tipo di investigazioni tramite una singola esposizione utilizzando una singola cassetta, di dimensione 35 x 84 cm (Kodak) oppure 35 x 91 cm (Carestream) oppure 43 x 129 cm (Carestream). Con i pannelli digitali (DR), a causa della loro dimensione ridotta e dei loro alti costi, allo stato dell’arte non è possibile fare tali investigazioni con una unica esposizione, ma è necessario effettuare diverse esposizioni in accordo con l’estensione dell’area esaminata, unendo due o più immagini radiologiche. Questa operazione è chiamata “stitching” poiché varie immagini sono unite (stitch ossia cucite) insieme. Sono state introdotte e sviluppate tre tecniche che permettono di effettuare un esame di stitching con pannelli DR: rotazionale, lineare e wide. L’obbiettivo di questo lavoro è quello di sottolineare le differenze e di evidenziare le problematiche, prendendo in considerazione la qualità dell’immagine e la semplicità di utilizzo, con lo scopo di trovare la tecnica migliore. Le metodologie sono state valutate tramite tre diversi parametri: la qualità dell’immagine, la comodità d’uso (prendendo in considerazione il tempo per effettuare un esame) e la semplicità dello sviluppo meccanico ed elettronico della tecnica. Ciascun metodo possiede delle qualità buone, ma anche criticità: pertanto la scelta della tecnica migliore non è semplice perché ognuna ha i suoi vantaggi e i suoi svantaggi. Al giorno d’oggi lo stitching rotazionale è il più utilizzato perché la qualità delle immagini è molto buona e non ci sono errori di parallasse. Però non è un sistema semplice da sviluppare perché ci sono due diversi movimenti meccanici da gestire. Per questa ragione è stato introdotto il sistema di stitching lineare, che è più semplice dal punto di vista meccanico ed elettronico, ma peggiora la qualità delle immagini. Il wide stitching è la tecnica più vicina alla tecnica con cassetta CR e possiede una qualità dell’immagine molto buona, ma la difficoltà nello sviluppo di un collimatore in grado di effettuare questa tecnica e la richiesta di alte prestazioni per il tubo radiogeno sono un grande ostacolo. La conclusione è che sebbene sia complesso e costoso, lo stitching rotazionale risulta la tecnica migliore tra quelle investigate. Un ruolo importante, che è andato perso con l’avvento dei sistemi digitali, era offerto dai filtri compensativi. Essi avevano il compito di proteggere i tessuti che assorbono di più la radiazione (es. ossa) rispetto alle parti del corpo più trasparenti (tessuti molli), permettendo di avere in un’unica esposizioni entrambe le tipologie di tessuti senza zone di sovra o sotto esposizione nell’immagine. Con i sistemi CR i filtri a cuneo erano di uso comune nelle investigazioni agli arti inferiori che comprendevano anche la zona del bacino, per esaminare patologie come l’inclinazione dell’anca. Con i sistemi DR i filtri a cuneo non sono utilizzati e sono dannosi per i metodi di stitching lineare e rotazionale. Per simulare l’effetto del filtro a cuneo, nei sistemi DR deve essere applicato un filtraggio, ad esempio la LUT logaritmica, prima di procedere con l’unione delle immagini. Rispetto ai sistemi CR, la dinamica più ampia dei sistemi DR riduce le zone dell’immagine sovra o sotto esposte, rendendo quindi possibile il recupero dell’immagine di quasi tutti i tessuti. Questo, unito alla filtrazione software, ha consentito di evitare il filtro compensativo per i sistemi DR negli esami di stitching. Allo stato attuale dell’arte le cassette CR che vengono utilizzate per esami di stitching hanno un grosso vantaggio rispetto ai sistemi DR: l’esame viene effettuato in un’unica esposizione, annullando tutti gli artefatti da movimento. Tuttavia le prestazioni dei pannelli, in termini di contrasto, risoluzione e dose, sono migliori rispetto a quelle delle CR e compensano lo svantaggio delle esposizioni multiple. Lo studio e il miglioramento dei pannelli digitali è rivolto a superare i limiti di questa tecnica. A tale scopo sono da poco tempo in commercio sistemi digitali cassette size, con sistema di scaricamento delle immagini wi-fi, che sono competitivi con le cassette CR

Imaging Pressure, Cells and Light Fields

Imaging systems often make use of macroscopic lenses to manipulate light. Modern microfabrication techniques, however, have opened up a pathway to the development of novel arrayed imaging systems. In such systems, centimeter-scale areas can contain thousands to millions of micro-scale optical elements, presenting exciting opportunities for new imaging applications. We show two such applications in this thesis: pressure sensing in microfluidics and high throughput fluorescence microscopy for high content screening. Conversely, we show that arrayed elements are not always needed for three dimensional light field imaging.Engineering and Applied Science

Modeling and Simulation in Engineering

This book provides an open platform to establish and share knowledge developed by scholars, scientists, and engineers from all over the world, about various applications of the modeling and simulation in the design process of products, in various engineering fields. The book consists of 12 chapters arranged in two sections (3D Modeling and Virtual Prototyping), reflecting the multidimensionality of applications related to modeling and simulation. Some of the most recent modeling and simulation techniques, as well as some of the most accurate and sophisticated software in treating complex systems, are applied. All the original contributions in this book are jointed by the basic principle of a successful modeling and simulation process: as complex as necessary, and as simple as possible. The idea is to manipulate the simplifying assumptions in a way that reduces the complexity of the model (in order to make a real-time simulation), but without altering the precision of the results

"Hallazgos característicos del examen de medición de los miembros inferiores por rayos X” Clínica Ricardo Palma, enero – marzo 2019

Determina los hallazgos característicos del examen de medición de miembros inferiores usando del sistema Long-Length Imaging. El estudio de tipo observacional, con diseño descriptivo, retrospectivo de corte transversal. La muestra estuvo conformada por 151 informes radiológicos de exámenes de medición de miembros inferiores por rayos X de pacientes atendidos en el Servicio de Radiología de la Clínica Ricardo Palma de enero a marzo de 2019. En el presente estudio se revisaron los informes de 151 pacientes. Encuentra que de la población total, 82 (54,3%) fueron mujeres. La edad promedio fue de 30,83 años y el rango de edad más frecuente de pacientes atendidos fue el de 10 a 19 años (32,5%). La diferencia promedio entre ambos miembros inferiores fue de 5,77 mm, siendo el miembro inferior derecho en promedio más largo (84,77 cm) que el miembro inferior izquierdo (84,71 cm). La dismetría de miembros inferiores superior a 5 mm se presentó con más frecuencia en el grupo de 20 a 29 años con un 53,3% seguido del grupo de 30 a 39 años con 41,2%. Concluye que se encontró 65 casos de dismetrías de más de 5 mm, siendo el 43%. Se encontró en promedio una diferencia de 5,77 mm de diferencia entre los miembros inferiores de los pacientes. Es una condición que no afecta a ningún sexo en particular. Los grupos etarios más afectados son los de 20 a 29 y de 30 a 39 años. La frecuencia de dismetrías es del 31,1% para diferencias de 5 a 10 mm y de 11,9% para diferencias de 11 mm o más.Tesi