Correlated Multimodal Imaging in Life Sciences:Expanding the Biomedical Horizon

International audienceThe frontiers of bioimaging are currently being pushed toward the integration and correlation of several modalities to tackle biomedical research questions holistically and across multiple scales. Correlated Multimodal Imaging (CMI) gathers information about exactly the same specimen with two or more complementary modalities that-in combination-create a composite and complementary view of the sample (including insights into structure, function, dynamics and molecular composition). CMI allows to describe biomedical processes within their overall spatio-temporal context and gain a mechanistic understanding of cells, tissues, diseases or organisms by untangling their molecular mechanisms within their native environment. The two best-established CMI implementations for small animals and model organisms are hardware-fused platforms in preclinical imaging (Hybrid Imaging) and Correlated Light and Electron Microscopy (CLEM) in biological imaging. Although the merits of Preclinical Hybrid Imaging (PHI) and CLEM are well-established, both approaches would benefit from standardization of protocols, ontologies and data handling, and the development of optimized and advanced implementations. Specifically, CMI pipelines that aim at bridging preclinical and biological imaging beyond CLEM and PHI are rare but bear great potential to substantially advance both bioimaging and biomedical research. CMI faces three mai

Current data processing strategies for cryo-electron tomography and subtomogram averaging

Cryo-electron tomography (cryo-ET) can be used to reconstruct three-dimensional (3D) volumes, or tomograms, from a series of tilted two-dimensional images of biological objects in their near-native states in situ or in vitro. 3D subvolumes, or subtomograms, containing particles of interest can be extracted from tomograms, aligned, and averaged in a process called subtomogram averaging (STA). STA overcomes the low signal to noise ratio within the individual subtomograms to generate structures of the particle(s) of interest. In recent years, cryo-ET with STA has increasingly been capable of reaching subnanometer resolution due to improvements in microscope hardware and data processing strategies. There has also been an increase in the number and quality of software packages available to process cryo-ET data with STA. In this review, we describe and assess the data processing strategies available for cryo-ET data and highlight the recent software developments which have enabled the extraction of high-resolution information from cryo-ET datasets

Optimizing MRI-guided prostate ultrasound ablation therapy using retrospective analyses and artificial intelligence

Magnetic resonance imaging (MRI)-guided transurethral ultrasound ablation (TULSA) is an emerging therapy that has been used to treat prostate cancer (PCa). TULSA destroys prostate tissue with heat using therapeutic ultrasound. The heating is monitored in real-time using MRI thermometry. Despite TULSA’s promise, there are several challenges that have slowed its widespread adoption. Fortunately, MRI images and heating parameters from all TULSA treatments are stor ed. By conducting detailed retrospective analyses and applying deep learning on existing treatments, we can extract valuable information and then leverage this knowledge to optimize future TULSA treatments. One major challenge occurs for those patients who had PCa radiation therapy failure and are seeking salvage treatment with TULSA. Many of these patients have leftover metal markers in the prostate. These markers can hamper subsequent TULSA therapy because they introduce susceptibility artifacts in the MRI image and may also block the ultrasound, which may compromise treatment safety and efficacy. Through an extensive retrospective analysis, we have determined that gold markers tend not to affect the treatment outcome, except when located simultaneously close to the urethra and far from the target boundary, or when located directly on the target boundary itself. Clinically, gold markers had no apparent effect on treatment safety and efficacy compared to a control cohort without markers at the 12-month follow-up. Conversely, nitinol markers are generally problematic for TULSA. A second major challenge applies to all TULSA treatment indications. Immediately after TULSA therapy, MRI contrast agents are used to visualize the non-perfused volume, an objective measure of the ablation outcome. Unfortunately, even if undertreatment is observed, retreatment is not possible, forcing an additional treatment several months later, and with it the associated risks of a second intervention. By training a deep learning model with existing TULSA treatment-day, contrast-free MRI image sets, we have predicted the non-perfused volume with an accuracy comparable to modern-day deep learning prostate segmentation methods. Overall, this work will help daily clinical practice and increase the odds of a successful TULSA therapy.MRI-ohjatun eturauhasen ultraääniablaatiohoidon optimointi retrospektiivisten analyysien ja tekoälyn avulla Magneettikuvaus(MRI)-ohjattu virtsaputken kautta annettu ultraääniablaatio (TULSA) on uusi primaarin ja sädehoidon jälkeen paikallisesti uusiutuneen eturauhassyövän (PCa) hoitomuoto. Menetelmässä eturauhaskudosta koaguloidaan korkean intensiteetin ultraäänellä reaaliaikaisessa MRI-ohjauksessa, mikä parantaa hoidon tarkkuutta. Lupaavista kliinisistä tuloksista huolimatta MRI-ohjaus altistaa teknisille ja kliinisille haasteille, mitkä ovat hidastaneet TULSA-hoidon laajempaa käyttöönottoa. TULSA-hoidossa jokainen vaihe rekisteröidään MRI-kuvin. Koneoppimista hyödyntämällä voidaan retrospektiivisesti analysoida näitä MRI-kuvia TULSA-hoitotulosten optimoimiseksi. Sädehoidon ohjauksessa käytetyt eturauhaseen asetetut merkkijyvät saattavat vaikuttaa TULSA-hoidon tehoon ja turvallisuuteen uusiutuneessa PCa:ssä, koska ne voivat aiheuttaa artefaktoja MRI-kuvaan ja estää ultraäänen etenemisen. Laajassa retrospektiivisessa analyysissä todettiin, että kultamerkkijyvät eivät yleensä vaikuta hoitotulokseen, elleivät ne sijaitse samanaikaisesti lähellä virtsaputkea ja kaukana hoitokohteesta tai suoraan kohteen edessä. Kultamerkkijyvillä ei ollut ilmeistä vaikutusta hoidon turvallisuuteen ja tehokkuuteen verrattuna kontrolliryhmään ilman merkkijyviä 12 kuukauden seurannassa. Välittömästi TULSA-hoidon jälkeen hoitotulos varmistetaan merkkiainetehosteisilla MRI-kuvilla, joilla visualisoidaan verenkierroton alue, mikä korreloi akuuttiin kudosvaurioon eli onnistuneeseen hoitovasteeseen. Ongelmana on, että vaikka merkkiainetehosteisissa MRI-kuvissa todettaisiin riittämätön hoitovaste, uudelleenhoito ei ole samalla hoitokerralla mahdollista, koska eturauhaseen kerääntynyt merkkiaine estää hoidon. Tällöin tarvitaan uusi hoitokerta kuukausien kuluttua toimenpiteen sisältämineen riskeineen, mikä viivästyttää hoitoa ja kuormittaa potilasta. Tässä tutkimuksessa onnistuttiin tarkasti ennustamaan verenkierroton alue hoidonaikaisista merkkiainetehostamattomista MRI-kuvista hyödyntämällä syväoppimismallia. Näillä havainnoilla on tärkeä kliininen merkitys TULSA-hoitotulosten parantamisessa

Integrated Array Tomography for 3D Correlative Light and Electron Microscopy

Volume electron microscopy (EM) of biological systems has grown exponentially in recent years due to innovative large-scale imaging approaches. As a standalone imaging method, however, large-scale EM typically has two major limitations: slow rates of acquisition and the difficulty to provide targeted biological information. We developed a 3D image acquisition and reconstruction pipeline that overcomes both of these limitations by using a widefield fluorescence microscope integrated inside of a scanning electron microscope. The workflow consists of acquiring large field of view fluorescence microscopy (FM) images, which guide to regions of interest for successive EM (integrated correlative light and electron microscopy). High precision EM-FM overlay is achieved using cathodoluminescent markers. We conduct a proof-of-concept of our integrated workflow on immunolabelled serial sections of tissues. Acquisitions are limited to regions containing biological targets, expediting total acquisition times and reducing the burden of excess data by tens or hundreds of GBs

Integrated Array Tomography for 3D Correlative Light and Electron Microscopy

Volume electron microscopy (EM) of biological systems has grown exponentially in recent years due to innovative large-scale imaging approaches. As a standalone imaging method, however, large-scale EM typically has two major limitations: slow rates of acquisition and the difficulty to provide targeted biological information. We developed a 3D image acquisition and reconstruction pipeline that overcomes both of these limitations by using a widefield fluorescence microscope integrated inside of a scanning electron microscope. The workflow consists of acquiring large field of view fluorescence microscopy (FM) images, which guide to regions of interest for successive EM (integrated correlative light and electron microscopy). High precision EM-FM overlay is achieved using cathodoluminescent markers. We conduct a proof-of-concept of our integrated workflow on immunolabelled serial sections of tissues. Acquisitions are limited to regions containing biological targets, expediting total acquisition times and reducing the burden of excess data by tens or hundreds of GBs.</p

PREDICTION OF RESPIRATORY MOTION

Radiation therapy is a cancer treatment method that employs high-energy radiation beams to destroy cancer cells by damaging the ability of these cells to reproduce. Thoracic and abdominal tumors may change their positions during respiration by as much as three centimeters during radiation treatment. The prediction of respiratory motion has become an important research area because respiratory motion severely affects precise radiation dose delivery. This study describes recent radiotherapy technologies including tools for measuring target position during radiotherapy and tracking-based delivery systems. In the first part of our study we review three prediction approaches of respiratory motion, i.e., model-based methods, model-free heuristic learning algorithms, and hybrid methods. In the second part of our work we propose respiratory motion estimation with hybrid implementation of extended Kalman filter. The proposed method uses the recurrent neural network as the role of the predictor and the extended Kalman filter as the role of the corrector. In the third part of our work we further extend our research work to present customized prediction of respiratory motion with clustering from multiple patient interactions. For the customized prediction we construct the clustering based on breathing patterns of multiple patients using the feature selection metrics that are composed of a variety of breathing features. In the fourth part of our work we retrospectively categorize breathing data into several classes and propose a new approach to detect irregular breathing patterns using neural networks. We have evaluated the proposed new algorithm by comparing the prediction overshoot and the tracking estimation value. The experimental results of 448 patients’ breathing patterns validated the proposed irregular breathing classifier

Real-time intrafraction motion monitoring in external beam radiotherapy

© 2019 Institute of Physics and Engineering in Medicine. Radiotherapy (RT) aims to deliver a spatially conformal dose of radiation to tumours while maximizing the dose sparing to healthy tissues. However, the internal patient anatomy is constantly moving due to respiratory, cardiac, gastrointestinal and urinary activity. The long term goal of the RT community to 'see what we treat, as we treat' and to act on this information instantaneously has resulted in rapid technological innovation. Specialized treatment machines, such as robotic or gimbal-steered linear accelerators (linac) with in-room imaging suites, have been developed specifically for real-time treatment adaptation. Additional equipment, such as stereoscopic kilovoltage (kV) imaging, ultrasound transducers and electromagnetic transponders, has been developed for intrafraction motion monitoring on conventional linacs. Magnetic resonance imaging (MRI) has been integrated with cobalt treatment units and more recently with linacs. In addition to hardware innovation, software development has played a substantial role in the development of motion monitoring methods based on respiratory motion surrogates and planar kV or Megavoltage (MV) imaging that is available on standard equipped linacs. In this paper, we review and compare the different intrafraction motion monitoring methods proposed in the literature and demonstrated in real-time on clinical data as well as their possible future developments. We then discuss general considerations on validation and quality assurance for clinical implementation. Besides photon RT, particle therapy is increasingly used to treat moving targets. However, transferring motion monitoring technologies from linacs to particle beam lines presents substantial challenges. Lessons learned from the implementation of real-time intrafraction monitoring for photon RT will be used as a basis to discuss the implementation of these methods for particle RT

Multimodal optical systems for clinical oncology

This thesis presents three multimodal optical (light-based) systems designed to improve the capabilities of existing optical modalities for cancer diagnostics and theranostics. Optical diagnostic and therapeutic modalities have seen tremendous success in improving the detection, monitoring, and treatment of cancer. For example, optical spectroscopies can accurately distinguish between healthy and diseased tissues, fluorescence imaging can light up tumours for surgical guidance, and laser systems can treat many epithelial cancers. However, despite these advances, prognoses for many cancers remain poor, positive margin rates following resection remain high, and visual inspection and palpation remain crucial for tumour detection. The synergistic combination of multiple optical modalities, as presented here, offers a promising solution. The first multimodal optical system (Chapter 3) combines Raman spectroscopic diagnostics with photodynamic therapy using a custom-built multimodal optical probe. Crucially, this system demonstrates the feasibility of nanoparticle-free theranostics, which could simplify the clinical translation of cancer theranostic systems without sacrificing diagnostic or therapeutic benefit. The second system (Chapter 4) applies computer vision to Raman spectroscopic diagnostics to achieve spatial spectroscopic diagnostics. It provides an augmented reality display of the surgical field-of-view, overlaying spatially co-registered spectroscopic diagnoses onto imaging data. This enables the translation of Raman spectroscopy from a 1D technique to a 2D diagnostic modality and overcomes the trade-off between diagnostic accuracy and field-of-view that has limited optical systems to date. The final system (Chapter 5) integrates fluorescence imaging and Raman spectroscopy for fluorescence-guided spatial spectroscopic diagnostics. This facilitates macroscopic tumour identification to guide accurate spectroscopic margin delineation, enabling the spectroscopic examination of suspicious lesions across large tissue areas. Together, these multimodal optical systems demonstrate that the integration of multiple optical modalities has potential to improve patient outcomes through enhanced tumour detection and precision-targeted therapies.Open Acces