Development of novel multimodal light-sheet fluorescence microscopes for in-vivo imaging of vertebrate organisms

The observation of biological processes in their native environments is of critical importance for life science. While substantial information can be derived from the examination of in-vitro biological samples, in-vivo studies are necessary to reveal the complexity of the dynamics happening in real-time within a living organism. Between the possible biological model choices, vertebrates represent an important family due to the various characteristics they share with the human organism. The development of an embryo, the effect of a drug, the interaction between the immune system and pathogens, and the cellular machinery activities are all examples of highly-relevant applications requiring in-vivo observations on broadly used vertebrate models such as the zebrafish and the mouse. To perform such observations, appropriate devices have been devised. Fluorescence microscopy is one of the main approaches through which specific sample structures can be detected and registered in high-contrast images. Through micro-injections or transgenic lines, a living specimen can express fluorescence and can be imaged through such microscopes. Various fluorescence microscopy techniques have been developed, such as Widefield Microscopy (WM) and Laser Scanning Confocal Microscopy (LSCM). In WM the entire sample is visualized in a single 2D image, therefore losing the depth information, while LSCM can recover the 3D information of the sample but with inherent limitations, such as phototoxicity and limited imaging speed. In the last two decades, Light-Sheet Fluorescence Microscopy (LSFM) emerged as a technique providing fast and 3D imaging, while minimizing collateral damages to the specimen. However, due to the particular configuration of the microscope’s components, LSFM setups are normally optimized for a single application. Also, sample management is not trivial, as controlling the specimen positioning and keeping it alive for a long time within the microscope needs dedicated environmental conditioning. In this thesis, I aimed at advancing the imaging flexibility of LSFM, with particular attention to sample management. The conjugation of these aspects enabled novel observations and applications on living vertebrate samples. In Chapter 1, a brief review of the concepts employed within this thesis is presented, also pointing to the main challenges that the thesis aims to solve. In Chapter 2, a new design for multimodal LSFM is presented, which enables performing different experiments with the same instrument. Particularly, high-throughput studies would benefit from this imaging paradigm, conjugating the need for fast and reproducible mounting of multiple samples with the opportunity to image them in 3D. Additionally, from this design, a transportable setup has also been implemented. With these systems, I studied the dynamics of the yolk’s microtubule network of zebrafish embryos, describing novel features and underlining the importance of live imaging to have a whole view of the sample’s peculiarities. This is described in Chapter 3. Further applications on challenging live samples have been implemented, monitoring the macrophage recruitment in zebrafish and the development of mouse embryos. For these applications, described in Chapter 4, I devised specific mounting protocols for the samples, keeping them alive during the imaging sessions. In Chapter 5, an additional LSFM system is described, which allows for recording the sub-cellular machinery in a living vertebrate sample, while avoiding its damage thanks to the devised sample mounting. Through this, single-molecule microscopy (SMM) studies, normally performed on cultured cells, can be extended to the nuclei of living zebrafish embryos, which better recapitulate the native environment where biological processes take place. Finally, Chapter 6 recapitulates the conclusions, the impacts, future integrations, and experimental procedures that would be enabled by the work resumed in this thesis.La observación de los procesos biológicos en su entorno es de vital importancia para las ciencias de la vida. Si bien se puede derivar información sustancial desde muestras biológicas in-vitro, los estudios in-vivo son necesarios para revelar la complejidad de la dinámica que ocurre, en tiempo real, dentro de un organismo vivo. Entre las posibles elecciones de modelos biológicos, los vertebrados representan una familia importante debido a las diversas características que comparten con el organismo humano. El desarrollo de un embrión, la interacción entre el sistema inmunitario y los patógenos, el efecto de un fármaco y las actividades celulares son ejemplos de aplicaciones que requieren observaciones in-vivo en modelos de vertebrados, como el pez cebra y el ratón. La microscopía de fluorescencia es uno de los principales métodos mediante los cuales se pueden grabar imágenes, de alto contraste, de estructuras biológicas específicas. Utilizando microinyecciones o líneas transgénicas, es posible inducir una expresión de proteínas fluorescentes en la muestra y entonces puede ser observada a través de dichos microscopios. Existen varias técnicas de microscopía de fluorescencia, entre ellas las más utilizadas son la microscopía ¿widefield¿ (WM) y la microscopía ¿confocal¿ (LSCM). En WM, una sola imagen en 2D representa el volumen entero de la muestra, por lo cual la información de profundidad se pierde. Por otro lado, LSCM puede recuperar la información en 3D con algunas limitaciones como la fototoxicidad y una velocidad de generación de las imágenes limitada. En las últimas dos décadas, la microscopía de fluorescencia de hoja de luz (LSFM) surgió como técnica que ofrece imágenes de manera rápidas y en 3D, y que al mismo tiempo minimiza los daños colaterales de la muestra. Sin embargo, debido a la geometría de los componentes del microscopio, las configuraciones de LSFM normalmente se optimizan para una sola aplicación. Además, la gestión de las muestras no es trivial, ya que controlar su posición y mantenerlas vivas durante largos periodos de tiempo dentro del microscopio requiere una atención especifica. En esta tesis, me propuse mejorar la versatilidad que LSFM puede ofrecer, con especial atención a la gestión de muestras vivas. La conjugación de estos aspectos permitió nuevas observaciones y nuevas aplicaciones en vertebrados vivos. En el Capítulo 1, se presenta un breve resumen de los conceptos empleados dentro de esta tesis, señalando también los principales desafíos que la tesis pretende resolver. En el Capítulo 2, se presenta un nuevo diseño para un LSFM multimodal, que permite realizar diferentes experimentos con el mismo instrumento. Los estudios de High-Throughput se beneficiarían de este diseño, ya que conjuga la necesidad de un montaje rápido y reproducible de varias muestras con las ventajas de LSFM. Además, a partir de este diseño, también se ha desarrollado un otro microscopio LSFM transportable. Con estos sistemas, se estudió la dinámica de la red de microtúbulos en embriones de pez cebra, describiendo características nuevas y acentuando la importancia de los experimentos in-vivo para obtener una visión completa de la muestra. Esto se describe en el Capítulo 3. Para realizar otras aplicaciones, como la observación de la dinámica de macrófagos en el pez cebra y del desarrollo de embriones de ratón, descritas en el Capítulo 4, se establecieron protocolos de montaje específicos para las muestras, manteniéndolas vivas durante las sesiones experimentales. En el Capítulo 5, se describe otro sistema LSFM, que permite extender los estudios de microscopía de moléculas individuales (SMM), normalmente realizados en cultivos de células, a núcleos de embriones de pez cebra vivos, que recrean mejor el entorno natural de los procesos biológicos. Finalmente, el Capítulo 6 recapitula las conclusiones, los impactos, las integraciones futuras y los procedimientos experimentales que serían posibilitados por el trabajo resumido en esta tesis.Postprint (published version

Napodobení a výroba vzhledu pomocí diferencovatelných materiálových modelů

Výpočetní deriváty kódu - s kódem - jsou jedním z klíčových aktivátorů revoluce strojového učení. V počítačové grafice umožňuje automatická diferenciace řešit problémy s inverzním renderingem, kde se z jednoho nebo několika vstupních snímků získávají parametry jako je odrazovost objektu, poloha nebo koeficienty rozptylu a absorpce ob- jemu. V této práci zvažujeme problémy s přizpůsobením vzhledu a s výrobou, které lze uvést jako příklady problémů s inverzním renderingem. Zatímco optimalizace založená na gradientu, kterou umožňují diferencovatelné programy, má potenciál přinést velmi dobré výsledky, vyžaduje správné využití. Diferenciovatelný rendering není řešením problémů typu brokovnice. Diskutujeme jak teoretické koncepty, tak praktickou implementaci dife- rencovatelných renderingových algoritmů a ukazujeme, jak se spojují s různými problémy s přizpůsobením vzhledu. 1Computing derivatives of code - with code - is one of the key enablers of the machine learning revolution. In computer graphics, automatic differentiation allows to solve in- verse rendering problems. There, parameters such as an objects reflectance, position, or the scattering- and absorption coefficients of a volume, are recovered from one or several input images. In this work, we consider appearance matching and fabrication problems, that can be cast as instances of inverse rendering problems. While gradient-based opti- mization that is enabled by differentiable programs has the potential to yield very good results, it requires proper handling - differentiable rendering is not a shotgun-type prob- lem solver. We discuss both theoretical concepts and the practical implementation of differentiable rendering algorithms, and show how they connect to different appearance matching problems. 1Katedra softwaru a výuky informatikyDepartment of Software and Computer Science EducationMatematicko-fyzikální fakultaFaculty of Mathematics and Physic

3D Volumetric Reconstruction for Light-Field SPECT

Preclinical research on single-photon emission computed tomography (SPECT) imaging is now well acknowledged for its critical role. It is fundamental for functional imaging and is a well-researched area of nuclear medicine emission tomography. Numerous efforts were made to provide an optimized SPECT collimator and detector design. However, these approaches suffer from limited sensitivity and resolution, demanding an efficient reconstruction algorithm development. Moreover, due to the image deterioration induced by the non-stationary collimator-detector response and the single-photon emitting nature of SPECT, it is difficult to quantify the 3D radiopharmaceutical distribution within the patient quantitatively. This dissertation's primary incentive is to design and develop a complete computational framework for the newly proposed L-SPECT scan procedure from the image acquisition to the image reconstruction. Using this framework, I solve several challenging problems related to implementing a dedicated novel 3D L-SPECT image reconstruction algorithm. In particular, a volumetric reconstruction algorithm for L-SPECT system is developed by considering the system configurations. Also, an in-depth analysis of the SPECT imaging system based on the light field concept using the micro pinhole range collimator is presented in this thesis. Moreover, I evaluate the performance of the developed reconstruction algorithms under various imaging circumstances in terms of image quality, computational complexity, and resolution. A Monte Carlo simulation environment for L-SPECT was developed by modelling the properties of the SPECT imaging setup. By examining the existing limitations in the proposed L-SPECT, an improved collimator-detector geometry for the micro-pinhole arrays was introduced in this thesis as one of the main contributions. The modular L-SPECT with the detector heads in a partial ring geometry achieved higher sensitivity and resolution than the planer L-SPECT. The modular L-SPECT was further improved by shifting the centre of the scanning detectors to eliminate the artifacts in the reconstructed images. A dedicated reconstruction algorithm for the modular L-SPECT was developed as proof of concept. In SPECT reconstruction, identification of uncertainty information would help to enhance and mitigate the limitations of the existing reconstruction algorithms. The critical contribution of this thesis is manifested in the development of an image reconstruction algorithm based on Bayesian probabilistic programming for SPECT and L-SPECT. A NUTS based MCMC algorithm is used for probabilistic programming-based reconstruction. The uncertainty associated with the radiation measurement is identified as a distribution from the posterior samples generated using the MCMC algorithm. The performance of the NUTS algorithm improved by using reverse-mode automatic differentiation and distributed programming. The automatic differentiation variational inference-based SPECT reconstruction algorithm is developed to reduce the computational cost in NUTS based reconstruction and uncertainty analysis. Further in this thesis, the L-SPECT simulations are calibrated by comparing with GATE simulations, which are the gold standard in this field. The projection results of MATLAB based simulations are comparable with GATE simulations. The system performance for the proposed different configurations was investigated and contrasted against the existing SPECT modalities and systems, such as LEHR and Inveon SPECT, respectively. The performance analysis of the L-SPECT revealed the system is able to achieve improved sensitivity and better field of view compared to the existing systems. The essential characteristics of this L-SPECT system based on the reconstructed images were assessed with pinhole radii of 0.1 mm and 0.05 mm. In addition, the system sensitivity, spatial resolution, and image quality are appraised from the 3D reconstructed images. The maximum achieved system’s sensitivity was 1000 Cps/Bbq using arrays with a pinhole radius of 0.1 mm at 1 mm pitch, while the highest resolution was obtained using arrays with 0.05 mm pinhole and 3 mm pitch. The designed L-SPECT with different configurations and the developed 3D reconstruction algorithms yielded superior image quality compared with LEHR reconstructions

Anatomical Segmentation of CT images for Radiation Therapy planning using Deep Learning

Radiation therapy is one of the key cancer treatment options. To avoid adverse effect in tissue surrounding the tumor, the treatment plan needs to be based on accurate anatomical models of the patient. In this thesis, an automatic segmentation solution is constructed for the female breast, the female pelvis and the male pelvis using deep learning. The deep neural networks applied performed as well as the current state of the art networks while improving inference speed by a factor of 15 to 45. The speed increase was gained through processing the whole 3D image at once. The segmentations done by clinicians usually take several hours, whereas the automatic segmentation can be done in less than a second. Therefore, the automatic segmentation provides options for adaptive treatment planning

Development of optical methods for real-time whole-brain functional imaging of zebrafish neuronal activity

Each one of us in his life has, at least once, smelled the scent of roses, read one canto of Dante’s Commedia or listened to the sound of the sea from a shell. All of this is possible thanks to the astonishing capabilities of an organ, such as the brain, that allows us to collect and organize perceptions coming from sensory organs and to produce behavioural responses accordingly. Studying an operating brain in a non-invasive way is extremely difficult in mammals, and particularly in humans. In the last decade, a small teleost fish, zebrafish (Danio rerio), has been making its way into the field of neurosciences. The brain of a larval zebrafish is made up of 'only' 100000 neurons and it’s completely transparent, making it possible to optically access it. Here, taking advantage of the best of currently available technology, we devised optical solutions to investigate the dynamics of neuronal activity throughout the entire brain of zebrafish larvae

Modular multimodal platform for classical and high throughput light sheet microscopy

Light-sheet fluorescence microscopy (LSFM) has become an important tool for biological and biomedical research. Although several illumination and detection strategies have been developed, the sample mounting still represents a cumbersome procedure as this is highly dependent on the type of sample and often this might be time consuming. This prevents the use of LSFM in other promising applications in which a fast and straightforward sample-mounting procedure and imaging are essential. These include the high-throughput research fields, e.g. in drug screenings and toxicology studies. Here we present a new imaging paradigm for LSFM, which exploits modularity to offer multimodal imaging and straightforward sample mounting strategy, enhancing the flexibility and throughput of the system. We describe its implementation in which the sample can be imaged either as in any classical configuration, as it flows through the light-sheet using a fluidic approach, or a combination of both. We also evaluate its ability to image a variety of samples, from zebrafish embryos and larvae to 3D complex cell cultures.The authors acknowledge financial support from the Spanish Ministerio de Economía y Competitividad (MINECO) through the “Severo Ochoa” program for Centres of Excellence in R&D (CEX2019-000910-S [MCIN/ AEI/10.13039/501100011033]), Fundació Privada Cellex, Fundació Mir-Puig, and Generalitat de Catalunya through CERCA program; MINECO/FEDER Ramón y Cajal program (RYC-2015-17935); Laserlab- Europe EU-H2020 GA no. 871124; European Union’s Horizon 2020 Framework Programme (H2020 Marie Skłodowska-Curie Innovative Training Networks ImageInLife N. 721537). We thank Verena Ruprecht (CRG- Center of Genomic Regulation, Barcelona), Paz Herráez (Universidad de León), Ester Antón-Galindo and Noelia Fernández-Castillo (Universitat de Barcelona), Marymar Becerra (Universidad Nacional Autónoma de México), Georges Lutfalla, Mai Nguyen Chi and Tamara Sipka (Université de Montpellier), Catarina Brito (ITQB/IBEQ, Lisbon), Antonia Weberling and Magdalena Zernicka-Goetz (University of Cambridge), and Corinne Lorenzo (ITAV – CNRS, Toulouse) for the samples provided. We also thank Maria Marsal and Jordi Andilla for many fruitful discussions.Postprint (published version

Validation of 3d radiative transfer in coastal-ocean water systems as modeled by DIRSIG

The radiative transfer equation (RTE) is a mathematical description of radiative gains and losses experienced by a propagating electromagnetic wave in a participating medium. Except for an isotropic lossless vacuum, all other volumes have the potential to scatter, absorb and emit radiant energy. Of these possible events, the global scattering term is the greatest obstacle between a radiative transfer problem and its solution. Historically, the RTE has been solved using a host of analytical approximations and numerical methods. Typical solution models exploit plane-parallel assumptions where it is assumed that optical properties may vary vertically with depth, but have an infinite horizontal extent. For more complicated scenarios that include pronounced 3D variability, a Monte Carlo statistical approach to the radiative transfer solution is often utilized. This statistical approach has been integrated within the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model in the form of photon mapping. Photon mapping provides a probabilistic solution to the in-scattered radiance problem, by employing a two-pass technique that first populates a photon map based on a Monte Carlo solution to the global scattering term, and then later uses this map to reconstruct the in-scattered radiance distribution during a traditional raytracing pass. As with any computational solution, the actual implementation of the technique requires assumptions, simplifications and integration within a cohesive rendering model. Moreover, the realistic simulation of any environment requires several other radiometric solutions that are not directly related to the photon mapped in-scattered radiance. This research attempts to validate raytraced and photon mapped contributions to sensor reaching radiance that can be expected in typical littoral environments, including boundary interface, medium and submerged or floating object effects. This is accomplished by comparing DIRSIG modeled results to those predicted analytically, by comparison to other numerical models, and by comparison to observed field phenomenology. When appropriate, first-order estimates of a computational solution\u27s ability to render a given phenomenon are provided, including any variance or bias that may result as a function of the user-specified solution configuration

Artistic Path Space Editing of Physically Based Light Transport

Die Erzeugung realistischer Bilder ist ein wichtiges Ziel der Computergrafik, mit Anwendungen u.a. in der Spielfilmindustrie, Architektur und Medizin. Die physikalisch basierte Bildsynthese, welche in letzter Zeit anwendungsübergreifend weiten Anklang findet, bedient sich der numerischen Simulation des Lichttransports entlang durch die geometrische Optik vorgegebener Ausbreitungspfade; ein Modell, welches für übliche Szenen ausreicht, Photorealismus zu erzielen. Insgesamt gesehen ist heute das computergestützte Verfassen von Bildern und Animationen mit wohlgestalteter und theoretisch fundierter Schattierung stark vereinfacht. Allerdings ist bei der praktischen Umsetzung auch die Rücksichtnahme auf Details wie die Struktur des Ausgabegeräts wichtig und z.B. das Teilproblem der effizienten physikalisch basierten Bildsynthese in partizipierenden Medien ist noch weit davon entfernt, als gelöst zu gelten. Weiterhin ist die Bildsynthese als Teil eines weiteren Kontextes zu sehen: der effektiven Kommunikation von Ideen und Informationen. Seien es nun Form und Funktion eines Gebäudes, die medizinische Visualisierung einer Computertomografie oder aber die Stimmung einer Filmsequenz -- Botschaften in Form digitaler Bilder sind heutzutage omnipräsent. Leider hat die Verbreitung der -- auf Simulation ausgelegten -- Methodik der physikalisch basierten Bildsynthese generell zu einem Verlust intuitiver, feingestalteter und lokaler künstlerischer Kontrolle des finalen Bildinhalts geführt, welche in vorherigen, weniger strikten Paradigmen vorhanden war. Die Beiträge dieser Dissertation decken unterschiedliche Aspekte der Bildsynthese ab. Dies sind zunächst einmal die grundlegende Subpixel-Bildsynthese sowie effiziente Bildsyntheseverfahren für partizipierende Medien. Im Mittelpunkt der Arbeit stehen jedoch Ansätze zum effektiven visuellen Verständnis der Lichtausbreitung, die eine lokale künstlerische Einflussnahme ermöglichen und gleichzeitig auf globaler Ebene konsistente und glaubwürdige Ergebnisse erzielen. Hierbei ist die Kernidee, Visualisierung und Bearbeitung des Lichts direkt im alle möglichen Lichtpfade einschließenden "Pfadraum" durchzuführen. Dies steht im Gegensatz zu Verfahren nach Stand der Forschung, die entweder im Bildraum arbeiten oder auf bestimmte, isolierte Beleuchtungseffekte wie perfekte Spiegelungen, Schatten oder Kaustiken zugeschnitten sind. Die Erprobung der vorgestellten Verfahren hat gezeigt, dass mit ihnen real existierende Probleme der Bilderzeugung für Filmproduktionen gelöst werden können