Development of Microscopy Systems for Super-Resolution, Whole-Slide, Hyperspectral, and Confocal Imaging

Optical microscope is an important tool for researchers to study small objects. In this thesis, we will focus on the improvement of traditional microscope systems from several aspects including resolution, field of view, speed, cost, compactness, multimodality. In particular, we will investigate computational imaging methods that bypass the limitations with traditional microscope systems by combining the optical hardware design and image processing algorithm. Examples will include optimizing illumination strategy for the Fourier ptychography (FP), developing field-portable high-resolution microscope using a cellphone lens, investigating pattern-illuminated FP for fluorescence microscopy, demonstrating multimodal microscopic imaging with the use of liquid crystal display, achieving fast and accurate autofocusing for whole slide imaging system

Development of quality assurance procedures and methods for the CBM Silicon Tracking System

The Compressed Baryonic Matter (CBM) experiment at the future Facility for Antiproton and Ion Research (FAIR) aims to study the properties of nuclear matter at high net-baryon densities and moderate temperatures. It is expected that, utilizing ultra-relativistic heavy-ion collisions, a phase transition from hadronic matter to QCD matter will be probed. Among the key objectives are the determination of the nature and order of the transition (deconfinement and/or chiral) and the observation of a critical end-point. To measure and determine the physics phenomena occurring in these collisions, appropriate detectors are required. The Silicon Tracking System (STS) is the key detector to reconstruct charged particle tracks created in heavy-ion collisions. In order to assure the necessary detector performance, about 900 silicon microstrip sensors must be checked and tested for their quality. For these tasks highly efficient and highly automated procedures and methods have to be developed. The first part of this dissertation reports on a novel automated inspection system developed for the optical quality control of silicon microstrip sensors. Proposed methods and procedures allow to scan along the individual sensors to recognize and classify sensor defects. Examples of these defects are: surface scratches, implant defects, metalization layer lithography defects and others. In order to separate and classify these defects various image-processing algorithms based on machine vision are used. The silicon sensors are also characterized geometrically to ensure the mechanical precision targeted for the detector assembly procedures. Since the STS detector will be operated in a high radiation environment with a total non-ionizing radiation dose up to 1x10^14 n_eq/cm^2 over 6 years of operation, the silicon sensors need to be kept in the temperature range of -5 to -10 °C at all times to minimize reverse annealing effects and to avoid thermal runaway. The second part of this work is devoted to the development and optimization of the design of cooling bodies, which remove the thermal energy of overall more than 40 kW produced by the front-end readout electronics. In particular, thermodynamical models were developed to estimate the cooling regimes and thermal simulations of the cooling bodies were carried out. Based on the performed calculations an innovative bi-phase CO2 cooling system of up to 200 W cooling power was built and allowed to verify the simulated cooling body designs experimentally.In der geplanten Experimentieranlage für Antiprotonen- und Ionenforschung (Facility for Antiproton and Ion Research, FAIR) wird das Compressed Baryonic Matter Experiment (CBM) nukleare Materie bei hoher Baryonendichte und moderaten Temperaturen untersuchen. Der Phasenübergang zwischen hadronischer und QCD-Materie kann mithilfe von ultrarelativistischen Schwerionenkollisionen untersucht werden. Die wichtigsten Ziele sind die Bestimmung der Art des Übergangs (Deconfinement- und/oder chiraler Phasenübergang) und die Untersuchung des kritischen Endpunktes im Phasendiagramm. Um diese Phänomene zu untersuchen, sind geeignete Detektorsysteme notwendig. Das Silicon Tracking System (STS) ist der zentrale Detektor, mit Hilfe dessen die Spuren der in den Schwerionenkollisionen erzeugten geladenen Teilchen rekonstruiert werden. Um die volle Funktionsfähigkeit des STS sicherzustellen, müssen die mehr als 900 Siliziumstreifensensoren vor dem Zusammenbau überprüft und getestet werden. Hierfür müssen die hocheffiziente und automatisierte Prozeduren und Methoden entwickelt werden. In erstem Teil dieser Dissertation wird über ein automatisiertes optisches Inspektionssystem berichtet. Das System erlaubt es, die einzelnen Siliziumsensoren auf potentielle vorhandene Oberflächendefekte zu untersuchen und sie zu klassifizieren. Beispiele hierfür sind: Kratzer auf der Oberfläche, Implantierungsdefekte oder Lithographiedefekte der Metallisierungsschicht. Für das Erkennen dieser Defekte werden mehrere “Machine Vision” Bildbearbeitungsalgorithmen benutzt. Außerdem werden die geometrischen Parameter der Sensoren, die für den Zusammenbau des STS wichtig sind, optisch kontrolliert. Der STS Detektor wird bei extrem hohen Kollisionsraten betrieben. Innerhalb einer Betriebsbszeit von 6 Jahren wird eine Strahlungsdosis von bis zu 1x10^14 n_eq/cm^2 akkumuliert, was zu einer deutlichen Erhöhung des Dunkelstrom führt und letztlich des “end-of-life” Kriterium darstellt. Die Siliziumsensoren müssen deswegen auf -5 bis -10 °C gekühlt werden, um “reverse Annealing” Effekte zu minimieren und das “Thermal Runaway” Phänomen zu verzögern. Durch die Ausleselektronik werden andererseits mehr als 40 kW an thermischer Energie nahe der Sensoren produziert, die deshalb mit Kühlkörpern komplett abgeleitet werden muß. Das zweite Teil dieser Dissertation wurde der Optimierung von Kühlkörpern gewidmet. Dafür wurden thermodynamische Modelle implementiert und entsprechende thermische Simulationen durchgeführt. Im Rahmen der Arbeit wurde ein 200 W CO2 Kühlungssystem gebaut, das es erlaubt, die Modellberechnungen und Simulationen einer Kühlung mit 2-phasigem CO2 zu überprüfen

Computational Imaging for Phase Retrieval and Biomedical Applications

In conventional imaging, optimizing hardware is prioritized to enhance image quality directly. Digital signal processing is viewed as supplementary. Computational imaging intentionally distorts images through modulation schemes in illumination or sensing. Then its reconstruction algorithms extract desired object information from raw data afterwards. Co-designing hardware and algorithms reduces demands on hardware and achieves the same or even better image quality. Algorithm design is at the heart of computational imaging, with model-based inverse problem or data-driven deep learning methods as approaches. This thesis presents research work from both perspectives, with a primary focus on the phase retrieval issue in computational microscopy and the application of deep learning techniques to address biomedical imaging challenges. The first half of the thesis begins with Fourier ptychography, which was employed to overcome chromatic aberration problems in multispectral imaging. Then, we proposed a novel computational coherent imaging modality based on Kramers-Kronig relations, aiming to replace Fourier ptychography as a non-iterative method. While this approach showed promise, it lacks certain essential characteristics of the original Fourier ptychography. To address this limitation, we introduced two additional algorithms to form a whole package scheme. Through comprehensive evaluation, we demonstrated that the combined scheme outperforms Fourier ptychography in achieving high-resolution, large field-of-view, aberration-free coherent imaging. The second half of the thesis shifts focus to deep-learning-based methods. In one project, we optimized the scanning strategy and image processing pipeline of an epifluorescence microscope to address focus issues. Additionally, we leveraged deep-learning-based object detection models to automate cell analysis tasks. In another project, we predicted the polarity status of mouse embryos from bright field images using adapted deep learning models. These findings highlight the capability of computational imaging to automate labor-intensive processes, and even outperform humans in challenging tasks.</p

Adaptive optics wavefront compensation for solid immersion microscopy in backside imaging

Thesis (Ph.D.)--Boston UniversityThis dissertation concerns advances in high-resolution optical microscopy needed to detect faults in next generation semiconductor chips. In this application, images are made through the chips' back side to avoid opaque interconnect metal layers on the frontside. Near infrared wavelengths are required, since the silicon is relatively transparent at these wavelengths. A significant challenge in this technique is to resolve features as small as 200nm using wavelengths exceeding 1OOOnm. The highest imaging resolution achievable with refractive optics at infrared wavelengths is demonstrated in this dissertation using an aplanatic solid immersion lens (SIL). This is the only method that has been found to be of sufficient resolution to image the next generation of integrated circuits. While the use of an aplanatic solid immersion lens theoretically allows numerical aperture far in excess of conventional microscopy (NASIL ~ 3.5), it also makes the system performance particularly sensitive to aberrations, especially when the samples have thicknesses that are more than a few micrometers thicker or thinner than designed thickness, or when the refractive index of the SIL is slightly different than that of the sample. In the work described here, practical design considerations of the SILs are examined. A SIL-based confocal scanning microscope system is designed and constructed. The aberrations of the system due to thickness uncertainty and material mismatch are simulated using both analytical model and ray-tracing software, and are measured in the SIL experimental apparatus. The dominant aberration for samples with thickness mismatch is found to be spherical aberration. Wavefront errors are compensated by a microelectromechanical systems deformable mirror (MEMS DM) in the optical system's pupil. The controller is implemented either with closed-loop real time sensor feedback or with predictive open-loop estimation of optical aberrations. Different DM control algorithms and aberration compensation techniques are studied and compared. The experimental results agree well with simulation and it has been demonstrated through models and experiments in this work that the stringent sample thickness tolerances previously needed for high numerical aperture SIL microcopy can be relaxed considerably through aberration compensation. Near-diffraction-limited imaging performance has been achieved in most cases that correspond to practical implementation of the technique

Automating Inspection of Tunnels With Photogrammetry and Deep Learning

Asset Management of large underground transportation infrastructure requires frequent and detailed inspections to assess its overall structural conditions and to focus available funds where required. At the time of writing, the common approach to perform visual inspections is heavily manual, therefore slow, expensive, and highly subjective. This research evaluates the applicability of an automated pipeline to perform visual inspections of underground infrastructure for asset management purposes. It also analyses the benefits of using lightweight and low-cost hardware versus high-end technology. The aim is to increase the automation in performing such task to overcome the main drawbacks of the traditional regime. It replaces subjectivity, approximation and limited repeatability of the manual inspection with objectivity and consistent accuracy. Moreover, it reduces the overall end-to-end time required for the inspection and the associated costs. This might translate to more frequent inspections per given budget, resulting in increased service life of the infrastructure. Shorter inspections have social benefits as well. In fact, local communities can rely on a safe transportation with minimum levels of disservice. At last, but not least, it drastically improves health and safety conditions for the inspection engineers who need to spend less time in this hazardous environment. The proposed pipeline combines photogrammetric techniques for photo-realistic 3D reconstructions alongside with machine learning-based defect detection algorithms. This approach allows to detect and map visible defects on the tunnel’s lining in local coordinate system and provides the asset manager with a clear overview of the critical areas over all infrastructure. The outcomes of the research show that the accuracy of the proposed pipeline largely outperforms human results, both in three-dimensional mapping and defect detection performance, pushing the benefit-cost ratio strongly in favour of the automated approach. Such outcomes will impact the way construction industry approaches visual inspections and shift towards automated strategies

Integrated Data and Energy Communication Network: A Comprehensive Survey

OAPA In order to satisfy the power thirsty of communication devices in the imminent 5G era, wireless charging techniques have attracted much attention both from the academic and industrial communities. Although the inductive coupling and magnetic resonance based charging techniques are indeed capable of supplying energy in a wireless manner, they tend to restrict the freedom of movement. By contrast, RF signals are capable of supplying energy over distances, which are gradually inclining closer to our ultimate goal &#x2013; charging anytime and anywhere. Furthermore, transmitters capable of emitting RF signals have been widely deployed, such as TV towers, cellular base stations and Wi-Fi access points. This communication infrastructure may indeed be employed also for wireless energy transfer (WET). Therefore, no extra investment in dedicated WET infrastructure is required. However, allowing RF signal based WET may impair the wireless information transfer (WIT) operating in the same spectrum. Hence, it is crucial to coordinate and balance WET and WIT for simultaneous wireless information and power transfer (SWIPT), which evolves to Integrated Data and Energy communication Networks (IDENs). To this end, a ubiquitous IDEN architecture is introduced by summarising its natural heterogeneity and by synthesising a diverse range of integrated WET and WIT scenarios. Then the inherent relationship between WET and WIT is revealed from an information theoretical perspective, which is followed by the critical appraisal of the hardware enabling techniques extracting energy from RF signals. Furthermore, the transceiver design, resource allocation and user scheduling as well as networking aspects are elaborated on. In a nutshell, this treatise can be used as a handbook for researchers and engineers, who are interested in enriching their knowledge base of IDENs and in putting this vision into practice

Mapping Trabecular Bone Fabric Tensor by in Vivo Magnetic Resonance Imaging

The mechanical competence of bone depends upon its quantity, structural arrangement, and chemical composition. Assessment of these factors is important for the evaluation of bone integrity, particularly as the skeleton remodels according to external (e.g. mechanical loading) and internal (e.g. hormonal changes) stimuli. Micro magnetic resonance imaging (µMRI) has emerged as a non-invasive and non-ionizing method well-suited for the repeated measurements necessary for monitoring changes in bone integrity. However, in vivo image-based directional dependence of trabecular bone (TB) has not been linked to mechanical competence or fracture risk despite the existence of convincing ex vivo evidence. The objective of this dissertation research was to develop a means of capturing the directional dependence of TB by assessing a fabric tensor on the basis of in vivo µMRI. To accomplish this objective, a novel approach for calculating the TB fabric tensor based on the spatial autocorrelation function was developed and evaluated in the presence of common limitations to in vivo µMRI. Comparisons were made to the standard technique of mean-intercept-length (MIL). Relative to MIL, ACF was identified as computationally faster by over an order of magnitude and more robust within the range of the resolutions and SNRs achievable in vivo. The potential for improved sensitivity afforded by isotropic resolution was also investigated in an improved µMR imaging protocol at 3T. Measures of reproducibility and reliability indicate the potential of images with isotropic resolution to provide enhanced sensitivity to orientation-dependent measures of TB, however overall reproducibility suffered from the sacrifice in SNR. Finally, the image-derived TB fabric tensor was validated through its relationship with TB mechanical competence in specimen and in vivo µMR images. The inclusion of trabecular bone fabric measures significantly improved the bone volume fraction-based prediction of elastic constants calculated by micro-finite element analysis. This research established a method for detecting TB fabric tensor in vivo and identified the directional dependence of TB as an important determinant of TB mechanical competence