Content aware multi-focus image fusion for high-magnification blood film microscopy

Automated digital high-magnification optical microscopy is key to accelerating biology research and improving pathology clinical pathways. High magnification objectives with large numerical apertures are usually preferred to resolve the fine structural details of biological samples, but they have a very limited depth-of-field. Depending on the thickness of the sample, analysis of specimens typically requires the acquisition of multiple images at different focal planes for each field-of-view, followed by the fusion of these planes into an extended depth-of-field image. This translates into low scanning speeds, increased storage space, and processing time not suitable for high-throughput clinical use. We introduce a novel content-aware multi-focus image fusion approach based on deep learning which extends the depth-of-field of high magnification objectives effectively. We demonstrate the method with three examples, showing that highly accurate, detailed, extended depth of field images can be obtained at a lower axial sampling rate, using 2-fold fewer focal planes than normally required

COMPREHENSIVE PERFORMANCE EVALUATION AND OPTIMIZATION OF HIGH THROUGHPUT SCANNING MICROSCOPY FOR METAPHASE CHROMOSOME IMAGING

Specimen scanning is a critically important tool for diagnosing the genetic diseases in today’s hospital. In order to reduce the clinician’s work load, many investigations have been conducted on developing automatic sample screening techniques in the last twenty years. However, the currently commercialized scanners can only accomplish the low magnification sample screening (i.e. under 10× objective lens), and still require clinicians’ manual operation for the high magnification image acquisition and confirmation (i.e. under 100× objective lens). Therefore, a new high throughput scanning method is recently proposed to continuously scan the specimen and select the clinically analyzable cells. In the medical imaging lab, University of Oklahoma, a prototype of high throughput scanning microscopy is built based on the time delay integration (TDI) line scanning detector. This new scanning method, however, raises several technical challenges for evaluating and optimizing the performance. First, we need to use the clinical samples to compare this new prototype with the conventional two-step scanners. Second, the system DOF should be investigated to assess the impact on clinically analyzable metaphase chromosomes. Further, in order to achieve the optimal results, we should carefully assess and select the auto-focusing methods for the high throughput scanning system. Third, we need to optimize the scanning scheme by finding the optimal trade-off between the image quality and efficiency. Finally, analyzing the performance of the various image features is meaningful for improving the performance of the computer aided detection (CAD) scheme under the high throughput scanning condition. The purpose of this dissertation is to comprehensively evaluate the performance of the high throughput scanning prototype. The first technical challenge was solved by the first investigation, which utilized a number of 9 slides from five patients to compare the detecting performance of the high throughput scanning prototype. The second and third studies were performed for the second technical challenge. In the second study, we first theoretically computed the DOF of our prototype and then experimentally measured the system DOF. After that, the DOF impact was analyzed using cytogenetic images from different pathological specimens, under the condition of two objective lenses of 60× (dry, N.A. = 0.95) and 100× (oil, N.A. = 1.25). In the third study, five auto-focusing functions were investigated using metaphase chromosome images. The performance of these different functions was compared using four widely accepted criteria. The fourth and fifth investigations were designed for the third technical challenge. The fourth study objectively assessed chromosome band sharpness by a gradient sharpness function. The sharpness of the images captured from standard resolution target and several pathological chromosomes was objectively evaluated by the gradient sharpness function. The fifth study presented a new slide scanning scheme, which only applies the auto-focusing operations on limited locations. The focusing position was adjusted very quickly by linear interpolation for the other locations. The sixth study was aimed for the fourth technical challenge. The study investigated 9 different feature extraction methods for the CAD modules applied on our high throughput scanning prototype. A certain amount of images were first acquired from 200 bone marrow cells. Then the tested features were performed on these images and the images containing clinically meaningful chromosomes were selected using each feature individually. The identifying accuracy of each feature was evaluated using the receiver operating characteristic (ROC) method. In this dissertation, we have the following results. First, in most cases, we demonstrated that the high throughput scanning can select more diagnostic images depicting clinically analyzable metaphase chromosomes. These selected images were acquired with adequate spatial resolution for the following clinical interpretation. Second, our results showed that, for the commonly used pathological specimens, the metaphase chromosome band patterns are clinically recognizable when these chromosomes were obtained within 1.5 or 1.0 μm away from the focal plane, under the condition of applying the two 60× or 100× objective lenses, respectively. In addition, when scanning bone marrow and blood samples, the Brenner gradient and threshold pixel counting methods can achieve the optimal performance, respectively. Third, we illustrated that the optimal scanning speed of clinical samples is 0.8 mm/s, for which the captured image sharpness is optimized. When scanning the blood sample slide with an auto-focusing distance of 6.9 mm, the prototype obtained an adequate number of analyzable metaphase cells. More useful cells can be captured by increasing the auto-focusing operations, which may be needed for the high accuracy diagnosis. Finally, we found that the optimal feature for the online CAD scheme is the number of the labeled regions. When applying the offline CAD scheme, the satisfactory results can be achieved by combining four different features including the number of the labeled regions, average region area, average region pixel value, and the standard deviation of the either region circularity or distance. Although these investigations are encouraging, there exist several limitations. First, the number of the specimens is limited in most of the assessments. Second, some important impacts, such as the DOF of human eye and the sample thickness, are not considered. Third, more recently proposed algorithms and image features are not used for the evaluation. Therefore, several further studies are planned, which may provide more meaningful information for improving the scanning efficiency and image quality. In summary, we believe that the high throughput scanning may be extensively applied for diagnosing genetic diseases in the future

Automatic Segmentation and Classification of Red and White Blood cells in Thin Blood Smear Slides

In this work we develop a system for automatic detection and classification of cytological images which plays an increasing important role in medical diagnosis. A primary aim of this work is the accurate segmentation of cytological images of blood smears and subsequent feature extraction, along with studying related classification problems such as the identification and counting of peripheral blood smear particles, and classification of white blood cell into types five. Our proposed approach benefits from powerful image processing techniques to perform complete blood count (CBC) without human intervention. The general framework in this blood smear analysis research is as follows. Firstly, a digital blood smear image is de-noised using optimized Bayesian non-local means filter to design a dependable cell counting system that may be used under different image capture conditions. Then an edge preservation technique with Kuwahara filter is used to recover degraded and blurred white blood cell boundaries in blood smear images while reducing the residual negative effect of noise in images. After denoising and edge enhancement, the next step is binarization using combination of Otsu and Niblack to separate the cells and stained background. Cells separation and counting is achieved by granulometry, advanced active contours without edges, and morphological operators with watershed algorithm. Following this is the recognition of different types of white blood cells (WBCs), and also red blood cells (RBCs) segmentation. Using three main types of features: shape, intensity, and texture invariant features in combination with a variety of classifiers is next step. The following features are used in this work: intensity histogram features, invariant moments, the relative area, co-occurrence and run-length matrices, dual tree complex wavelet transform features, Haralick and Tamura features. Next, different statistical approaches involving correlation, distribution and redundancy are used to measure of the dependency between a set of features and to select feature variables on the white blood cell classification. A global sensitivity analysis with random sampling-high dimensional model representation (RS-HDMR) which can deal with independent and dependent input feature variables is used to assess dominate discriminatory power and the reliability of feature which leads to an efficient feature selection. These feature selection results are compared in experiments with branch and bound method and with sequential forward selection (SFS), respectively. This work examines support vector machine (SVM) and Convolutional Neural Networks (LeNet5) in connection with white blood cell classification. Finally, white blood cell classification system is validated in experiments conducted on cytological images of normal poor quality blood smears. These experimental results are also assessed with ground truth manually obtained from medical experts

Automated Raman cytology system for cancer diagnostics

Raman spectroscopy is a promising optical diagnostic tool that can be applied to un-stained cells in order to detect changes in molecular composition. The data collected can be described as a chemical fingerprint of the sample under investigation. Thus it is very popular in the recent times to use Raman spectroscopy in cytology to increase diagnostic sensitivity and specificity for early stage cancer. In this thesis, I introduce an automated Raman cytology system for cancer diagnostics which integrates all the hardware and software in Micro-manager and operates them in a specific order. An autofocus algorithm for unstained cells and a three-dimensional morphology recovery algorithm are also investigated and contributed to the final automated system.With increasing usage of Raman cytology systems, automation is a solution to limit data variabilities which is a major problem at the moment. In addition, a higher throughput of cellular analysis and a reduction in manpower could be expected from the proposed automation system

Inhaltsbasierte Autofokussierung in der automatisierten Mikroskopie

Die automatische Fokussierung ist ein grundlegender Arbeitsschritt für die Bildaufnahme und Auswertung mit motorgetriebenen Mikroskopen. Auch wenn die Forschungs- und Entwicklungsarbeit auf dem Gebiet des kontrastbasierten Autofokus nunmehr auf eine viele Dekaden lange Geschichte zurückblicken kann, fehlt es selbst aktuellen Methoden an Robustheit gegenüber Bildstörungen und der Handhabung komplexerer Präparatstrukturen. Diese Dissertation stellt einen neuen Autofokusansatz vor, der grundsätzlich mit jeder Mikroskopieart wie unter anderem Fluoreszenz-, Hellfeld- oder auch Phasenkontrast-Mikroskopie verwendet werden kann. Die Neuheit der Methode besteht in einer inhaltsbasierten Fokussuche, die für eine gezieltere Autofokussierung Vorwissen über das zu untersuchende Präparat verwendet. Dabei stellen die von den Trainingsdaten extrahierten und per Boosting selektierten lokalen Haar-Merkmale die Wissensbasis. Die im folgenden als Inhaltsbasierte Autofokus (IB-AF) bezeichnete Methode verfährt in drei Schritten: Zuerst werden an beliebiger z-Koordinate innerhalb des Präparats Regions-of-Interest (ROI) ermittelt, die starke Objekthypothesen enthalten. Danach wird nur auf diesen Regionen eine Kontrastmessung zur Fokuslagenndung der jeweiligen Region entlang der z-Achse durchgeführt. Im letzten Schritt werden die Regionen in ihrer Fokuslage einer genaueren Verikation unterzogen, um ergänzend etwaige uninteressante Objekte auszuschließen. Mit dieser Herangehensweise wendet sich der IB-AF von traditionellen Methoden ab, welche den gesamten Bildbereich einer Fokusmessung unterziehen. Dadurch ist es möglich, sowohl Artefakte aus der Schärfemessung auszuschließen, als auch gezielt spezische Objekte in den Fokus zu bringen. Die vorgestellte Methode wurde auf Präparaten mit unterschiedlichen Herausforderungen getestet und erzielte ein erfolgreiches Fokussieren, wo andere Methoden bisher scheiterten

High throughput platform for multiscale quantitative phase imaging

Quantitative phase imaging (QPI) yields the spatial phase map of the object’s scattering potential. QPI has enabled unprecedented label-free studies in biomedicine, ranging from cell dynamics and growth to cancer diagnosis and prognosis. The field is currently transitioning from technology-driven to application-driven research and from engineering-background users to biomedical-background users. Aligned with these efforts, we present our recent advances in high-throughput, user-friendly QPI technology for multiscale spatiotemporal imaging

Holography

Holography - Basic Principles and Contemporary Applications is a collection of fifteen chapters, describing the basic principles of holography and some recent innovative developments in the field. The book is divided into three sections. The first, Understanding Holography, presents the principles of hologram recording illustrated with practical examples. A comprehensive review of diffraction in volume gratings and holograms is also presented. The second section, Contemporary Holographic Applications, is concerned with advanced applications of holography including sensors, holographic gratings, white-light viewable holographic stereograms. The third section of the book Digital Holography is devoted to digital hologram coding and digital holographic microscopy