Automated focusing and astigmatism correction in electron microscopy

Nowadays electron microscopy still requires an expert operator in order to manually obtain in-focus and astigmatism-free images. Both the defocus and the twofold astigmatism have to be adjusted regularly during the image recording process. Possible reasons are for instance the instabilities of the environment and the magnetic nature of some samples. For some applications the high level of repetition severely strains the required concentration. Therefore, a robust and reliable autofocus and twofold astigmatism correction algorithm is a necessary tool for electron microscopy automation. Most of the automatic focusing methods are based on a sharpness function, which delivers a real-valued estimate of an image quality. In this thesis we study sharpness functions based on image derivative, image Fourier transform, image variance, autocorrelation and histogram. A new method for rapid automated focusing is developed, based on a quadratic interpolation of the derivative-based sharpness function. This function has been already used before on heuristic grounds. In this thesis we give a more solid mathematical foundation for this function and get a better insight into its analytical properties. Further we consider a focus series method, which could act as an extension for an autofocus technique. The method is meant to obtain the astigmatism information from the focus series of images. The method is based on the moments of the image Fourier transforms. After all the method of simultaneous defocus and astigmatism correction is developed. The method is based on a three-parameter optimization (Nelder-Mead simplex method or interpolation-based trust region method) of a sharpness function. All the three methods are employed for the scanning transmission electron microscopy. To be more specific, we have implemented them in the FEI scanning transmission electron microscope and successfully tested their performance as a part of a real-world application

COMPREHENSIVE PERFORMANCE EVALUATION AND OPTIMIZATION OF HIGH THROUGHPUT SCANNING MICROSCOPY FOR METAPHASE CHROMOSOME IMAGING

Specimen scanning is a critically important tool for diagnosing the genetic diseases in today’s hospital. In order to reduce the clinician’s work load, many investigations have been conducted on developing automatic sample screening techniques in the last twenty years. However, the currently commercialized scanners can only accomplish the low magnification sample screening (i.e. under 10× objective lens), and still require clinicians’ manual operation for the high magnification image acquisition and confirmation (i.e. under 100× objective lens). Therefore, a new high throughput scanning method is recently proposed to continuously scan the specimen and select the clinically analyzable cells. In the medical imaging lab, University of Oklahoma, a prototype of high throughput scanning microscopy is built based on the time delay integration (TDI) line scanning detector. This new scanning method, however, raises several technical challenges for evaluating and optimizing the performance. First, we need to use the clinical samples to compare this new prototype with the conventional two-step scanners. Second, the system DOF should be investigated to assess the impact on clinically analyzable metaphase chromosomes. Further, in order to achieve the optimal results, we should carefully assess and select the auto-focusing methods for the high throughput scanning system. Third, we need to optimize the scanning scheme by finding the optimal trade-off between the image quality and efficiency. Finally, analyzing the performance of the various image features is meaningful for improving the performance of the computer aided detection (CAD) scheme under the high throughput scanning condition. The purpose of this dissertation is to comprehensively evaluate the performance of the high throughput scanning prototype. The first technical challenge was solved by the first investigation, which utilized a number of 9 slides from five patients to compare the detecting performance of the high throughput scanning prototype. The second and third studies were performed for the second technical challenge. In the second study, we first theoretically computed the DOF of our prototype and then experimentally measured the system DOF. After that, the DOF impact was analyzed using cytogenetic images from different pathological specimens, under the condition of two objective lenses of 60× (dry, N.A. = 0.95) and 100× (oil, N.A. = 1.25). In the third study, five auto-focusing functions were investigated using metaphase chromosome images. The performance of these different functions was compared using four widely accepted criteria. The fourth and fifth investigations were designed for the third technical challenge. The fourth study objectively assessed chromosome band sharpness by a gradient sharpness function. The sharpness of the images captured from standard resolution target and several pathological chromosomes was objectively evaluated by the gradient sharpness function. The fifth study presented a new slide scanning scheme, which only applies the auto-focusing operations on limited locations. The focusing position was adjusted very quickly by linear interpolation for the other locations. The sixth study was aimed for the fourth technical challenge. The study investigated 9 different feature extraction methods for the CAD modules applied on our high throughput scanning prototype. A certain amount of images were first acquired from 200 bone marrow cells. Then the tested features were performed on these images and the images containing clinically meaningful chromosomes were selected using each feature individually. The identifying accuracy of each feature was evaluated using the receiver operating characteristic (ROC) method. In this dissertation, we have the following results. First, in most cases, we demonstrated that the high throughput scanning can select more diagnostic images depicting clinically analyzable metaphase chromosomes. These selected images were acquired with adequate spatial resolution for the following clinical interpretation. Second, our results showed that, for the commonly used pathological specimens, the metaphase chromosome band patterns are clinically recognizable when these chromosomes were obtained within 1.5 or 1.0 μm away from the focal plane, under the condition of applying the two 60× or 100× objective lenses, respectively. In addition, when scanning bone marrow and blood samples, the Brenner gradient and threshold pixel counting methods can achieve the optimal performance, respectively. Third, we illustrated that the optimal scanning speed of clinical samples is 0.8 mm/s, for which the captured image sharpness is optimized. When scanning the blood sample slide with an auto-focusing distance of 6.9 mm, the prototype obtained an adequate number of analyzable metaphase cells. More useful cells can be captured by increasing the auto-focusing operations, which may be needed for the high accuracy diagnosis. Finally, we found that the optimal feature for the online CAD scheme is the number of the labeled regions. When applying the offline CAD scheme, the satisfactory results can be achieved by combining four different features including the number of the labeled regions, average region area, average region pixel value, and the standard deviation of the either region circularity or distance. Although these investigations are encouraging, there exist several limitations. First, the number of the specimens is limited in most of the assessments. Second, some important impacts, such as the DOF of human eye and the sample thickness, are not considered. Third, more recently proposed algorithms and image features are not used for the evaluation. Therefore, several further studies are planned, which may provide more meaningful information for improving the scanning efficiency and image quality. In summary, we believe that the high throughput scanning may be extensively applied for diagnosing genetic diseases in the future

Measurement and correction of aberrations in light and electron microscopy

Imperfections in image formation, called aberrations, often preclude microscopes from reaching diffraction-limited resolution. Aberrations can be caused either by the microscope itself or by the sample and can be compensated for by using an active element integrated into the beam path which is functioning as a corrector. The optimal settings for this corrector need to be determined without excessive damage to the sample. In particular, for sensitive biological samples, the potential gain for signal and/or resolution needs to be weighed against sample damage. Here I present the development of a special type of optical coherence microscopy (called deep-OCM), which allows the precise determination of the average rat brain refractive index in vivo. The conclusion is that two-photon microscopy is affected by optical aberrations in this sample starting at depths around 200 micrometers. Deep-OCM is well suited for imaging myelinated nerve fibers. Individual fibers can be visualized in the living brain in unprecedented depths beyond 300 micrometers. In the second part of this thesis I describe the development and testing of an auto-focuser and auto-stigmator (called MAPFoSt) for a scanning electron microscope to ensure optimal imaging quality after switching samples or during long acquisition series. MAPFoSt determines the three focus and stigmation parameters from only two test images

A derivative-based fast autofocus method in electron microscopy

Most automatic focusing methods are based on a sharpness function, which delivers a real-valued estimate of an image quality. In this paper, we study an L 2-norm derivative-based sharpness function, which has been used before based on heuristic consideration. We give a more solid mathematical foundation for this function and get a better insight into its analytical properties. Moreover an efficient autofocus method is presented, in which an artificial blur variable plays an important role. We show that for a specific choice of the artificial blur control variable, the function is approximately a quadratic polynomial, which implies that after the recording of at least three images one can find the approximate position of the optimal defocus. This provides the speed improvement in comparison with existing approaches, which usually require recording of more than ten images for autofocus. The new autofocus method is employed for the scanning transmission electron microscopy. To be more specific, it has been implemented in the FEI scanning transmission electron microscope and its performance has been tested as a part of a particle analysis application