Journal ArticleTransmission electron microscopy (TEM) is an important  modality for the analysis of cellular structures in neurobiology. The computational analysis of neurons entail their segmentation and reconstruction from TEM images. This problem is complicated by the heavily textured nature of cellular TEM images and typically low signal-to-noise ratios. In this paper, we propose a new partial differential equation for enhancing the contrast and continuity of cell membranes in TEM images

Tasdizen, Tolga

English

The University of Utah: J. Willard Marriott Digital Library

ENHANCEMENT OF CELL BOUNDARIES IN TRANSMISSION ELECTRONMICROSCOPY IMAGEST o lg a  T a s d iz e n ,  R o s s  W h i ta k e r  R o b e r t  M a r c ,  B r y a n  J o n e s  Scientific Computing and Imaging Institute Moran Eye Center University of Utah University of Utah Salt Lake City, UT Salt Lake City, UTABSTRACTTransmission electron microscopy (TEM) is an important modality for the analysis of cellular structures in neurobi­ology. The computational analysis of neurons entail their segmentation and reconstruction from TEM images. This problem is complicated by the heavily textured nature of cellular TEM images and typically low signal-to-noise ra­tios. In this paper, we propose a new partial differential equation for enhancing the contrast and continuity of cell membranes in TEM images.1. INTRODUCTIONAn important goal of neurobiological research is to decipher the patterns of neuronal connections that govern vertebrate behavior. Medical imaging modalities such as magnetic resonance imaging (MRI) provide 3D measurements of the brain with resolutions on the order of 1 mm. This resolution provides macroscopic information about brain organization, but does not allow analysis of individual neurons. Confocal light microscopy provides 3D measurements with a reso­lution on the order of 1 micron, which is still insufficient to reconstruct synaptic connections of individual neurons. Understanding brain function at the cellular level requires a detailed analysis of connectivity of individual neurons in regions that consist of densely packed cells and processes. Electron microscopy, which can provide resolutions on the order of 1 nanometer, remains the primary tool for resolving neurons, their sub-cellular 3D structures, and their connec­tions. However, compared to the vast amount of research in medical imaging modalities such as MRI and CT, the num­ber of scientific papers on electron microscopy applications in the image processing community has been very limited.Detailed, data-driven descriptions of cells are especially important in neurobiology [1, 2], Several methods have been proposed in the literature for the segmentation and reconstruction of neurons from transmission electron mi­croscopy (TEM) images. Vazquez et  al. introduced a semi­automatic, differential geometric method for segmenting neu­rons in EM images [3], In their method, a user initial­izes points on the boundary of the neuron, then a minimal length geodesic criteria is used to complete the boundary. Deformable curve model based segmentation methods have also been proposed [4, 5], All of the methods discussed above use cell membranes in some way to direct the seg­mentation; hence, they would benefit from a filtering ap­proach that can reduce noise while enhancing membrane structures.2. RELATED WORKIn TEM, a thin specimen is cut and stained, then is sus­pended in an electron beam. The staining agent, which blocks the electron beam, is selectively picked up by differ­ent structures such as membranes. The source of contrast in TEM images is the darker appearance of stained structures, which include cell and organelle membranes, and other struc­tures such a synaptic vesicles in images of neuronal cells. Cells can be segmented by detecting their membranes; how­ever, this process is complicated by the texture generated by other stained structures. In this paper, we propose a partial differential equation (PDE) for enhancing the contrast and continuity of cell membranes in TEM images.Perona and Malik proposed a non-linear diffusion PDE which restricts diffusion around image edges [6]. This PDE is particularly suitable for denoising images composed of piecewise homogeneous intensity regions. It fails in tex­tured images and more generally, in images with structures that do not conform to the piecewise homogeneity assump­tion. As will be shown in Section 4, the Perona and Malik PDE is not suitable for filtering TEM images.In TEM images, cell membranes appear as noisy, one­dimensional structures (curves), and in 3D they appear as sheets. A PDE that can be used to enhance curvilinear co­herence in images was proposed by Weickert [7], This PDE performs directional diffusion aligned with anisotropic struc­tures in images. The image anisotropy is computed us­ing intensity gradient distributions of local neighborhoods0-7803-9134-9/05/$20.00 ©2005 IEEE II-129(structure tensor). A PDE based on the structure tensor was used to remove speckle noise from ultrasound images [8], Carmona and Zhong proposed using the eigenvalues of the Hessian to limit the diffusion around features [9]; however, their method is for denoising only and is not a coherence- enhancing PDE. In this paper, we propose a modification of Weickert’s coherence enhancing diffusion by replacing the information from the structure tensor by information from the Hessian (Section 3). This new PDE is then used to en­hance cell membranes is TEM images and compared against the previously discussed PDEs (Section 4).3. M ETHODSThe principles of coherence enhancing diffusion was intro­duced in [7], Given an image f  (x) : Q ^  IR, a filtered version u(x,  t )  is computed as the solution to the following PDE:du—  =  V  • (D V « ) , (1)with the initial condition u(x , 0) =  f  (x) and homogeneous Neumann boundary conditions on dQ. The diffusion tensor D  specifies the anisotropic directionality of the diffusion.tLet u a denote the convolution of u with K a, a Gaus­sian kernel with standard deviation a.  Let K p be another Gaussian kernel with standard deviation p. Then a structure tensor can be defined as K p * ( V u a <g> V u a ) [7], Typically, a  is 1 pixel or less, whereas p is chosen to define a neigh­borhood size. Then, this tensor summarizes the first order structure of neighborhoods: two small eigenvalues in flat in­tensity areas, one large eigenvalue around linear structures (ridges and edges) and two large eigenvalues at junctions,Dto have the same eigenvectors as the structure tensor, but with modified eigenvalues that are chosen to concentrate diffusion along the directions of linearity, i.e. tangential to an edge. Let ^ ^ d  m2 be the eigenvalues of D  with Mi >  M2 • Then the eigenvalues of D  are chosen asAi :=  a  and A2 :=  a + (1  — a ) , exp (—C / ( mi  — M2 )2) (^)where C  is a threshold parameter and a  is a regularization parameter that introduces a limited amount of isotropic dif­fusion to the system. Typical values for C m  d a  are 1 and 0.001, respectively. While, this choice of D  leads to a co­herence enhancing PDE, we found that constructing the dif­fusion tensor based on the Hessian instead of the structure tensor gave better results when applied to cellular TEM im­ages (see Section 4). Define a Gaussian filtered version of the Hessian:Hp :=  a 2d2x d2up \ dxdydAup ' dxdy d2up d2y /(3)Mi M2 H pDH ption 2. The numerical implementation of the diffusion PDE is beyond the scope of this paper, and is given in detail in [7],4. EXPERIM ENTSFigure 1(a) shows a portion of a TEM image of a rabbit retina. Membranes of neurons and organelles inside the neuron cells, such as micro-filaments and vesicles, appear darker due to the staining process. The enhancement of cell membranes requires: (i) closing of gaps in the membranes due to non-uniform staining, and (ii) removal of noise and enhancement of contrast. Filtering with a Gaussian ker­nel removes noise; however, it also blurs membranes, Fig­ure 1(b). As expected, the Perona and Malik PDE [6] moves the input image toward a piecewise constant image, Fig­ure 1(c). The continuity of membranes is not enhanced; in fact, it is even disrupted in certain places such as the long, horizontally oriented cell on the right side of the image. The geometry of the membranes and the texture inside the cells make this PDE unsuitable for cellular TEM images.The results of Weickert’s coherence enhancing diffu­sion PDE is shown in Figure 3(a). We experimented witha =0.001, a  =  1 .0 ,C  =  1.0,p =  3.0, t  =  40). The con­tinuity of membranes is enhanced, but their contrast from surrounding textures is not improved. The eigenvector of the structure tensor with the smaller eigenvalue is aligned with linear structures (edges, ridges) by construction. Fig­ure 2(a) shows the angle of this eigenvector; it does not closely match the membrane structures.The second-order derivative response of ridge-like fea­tures is stronger than their first-order derivative response. Carmona and Zhong proposed a diffusion PDE with a scalar diffusion coefficient based on the smaller eigenvalue of the Hessian [9], However, their method differs from ours in two important respects: (i) the Hessian is not convolved with a smoothing kernel as in equation 3 and (ii) their approach is for limiting the diffusion around linear features only; it is not a explicitly directional, coherence-enhancing PDE. In this paper, we propose to use the directionality (eigenvec­tors) as well as the eigenvalues of the Hessian to substi­tute the structure tensor in Weickert’s coherence enhance­ment framework. Figure 2(b) and (c) show the angle of the Hessian eigenvector (for a small portion of the image) with the smaller eigenvalue (in magnitude) computed with p =  0 (no smoothing) and p =  3.0, respectively. These re­sults illustrate that smoothing of the Hessian is essential be­fore the computation of eigenvectors. The parameters usedpII-130J T  ■- r J  ' iiI ;«*(c)Fig. i .  (a) A portion of a TEM image of a rabbit retina, and results of processing with (b) Gaussian filtering, (c) Perona- Malik PDE.Fig. 2. Angle of eigenvector (for a small portion of the TEM image) from (a) the structure tensor, (b) Hessian with no smoothing, and (c) Hessian with smoothing p =  3 pixels.Fig. 3. (a) Weickert PDE and (b) proposed Hessian PDE.are the same as in the structure tensor computation, i.e the pboth cases. Compared to the angle from the structure ten­sor, the angle from the Hessian follows the directions of the membranes much more closely. This better representation of membrane angles results in a better enhancement result shown in Figure 3(b). This result (image size 528 x 339)3single-processor PC.In Figure 3(b), the contrast of the membranes and their continuity are enhanced. To verify the contrast enhance­ment claim, we performed the following membrane detec­tion experiment. Membranes are among the darkest objects in the image. Figure 4(a-c) show the intensity histograms of the original image, the image processed with the Weickert PDE and the proposed PDE. Figure 4(d-f) show segmen­tation by simple intensity thresholding. The original his­togram hints at a subtle two-class intensity distribution, but this is smoothed into a single class distribution by the We­ickert PDE. No threshold value can produce a satisfactory result for detecting membranes. On the other hand, the his­togram of the image processed with the proposed method has a clear two-class nature. Due to the better diffusion alignment along membranes, their intensities are not mixed with the brighter surroundings. Consequently, a threshold can be chosen to detect membranes with a relatively smallII-131am ount of false-positives. M ore com plicated algorithms could be applied to the processed image for better detection results.5. C O N C L U SIO N SIn this paper, we introduced a new way of enhancing the contrast of cell m em branes in TEM  images. We adopt the fram ework of Weiclcert’s coherence enhancing PDE. How­ever, the diffusion directions are com puted from  a smoothed Hessian matrix (second-order derivative structure) rather than the structure tensor (first-order derivative structure). The second-order derivatives are better indicators of elongated, thin structures in images; therefore, better filtering results are obtained with the proposed method. In this paper, this is dem onstrated in the context of enhancing m em branes of cells in TEM  images which is very im portant for neurobio- logical research that aims to reconstruct shapes and connec­tivities of neurons. O ther applications with sim ilar image features, such as road finding, can also benefit from  using the proposed m ethod as a pre-processing step. A cknow ledgem ents: This research was partially supported by NSF E IA 0313268, NIH  EY002576, and NIH EYO15128.6. R E F E R E N C E S[1] G. Randall, A. Fernandez, O. Trujillo-Cenoz, G. Apel- baum , M. Bertalm io, L. Vazquez, F. M alm ierca, and P. M orelli, “Neuro3d: an interactive 3d reconstruc­tion system of serial sections using automatic registra­tion,” in SPIE Proc. Three-Dimensional and M ulti­dimensional Microscopy: Image Acquisition and Pro­cessing V, 1998, vol. 3261.[2] J. C. F iala and K. M. Harris, “Extending unbiased stere- ology of brain ultrastructure to three-dim ensional vol­umes,” J. o f  the American Medical Inf. Assoc., vol. 8, no. 1, pp. 1-16, 2001.[3] L. Vazquez, G. Sapiro, and G. Randall, “Segmenting neurons in electronic m icroscopy via geom etric trac­ing,” in Proc. o f Int. Conf. on Image Processing, 1998, pp. 814-818.[4] I. Carlbom, D. Terzopoulos, and K. M. Harris, “Com puter-assieted registration, segmentation, and 3d reconstruction from  images of neuronal tissue sec­tions,” IEEE Trans, on Medical Imaging, vol. 13, no.2, 1994.[5] M. Bertalm io, G. Sapiro, and G. Randall, “M orphing active contours: A  geom etric approach to topology- independent image segm entation and tracking,” in Proc. o f Int. Conf. on Image Processing, 1998, pp. 318­322.Fig. 4. Feature detection: H istogram s of (a) original image, (b) Weiclcert’s PDE, and (c) proposed Hessian PDE. Results of thresholding the respective images: (d-f).[6] P. Perona and J. M alik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. In tell, vol. 12, no. 7, pp. 629-639, 1990.[7] J. Weiclcert, “Coherence-enhancing diffusion filtering,” Int. J. Computer Vision, vol. 31, pp. 111-127, 1999.[8] K. Z. A bd-Elm oniem , A.-B. M. Youssef, and Y. M. Kadah, “Real-time speckle reduction and coher­ence enhancem ent in ultrasound imaging via nonlinear anisotropic diffusion,” IEEE Trans, on Biomedical En­gineering, vol. 49, no. 9, pp. 997-1014, 2002.[9] R. A. Carm ona and S. Zhong, “Adaptive sm oothing respecting feature directions,” IEEE Trans, on Image­Processing, vol. 7, no. 3, pp. 353-358 ,1998 .II-132

Enhancement of cell boundaries in transmission electon microscopy images

https://collections.lib.utah.edu/dl_files/00/49/0049ab7c51925b45343598f4fa99396fd20cee57.pdf

Enhancement of cell boundaries in transmission electon microscopy images

Abstract

Similar works

Full text

Available Versions

The University of Utah: J. Willard Marriott Digital Library