Motivation: Modern anatomical and developmental studies often require high-resolution imaging of large specimens in three dimensions (3D). Confocal microscopy produces high-resolution 3D images, but is limited by a relatively small field of view compared with the size of large biological specimens. Therefore, motorized stages that move the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow direct reconstruction (Stitching) of the whole image from individual image stacks

Preibisch, Stephan

Saalfeld, Stephan

Tomancak, Pavel

English

PubMed

Motivation: Modern anatomical and developmental studies often require high-resolution imaging of large specimens in three dimensions (3D). Confocal microscopy produces high-resolution 3D images, but is limited by a relatively small field of view compared to the size of large biological specimens. Therefore, motorized stages that move the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow direct reconstruction (Stitching) of the whole image from individual image stacks. Results: To optimally stitch a large collection of 3D confocal images, we developed a method that, based on the Fourier Shift Theorem, computes all possible translations between pairs of 3D images, yielding the best overlap in terms of the cross correlation measure and subsequently finds the globally optimal configuration of the whole group of 3D images. This method avoids the propagation of errors by consecutive registration steps. Additionally, to compensate the brightness differences between tiles, we apply a smooth, non-linear intensity transition between the overlapping images. Our stitching approach is fast, works on 2D and 3D images, and for small image sets doesn't require prior knowledge about the tile configuration. Availability: The implementation of this method is available as an ImageJ plugin distributed as a part of the Fiji project.1 1 Fiji is just ImageJ: http://pacific.mpi-cbg.de

Preibisch, S.

Saalfeld, S.

Tomancak, P.

MPG.PuRe

Globally Optimal Stitching of Tiled 3D Microscopic Image Acquisitions

S. Preibisch

S. Saalfeld

P. Tomancak

Crossref

Globally optimal stitching of tiled 3D microscopic image acquisitions

Motivation: Modern anatomical and developmental studies often require high-resolution imaging of large specimens in three dimensions (3D). Confocal microscopy produces high-resolution 3D images, but is limited by a relatively small field of view compared with the size of large biological specimens. Therefore, motorized stages that move the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow direct reconstruction (Stitching) of the whole image from individual image stacks. Results: To optimally stitch a large collection of 3D confocal images, we developed a method that, based on the Fourier Shift Theorem, computes all possible translations between pairs of 3D images, yielding the best overlap in terms of the cross-correlation measure and subsequently finds the globally optimal configuration of the whole group of 3D images. This method avoids the propagation of errors by consecutive registration steps. Additionally, to compensate the brightness differences between tiles, we apply a smooth, non-linear intensity transition between the overlapping images. Our stitching approach is fast, works on 2D and 3D images, and for small image sets does not require prior knowledge about the tile configuration. Availability: The implementation of this method is available as an ImageJ plugin distributed as a part of the Fiji project (Fiji is just ImageJ: http://pacific.mpi-cbg.de/)

MDC Repository

[11:49 14/5/2009 Bioinformatics-btp184.tex] Page: 1463 1463–1465BIOINFORMATICS APPLICATIONS NOTE Vol. 25 no. 11 2009, pages 1463–1465doi:10.1093/bioinformatics/btp184Systems biologyGlobally optimal stitching of tiled 3D microscopic imageacquisitionsStephan Preibisch, Stephan Saalfeld and Pavel Tomancak∗Max Planck Institute of Molecular Cell Biology and Genetics, Pfotenhauerstrasse 108, Dresden, GermanyReceived and revised on January 30, 2009; accepted on March 26, 2009Advance Access publication April 3, 2009Associate Editor: Trey IdekerABSTRACTMotivation: Modern anatomical and developmental studies oftenrequire high-resolution imaging of large specimens in threedimensions (3D). Confocal microscopy produces high-resolution 3Dimages, but is limited by a relatively small field of view compared withthe size of large biological specimens. Therefore, motorized stagesthat move the sample are used to create a tiled scan of the wholespecimen. The physical coordinates provided by the microscopestage are not precise enough to allow direct reconstruction (Stitching)of the whole image from individual image stacks.Results: To optimally stitch a large collection of 3D confocal images,we developed a method that, based on the Fourier Shift Theorem,computes all possible translations between pairs of 3D images,yielding the best overlap in terms of the cross-correlation measureand subsequently finds the globally optimal configuration of thewhole group of 3D images. This method avoids the propagation oferrors by consecutive registration steps. Additionally, to compensatethe brightness differences between tiles, we apply a smooth, non-linear intensity transition between the overlapping images. Ourstitching approach is fast, works on 2D and 3D images, and forsmall image sets does not require prior knowledge about the tileconfiguration.Availability: The implementation of this method is available as anImageJ plugin distributed as a part of the Fiji project (Fiji is justImageJ: http://pacific.mpi-cbg.de/).Contact: tomancak@mpi-cbg.de1 INTRODUCTIONThere is an increasing demand to image complete biologicalspecimens with high resolution in two and three dimensions.Biologists use Confocal, Bright field or Electron microscopesequipped with motorized stages to stepwise image large areas usinghigh magnifications. The acquired image tiles have to be combinedinto one final output image by a process usually referred to asStitching. Motorized stages used for such acquisitions operate withhigh precision, however, in most cases the physical coordinates fromthe stage are not precise enough to provide a proper reconstructionof the whole acquisition on the pixel level. Moreover, in some casesthe approximate configuration of the tiles may be unknown.To overcome these problems, we present a method capableof stitching sets of 2D and 3D images. The method uses phase∗To whom correspondence should be addressed.Fig. 1. Stitching results. (A) The 3D visualization of stitched 2×3 mosaicof the central nervous system of a Drosophila larva (Table 1, row 2). Thedisplacements of individual 3D stacks are artificially exaggerated. (B,C) Aclose up of the border between adjacent tiles as recorded by the microscopestage (B) and after the application of the stitching method (C).correlation (Kuglin and Hines, 1975) to find the translationbetween all image pairs and registers multi-tile acquisitions globallyminimizing all pairwise registration errors. The global approachallows stitching even without prior knowledge of tile configuration.Additionally, we implemented non-linear blending to compensatebrightness differences between the image tiles to create artifact-free,stitched output images.2 METHODSThe acquisition results in a set of n image tiles where a subset overlaps suchthat all images form a connected graph. The method has to determine thecorrect overlap for all tiles and create a properly registered output imagecontaining all image tiles that are connected to the graph.We assume all transformations between overlapping tiles to be translationonly which is reasonable for most commonly used microscopy techniqueswhere the static, fixed sample rests on a level microscope slide moved by amotorized stage.2.1 Phase correlation and fast Fourier transformWe use the Fourier transform (F )-based phase correlation method (PCM)(Kuglin and Hines, 1975) to compute translational offsets between images.If two images A and B are related by translation, the shift (x,y,z)T can becomputed from inverse Fourier transform (F−1) of the phase correlation© 2009 The Author(s)This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/2.0/uk/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. at FAK/MDC on July 10, 2015http://bioinformatics.oxfordjournals.org/Downloaded from [11:49 14/5/2009 Bioinformatics-btp184.tex] Page: 1464 1463–1465S.Preibisch et al.matrix Q(F (A),F (B)). It is theoretically represented by a single peak (δ-function) whose location defines the correct translation vector between theimages.In real images, however, F−1(Q) contains several peaks marking differenttranslations with high correlation. Moreover, each peak describes eightdifferent possible translations (in 3D) due to the periodicity of the Fourierspace. To determine the correct shift, we select the n highest local maxima(3×3×3 neighborhood) from F−1(Q) and evaluate their eight possibletranslations by means of cross-correlation on the overlapping area of theimages A,B. The peak with the highest correlation is selected as translationbetween the two images. If none of the peaks is above a certain limit thetiles are assumed to be non-overlapping.The Fourier transform F is computed using the multi-threaded PrimeFactor Fast Fourier Transform (PFFFT) (Good, 1958). Prior to performingthe PFFFT the input images have to be extended to a size supported by thatalgorithm. To overcome sharp edges resulting from zero padding, we extendthe images by the mirrored image content faded from its real intensity toblack using an exponential windowing function. In this way, the periodicityis not broken and real image information is preserved as only the mirroredparts are faded out. The resulting images are zero-padded to dimensionssupported by the PFFFT algorithm.2.2 Globally optimized registrationThe PCM results in a set of translations P= (pAB :A,B∈V ) with V beingthe set of all tiles, and each pAB mapping tile A to an overlapping tile Bmaximizing the pairwise alignment quality. Consecutive placement of tilesinto the output image would lead to the propagation of alignment errors asthe independent pairwise transforms may conflict when the position of atile is defined by more than one neighbor. We avoid error propagation byminimizing the sum of all pairwise transfer errors by means of least squares.Given that F ∈V is a fixed tile that defines the common reference frame, theoptimal configuration TVF ={tAF :A,F ∈V} is defined asargminTVF∑A∈V\{F}⎛⎝ ∑B∈V\{A}‖tBF −tAF −pAB‖2⎞⎠The solution of this term gives the set of all translational parametersand the displacement relative to the initially estimated pairwise translation(Table 1). Overlapping tiles for which no initial translation was found maystill be connected through other images. Additionally, by checking the ratiobetween maximal and average displacement, we are able to detect misalignedtiles, where pairwise PCM failed to identify correct translation. If themaximal displacement is significantly larger than the average displacementthen the alignment of the image pair with the highest displacement is removedand TVF is recalculated. If tiles become disconnected from all their neighborsthey will not contribute to the output image. These are usually tiles withoutcontent.2.3 Non-linear blendingMicroscopic acquisitions suffer from shading, a position-dependent additiveterm to the pixel intensity resulting in clearly visible brightness differencesin the overlapping areas of adjacent tiles. Background subtraction prior tostitching is problematic for confocal microscopy images that lack backgroundintensities in the absence of a specimen.To compensate for shading differences between tiles, we apply a weightingfactor to each pixel in each contributing tile. The applied factor is based onthe exponentially weighted distance of the current pixel coordinate fromits own tile boundary. Thus, the non-linear blending creates a weightingmap for each tile that has high influence on the central pixels and zeroinfluence on pixels at the edge of the image. The non-linearity of the 3Dweight function is determined by the tunable parameter α [α=0 blending byTable 1. Examples of stitching performance on tiled microscopy datacomputed on an Intel® Quad-Core CPU machine with 2.67GHz and 24GBof RAMTiles Single-tile Output image Comp. time Min/avg/maxdimension size (min:sec) displacement (pixel)3 10242 ×42 108 MB 0:42 0.00/0.00/0.006 5122 ×86 350 MB 1:20 0.60/0.77/1.0524 10242 ×68 1.2 GB 22:43 0.49/0.76/0.9972 5122 ×122 1.9 GB 43:10 0.00/0.39/0.6463 10242 ×92 5.4 GB 178:57 0.00/0.66/1.18The global alignment adjusts the position of individual tiles up to about 1pixel.The sixtile dataset in row 2 is RGB all other are gray-scale.averaging, α=1 linear blending (Brown and Lowe, 2007), α>1 non-linearblending).3 RESULTS AND DISCUSSIONWe used the presented method to reconstruct several types of tiledmicroscopy acquisitions (Table 1, Fig. 1) ranging from mosaics ofhistological 2D images to sets of gray-scale and RGB 3D confocalstacks. The reconstructed images show no stitching artifacts, eitherfrom misalignments or from intensity leaps in the overlapping areas(http://fly.mpi-cbg.de/preibisch/stitching.html).The globally optimized registration finishes with low average andmaximal displacement and does not discard any non-uniform imagetiles (Table 1). Moreover, the method is capable of reconstructingeven mosaics with an unknown configuration of the tiles. Inthis case, the PCM is computed for all tile pairs (i.e. all tilesare assumed to overlap), connections that result in inconsistentor low-quality alignments are sequentially dropped from theconnection graph until only the truly overlapping tiles remain inthe graph.The program is fully multi-threaded taking advantage of multi-core CPUs. Typically, registration consumes significantly less timethan image acquisition. The method is applicable to virtually anytype of microscopy image data as it uses LOCI bioformat standardfor import. The method is implemented as a plugin for the fullyopen source ImageJ (Rasband, 2008) distribution called Fiji that isactively maintained and expanded by an international community ofdevelopers ensuring long-term availability and future improvementsof the stitching method. All operations of the plugin and therequired parameters are extensively documented (http://fly.mpi-cbg.de/preibisch/manual).The described stitching algorithm represents an open sourcesolution for efficient reconstruction of large 2D and 3D imagemosaics, which will become increasingly important for analysisof large biological specimens with the high resolution offered bymodern microscopy techniques.ACKNOWLEDGEMENTSWe thank James W. Truman, Nicolas Gompel and Simone Fietz forproviding unpublished images. We also thank Benjamin Schmidtand the Fiji team for providing programming libraries.Conflict of Interest: none declared.1464 at FAK/MDC on July 10, 2015http://bioinformatics.oxfordjournals.org/Downloaded from [11:49 14/5/2009 Bioinformatics-btp184.tex] Page: 1465 1463–1465Globally optimal stitchingREFERENCESBrown,M. and Lowe,D.G. (2007) Automatic panoramic image stitching using invariantfeatures. Int. J. Comput. Vis., 74, 59–73.Good,I.J. (1958) The interaction algorithm and practical Fourier analysis. J. R. Stat.Soc. B, 20, 361–372.Kuglin,C.D. and Hines,D.C. (1975) The phase correlation image alignment method.In Proceedings of the IEEE, International Conference on Cybernetics and Society,pp. 163–165.Rasband,W. (1997–2008) ImageJ: image processing and analysis in Java[version 1.40g].1465 at FAK/MDC on July 10, 2015http://bioinformatics.oxfordjournals.org/Downloaded from 

Motivation: Modern anatomical and developmental studies often require high-resolution imaging of large specimens in three dimensions (3D). Confocal microscopy produces high-resolution 3D images, but is limited by a relatively small field of view compared to the size of large biological specimens. Therefore, motorized stages that move the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow direct reconstruction (Stitching) of the whole image from individual image stacks.

Results: To optimally stitch a large collection of 3D confocal images, we developed a method that, based on the Fourier Shift Theorem, computes all possible translations between pairs of 3D images, yielding the best overlap in terms of the cross correlation measure and subsequently finds the globally optimal configuration of the whole group of 3D images. This method avoids the propagation of errors by consecutive registration steps. Additionally, to compensate the brightness differences between tiles, we apply a smooth, non-linear intensity transition between the overlapping images. Our stitching approach is fast, works on 2D and 3D images, and for small image sets doesn't require prior knowledge about the tile configuration.

Availability: The implementation of this method is available as an ImageJ plugin distributed as a part of the Fiji project.1

1 Fiji is just ImageJ: http://pacific.mpi-cbg.de

Globally optimal stitching of tiled 3D microscopic image acquisitions

Abstract

Similar works

Full text

Available Versions

MPG.PuRe

Crossref

MDC Repository

Max Planck Society eDoc Server