Wedekind, Jan

English

Sheffield Hallam University Research Archive

Focus set based reconstruction of micro-objectsWEDEKIND, JanAvailable from Sheffield Hallam University Research Archive (SHURA) at:http://shura.shu.ac.uk/3737/This document is the author deposited version.  You are advised to consult the publisher's version if you wish to cite from it.Published versionWEDEKIND, Jan (2004). Focus set based reconstruction of micro-objects. International IEEE Conference on Mechatronics & Robotics (MechRob'04), 754-756.Repository use policyCopyright © and Moral Rights for the papers on this site are retained by the individual authors and/or other copyright owners. Users may download and/or print one copy of any article(s) in SHURA to facilitate their private study or for non-commercial research. You may not engage in further distribution of the material or use it for any profit-making activities or any commercial gain.Sheffield Hallam University Research Archivehttp://shura.shu.ac.uk Focus set based reconstruction of micro-objectsJan Wedekind∗.Microsystems and Machine Vision LaboratorySchool of Engineering, Sheffield Hallam UniversityPond Street, Sheffield S1 1WB, UK.j.wedekind@shu.ac.ukhttp://www.shu.ac.uk/mmvl/21.06.2004Abstract - With a light optical microscope and a motorizedZ-table, the focus depth of a micro-object can be determined.This paper presents an algorithm for determining the depthmap of the surface of a non-translucent micro-object based onfocus set data.If a robot-manipulator wants to grip a known object, such adepth map can be used to determine its position. Surface datais also useful for automatically determining gripping points.Problems such as this are addressed by the current MiCRoN1project.Many algorithms addressing the problem of extracting depthmaps can be found. However, most of them suffer from system-atic errors. The algorithm presented in this paper efficientlysuppresses these systematic errors by using a multi-scale ap-proach.1 INTRODUCTION1.1 MOTIVATIONThere are many differences between macro- and micro-scaletasks using robots. While task planning has to cope withproblems like adhesion, vision sensors encounter the prob-lem of limited depth of focus.In practice, the limited depth of focus can be used to obtaindepth information. Two images with different focus settingsare sufficient for measuring the surface of a non-translucentobject[6]. This is more feasible than stereo-vision, exspe-cially in the micro-world.If a micro-robot needs to perform a gripping operation, ithas to sense the object’s position. [2] describes a stable algo-rithm for recognizing objects using depth maps. The depthmap could also be used for extracting information as whereto grip the micro-object.To use the algorithm described in [2], a set of points {Pj},that are evenly distributed on the surfaces of the objects isrequired. Furthermore the corresponding normal vectors ~njhave to be estimated:{(~pj , ~nj)}, ~pj := ~0Pj , |~n| != 1 (1)The spin image for a pointQwith normal vector ~m is defined∗Thanks to: Balasundram Amavasai, Manuel Boissenin, Axel Bu¨rkle,Fabio Caparrelli, Arul Selvan, Jon Travis and everyone else who helped mewriting this paper1IST-2001-33567, http://wwwipr.ira.uka.de/˜micron/as a histogram of the function S : R2 7→ R:SQ(~x) :=∑jδ(~x−(√|~pj − ~q|2 − (~m · (~pj − ~q))2~m · (~pj − ~q))) (2)Spin images can be used as features for image recognition be-cause they are rotationally invariant. The similarity measurefor these features is the pixel-wise cross-correlation of twohistograms.Using these features, the 3D pose of objects in clutteredscenes can be obtained with a high level of confidence.Object recognition using spin images combined with thealgorithm presented in this paper could prove to be a power-ful vision algorithm.There are various algorithms for computing a depth mapwhich maximise a local sharpness measure on a focus set [4],but every local sharpness measure suffers from systematic er-rors. [5] presents a more sophisticated algorithm, discardingpixels of the depth map which are considered to be biased.The algorithm presented in this paper does not discard pix-els. Instead, a heuristic method for suppressing these errorsis introduced. This paper is based on research initially pub-lished in [7] and [1].1.2 ENVIRONMENTTo obtain the images, a microscope with a CCD-camera wasused. A set of gray-level images of the micro-object wastaken whilst moving the focal plane upwards relative to theobject. Using a piezo-driven Z-table it is possible to take afocus set with equidistant focal planes.The focus setg Z× Z× Z 7→ R(x1, x2, z) → luminosity of pixel (x1, x2)in zth imageis the input for the algorithm. The algorithm has to computedepth map from this (see figure 1).1.3 STATE OF THE ART1.3.1 COMPUTING A DEPTH MAPMaximising the sharpness of a local image region is a widelyused approach to acquire a depth map of an object.A simple but powerful sharpness measure is the varianceof the set of gray-level values taken from a local region of animage.Figure 1: focus set⇒ depth mapTo generate the results shown in fig. 2, the weighted vari-anceσl(x1, x2, z) := α(2)l (x1, x2, z)−(α(1)l (x1, x2, z))2with weighted moments α(1) and α(2)α(i)l (x1, x2, z) :=1n∑(x′1,x′2)∈Z×Zwl(x′1 − x1, x′2 − x2) g(x′1, x′2, z)i (3)was used, where wl is defined recursively as followsw1(dx1,dx2) := 4 dx1 = 0 ∧ dx2 = 01 max(|dx1|, |dx2|) = 10 otherwisewl(dx1,dx2) :=∑−1 ≤ du1 ≤ 1,−1 ≤ du2 ≤ 1wl−1(dx1 + du1,dx2 + du2)(4)with l determining the size of the local region.The depth map is obtained by maximising the sharpnessmeasure:dml(x1, x2) := argmaxz∈Z(σl(x1, x2, z))(5)1.3.2 EXTENDED DEPTH OF FOCUSUsing the depth map from equation (5), an image with ex-tended depth of focus (=:”deep view”) can be computed bysimply taking the gray-level values of the pixels, which areconsidered to be on the surface of the object:dvl(x1, x2) := g(x1, x2, dml(x1, x2))Often a depth map dml of poor quality can still result in auseful deep view dvl.2 MULTI-SCALE APPROACH2.1 SYSTEMATIC ERRORSFig. 2 shows the reconstruction of a micro-sphere with a di-ameter of about 0.6mm. The depth map was computed fromequation (5) for l = 2.The resulting depth map is of poor quality.By choosing higher values of l, a less noisy depth mapcan be generated while losing higher frequency details. AnyFigure 2: Reconstruction of a micro-sphere based on max-imisation of a sharpness measuresystematic errorinsufficient contrastimage index zpixel−position in image yFigure 3: Systematic error of depth mapchoice of l will always be a trade-off between noisiness andresolution. To tackle this problem, an adaptive algorithmcould be used.Fig. 3 shows a vertical slice through a focus set of a planarmicro-grid. The red dots are the intersections with the com-puted depth map. There are errors not only resulting frominsufficient contrast but also systematic errors, because theregions of maximum sharpness are not coinciding with theobjects surface..Fig. 4 shows an illustration of the problem. The pulseresponse of the light optical microscope2 is a double cone.At the (x1, x2) position of the peak, the sharpness measurewill have a maximum at the expected depth (see b and d). Incontrast at neighboring positions the sharpness measure willhave two local maxima (see a and c).Any algorithm which solves these problems, will give re-sults of much higher quality.2.2 MULTI-SCALE APPROACHThe algorithm presented here computes a sharpness measureand a depth map for different sizes of the local region l.To save computing time the resolution is reduced whilst thefilter-size is increased by sub-sampling with 2l−1:σ∗l (x1, x2, z) := σl(2l−1 x1, 2l−1 x2, z) (6)2Here the pulse response is the focus set of a “tiny” dot. The verticalslice displayed in fig. 3 f. e. shows several structures, which are lookingsimilar to double cones.PSFccabddFigure 4: Schematic behaviour of the sharpness measure forthe focus set of a peakcmaximum*factorsharpness measuremaximumdepth estimatesearch intervaldepth in focus setFigure 5: Definition of search intervalAlso the possibility of recursively computing (6) using (3)and (4) is exploited. Usingidm∗l (x1, x2) := argmaxz∈Z(σ∗l (x1, x2, z))a set of intermediate depth maps{idm∗l |l ∈ {1, 2, . . . , lmax}}(7)is generated afterwards.To merge the depth maps, sharpness intervals are com-puted, as shown in figure 5, using a fixed parameter c ∈(0, 1]:I∗l (x1, x2) :=[min(M∗l),max(M∗l)]withM∗l :={z∣∣∣σ∗l (x1, x2, z) ≥c σ∗l(x1, x2, idm∗l (x1, y1))}Starting at the coarsest level lmax (f.e. lmax = 8), thedepth map dm∗ is computed by merging the set of depthmaps from equation (7) as follows:dm∗lmax := idm∗lmaxdm∗l−1(x1, x2) := idm∗l−1(x1, x2)I∗l−1(x1, x2) ⊆I∗l (bx12 c, bx22 c)dm∗l (bx12 c, bx22 c) otherwisedm∗ := dm∗1(8)Figure 6: 1th, 50th and 99th image of a simulated focus setgenerated with povray[3]Table 1: Results of test on simulated focus set(c,lmax) variance(0.0,1) 21.6531(0.4,6) 1.413543 RESULTSIt is difficult to quantise the quality of the algorithm, becausethe accuracy of the resulting estimation depends on the spa-tial frequencies of the object’s surface and texture. It alsodepends on the parameters c and lmax.The algorithm was tested on a simulated focus set of aplane with an inclination of0.8 focal planespixeland a checkerboard as texture (see fig. 6). To measure its performance thealgorithm was parametrised once with (c, lmax) = (0.0, 1)and once with (c, lmax) = (0.4, 6). A regression plane wasfitted to the results and the variance of the difference betweenthe result and the fitted plane was computed. The results areshown in table 1.Fig. 7 shows the reconstruction of the micro-sphere usingthe improved algorithm. No filtering was applied so that theresult could be compared to fig. 2 more easily. The algorithmalso does not interpolate between the focal planes so that thedepth map is showing steps.Figure 7: Reconstruction of the micro-sphere using the im-proved algorithm4 CONCLUSIONSimply maximising a sharpness measure is not sufficient inmost cases, because the regions of maximum sharpness andthe object’s surface are not necessarily overlapping. Alreadyby applying the multi-scale approach presented in this paper,the quality of the reconstruction is improved significantly.By improving the method of computing and composing thedepth maps (see equ. (8)) or by filtering the result, the qualityof the reconstruction could be improved even further.Whilst aiming to improve the overall quality of the 3Dobject reconstruction, this method is already being used inthe MiCRoN project to address the object recognition sub-task as part of a more complex micromanipulation task. Thisdemonstration consists of a pick-and-place manipulation ofmicron-sized 3D parts by using cm3-sized robots equippedwith specialised manipulation tools. The whole manipula-tion task is supervised by a global positioning system (used totrack the position of the robots) and a local mobile miniatureCCD camera mounted on a larger robot. Because of the con-strained environment and the relative size of the objects withrespect to the CCD camera, other 3D reconstruction methodssuch as depth from stereo are not physically achievable. Inthis case, as well as in many other applications at the micro-scale, 3D object reconstruction by using single viewpoint im-ages is an obliged choice.The software can be downloaded from http://www.shu.ac.uk/mmvl/˜engjw/surftex-1.6.tar.gz.References[1] Axel Bu¨rkle. Optische 2,5D-Rekonstruktionmikroskopischer Objekte. Logos Verlag Berlin,2004. ISBN 3-8325-0468-0.[2] Andrew E. Johnson. Spin-images: A representation for3-d surface matching. PDF-Dokument, March 1997.http://www.ri.cmu.edu/pub_files/pub2/johnson_andrew_1997_3/johnson_andrew_1997_3.pdf.[3] Persistence OF Vision RAYtracer. WWW-Seite. http://www.povray.org/.[4] Gerrit Ro¨nneberg. Mikrostrukturen im Fokus. F&MZeitschrift fu¨r Elektronik, Optik und Mikrosystemtech-nik, 104(10):715–720, 1996. Also see http://www.emk.e-technik.tu-darmstadt.de/forsch/publikationen/ro_int_ver.pdf.[5] Torsten Scheuermann. Beru¨hrungslose Gestaltvermes-sung von Mikrostrukturen durch Fokussuche, volume 13of Wissenschaftliche Schriftenreihe des ICT. FraunhoferInstitut fu¨r Chemische Technologie (ICT), Joseph-von-Fraunhofer-Straße 7, 76327 Karlsruhe, February 1997.engl. title: Profilometry of microstructures by depth fromfocus techniques.[6] Masahiro Watanabe and Shree K. Nayar. Ratio-nal Filters for Passive Depth from Defocus. Inter-national Journal of Computer Vision, 27(3):203–225,1998. http://www.cs.columbia.edu/CAVE/publinks/watanabe_IJCV_1997.pdf.[7] Jan Wedekind. Fokusserien-basierte Rekon-struktion von Mikroobjekten. Master’s the-sis, University of Karlsruhe (TH), May 2002.http://www.ubka.uni-karlsruhe.de/vvv/2002/informatik/2/2.pdf.

Focus set based reconstruction of micro-objects

Abstract

Similar works

Full text

Available Versions

Sheffield Hallam University Research Archive