Fisher, Bob

Oscar, M.

Xiao, Y.

English

There has been an increasing interest in studying animal behaviours [1]. In the EU funded project “ChiRoPing: Developing Versatile and Robust perception using Sonar Systems that Integrate Active Sensing, Morphology and Behaviour” [2], the interest is placed on Chiroptera (bats) as bats are the nature’s expert in active sonar sensing [3]. Study of bats’ behaviour is not only of biological interest in the project but also helpful in engineering an active sonar sensing system similar to what bats possess.One source of information to be investigated in this study is 3D morphology, and in particular how the morphologyof a bat species is related to their echolocation behaviours. To this end, dynamic 3D surface capture of bat heads in flight during echolocation is required. Together with sonar data collected by acoustic sensors, the dynamic 3D shape data can enable biologists and acoustic experts to examine the details of bats’ sonar system in vivo. A high-speed stereo photogrammetry based 3D scanner is employed to perform the 3D capture. The main reasons are twofold. First for capture speed, stereo photogrammetry only requires passive capture of images from two views, thereby allowing fast data recording; second for data consistency, stereo photogrammetry could generate effective establishment of 3D point correspondence between the captured frames allowing a consistent shape analysis over a entire 3D sequence. In this paper we report our scannercharacterisation experiments

Edinburgh Research Explorer

 1Characterising a High-Speed 3D Surface Scanner for Bat Behaviour Study  Y. Xiao, R.B. Fisher, M. Oscar  School of Informatics, University of Edinburgh, UK  I. INTRODUCTION There has been an increasing interest in studying animal behaviours [1].  In the EU funded project “ChiRoPing: Developing Versatile and Robust perception using Sonar Systems that Integrate Active Sensing, Morphology and Behaviour” [2], the interest is placed on Chiroptera (bats) as bats are the nature’s expert in active sonar sensing [3]. Study of bats’ behaviour is not only of biological interest in the project but also helpful in engineering an active sonar sensing system similar to what bats possess.  One source of information to be investigated in this study is 3D morphology, and in particular how the morphology of a bat species is related to their echolocation behaviours. To this end, dynamic 3D surface capture of bat heads in flight during echolocation is required. Together with sonar data collected by acoustic sensors, the dynamic 3D shape data can enable biologists and acoustic experts to examine the details of bats’ sonar system in vivo. A high-speed stereo photogrammetry based 3D scanner is employed to perform the 3D capture. The main reasons are twofold. First for capture speed, stereo photogrammetry only requires passive capture of images from two views, thereby allowing fast data recording; second for data consistency, stereo photogrammetry could generate effective establishment of 3D point correspondence between the captured frames allowing a consistent shape analysis over a entire 3D sequence. In this paper we report our scanner characterisation experiments.  II. THE HIGH-SPEED 3D SCANNER A. System Overview The high-speed scanner (DI3DTM) is manufactured by Dimensional Imaging Ltd. Its hardware mainly comprises two MikrotronTM high-speed cameras, two infrared lights and two processing computers. The cameras are mounted to form a stereo rig (see Figure 1). The distance between the cameras can be adjusted to suit 3D capture of different scenes. Specially designed cables, along with frame grabber, allow image capture up to 500 fps (frames per second). The infrared lights are used to illuminate the capture scene without distracting the bats. The infrared wavelength is carefully selected to overlap the visibility spectrum of the cameras. The computers are responsible for storing and processing raw data of images captured by the stereo cameras, and they share buffers so that the data can be processed in parallel when performing stereo matching. The computers are also synchronized through two synchronization boards connected externally by a synchronization cable. The synchronization is required when recording stereo images. The software consists of three major modules: image capture, 3D reconstruction, and 3D viewing. The image capture module allows users to trigger image capture simultaneously for the stereo cameras. The captured images are processed by the 3D reconstruction module and the results can viewed using the 3D viewing module.   Fig. 1.  Stereo rig of the high-speed scanner  B. Acquisition Set-up In the ChiRoPing project, two groups of bats (insect gleaning and water trawling) are planned for study, for each of which a capturing scenario has been considered. An insect gleaning bat usually hovers in front of a prey on a leaf for a few seconds before performing capture. In this scenario we set up the stereo rig in a small bush where prey is placed on some leaves. When the bat is hovering within the working range of the stereo cameras, a capture session will be triggered to record stereo images of the bat. The distance between the bat and the stereo rig is expected to be 80cm. To suit this capture scenario, Fujinon CF50HA-1 50mm lenses are chosen. At the working distance of 80cm, a single CF50HA-1 lens allows a capture window of 20.2cm X 15.4cm which is about 2-3 times bigger than the insect gleaning bat. The other capture scenario is for water trawling bats. The working distance is expected to be 2m in this case. Fujinon CF75HA-1 75mm lenses are chosen for this application.  At the working distance of 2m, a single CF75HA-1 lens allows a capture window of 30.6cm X 22.9cm which suits the bigger size of  a water trawling bat.  2III. PERFORMANCE TEST OF THE SCANNER We recently received the DI3DTM high-speed 3D scanner. Validation of DI3DTM earlier products for static scanning has been reported in clinical context [4,5], however it is unknown whether or not those validation results are applicable to dynamic scanning as required in the Chiroping project. Therefore our first goal is to test the performance of the new high-speed scanner in various conditions in order to understand its capabilities and limitations. We designed initially two groups of experiments to discover the scanner’s performance. The first group of experiments considered static scenes. The purpose of the static experiments is to find out how scanner parameters such as baseline, aperture, and working distance affect scanning performance and then figure out a reasonable range of these parameters for the real capture. The second group of experiments concerns the speed of the object to be captured. We wanted to understand how the scanner performs regarding to object speed including velocity and direction, and then decide what is the range of object speed within which the scanner can effectively capture 3D shapes. A. Static Experiments Our first group of experiments used a static plane. The plane is placed in front of the stereo rig and adjusted to be roughly perpendicular to the optical axes of the cameras. Once a 3D image of the plane is obtained, we can calculate the variation of the 3D image against the plane, which can be used as an indicator of the performance of the scanner. 1) Working Range Test -12-10-8 -6 -4 -2 0 2 4 6 8 10 1214 1618 200.00000.05000.10000.15000.20000.25000.30000.35000.40000.45000.50000.5500Region 1Region 2Region 3Region 4Region 5 Fig. 2.  (a) test plane; (b) RMS error (mm) for regions in (a)  The term “working distance” is used in our study to describe the distance between the object and the camera baseline. We tested the hypothesis that there is a range of working distance (working range) beyond which the scanner cannot produce acceptable 3D images. In the test, 50mm lenses are used and the stereo cameras were converged to 80cm (which is the set-up to capture insect gleaning bats) with baseline of 15cm. The working distance was tested between -14cm and +20cm (centred at 80cm).  The test plane has 5 textured regions (labelled 1-5 in Figure 2(a)) and RMS (Root Mean Square) errors of fitting a plane to 3D images in these regions are calculated. As shown in Figure 2(b), the RMS curves for all regions exhibit basin shapes around working distance of 0cm. It indicates that there exists a valid working range within which the 3D measurements of the test plane can be considered “good” measurements. On the other hand, how to define a working range should depend on the specific criteria in different applications. 2) Aperture Test The Fujinon CF50HA-1 50mm lenses have F stops from F1.8 – F22 which correspond to different apertures. Given the same baseline and working distance as in the experiment A-1, we vary the aperture in 3 F stops (F4, F5.6,F8) to test how the scanner performs. The reason why we only chose 3 F stops is because the other F stops either make the images too dark or too bright which hampers the stereo matching. Again we calculated RMS errors for regions 1-5 of the test plane and the RMS errors at F5.6 exhibit a 2cm wider basin of working distance than those at F4 and F8. The result indicates that a proper exposure of the capturing scene can maximize the scanner’s working range.  3) Baseline Test The baseline of the stereo cameras is another parameter to tune in real capture. In this experiment, we tested the scanner using baseline lengths 130mm, 195mm and 260mm for 50mm lenses and baseline lengths 130mm, 260mm and 390mm for 75mm lenses. The centres of working distance have been chosen as 80cm for 50mm lenses and 200cm for 75mm lenses because these are the two proposed distances to capture bats. The test object was the same plane as shown in Fig. 2(a). In the previous working range test (Fig. 2(b)), we observed that RMS errors for the 5 regions of the plane all exhibit a “basin” shape although the RMS error for each region has a degree of randomness on its own. To overcome the effect of the randomness, we calculated the average of the RMS errors of the 5 regions for all distances with different baseline lengths and used it as an indicator of the scanner performance.  The averaged RMS errors are illustrated in Figure 3 ((a) for 50mm lenses and (b) for 75 mm lenses). It can be seen clearly that wider baselines generate less RMS errors. However, the valid working range (we mean the valid working range by the range of working distance within which the scanner can produce valid 3D measurements, e.g., shape of the test object is preserved in the captured data) may become shorter when the baseline is longer. For instance, for 50mm lenses with baseline length 260mm, the 3D measurements were observed with low RMS errors (below 0.1mm, see the blue curve in Fig. 3(a)) in the range [-6cm,8cm]. When the working distance exceeded the range, the scanner was not able to output valid 3D images representing the test plane. In comparison, the 195mm baseline could allow valid 3D measurement in the range [-10cm,10cm] with slightly increased RMS errors and the 260mm baseline can even achieve longer working range of [-8cm, 16cm] though the price is the even higher level of RMS errors. The experiment with 75mm confirms that wider baselines produce smaller RMS errors but have shorter working ranges. Therefore the indication is that the length of baseline is an important factor for the stereo system and it should be selected to reflect the balance of the 3D measurement accuracy and the valid working range. Our experiments show that 195mm baseline for 80cm working  3distance with 50mm lenses and 260mm baseline for 200cm working distance are good choice to have a good working range and yet still are able to achieve reasonably low measurement error levels.    (a)RMS errors with different baselines and working distances using 50mm lenses  (b)RMS errors with different baselines and working distances using 75mm lenses Figure 3 Baseline test  4) Separability test  Separating fine details is often desirable for a 3D scanner. We tested the DI3D scanner using two threads which form a cross on top of a textured plane (see Fig. 4(a)). The threads are about 0.5mm in diameter and were placed about 80cm away from the stereo cameras. Using 50mm lenses, the threads were captured in the intensity images by the cameras with widths of 1-2 pixels (0.5-1mm). In the output 3D images from the scanner, the threads were also visible with smoothed shapes and enlarged widths. To measure more accurately the widths of the threads in the 3D images, we fitted the 3D images with a plane and calculated the residual map. As shown in Fig. 4(b), the profile of the threads is clearly reflected in the residual map. Also from the profile of threads derived from the residual map, we can identify the minimum separation distance between the threads before the threads are merged together. It was observed in our experiments that the minimum distance is about 7mm, which indicates that fine shape details less than 7mm apart could be missed at the working distance 80cm using 50mm lenses.        (a) intensity image of threads (b) residual map of the 3D image of (a) when fitted with a plane Figure 4 Separability test  B. Dynamic Experiments We have also tested the scanner with moving objects. Figure 5 illustrates the 3D images (rendered with shading and texture) of a moving furry toy (we won’t be able to capture real bats until November so a furry toy is considered in our test) at speeds 1,2,4m/s. It can be seen that the 3D shape of the toy is recovered well at  1m/s. At 2m/s, there exists slight blur in the 3D images, and also we observed small degree of inconsistency in the recovered 3D shapes. At 4m/s, the recovered 3D shapes are seriously deteriorated and inconsistency of shapes is evident. To characterize quantitatively variation of 3D measurements produced by the scanner, we used a textured ball (see Figure 6(a)) as the test object. The ball was moving toward and away from the scanner. We fit a sphere to the 3D images captured and applied RMS errors of the fitting as indicator of measurement variation. As shown in Figure 6(b) (red curves denote RMS errors, blue dots illustrate positions of the ball, z is the direction of working distance), the RMS errors reach their minimum when the ball passes the point on which the cameras are focussed and increase sharply when the ball exceeds the scanner’s working range. It suggests that at object speeds of 2-3m/s the scanner is able to capture good data if the object is within its working range. We plan to test the scanner with higher speeds, and results will be obtained soon later. 1m/s 2m/s 4m/s  Figure 5 3D images of a falling toy at various speeds  4 (a) test ball   (b) RMS errors of 3D measurements of the dynamic ball at different speeds Figure 6 Experiment with a dynamic ball IV. CONCLUSION AND FUTURE WORK Our experiments with the DI3D high-speed 3D scanner have suggested that 1) the working range of the scanner varies depending on the baseline length and the focus length of lenses; 2) a proper aperture setting is vital for good capture; 3) a wider baseline can improve measurement accuracy but may reduce working range; 4) when object speed is above a threshold, motion blur seriously affects 3D measurement. However whether these findings are applicable to capturing 3D shapes of a flying bat is still questionable as bats have articulated shapes, dark textures and varying speeds. Therefore we propose a data processing method (shown in Figure 7) to compute dynamic 3D models of flying bats. Raw stereo data is first processed by a deblurring module to reduce motion blur. The deblurred images are fed to the DI3D software to get 3D images. A generic 3D bat head model is fitted to the 3D data to obtain individualized 3D models of the bats. For each bat species in the study, we plan to collect a few volumetric scans of dead bats. From these volumetric scans 3D surfaces of the bat heads can be obtained, which can be then used to generate a 3D generic bat head model for the species. Moreover, we hope to combine shape information collected from different bat scans within the same species to build a shape space to guide the model fitting process.   Figure 7 Diagram of recovering 3D bat shapes  REFERENCES [1] C.F. Graetzel, S.N. Fry, B.J. Nelson, “A 6000 Hz computer vision system for real-time wing beat analysis of Drosophila”, The First Conference on Biomedical Robotics and Biomechatronics, 2006 [2] http://www.chiroping.org/ [3] N. Suga, “Biosonar and neural computation in bats”, Scientific American, 262(6):60-8, June 1990 [4] R.J. Winder, T.A. Darvann, W. McKnight, J.D.M. Magee, P. Ramsay-Baggs, “Technical validation of the Di3D stereophotogrammetry surface imaging system”, British Journal of Oral and Maxillofacial Surgery, 46:1 (33-37), 2008 [5] B. Khambay, N. Nairn, A. Bell, J. Miller, A. Bowman, A.F. Ayoub, “Validation and reproducibility of a high-resolution three-dimensional imaging system”, British Journal of Oral and Maxillofacial Surgery, 46:1 (27-32), 2008  

A 6000 Hz computer vision system for real-time wing beat analysis of Drosophila”,

Biosonar and neural computation in bats”,

Technical validation of the Di3D stereophotogrammetry surface imaging system”,

Validation and reproducibility of a high-resolution three-dimensional imaging system”,

Characterising a High-Speed 3D Surface Scanner for Bat Behaviour Study

Characterising a High-Speed 3D Surface Scanner for Bat Behaviour Study

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer