Male mosquitoes swarm to find mates, but the characteristics of these swarms have not yet been quantitatively analyzed in the field. This paper describes a simple method used recently to localize individual Anopheles gambiae in a swarm via stereoscopic image analysis of video footage swarms recorded in Doneguebougou, Mali. The methods described here represent a streamlining of simple triangulation for insect localization in the hope that other researchers will be able to apply the method for studying fundamental questions about flying insects or other taxa.&#xd;&#xa

Nicholas C. Manoukis

English

Nature Precedings

Description of a Method for Localizing Swarming Mosquitoes and Other
Insects in 3D Space with Visualizations
Nicholas C. Manoukis†
†Section of Vector Biology, Laboratory of Malaria and Vector Research
National Institute of Allergy and Infectious Diseases, National Institutes of Health
12735 Twinbrook Parkway, Bethesda, MD 20892 USA
manoukisn@niaid.nih.gov
November 2008
Abstract
Male mosquitoes swarm to find mates, but the characteristics of these swarms have not yet
been quantitatively analyzed in the field. This paper describes a simple method used recently
to localize individual Anopheles gambiae in a swarm via stereoscopic image analysis of video
footage swarms recorded in Done´gue´bougou, Mali. The methods described here represent a
streamlining of simple triangulation for insect localization in the hope that other researchers
will be able to apply the method for studying fundamental questions about flying insects or
other taxa.
Key-Words : Stereoscopic image analysis, 3D localization, Insect tracking, Triangulation
1 Methodological overview
Estimating the position of an individual mosquito in
video footage involved 1) preparing a stereo image
from the movies of two cameras, 2) measuring the
location of the individual in the images and 3) us-
ing simple geometry to obtain the estimate of posi-
tion in three dimensions. I describe each of these
steps below, as we performed them during a study
of mosquito swarms in Mali [Manoukis et al.(2009)].
Figure 1 gives an illustration of the camera orienta-
tions and how they are related to world coordinates.
I will use that notation throughout the following dis-
cussion.
The camera lenses are aligned so as to present
a parallel projection. I could identify individual
mosquitoes (correspond) using the epipolar con-
straint method [Hartley and Zisserman(2004)] and
by stepping through the images to confirm that both
points representing a single individual were moving
in similar directions.
2 Creating stereo images
Our Sony HDR-HC7 high definition digital cam-
corders (Sony Corp., Tokyo Japan) produced HDV
compressed video files which could be transferred
to the computer. These files are not accessible
via most free video processing applications so I de-
muxed them using ProjectX (http://project-x.
sourceforge.net/) to extract an MPEG-2 com-
pressed video stream.
The MPEG-2 compressed clips could be used
directly with StereoMovieMaker (http://stereo.
jpn.org/eng/stvmkr/). This application allowed
fine adjustments and verification of time synchroniza-
tion between clips, as well as basic modification of the
levels of each image.
1
N
at
ur
e 
Pr
ec
ed
in
gs
 : 
hd
l:1
01
01
/n
pr
e.
20
08
.2
59
9.
1 
: P
os
te
d 
3 
De
c 
20
08
Figure 1: Camera and axis orientation
Once alignment was confirmed to be ac-
curate, greyscale stereo movies were rendered
in an intermediate, losslessly compressed for-
mat (Huffyuv: http://neuron2.net/www.math.
berkeley.edu/benrg/huffyuv.html). These stere-
omovies showed mosquitoes from the right cam-
era’s footage in cyan and those from the left in
red. I extracted still images from these stereo
movies at the native frame rate of 29.97 images per
second using VirtualDub (http://www.virtualdub.
org/) or Cinelerra (http://cinelerra.org/); these
formed the basis for the analysis described in
[Manoukis et al.(2009)] and the visualization pre-
sented below.
3 Localizing individuals
Localization of individual mosquitoes in three dimen-
sional space once stereo images were available con-
sisted of two discrete steps. First, I estimated the
distance from the cameras of the mosquito using tri-
angulation on measurements taken from the stereo-
scopic image. Second, I utilized a pin-hole camera
model on one of the images from the stereo pair to
determine the ray along which the mosquito lay. Us-
ing the estimate of distance and knowing the direc-
tion, I could obtain an estimate of the position of
the individual in three dimensions. I describe each of
the steps in detail below, first describing the camera
model, as this is fundamental to the discussion below.
3.1 Camera model
The camera model I used is shown graphically in Fig-
ure 2. I require f , the focal length of the lens in mm,
the size of the sensor (U = horizontal size in mm, V
= vertical size in mm), the position of a mosquito in
the image ({up, vp}, in pixels) and the number of pix-
els in the image horizontally and vertically (hr and
vh, respectively).
Figure 2: Camera model
I first calculate u, v in mm, as follows; note that
u is relative to the image center, which I defined as
{hr/2, vr/2}
u = up
vh
U
(1)
v = vp
hh
V
(2)
3.2 Estimating positions
Distance from the camera (Z, Figure 2)was estimated
using simple triangulation, as shown below
Z =
bf
ul − ur (3)
where b is the baseline (distance between the optic
centers of each camera, in m), f is the focal length
(in m) and ul and ur are the size of u for the left
and right cameras, respectively, in m. I note that
since the projections are parallel, I expect that vl =
vr (where vl, vr are v for the right and left cameras).
Once Z is known, X and Y could be estimated as
follows:
2
N
at
ur
e 
Pr
ec
ed
in
gs
 : 
hd
l:1
01
01
/n
pr
e.
20
08
.2
59
9.
1 
: P
os
te
d 
3 
De
c 
20
08
X = ur
Z
f
(4)
Y = vr
Z
f
(5)
I used the right camera for u and v, as shown above,
but the left also could have been used.
4 Software availability
An R [R Development Core Team(2008)]function, to
calculate positions from stereo measurements, cal-
cxyz.R is available from http://exon.niaid.nih.
gov/3dswarms. Also available at that site is a conve-
nience plugin for ImageJ which may be used to easily
gather these data.
5 Swarm visualizations
I have prepared two complementary types of visual-
izations of the swarm structure and dynamics based
on our first results [Manoukis et al.(2009)]. The first
are manipulable virtual reality “worlds” which rep-
resent the locations at which individual mosquitoes
were seen relative to the swarm centroid at that mo-
ment. The mosquitoes are colored to represent their
proximity to the nearest neighbor at the moment
when they were seen. A sample is given below, in
Figure 3.
The second set of visualizations are a pair of an-
imated plots for each swarm depicting the locations
at which mosquitoes were seen over time. In the first
plot, “Positions per time step”, mosquitoes appear in
the configurations they were seen for each image ana-
lyzed. In the second, “Accumulation of positions over
time”, mosquitoes appear as they did in each image
but are not removed between images. This allows a
rough comparison of where individuals were being lo-
calized relative to those seen in other images. Playing
both files at approximately the same time is helpful to
understanding the positions where mosquitoes were
observed.
3D visualizations for the remaining swarms and
movies are available on line at the following URL:
http://exon.niaid.nih.gov/3dswarms/Vis
6 Conclusion
I have presented a streamlined method for localiz-
ing flying insects in 3D space based on simple prin-
ciples of triangulation and a basic model of camera
optics. In addition, I have provided a sample visual-
ization based on the first analysis using this method
[Manoukis et al.(2009)], which has provided some in-
sight into mating aggregations of Anopheles gambiae,
a vector of medical importance. It is my hope that
the approach described here will be useful to other
researchers studying flying insects or perhaps other
taxa.
7 Acknowledgments
Sincere thanks to the many colleagues involved in the
study of An. gambiae swarms for their patience and
inspiration, especially Abdoulaye Diabate, Adamou
Abdoulaye, Moussa Diallo, Adama Dao, Alpha S.
Yaro, Jose´ M.C. Ribeiro and Tovi Lehmann. I would
also like to thank Daniel C. Medina for helpful dis-
cussions on stereoscopic image analysis, mosquito be-
havior and history.
References
[Hartley and Zisserman(2004)] R. I. Hartley and
A. Zisserman. Multiple View Geometry in Com-
puter Vision. Cambridge University Press, ISBN:
0521540518, second edition, 2004.
[Manoukis et al.(2009)] N.C. Manoukis, A. Diabate,
A. Abdoulaye, M. Diallo, A. Dao, A.S. Yaro,
J.M.C. Ribeiro, and T. Lehmann. Structure and
dynamics of male swarms of Anopheles gambiae.
Journal of Medical Entomology, In Press, 2009.
[R Development Core Team(2008)] R Development
Core Team. R: A Language and Environment
for Statistical Computing. R Foundation for Sta-
tistical Computing, Vienna, Austria, 2008. URL
http://www.R-project.org. ISBN 3-900051-07-
0.
3
N
at
ur
e 
Pr
ec
ed
in
gs
 : 
hd
l:1
01
01
/n
pr
e.
20
08
.2
59
9.
1 
: P
os
te
d 
3 
De
c 
20
08
Figure 3: Cumulative 3D locations of individuals detected in the swarm of 16 September 2007. All positions are given relative
to the swarm centroid at the moment of detection , to control for the effect of camera movement. Colors indicate the natural
logarithm of the distance to nearest neighbor (DNN): Red = closer, blue = further. Axis orientation relative to camera position
is as follows: Y=vertical axis, X=horizontal, left/right and Z=horizontal, distance from the camera At this time you must use
Adobe Reader 8 or later in order to view and manipulate the figure. You can download this software for many platforms free
of cost from the following URL: http://www.adobe.com/products/reader. A fast computer and accelerated graphics card are
helpful to seeing the figure move clearly.
4
N
at
ur
e 
Pr
ec
ed
in
gs
 : 
hd
l:1
01
01
/n
pr
e.
20
08
.2
59
9.
1 
: P
os
te
d 
3 
De
c 
20
08


A Language and Environment for Statistical Computing. R Foundation for Statistical Computing,

Multiple View Geometry in Computer Vision.

Structure and dynamics of male swarms of Anopheles gambiae.

Description of a method for localizing swarming mosquitoes and other insects in 3D space with visualizations

Male mosquitoes swarm to find mates, but the characteristics of these swarms have not yet been quantitatively analyzed in the field. This paper describes a simple method used recently to localize individual Anopheles gambiae in a swarm via stereoscopic image analysis of video footage swarms recorded in Donéguébougou, Mali. The methods described here represent a streamlining of simple triangulation for insect localization in the hope that other researchers will be able to apply the method for studying fundamental questions about flying insects or other taxa

CiteSeerX

Description of a Method for Localizing Swarming Mosquitoes and Other Insects in 3D Space with Visualizations

http://precedings.nature.com/documents/2599/version/1/files/npre20082599-1.pdf

Description of a method for localizing swarming mosquitoes and other insects in 3D space with visualizations

Abstract

Similar works

Full text

Available Versions

Nature Precedings

CiteSeerX