This paper describes how we deploy video surveillance techniques to monitor the activities within a sensor network in order to detect environmental events. This approach combines video and sensor networks in a completely different

way to what would be considered the norm. Sensor networks

consist of a collection of autonomous, self-powered

nodes which sample their environment to detect anything

from chemical pollutants to atypical sound patterns which

they report through an ad hoc network. In order to reduce

power consumption nodes have the capacity to communicate

with neighbouring nodes only. Typically these communications

are via radio waves but in this paper the sensor nodes communicate to a base station through patterns emitted

by LEDs and captured by a video camera. The LEDs are chemically coated to react to their environment and on doing so emit light which is then picked up by video analysis.

There are several advantages to this approach and to demonstrate we have constructed a controlled test environment.

In this paper we introduce and briefly describe this

environment and the sensor nodes but focus mainly on the

video capture, image processing and data visualisation techniques

used to indicate these events to a user monitoring the

network

Beirne, Stephen

Cooke, Eddie

Corcoran, Brian

Diamond, Dermot

O'Connor, Noel E.

Shepherd, Roderick

Smeaton, Alan F.

English

DCU Online Research Access Service

VIDEO ANALYSIS OF EVENTS WITHIN CHEMICAL SENSOR NETWORKSE. Cooke†, N.E. O’Connor†, A.F. Smeaton†, D. Diamond‡, R. Shepherd‡, S. Beirne∗, and B. Corcoran∗†Centre for Digital Video Processing ‡National Centre for Sensors Research∗School of Mechanical EngineeringAdaptive Information Cluster, Dublin City UniversityGlasnevin, Dublin 9, IrelandABSTRACTThis paper describes how we deploy video surveillance tech-niques to monitor the activities within a sensor network inorder to detect environmental events. This approach com-bines video and sensor networks in a completely differentway to what would be considered the norm. Sensor net-works consist of a collection of autonomous, self-powerednodes which sample their environment to detect anythingfrom chemical pollutants to atypical sound patterns whichthey report through an ad hoc network. In order to reducepower consumption nodes have the capacity to communi-cate with neighbouring nodes only. Typically these com-munications are via radio waves but in this paper the sensornodes communicate to a base station through patterns emit-ted by LEDs and captured by a video camera. The LEDsare chemically coated to react to their environment and ondoing so emit light which is then picked up by video anal-ysis. There are several advantages to this approach and todemonstrate we have constructed a controlled test environ-ment. In this paper we introduce and briefly describe thisenvironment and the sensor nodes but focus mainly on thevideo capture, image processing and data visualisation tech-niques used to indicate these events to a user monitoring thenetwork.1. INTRODUCTIONSensor networks represent the development of communitiesof small, autonomous, self-powered sensor devices whichhave limited communications capability using wireless net-works but which can be deployed as collaborating commu-nities of nodes for low-bandwith monitoring and alert de-tection [12]. The current dominant approach to developingsensor networks is to configure groups of nodes, such asBerkeley motes [1]. Within such an network, along with thewireless messaging, each node has to be capable of dealingwith security issues and the networking protocols to be sup-ported [11], as well as the actual sensing technology to beembedded within the device, i.e. chemical sensing [5], au-dio sensing etc. This requires that each node has the com-puting power to assemble the communications infrastruc-ture and, more importantly, the battery power to consistentlyand reliably sustain such a network.In a conventional mote-based sensor node one of thebiggest users of power is the energy for communications.In this paper we present a network where communication isbased on light signals emitted by LEDs. The LEDs them-selves are chemically coated with sensing chemicals whichreact by changing colour on detection of whatever it is theyare monitoring. Such LED signals may be picked up by anexisting network of CCTV cameras where coverage of theenvironment to be monitored is already in place and wherethe capacity to detect “events” can be based on a combina-tion of LED signals from nodes and video processing of theoutputs of the CCTV network. Such coverage of the envi-ronment is certainly in place within the built environmentsuch as within city centres, towns, homes, strategic build-ings where people will congregate including subways, trainstations, airports, etc. This approach has the advantage ofreducing the cost of deployment since individual nodes willnot be required to create and maintain their own commu-nications infrastructure and so they can be built as simpler,cheaper, and less power consuming devices. We have builta laboratory-based simulation environment where we canlocate LED-based sensor network nodes and introduce gasplumes in a controlled manner in order to test the detectionand communication between nodes and the monitoring sta-tion.The rest of this paper is organised as follows; in Sec-tion 2 we give an outline of the sensor nodes we are usingand the environmental monitoring chamber we have builtfor testing. Section 3 gives the details of the video process-ing we perform in order to detect signals from the individ-ual nodes and covers the video capture, video processingand output display we have constructed. In Section 4 wepresent experimental results. While Section 5 outlines ourconclusions and future work.2. SYSTEM SET-UP2.1. Sensor NodesThe nodes we have developed are constrained by: process-ing power, amount of storage space, and the amount of powerthey can consume. These nodes use Mica2Dot MPR500CAmote devices [1] whose processors run at 4MHz. The sen-sors are powered using CR2354 3V Li coin cell batterieswith a capacity of around 560mA-hr and have a lifetime weestimate as follows. If we assume the duty cycle of a moteis that it is sleeping for 99% of the time and using about17uA/1% when it is active and using 22-25mA for the mainprocessor, radio and sensor. Then the mote and all extrasconsumes about 0.237mA-hr and on this basis a 250mA-hrbattery will have about 1.45 months of life, while the coincells we use extend this to about 3 months.The sensor comprises of 2× 1206 surface mount LEDs,both coated with a PVC polymer film containing a pH in-dictor dye, and use at most 5mA. The basic concept be-hind the sensing is that this PVC compound will sense somechange in the environment. For our environmental monitor-ing chamber the event we detect is a change in pH which weachieve by introducing acetic acid into the chamber. A pHindictor in the PVC coating, coated directly onto the lensof the LEDs, changes colour when acetic acid is introduced,and this change affects the filtering of emitted light from theLED. The way in which the chemical sensing uses the pairof LEDs on each node is that the circuitry in the node en-ables the LEDs to be switched between detect (reverse bias)or emit (forward bias) modes [7]. One LED is coated witha chemo-sensitive polymer coating incorporating a pH sen-sitive dye such as bromocresol green. This LED is operatedin detect mode to sense the status of the colour film and isinitially blue. A change in the external environment due tothe diffusion of a chemical plume generates a colour changein the chemo-sensitive film on the detector LED. This is de-tected by the circuit which turns on the other, emitter LED.Simple low-cost, low power platforms like this can be usedto detect a broad variety of chemical and biological speciesusing colorimetric approaches that are already well known[2, 8].2.2. Environmental Monitoring ChamberThere are two objectives behind the development of an en-vironmental monitoring chamber. The first, and more am-bitious, is the development of a controlled environment thatcan be used as a test rig to map and model fluid dynam-ics using a sensor network. Sensors are dispersed withinthe chamber and their detection of the spread of chemicalevents can be observed and analysed. The second objec-tive behind developing the chamber is to use it as a testbedfor the development of sensor nodes, in particular using thecombined LED sensing and communications approach. Thechamber is constructed from bonded 10mm thick clear Per-spexł sheets and has a dimension of 2m×1m×1m. Trans-parency of the chamber walls facilitates the use of cameraequipment to capture the sensor LEDs’ responses. A ren-dered representation of the designed test chamber is shownin Fig 1(a).(a) (b)Fig. 1. (a) Schematic of the Environmental MonitoringChamber. (b)Vector plot of the velocity dispersement of aplume introduced into the chamber.One important line of research which we are using thechamber for, is to study the dispersement and spread of gasplumes and our current test data have shown that it is possi-ble to activate the sensor devices by passing an acetic acidbased plume over them. This indicates that it will be pos-sible to use the sensors to map and predict plume develop-ment. A predicted vector plot example of the disbursementof a plume is shown in Fig 1(b) and by placement of sensorsat strategic points we would expect to confirm the disburse-ment by actual readings at such points.3. VIDEO MONITORING OF A SENSORNETWORKIt is worth noting that detecting plumes of acetic acid in aperspex chamber is far removed from monitoring and de-tecting real environmental events in the field. However,whilst our monitoring approach does not necessarily scaleto all unconstrained real-world situations, technological ad-vances and declining costs in capture technology, as wellas heightened security concerns have led to a rapid diffu-sion of CCTV surveillance in recent times [9]. Within Eu-rope, circa 30% of publicly accessible space, e.g. shops,banks, restaurants etc., contain some form of video surveil-lance [10]. Such camera networks could conceivably beused in the future for the kind of data harvesting proposedhere. In this section, we describe our capture system, howimage processing techniques can be used to identify sensoractivity, and the client-server architecture of the system de-veloped to facilitate low-cost low-bandwidth disseminationof the sensed data back to stake holders.3.1. Video ProcessingThe goal of video processing in monitoring the chamber isto identify when an LED is triggered to an on/off state. Inorder to capture video we use a low-end VGA 640 × 480resolution CCD camera, which is typical of most CCTVsurveillance set-ups. The camera used is equipped with a6mm lens and a polarized filter in order to reduce visualnoise due to lighting reflections from the chamber’s trans-parent and reflective walls.In general, CCD cameras images are represented withinthe RGB colour space. In an image pixel values representa point in this three dimensional RGB space and thereforethe difference between two colours can be determined as adistance within this space. The problem with RGB is thatthese distance measures are very different to the perceptionof the human eye and are susceptible to noise in the image.Hence, colours which are close together for the human vi-sual system aren’t necessarily close in RGB space and viceversus. For this reason the first step we perform is to con-vert the image to the CIELAB perceptually uniform colourspace. This space is optimised according to the perceptionof the human eye. This transformation involves convertingRGB to XYZ, the tristimulus values of CIE, using Eq (1).Then these values are transformed into CIELAB using Eq(2), where X0,Y0 and Z0 are the illumination tristimulusvalues, further details about this conversion can be foundin [4].X = 0.4303R+ 0.3416G+ 0.1784BY = 0.2219R+ 0.7068G+ 0.0713BZ = 0.0202R+ 0.1296G+ 0.9393B(1)L∗ = 116f(YY0)− 16a∗ = 500[f(XX0)− f(YY0)]b∗ = 200[f(YY0)− f(ZZ0)]wheref(q) = 3√q q > 0.008856f(q) = 7.787q + 16116 q ≤ 0.008856(2)The next step is to define a model within this space forthe colours of the LEDs we wish to identify. In order toincrease the flexibility of the filter we provide an initialisa-tion/training phase whereby valid filter colours within theimage can be defined by a superuser in order to build therequired filter. After the filter is defined all pixels in subse-quently captured frames can be classified on the basis of theLED model.The colour filter allows us to extract an area of inter-est (AOI), representing the coloured LED, from the back-ground. In the initialisation phase, the colour intensity ofLEDs in the on/off states is measured and a mean valuecomputed. We then generate a histogram of the intensity(a) (b)(c) (d)Fig. 2. (a) Manual refinement of automatically detect LEDlocations. (b) Set-up of information to be sent to client. (c)Client status. (d) 3D view as seen on client side.values over time within the various AOI. Analysis of lo-cal minimums and maximums within this histogram allowsus to determine the LEDs current state, i.e. on or off, in-dicating the triggering of specific events in the monitoredenvironment.3.2. Capture Control System and Data VisualisationThe design of the interface to control the video capture sys-tem and display sensed data follows a client-server architec-ture. This facilitates full system control at the server whilstsupporting multiple remote clients for viewing the output ofthe video capture system. At the server side, full control isprovided over the initialisation phase that is run once priorto any experimentation. This allows setting of various de-tection thresholds (as outlined above) and visualisation andmanual adjustment of automatically detected sensor loca-tions within the chamber. This is shown in Fig 2(a). Theinterface also provides statistics on the data and this can belogged during an experiment (e.g. colour, 2D/3D coordi-nates, etc) – see Fig 2(b). Finally, information on client dataviewers connected to the system is also provided – Fig 2(c).The output of video processing, i.e. the location and sta-tus of LEDs, is visualised via a 3D wireframe virtual modelof the chamber. This allows user navigation in three dimen-sions around the chamber, allowing free view point inspec-tion of the status of the sensors. This approach to visualisa-tion provides the user with more flexibility whilst requiringa much lower bandwidth transmission link than the origi-nal images. The interface, with multiple LEDs detected, isillustrated in Fig 2(d).(a) (b) (c) (d)Fig. 3. (a) Image of environmental chamber with 6 LEDs. (b) Yellow boxes indicate identified LEDs. (c) Environmentalchamber taken from a different viewpoint. (d) Correctly identified LEDs are indicated by yellow boxes.4. EXPERIMENTAL RESULTSIn Fig 3(a) we present an example of the scenario where6 LEDs are in the on state. Fig 3(b) illustrates the identi-fied LEDs by placing yellow boxes around them. Fig 3(c)presents the environmental chamber from another viewpointand correctly detects a different LED arrangement as indi-cated by the yellow boxes in Fig 3(d).5. CONCLUSIONSIn this paper we have introduced the idea of a sensor net-work for environmental monitoring that communicates backto a base station via video monitoring of LED signals emit-ted by the sensor nodes. We have demonstrated this work-ing in a chemical chamber which is now operational with adozen sensors in 3D space. The sensors have a chemically-coated LED for detection and messaging. Identification ofthe LEDs is done based on colour and analysis of intensityhistograms within identified AOI. We have also presented acapture and control system which provides information onvarious sensor parameters as well as 3D data visualisationfor remote monitoring. The advantages of this system arethat our nodes require much less power than radio-basedsolutions, a large number of CCTV networks already existin built-up areas, and the costs of CCTV installation andmonitoring are comparatively cheap.Our future work is in two directions: to deploy our sen-sors for actual environmental monitoring in operational sit-uations such as landfill sites, and to investigate using a ther-mal infra-red camera for video monitoring of the sensornodes. We will also continue to research technology for usein the chemical sensing which can be used to coat the LEDsand investigate security aspects as well. §6. REFERENCES[1] G. Anastasi, A. Falchi, A. Passarella, M. Conti, and E. Gre-gori. Performance measurements of motes sensor networks.§This work is partly supported by Science Foundation Ireland undergrant 03/IN.3/I361.In MSWiM ’04: Proceedings of the 7th ACM internationalsymposium on Modeling, analysis and simulation of wirelessand mobile systems, pages 174–181, New York, NY, USA,2004. ACM Press.[2] S. Baldwin, K. T. Lau, W. Yerazunis, R. Shepherd, and D. Di-amond. Colorimetric detection of iron (II) using novel pairedemitter detector diode (PEDD) based optical system. IEICETrans. Electron., E87-C(12):2099–2102, 2004.[3] S. Brady, D. Diamond, and K.-T. Lau. Inherently conduct-ing polymer modified polyurethane smart foam for pressuresensing. Sensors and Actuators A : Physical, 119(2):398–404, April 2005.[4] C. Connolly and T. Fleiss. A study of efficiency and accuracyin the transformation from rgb to cielab color space. IEEETransactions on Image Processing, 6(7):1046–1048, 1997.[5] D. Diamond. Internet scale sensing. Analytical Chemistry,76:278A–286A, 2004.[6] L. E. Dunne, S. Brady, B. Smyth, and D. Diamond. Initialdevelopment and testing of a novel foam-based pressure sen-sor for wearable sensing. J. Neuroengineering Rehabil., 2(1),2005.[7] K. T. Lau, S. Baldwin, R. L. Shepherd, P. H. Dietz, W. S.Yerazunis, and D. Diamond. Novel fused-LED devices asoptical sensors for colorimetric analysis. Talanta, 63:167–173, 2004.[8] K.-T. Lau, E. McHugh, S. Baldwin, and D. Diamond. Pairedemitter-detector LEDs for the measurement of Lead II andCadmium II. Sens. Actuator B, in press, 2005.[9] M. Nieto, K. Johnston-Dodds, and C. Simmons. Public andPrivate Applications of Video Surveillance and BiometricTechnologies. Californian Research Bureau, March 2002.[10] C. Norris, M. McCahill, and D. Wood. The growth of cctv:a global perspective on the international diffusion of videosurveillance in publicly accessible space. The Journal ofSurveillance and Society, 2 Summer:110–135, 2004.[11] A. Perrig, J. Stankovic, and D. Wagner. Security in wirelesssensor networks. Commun. ACM, 47(6):53–57, 2004.[12] R. Szewczyk, E. Osterweil, J. Polastre, M. Hamilton,A. Mainwaring, and D. Estrin. Habitat monitoring with sen-sor networks. Commun. ACM, 47(6):34–40, 2004.

A study of efﬁciency and accuracy in the transformation from rgb to cielab color space.

Habitat monitoring with sensor networks.

Inherently conducting polymer modiﬁed polyurethane smart foam for pressure sensing. Sensors and Actuators A : Physical,

Initial development and testing of a novel foam-based pressure sensor for wearable sensing.

Internet scale sensing. Analytical Chemistry,

Novel fused-LED devices as optical sensors for colorimetric analysis.

Paired emitter-detector LEDs for the measurement of Lead II and Cadmium II. Sens. Actuator B, in press,

Performance measurements of motes sensor networks. §This work is partly supported by Science Foundation Ireland under grant 03/IN.3/I361.

Public and Private Applications of Video Surveillance and Biometric Technologies. Californian Research Bureau,

Security in wireless sensor networks.

The growth of cctv: a global perspective on the international diffusion of video surveillance in publicly accessible space.

W.Yerazunis, R.Shepherd, andD.Diamond. Colorimetric detection of iron (II) using novel paired emitter detector diode (PEDD) based optical system.

Video analysis of events within chemical sensor networks

This paper describes how we deploy video surveillance techniques to monitor the activities within a sensor network in order to detect environmental events. This approach combines video and sensor networks in a completely different\ud
way to what would be considered the norm. Sensor networks\ud
consist of a collection of autonomous, self-powered\ud
nodes which sample their environment to detect anything\ud
from chemical pollutants to atypical sound patterns which\ud
they report through an ad hoc network. In order to reduce\ud
power consumption nodes have the capacity to communicate\ud
with neighbouring nodes only. Typically these communications\ud
are via radio waves but in this paper the sensor nodes communicate to a base station through patterns emitted\ud
by LEDs and captured by a video camera. The LEDs are chemically coated to react to their environment and on doing so emit light which is then picked up by video analysis.\ud
There are several advantages to this approach and to demonstrate we have constructed a controlled test environment.\ud
In this paper we introduce and briefly describe this\ud
environment and the sensor nodes but focus mainly on the\ud
video capture, image processing and data visualisation techniques\ud
used to indicate these events to a user monitoring the\ud
network

Name not available

This paper describes how we deploy video surveillance techniques to monitor the activities within a sensor network in order to detect environmental events. This approach combines video and sensor networks in a completely different
way to what would be considered the norm. Sensor networks
consist of a collection of autonomous, self-powered
nodes which sample their environment to detect anything
from chemical pollutants to atypical sound patterns which
they report through an ad hoc network. In order to reduce
power consumption nodes have the capacity to communicate
with neighbouring nodes only. Typically these communications
are via radio waves but in this paper the sensor nodes communicate to a base station through patterns emitted
by LEDs and captured by a video camera. The LEDs are chemically coated to react to their environment and on doing so emit light which is then picked up by video analysis.
There are several advantages to this approach and to demonstrate we have constructed a controlled test environment.
In this paper we introduce and briefly describe this
environment and the sensor nodes but focus mainly on the
video capture, image processing and data visualisation techniques
used to indicate these events to a user monitoring the
network

O\u27Connor, Noel E.

Irish Universities

Video analysis of events within chemical sensor networks

Abstract

Similar works

Full text

Available Versions

DCU Online Research Access Service

Name not available

Irish Universities