Oak Ridge National Laboratory staff have developed a video compression system for low-bandwidth remote operations. The objective is to provide real-time video at data rates comparable to available tactical radio links, typically 16 to 64 thousand bits per second (kbps), while maintaining sufficient quality to achieve mission objectives. The system supports both continuous lossy transmission of black and white (gray scale) video for remote driving and progressive lossless transmission of black and white images for remote automatic target acquisition. The average data rate of the resulting bit stream is 64 kbps. This system has been demonstrated to provide video of sufficient quality to allow remote driving of a High-Mobility Multipurpose Wheeled Vehicle at speeds up to 15 mph (24.1 kph) on a moguled dirt track. The nominal driving configuration provides a frame rate of 4 Hz, a compression per frame of 125:1, and a resulting latency of ~1s. This paper reviews the system approach and implementation, and further describes some of our experiences when using the system to support remote driving

DePiero, Fred W.

Noell, Timothy E.

English

DigitalCommons@CalPoly

REDUCED BANDWIDTH VIDEOFORREMOTE VEHICLE OPERATIONS*Submitted toAUVS-93 Unmanned Systems: ExpandingGovernment and Commercial ApplicationsWashington, D.C.June 28-30, 1993*Research sponsored by the Army Research Laboratory, Office of Technology Development, and theU.S. Department of Energy under contract DE-ACOS-84OR21400 with Martin Marietta Energy Systems, Inc.REDUCED BANDWIDTH VIDEOFORREMOTE VEHICLE OPERATIONS*Timothy E. NoeU and Frederick W. DePieroRobotics & Process Systems DivisionOak Ridge National LaboratoryP.O. Box 2008Oak Ridge, Tennessee 37831-6304ABSTRACTOak Ridge National Laboratory staff have developed a video compression system forlow-bandwidth remote operations. The objective is to provide real-time video at data ratescomparable to available tactical radio links, typically 16 to 64 thousand bits per second(kbps), while maintaining sufficient quality to achieve mission objectives. The systemsupports both continuous lossy transmission of black and white (gray scale) video forremote driving and progressive lossless transmission of black and white images for remoteautomatic target acquisition. The average data rate of the resulting bit stream is 64 kbps.This system has been demonstrated to provide video of sufficient quality to allow remotedriving of a High-Mobility Multipurpose Wheeled Vehicle at speeds up to 15 mph(24.1 kph) ,_n a moguled dirt track. The nominal driving configuration provides a framerate of 4 Hz, a compression per frame of 125:1, and a result.ing latency of ~ls. This paperreviews the system approach and implementation, and further describes some of ourexperiences when using the system to support remote driving.BACKGROUNDThe use of untethered teleoperated vehicles for many remote operations is greatlylimited because of a need to use low-bandwidth communication links. For example,tactical operations that require a low signature give rise to low-bandwidth requirements.Low-bandwidth channels are also encountered in underwater operations and in spaceapplications. The most notable difficulty in using low-bandwicith channels for vehiclecontrol and mission package support is the problem of video transmissions from thevehicle to the driver's station. Namely, standard black and white video requires*Research sponsored by the Army Research Laboratory, Office of Technology Development, and theU.S. Department of Energy under contract DE-AC05-84OR21400 with Martin Marietta Energy Systems, Inc.-60 million bits per second. Currently available tactical communication links support only16 to 64 thousand bits per second (kbps). This equates to a minimum video compressionrequirement of -1000:1 for remote operations via these types of low-bandwidth channels.SUMMARY OF APPROACHThe Oak Ridge National Laboratory (ORNL) system [1] achieves video compressionby combining a multi-resolution decomposition (a Laplacian pyramid), simplified motionestimation, scalar quantization, foveation, zero-run-length coding, Huffman entropycoding, and frame rate reduction. A more complete description and discussion of thisapproach can be found in reference [2].The Laplacian pyramid approach to ima_ coding [3] decomposes an image into a setof subbands, each containint' a separate spatial frequency band. The generat_ion of subbandsis a process of recursive bandpass filtering A uniform quantizer is then applied to eachpixel of each subband. Subbands can be quantized more or less aggressively to provide afrequency selective distribution of the loss of image information.Foveation [4] provides a region of higher image quality in the resulting image. Thisis achieved by simply masking data from each subband as desired. The areas where alisubbands are included is the area of highest quality for a given set of quantizers. Variouseffects can be achieved with different subsets of the subbands present in any given area ofthe imaf_e. In general, the system is set up to provide the highest quality in the center ofthe field of view, with decreasing resolution at points removed from the center region.The system also provides dynamic adjustments that allow an operator to position thefoveal area on the output image. This effectively allows the operator to allocate bandwidthto the operationally significant areas of the scene, ihgure 1 shows a quantized and foveatedLaplacian pyramid together with a compressed and an original image. This image is takenfrom a 64-kbps compressed video bit stre_,n during a demonstration of the system at theU.S. Army's Churchville test facility near Churchville, Maryland, in May of 1992.Simple motion estimation is implemented by predicting the next instance of asubband with the current instance. When the next subband is calculated, the predictionerror is determined by subtraction, and only the error is transmitted. The decompressionalgorithm does this same prediction and adds the error to reconstruct the subband. Thesystem also implements a periodic transmission of an intact subband that serves to ensurea temporal limitation of channel errors. Experiments were performed to determine whichsubbands would provide a net compression gain if predicted in this manner and todetermine _ suitable refresh period. The final system used subband prediction on only thetwo lowest frequency bands because their image content varied slowly enough to makecompression gains realizable. A four-frame interval was chosen so that the temporalduration of a channel error was limited to 1 s (given ~4-Hz nominat frame rate).Fig. 1. An original image (upper left), a compressed image (upper right), anda quantized and foveated Laplacian pyramid.Finally, the quantized and foveated pyramid is zero-run-length encoded and thencompressed with a Huff'man entropy coder [5]. This maps the variable-sized portions ofthe subbands into a bit stream of variable-length code words. The Huffman code wordassignment is determined by an a priori analysis of representative driving imagery.The decompression side of the system decodes the bit stream, adds the predictionerrors (if appropriate), and paints the reconstructed pyramid into a buffer. The pyramid isthen "collapsed" by a series of expand and oFerations [3].IMPLEMENTATIONThe implementation of the ORNL system consists of two VersaModule European(VME) racks, one for compression and one for decompression. The compression rack ismounted on board a High-Mobility Multipurpose Wheeled Vehicle (HMMWV) outfittedfor teleoperation. The decompression rack is mounted in an environmental enclosureadjacent to a VME-based Sparc station. An operator interface runs on the Sparc, providingvehicle control functions. The driving station and teleoperated HMMWV were developedby the Information Processing Branch of the Army Research Laboratory (ARL), Adelphi,Maryland [6].Figure 2 shows the basic hardware components in the overall system. Six VMEsingle-board computers are arranged in a pipelined architecture, three in the compressionrack, and three in the decompression rack. Each rack also contains Datacube imageprocessing hardware. The first processor on board the vehicle (CPU 1) is responsible forcontrolling a Datacube image-processing board that provides real-time generation of thequantized and foveated Laplacian pyramids. CPU 2 performs the zero-run-length andHuffman coding. The third processor interfaces to a packet radio. The processors in thedecompression rack perform symmetrical functions. A more detailed description of thesystem implementation can be found in reference [2].The packet radios used are the ARLAN 620 Ethernet radios from Telesystems, Inc.,Toronto, Cat, ada. These radios use a spread-spectrum type of modulation and operate inthe 902- to 928-MHz band. The units provide a wireless Ethernet bridge between the VMEsystems. The low-bandwidth communication channel is emulated by using a real-timeclock to maintain an average data rate of 64 kbps. The channel's data rate is alsomonitored at the receiving unit for verification.When in driving mode with a 64-kbps channel, this implementation provides anominal performance of-125:1 compression per frame, 4 frames per second, and aprocessing latency of -1 s (roughly 0.4 s for compression processing, 0.3 s for low-bandwidth transmission, and 0.3 s for decompression processing). Ibis performance hasdemonstrated capabilities of supporting remote operation of the HMMWV at speeds up to15 mph (24.1 kph) on a moguled dirt track.The ORNL compression system also provides progressive transmission of stillimages and subimages in support of an Automatic Target Acquisition (ATA) missionpackage on board the HMMWV. The ATA system was developed also by the InformationProcessing Branch of the ARL [6]. At the start of targeting operations, the ATA systempasses to the compression rack seven images that cover the area to be monitored. Theseare transmitted at full quality and stored in video buffer:; in the decompression rack.When the ATA system acquires targets, it assigns a target number and passes the image,along with the target's pixel coordinates and size, to the compression system. Thecompression system then transmits only the small rectangular portions of the imagescontaining the tracked targets, which are t_pically -40 x 60 pixels. At the operator station,the decompression system pastes targets at the appropriate ons within the appropriateframe buffers. The ATA system keeps the compression system informed as to the pixellocation of a given target. As the target moves, the decompression rack pastes in thebackground at _he old location and pastes the updated target image at the new location.The contents of the tracking buffers are displayed on the operator control station at taeoperator's request. A socket-based custom protocol is used to communicate among theORNL compression system, the operator control station, and the ATA mission package.The system has been demonstrated to be capable of supporting remote ATA at 64 kbps byusing this approach.[_ Camera I linage FIFO IV'IF'_t.--'l pmcosin 8 _ ef ofSubbands Com premedELaFre PacketsTargeting LapLacian _ Encoding liD" RadioSubsystem Sublmnd InterfaceGmertiouand DataFIFO RadioControl Buffer Control BuffertssEQUIPMENTON BOARD VEHICLEmmmmmmmmmmmmmmmm TEQUIPM_TATDmVl'_SSTA_ON ] !_0 ] [ !'_0 ! controlControl Buffer Contrei Buffer and Dats[ Subband Decoding _ RadioEthernet Socket [Recombinatio_ Interface im • A ,, DriverRemote Semaphore " "[" Control- _ i 1_ StationVMEBass I/0Image HFO FIFOLow Bandwidth RF Processing of of..... 4_" Hardware Subbands CompressedPackets lVlordtorFig. 2. Block diagram of Oak Ridge National Laboratoryvideo compression system. (NTSC = National TelevisionStandard Code, and FIFO = First in, first out)DRIVING WITH THE ORNL SYSTEMIn April and May of 1992, the ORNL video compression system was tested anddemonstrated at the Office of the Secretary of Defense's Robotics Testbed Demo I at theU. S. Army's Churchville test facility near ChurchviUe, Maryland. During this andsubsequent testing, many practical lessons were learned about driving with the ORNLvideo compression system.The first task when initially setting up the system was to determine the best trade-offof quality vs frame rate for remote driving. Five parameter settings were determined thatprovided a range of quality vs frame rate operating po'rats at 64 kbps. The parametersinclude the size of each subband's foveal area and the bin width of the quantizers for eachsubband. Codebooks were determined for each subband/quantizer combination tooptimize the compression for each parameter setting. This was done using a video tapefrom the driving camera of the HMMWV while the test course was driven at 10 mph(16.1 kph). This optimization is exhibited by a maximized frame rate for a given parametersetting because the system is designed to transmit images as fast as possible for a givenparameter setting and bandwidth. Frame rates ranged from 1 to 6 frames per second. Itwas quickly determined that the image quality should be somewhat compromised so thatthe frame rate could approach the upper part of this range. The previously discussedsetting that provides a nominal frame rate of 4 Hz with a per frame compression of -125:1was determin.ed to be the most suitable choice for driving with the ORNL system. Thecorresponding latency was determined to be approximately I s.Early attempts at teleoperation made it obvious that speed control would be difficultgiven the 1 s latency. For example, as the operator released the brake, the vehicle began tomove, but this motion was not immediately evident in the low-bandwidth video due tothe latehcy. As the operator compensated, the vehicle soon went faster than desired. Thisled to a series of rapid starts and abrupt stops. Preoccupation with speed control detractedfrom the operator's attention to steering, and driving performance was unacceptable. Thiswas addressed by putting both the HMMWV's transfer case and transmission in theirlowest gears• This provided a more sensi_ve, limited range speed control and allowed theoperator to concentrate on the overall driving task. In general, latency should beminimized and stable. This permits the operator to anticipate driving conditions byvisualizing the vehicle slightly down range in the scene. Future systems would benefitfrom a cruise-control type of speed input as an operator aid.Another prol_lem that had to be overcome was the impact of competing radio-frequency (RF) traffic. The emphasis of the overall program effort was on theteleoperation of vehicles and demonstration of multiple integrated mission packages, noton the demonstration of radio technology. Unfortunately, this resulted in multiple radiosoperating in the 902 to 928-MHz band at the demonstration site. Although the spreadspectrum radios provide some channel separation via their spreading codes, the resultingloss in system performance was unacceptable. Namely, packets containing subbandsbecame garbled if another radio keyed up during the transmission, causing the built infeatures of the ARLAN radios to reject the subband, packet, which initiated aretransmission. This caused a drop in net available bandwidth below 64 kbps, as measuredat the decompression rack's radio interface CPU (CPU 4 in Fig. 2). The impact on thecompression system's performance was a reduced frame rate, and increased latency.To reduce the effects of RF collisions, the ARLAN radios were operated on a higherbandwidth channel. Also, the Ethernet driver of the compression rack's radio interfaceCPU (CPU 3 in Fig. 2) was modified to use a smaller.maximum packet size. However, thesystem still limited the video bandwidth to 64 kbps. These modifications lowered theamount of time the radio was transmitting for each physical packet. This, in turn, loweredthe probability that a packet would be corrupted by a competing radio transmission. Theimpact of the competing RF traffic on the system's performance was significantly reduced,but there were still rare instances when the throughput would suffer. With a formalcommunications plan, this might have been avoided, but it was an issue with theprototype fleet. Hence, it became important that the operator be able to tell when theimage was not being updated at some minimum rate (e.g., 2 Hz). To alert the operator, asmall bar graph would appear if a half second elapsed without a new image. The bar graphgrew in proportion to the age of the current image. When the bar graph appears, theoperator should consider taking an appropriate action, such as slowing down.In June of 1993, additional testing of the system was performed at the AberdeenProving Ground's Unmanned Vehicle Test Site. The test site is a13-acre grass field with sections of gravel road. At this particular site, the gray level of thegravel road and that of the adjacent grass was essentially the same in the black-and-whitedriving image. Even with full-bandwidth black-and-white video, it was difficult for theoperator to discern road from grass. This had not been a issue r,_ the Churchville site,where a dirt road resulted in a different gray level than _he surrounding grass.Experimenting with a number of color and infrared optical filters resulted in only limitedsuccess. This underscores the fact that good source video is very important for imagecompression systems. Adding color capability to the compression system will improveimage understandability but does increase bandwidth requirements. Another possibility isto use an infrared camera. This will not increase bandwidth requirements, but mayimprove image contrast for such operationally significant concerns, such as being able todiscern road from grass.CONCLUSIONSThe ORNL video compression system supports teleoperation of an HMMWV atspeeds up to 15 mph (24.1 kph) on a moguled dirt track. Experience in driving the systemidentified a quality vs frame rate operating point that favors frame rate to be suitable fordriving video over a 64-kbp3 channel. Latency of the system was determined to be --1 s,which affects driving performance. The system also supports remote automatic targetacquisition at 64 kbps. Future work will address improvements to the compressionalgorithm, which will improve latency and quality for a given bandwidth. The use ofcolor and/or infrared will be investigated to improve image contrast whenneeded.ACKNOWLEDGMENTSThe authors express their gratitude to C. M. Shoemaker and T. W. Haduch for theirprogrammatic guidance throughout this project and to the members of the AdvancedSensor Systems Branch of the Army Research Laboratory for their cooperation andexcellent development efforts. We thank Tim Gee of The Georgia Institute of Technologyfor his significant" contributions.REFERENCES1. F. W. DEPIERO, T. E. NOELL, and T. F. GEE, "Remote Driving with Reduced BandwidthCommunication," Proc. Space Operations, Application and Research Symposium,Houston, Tex. (1992).2. F. W. DEPIERO, T. E. NOELL, and T. F. GEE, "A Video Transmisbion System for Low-Bandwidth Remote Driving," Proc. American Nuclear Society Fifth Topical Meeting onRobotics & Remote Systems, Knoxville, Tenn. (1993).3. P. J. BURT and E. H. ADELSON, "The Laplacian Pyramid as a Compact Image Code,"IEEE Trans. Commun. 31, 532--40 (1983)o4. P. J. BURT, "Multiresolution Techniques for Image Representation, Analysis and'Smart' Transmission," Proc. SPIE Conf. 1199: Visual Communications and ImageProcessing IV, Philadelphia, Penn. (1989).5. A. K. JAIN, "Image Data Compression: A Review," Proc. of the IEEE, 69, 349-84 (1981).6. S. BALAKIRSKY, P. DAVID, F. FISHER, P. EMMERMAN, and P. GAYLORD, "Semi-Autonomous Mobile Target Engagement Systems," Proc. AUVS-93 Unmanned Systems:Expanding Government & Commercial Applications, Washington, D.C. (1993).

Reduced Bandwidth for Remote Vehicle Operations

https://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?article=1015&amp;context=eeng_fac

Reduced Bandwidth for Remote Vehicle Operations

Abstract

Similar works

Full text

Available Versions

DigitalCommons@CalPoly