Multi-camera RGB-D systems are becoming popular as sensor setups in Computer Vision applications but they are prone to cause interference between them, compromising their accuracy. This paper extends previous works on the analysis of the noise introduced by interference with new and more realistic camera configurations and different brands of devices. As expected, the detected noise increases as distance and angle grows, becoming worse when interference is present. Finally, we evaluate the effectiveness of the proposed solutions of using DC vibration motors to mitigate them. The results of this study are being used to assess the effect of interference when applying these setups to human motion tracking.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. Plan Propio de Investigación de la UMA. Junta de Andalucía, proyecto TEP2012-53

Fernandez, Mailys

Gonzalez-Jimenez, Antonio Javier

Monroy, Javier

Moreno, Francisco-Angel

Ruiz Sarmiento, José Raúl

English

Moreno-Dueñas, Francisco Ángel

Ruiz-Sarmiento, José Raúl

González-Jiménez, Antonio Javier

Repositorio Institucional Universidad de Málaga

September 2017Analyzing interference between RGB-Dcameras for human motion trackingFrancisco-Angel MORENO a,1, Jose-Raul RUIZ-SARMIENTO a, Javier MONROY a,Maı¨lys FERNANDEZ b and Javier GONZALEZ-JIMENEZ aaMAPIR Group, University of Malaga, Spainb E´cole d’inge´nieur Te´le´com Physique, University of Strasbourg, FranceAbstract. Multi-camera RGB-D systems are becoming popular as sensor setupsin Computer Vision applications but they are prone to cause interference betweenthem, compromising their accuracy. This paper extends previous works on the anal-ysis of the noise introduced by interference with new and more realistic cameraconfigurations and different brands of devices. As expected, the detected noise in-creases as distance and angle grows, becoming worse when interference is present.Finally, we evaluate the effectiveness of the proposed solutions of using DC vibra-tion motors to mitigate them. The results of this study are being used to assess theeffect of interference when applying these setups to human motion tracking.Keywords. RGB-D cameras, interference, Kinect, Asus, vibration motor, humanmotion tracking1. IntroductionNowadays, RGB-D cameras have become very popular sensors in Computer Vision andRobotics due to their direct and fast sensing of depth at a very affordable prize. Amongtheir shortcomings we find their relatively small field-of-view (∼50o) and a limited rangeof operation (∼1m-3.5m), so it is common to find setups with several devices workingjointly to overcome these limitations. Additionally, the combination of multiple devicescan avoid blind zones and potentially produce richer and more accurate scene represen-tations. In these configurations, however, since these cameras are active sensors, theymay interfere to each other, affecting the quality of the data coming from the overlappedareas. This paper presents an analysis of such interference in relation to the distance andangle to the measured surface. We also evaluate the effectiveness of the existing solutionsin the literature [1,2], based on asynchronous vibrations between the cameras to reduceinterference. Although a similar analysis has been carried out in [1], we’ve extended it byalso estimating the depth error in a setup with the cameras pointing with a certain angleto a large and flat surface. This has been repeated for several distances, aiming to repro-duce the conditions of a more realistic scenario. The experiments reveal that the noise ofthe measured points increases with distance and it is larger in the case of the camera notbeing perpendicular to the measured distance. Moreover, interference is shown to affect1Corresponding Author: Francisco-Angel Moreno, MAPIR Group, University of Ma´laga, ETSI Informa´tica,Lab 2.3.6i, Campus de Teatinos, 29071 Ma´laga, Spain; E-mail: famoreno@uma.esSeptember 20170.33D surface with one Kinect camera ONx3D surface with two Kinect cameras ON0.10 0.2 0.4 0.5 0.6 0.30.10 0.2 0.4 0.5 0.60.30.2-0.100.1-0.20.4yyx0.30.2-0.100.1-0.20.4Figure 1. 3D representation of a flat surface (a) without and (b) with interference.even more to the accuracy of the recorded data, although its effect can be significantlymitigated by using a DC vibration motor attached to one of the cameras. This analysisrepresents a primary step aimed to evaluate the effect of the interference when applyingmultiple RGB-D cameras to perform human motion tracking, this being of special in-terest in disciplines as physiotherapy or rehabilitation. Due to space limitations, though,this further analysis will be addressed in the full version.2. BackgroundKinect-type cameras, as the ones used in this paper, project a known light pattern on thescene that is deformed by the objects within it. This pattern is then captured by an infrared(IR) sensor and compared with the emitted one, hence being able to estimate the depthof every pixel according to the pattern deformation. The active nature of this workingprinciple induces interference in the overlapping areas where two or more devices areprojecting their patterns at the same time, leading to noisier measurements, outliers andeven points not being detected. This can be seen in Figure 1, where a Kinect device (C1)has scanned a flat surface while another one (C2) has been set to be directly pointing tothe same surface. When C2 is on, the recorded data are noticeably noisier and containmore spurious points (including holes in the surface representation) due to interference.3. ExperimentsIn this section we present a quantitative study of the magnitude of the noise introducedby interference in a two-camera setup according to the distance and angle between thecameras and the measured surface. For that, we have set two configurations: i) measuringperpendicularly and ii) with an angle of 45o with respect to the surface. The measure-ments have been repeated for two different brands of devices: Kinect and Asus.3.1. Setup and analysis of interferenceThe cameras for the experiments have been placed on tripods about 30 cm apart whilescanning a 2 m×3.5 m wall. Then, we have performed measurements every 25 cm from1 m to 3.5 m in order to cover most of the range operation of the devices. Finally, thisrecording has been repeated for the two above-mentioned configurations aiming to eval-uate the impact of both the distance and the angle of the surface in the measurements. InSeptember 2017● ● ● ●● ●● ●● ●●1.0 1.5 2.0 2.5 3.0 3.5Kinect● ●● ●● ●● ●● ●●1.0 1.5 2.0 2.5 3.0 3.5Asus0.040.030.020.0100.040.030.020.010Depth dispersion (m)Distance (m) Distance (m)Depth dispersion (m)● one devicetwo devices● one devicetwo devices● ●● ●●●●1.0 1.5 2.0 2.5● ●● ●●● ●1.0 1.5 2.0 2.5Kinect AsusDepth dispersion (m) Depth dispersion (m)0.030.020.0100.030.020.010Distance (m) Distance (m)● one devicetwo devices● one devicetwo devicesFigure 2. Depth noise with and without interference for the Kinect and Asus devices (top: perpendicular,bottom at 45o).the following, the data recorded by device C1 is employed as the control sample whilstdevice C2 only serves to create interference and, therefore, the information it provides isnot saved. Thus, similarly to the example presented in Section 2, at each distance, twomeasurements are grabbed, corresponding to the data from C1 while C2 is off and on.Then, the noise of the depth data is estimated by first computing the covariance matrix ofthe 3D points recorded by C1 and subsequently computing its Singular Value Decompo-sition (SVD) to get its eigenvalues. The square root of the smallest eigenvalue representsthe standard deviation of the measurements in the dimension perpendicular to the wall’ssurface, being considered the depth noise in this work.Figure 2 (top) shows the average depth noise (for the (left) Kinect and (right) Asusdevices) in a set of recorded frames for each of the considered distances with the cameraspointing perpendicularly to the surface. As can be seen, the noise ranges from 4 mm (ata distance of 1 m) to around 25 mm near the end of the working range when C2 is off (inred), while interference caused by C2 increases the noise in all the distances (in blue).Note that there is a peak in the graph at 1.25 m in the case of the Kinect devices, probablycaused by daylight interference to cameraC1 in that particular setup, although, it seemednot to affect the Asus devices.Besides, the results reveal that the Kinect devices are affected more strongly byinterference than the Asus ones, placing the latter in a favorable position to be used assensors. Similarly, Figure 2 (bottom) represents the same noise when the cameras areplaced with an angle of 45o with respect to the surface. In this case, we only show theresults for distances from 1 m to 2.5 m since, for larger distances, data were highlyinaccurate in both situations due to the pronounced inclination of the measured plane.Again, the influence of interference is clearly visible in the figure, specially for the Kinectsensors.3.2. Interference reductionIn order to mitigate the effect of interference in the measurements, we have followed thesolutions presented in [1,2] by attaching a DC vibration motor to one of the cameras andcomputing the noise in the depth data again. Since the IR light emitter and the sensorSeptember 2017● ● ● ●● ●● ●● ●●1.0 1.5 2.0 2.5 3.0 3.5● ●● ●● ●● ●● ●●1.0 1.5 2.0 2.5 3.0 3.5Kinect AsusDepth dispersion (m) Depth dispersion (m)0.040.030.020.0100.040.030.020.010● 1 device2 devices2 devices w/motor● 1 device2 devices2 devices w/motorDistance (m) Distance (m)● ●● ●●●●1.0 1.5 2.0 2.5● ●● ●●● ●1.0 1.5 2.0 2.5Kinect AsusDepth dispersion (m) Depth dispersion (m)0.030.020.0100.030.020.010Distance (m) Distance (m)● 1 device2 devices2 devices w/motor● 1 device2 devices2 devices w/motorFigure 3. Influence of the motor on the depth dispersion regarding the distance (top: perpendicular, bottom at45o).are rigidly connected and move jointly in a single device, inducing vibrations avoids thecamera to detect the pattern projected by other devices. The results of the experimentsreveal the efficacy of this solution, as the noise produced by interference is almost com-pletely removed in every setup (Figure 3), hence improving the accuracy of the multi-camera system when they are pointing to the same scene.4. ConclusionThis paper has analyzed the influence of interference between RGB-D cameras in theaccuracy of their measurements, one of the main problems of using multiple depth cam-eras to measure the same scene. For that, we have conducted a series of experiments withthe cameras at different distances and angles with respect a large flat surface, trying tosimulate realistic conditions of scanning. The experiments reveal that the depth noise in-creases as distance grows and it is higher when the camera optical axis is not perpendic-ular to the surface. Not only that, the effect of interference from other devices is clearlynoticeable in form of noisier points, increasing the number of outliers and holes in thedetected 3D point cloud. Besides, Asus cameras seem to be less affected by interferencethan Kinect ones. Finally, we have evaluated the effectiveness of the proposed solutionsin the literature based on DC vibration motors and validated their capability of alleviatingthe effects of interference. This analysis and evaluation is being employed to perform afurther study about the influence of interference in performing human motion tracking.References[1] D Alex Butler, Shahram Izadi, Otmar Hilliges, David Molyneaux, Steve Hodges, and David Kim.Shake’n’sense: reducing interference for overlapping structured light depth cameras. In Proceedings ofthe SIGCHI Conference on Human Factors in Computing Systems, pages 1933–1936. ACM, 2012.[2] Andrew Maimone and Henry Fuchs. Reducing interference between multiple structured light depth sen-sors using motion. In 2012 IEEE Virtual Reality, VR 2012, Costa Mesa, CA, USA, March 4-8, 2012,pages 51–54, 2012.

Analyzing interference between RGB-D cameras for human motion tracking

Repositorio Institucional de la Universidad de Málaga

https://riuma.uma.es/xmlui/bitstream/10630/15044/1/APPIS2018_Moreno_etal.pdf

Analyzing interference between RGB-D cameras for human motion tracking

Abstract

Similar works

Full text

Available Versions

Repositorio Institucional Universidad de Málaga

Repositorio Institucional de la Universidad de Málaga