The increasing importance of pollutant noise has led to the creation of many new noise testing laboratories in recent years. For this reason and due to the legal implications that noise  reporting may have, it is necessary to create procedures intended to guarantee the quality of  the testing and its results. For instance, the ISO/IEC standard 17025:2005 specifies general  requirements for  the  competence of testing laboratories. In this standard,  interlaboratory  comparisons are one of the main measures that must be applied to guarantee the quality of laboratories when applying specific methodologies for testing. In the specific case of environmental noise, round robin tests are usually difficult to design, as it is difficult to find scenarios that can be available and controlled while the participants  carry out the measurements.  Monitoring and controlling the factors that can influence the measurements (source emissions, propagation, background noise…) is not usually affordable, so the most extended solution is to create very effortless scenarios, where most of the factors that can have an influence  on the results are excluded (sampling, processing of results, background noise, source detection…) The new approach described in this paper only requires the organizer to make actual measurements (or prepare virtual ones). Applying and interpreting a common reference document  (standard, regulation…), the participants must analyze these input data independently to provide the results, which will be compared among the participants. The measurement costs are  severely reduced for the participants, there is no need to monitor the scenario conditions, and  almost any relevant factor can be included in this methodolog

Asensio Rivera, César

Pagán Muñoz, Raúl

Pavón García, Ignacio

The increasing importance of pollutant noise has led to the creation of many new noise testing laboratories in recent years. For this reason and due to the legal implications that noise reporting may have, it is necessary to create procedures intended to guarantee the quality of the testing and its results. For instance, the ISO/IEC standard 17025:2005 specifies general requirements for the competence of testing laboratories. In this standard, interlaboratory comparisons are one of the main measures that must be applied to guarantee the quality of laboratories when applying specific methodologies for testing. In the specific case of environmental noise, round robin tests are usually difficult to design, as it is difficult to find scenarios that can be available and controlled while the participants carry out the measurements. Monitoring and controlling the factors that can influence the measurements (source emissions, propagation, background noise...) is not usually affordable, so the most extended solution is to create very effortless scenarios, where most of the factors that can have an influence on the results are excluded (sampling, processing of results, background noise, source detection...) The new approach described in this paper only requires the organizer to make actual measurements (or prepare virtual ones). Applying and interpreting a common reference document (standard, regulation...), the participants must analyze these input data independently to provide the results, which will be compared among the participants. The measurement costs are severely reduced for the participants, there is no need to monitor the scenario conditions, and almost any relevant factor can be included in this methodology

Asensio, C.

Pavón, I.

Pagan Munoz, R.

Pure OAI Repository

English

E-comparisons: A new approach to round robin tests for environmental noise assessment in transport infrastructures

Archivo Digital UPM

Asensio, C

Pavón, I

Pagan Munoz, R Raul

Repository TU/e

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

  
ICSV18, Rio de Janeiro, Brazil, 10-14 July 2011  1 
E-COMPARISONS: A NEW APPROACH TO ROUND 
ROBIN TESTS FOR ENVIRONMENTAL NOISE 
ASSESSMENT IN TRANSPORT INFRASTRUCTURES 
C. Asensio, I. Pavón and  R. Pagán 
Universidad Politécnica de Madrid (CAEND), c/ Serrano 144,  C.P. 28006, Madrid, Spain. 
 e-mail: cesar.asensio@caend.upm-csic.es        
The increasing importance of pollutant noise has led to the creation of many new noise test-
ing laboratories in recent years. For this reason and due to the legal implications that noise 
reporting may have, it is necessary to create procedures intended to guarantee the quality of 
the testing and its results. For instance, the ISO/IEC standard 17025:2005 specifies general 
requirements for the competence of testing laboratories. In this standard, interlaboratory 
comparisons are one of the main measures that must be applied to guarantee the quality of la-
boratories when applying specific methodologies for testing. In the specific case of environ-
mental noise, round robin tests are usually difficult to design, as it is difficult to find scenari-
os that can be available and controlled while the participants carry out the measurements. 
Monitoring and controlling the factors that can influence the measurements (source emis-
sions, propagation, background noise…) is not usually affordable, so the most extended solu-
tion is to create very effortless scenarios, where most of the factors that can have an influence 
on the results are excluded (sampling, processing of results, background noise, source detec-
tion…) 
The new approach described in this paper only requires the organizer to make actual meas-
urements (or prepare virtual ones). Applying and interpreting a common reference document 
(standard, regulation…), the participants must analyze these input data independently to pro-
vide the results, which will be compared among the participants. The measurement costs are 
severely reduced for the participants, there is no need to monitor the scenario conditions, and 
almost any relevant factor can be included in this methodology. 
 
1. Introduction 
The increasing importance of pollutant noise has derived in the creation of regulations whose 
main objective is to reduce noise, and make different land-uses compatible. These regulations fol-
low two different approaches. The first one is concerned with environmental noise protection, and 
the second one with the noise disturbance caused by activities 
1-5
. The increasing demand on meas-
urements and noise inspections has led to the creation of many new noise testing laboratories and 
inspection bodies in recent years. Due to the legal implications that noise reporting may have, it is 
necessary to guarantee the quality of the measurements and the independence and expertise of the 
18
th
 International Congress on Sound and Vibration, Rio de Janeiro, Brazil, 10-14 July 2011 
 
 
2 
noise inspectors and practitioners. For instance, ISO/IEC standard 17025:2005
6
 specifies general 
requirements for the competence of testing laboratories. It establishes some general requisites re-
garding the instrumentation and its calibration, the expertise of the practitioners and the accordance 
of procedures with testing standards. Interlaboratory comparisons are one of the main measures 
intended to guarantee the quality, consistency and comparability of the results, tests and testing pro-
cedures. The development and operation of proficiency testing by interlaboratory comparisons is 
standardized in the ISO/IEC GUIDE 43-1
7
, which defines the main concepts and managing proce-
dures to be considered. 
In the specific case of environmental noise, round robin tests are usually difficult to design, as 
it is difficult to find scenarios that can be available and controlled while the participants carry out 
their measurements. Monitoring the factors that can influence the measurements (source emissions, 
propagation, background noise…) is not usually affordable, therefore the most extended solution is 
to create very effortless scenarios, where most of the factors that can have an influence on the re-
sults are excluded (sampling, processing of results, background noise, source detection…). But this 
is not the best solution, as practically the only thing checked under these circumstances is the in-
strumentation, which is just an alternative to calibration. It would be required to create more com-
plex scenarios to check the repeatability of the laboratories, and inter-laboratory reproducibility 
under more difficult and closer-to-reality scenarios. This is one way to achieve a continuous im-
provement in the quality of the methods and the laboratories. 
According to the e-comparisons approach described in this paper, only the organizer of the 
comparison makes actual measurements. By applying and interpreting a common reference docu-
ment (standard, regulation…), the participants will analyze real or virtual data and report the results, 
which will be compared among the participants. As long as the test scenario can be set up for any 
circumstance, and almost any relevant factor can be included in this methodology, this practice can 
be used to provide focused results, allowing fixing some of the factors and making them independ-
ent, as needed for every purpose. Measurement costs are severely reduced for the participants, as 
the physical process of measuring is avoided and data can be circulated using the Internet, and so, 
there is no need to control and monitor the scenario conditions. 
2. Methodology 
The design of an interlaboratory comparison starts by defining the precise objectives to be 
captured. It must set the focus on the capacities of the participating laboratories, and it needs to ex-
clude any supplementary elements that are not important, or those that can be easily considered in-
dependently. This will only be possible if the organizer of the comparison is an expert in the field of 
application, and is very familiar with the reference documents and associated procedures. 
E-comparisons spread huge potentials for many fields of acoustic testing. They can be used 
not only to estimate the uncertainty of the methods and rejection or acceptance of the results report-
ed by the laboratories, but they also allow easily checking the compliance of the requirements in the 
standards, so that the results of non-compliant laboratories can be easily excluded from the statisti-
cal analysis. The following tables illustrate some of the possibilities of e-comparisons regarding the 
fields of environmental and building acoustics. 
 
18
th
 International Congress on Sound and Vibration, Rio de Janeiro, Brazil, 10-14 July 2011 
 
 
3 
 
Table 1. E-comparisons applied to environmental noise measurements (ISO 1996) 
Test procedure Parameters to explore Notes for e-comparison design 
- Aircraft noise (ISO 20906) 
- Railway noise 
- Road traffic noise 
- Industrial noise 
 
- Noise event detection tech-
niques 
The organizer has to provide 
noise level profiles and addi-
tional information for the iden-
tification task (recordings, 
notes…) 
- Time sampling For a single location, the user 
has to report a single result 
from several measurements 
- Spatial sampling The organizer has to provide 
measurements for several loca-
tions, allowing the user to 
choose the correct ones 
- Low frequency correction 
- Tone correction 
- Impulse correction 
The measurements provided 
will include one or several of 
these factors, so that the exper-
tise of the laboratories can be 
checked  
- Background noise correc-
tion 
Background noise measure-
ments must be provided 
- Uncertainty calculations The user will be requested to 
report results and their related 
uncertainty. The organizer will 
provide information to allow 
the user to make the calcula-
tion for type A and type B con-
tributions. 
- Instrumentation Instrumentation could be also 
checked. It would be necessary 
to provide a recording of a ref-
erence signal for calibration 
purposes and the recordings of 
the noise to be measured. The 
recordings must be inserted 
into the measurement chain 
using a connection to the pre-
amplifier. Only the microphone 
remains outside the compari-
son. 
- Other data processing  
- Noise inspection - Compliance To report compliance, the 
noise inspector will have to 
check almost all the parameters 
indicated above, including un-
certainty. 
 
18
th
 International Congress on Sound and Vibration, Rio de Janeiro, Brazil, 10-14 July 2011 
 
 
4 
 
Table 2. E-comparisons applied to noise insulation testing (ISO 140 series, field measurements) 
Test procedure Parameters to explore Notes for e-comparison design 
- Airborne sound insulation 
between rooms 
- Airborne sound insulation 
of façade and façade ele-
ments 
- Impact of sound insulation 
- Positions of the source (fa-
cade and rooms) 
- Positions of the microphone 
in the emitting room 
- Positions of the microphone 
in the receiver room 
- Positions of the micro-
phones in front of the façade  
The organizer must provide 
results for many valid and non-
valid positions so that the ex-
pertise of the laboratories and 
the influence of the location on 
the results can be checked. 
The virtual measurements can 
be created from random Gauss-
ian variables, with known 
means and variances for each 
frequency band. 
- Background noise correc-
tions 
Background noise measure-
ments must be provided 
- Rating according to ISO 
717 
This overall result is directly 
derived from the spectra re-
sults, but it could be also in-
cluded in the comparison. 
- Uncertainty of the results The user will be requested to 
report spectra and overall re-
sults and their related uncer-
tainty. The organizer will pro-
vide information to allow the 
user to make the calculation for 
type A and type B contribu-
tions. 
 
In order to clarify the description of the methodology throughout the paper, we will describe 
the steps followed for a comparison on airport noise measurements. For this specific case, it is very 
difficult to create a traditional interlaboratory comparison, as there are many factors involved that 
influence the results: instrumentation, the number and type of aircraft, the dispersion on the flight 
paths, airport operability, meteorological factors, source variation factors… But all of these factors 
were outside our scope. The aim of the comparison is to compare the specific methods that every 
laboratory implements from the bases described in ISO 20906
8
. Therefore, the reference document 
is ISO 20906, and we wished to compare the results obtained by the laboratories for the same mete-
orological conditions and the same source. The influence of the instrument was also excluded, as it 
can be quantified separately for every laboratory. The influence of residual noise, and how each 
laboratory manages it could also have been included in the comparison, but we decided to exclude it 
for simplicity. After excluding all those factors, the only remaining one is that derived from the pro-
cessing of the measurements, which is the result of the methodology used for the marking of aircraft 
sound events (detection + classification + identification, according to ISO 20906, see Figure 1). 
18
th
 International Congress on Sound and Vibration, Rio de Janeiro, Brazil, 10-14 July 2011 
 
 
5 
 
Figure 1. Aircraft events identification scheme according to ISO 20906 
Once the objectives have been defined, we must find a way to meet them. In our example, we 
can achieve the objectives by just using a measurement file that can be processed by all the labora-
tories. The organizer made measurements and recordings, and selected the appropriate profile for 
the experiment. Then, this reference material (measurements + recordings) was circulated among 
the participants. As established by ISO 5725
9
, the test items remain identical for all of them. The 
organizer decided to set three different scenarios for the comparison according to the dynamic range 
of the aircraft noise events considered. Consequently, three different reference materials were tested 
by each laboratory in this e-comparison: 
 ENV1: Aircraft sound events easily detectable (from measurements and audio files) 
 ENV2: Aircraft sound events hard to detect in the measurement files, but clearly audible 
 ENV3: Aircraft sound events very difficult to detect, and the presence of other sound events. 
It was decided not to use files longer than 1 hour, as this duration was considered long enough 
to analyze the competence of the laboratory and validity of its procedures. Every test set consisted 
of an audio file and a measurement file, with a duration T (seconds).  
Every measurement file contained T one-second continuous equivalent sound pressure level 
measurements (LAeq,1s) that comprised the noise profile (see, Figure 2). In general, this profile will 
be used for event detection and classification tasks, while the recordings will be used for identifica-
tion. 
 
Figure 2. Example of measurements time history (blue) and recordings (green) 
18
th
 International Congress on Sound and Vibration, Rio de Janeiro, Brazil, 10-14 July 2011 
 
 
6 
The participants were asked to report results according to ISO 20906, in terms of equivalent 
noise level referred to the reference time interval T (which is also the duration of the measure-
ments). The requested results must be calculated according to equation 1, but this equation was not 
provided: 
  (1) 
where Li refers to the Ta measurements (  LAeq,1s ) marked as aircraft noise. 
Communicating with the participants, circulating the reference material and reporting the re-
sults or any other details, can be carried out using a collaborative Internet website, or simply by 
email. Confidentiality must be assured whatever method is used for communication. 
The statistical analysis of the reported results does not change for e-comparisons. One of the 
main issues refers to the detection of outliers and the rejection criteria for the laboratory’s results. 
3. Results 
The idea of e-comparisons was conceived while carrying out the Aircomp Project, intended to 
describe the influence of the human factor on the results of aircraft noise monitoring. Five European 
universities were involved in this project (Politecnico di Milano, Universidad Politécnica de Ma-
drid, University of Zagreb, Universidad Politécnica de Cataluña and the University of Sheffield). 
The results reported by the 64 participants on this project have been used to check the validity of 
this methodology for the production of e-comparisons. Most of the participants were engineering or 
master’s students, with a certain background regarding acoustics. But some professors, researchers 
and experts in environmental acoustics were also involved in the comparison. Figure 3 shows the 
box plot of the results reported, where some outliers can be clearly identified. A simple analysis of 
the data shows that the methodology used by the participants has an influence on the reported re-
sults, which means variability. If we exclude the outlier participants from this analysis, the variabil-
ity in the results increases as the acoustic environment becomes more difficult, and the dynamic 
range of the noise events is lower. 
 
Gráfico de Cajas y Bigotes
Laircraft
E
n
vi
ro
n
m
e
n
t 1
2
3
36 46 56 66 76
 
Figure 3. Box plot of the results reported by the participants for the three acoustic environments 
The results would allow the outlier participants to reject their results, and revise their proce-
dures. The rest of the participants reported comparable data within a coverage interval, which is 
evidence of the quality in their results. Furthermore, they could be used for estimating the uncer-
tainty of the method. 
18
th
 International Congress on Sound and Vibration, Rio de Janeiro, Brazil, 10-14 July 2011 
 
 
7 
 
4. Conclusions 
In this paper, we have presented a new approach to round robin tests for environmental noise 
assessment in transport infrastructures through e-comparisons. By using e-comparisons, it is possi-
ble to make an in-depth evaluation of the methodology, performance and expertise of laboratories, 
and set specific targets that can be configured in as complex a way as required. The reference mate-
rial used for the tests can be measured just by the organizer, but it is also possible to create virtual 
reference material artificially in the laboratory, so that comparisons can be made beyond the limits 
of real world acoustics. 
It has been demonstrated that the use of the new technologies can be very useful to simplify 
the process and reduce the costs of interlaboratory comparisons. E-comparisons have currently 
shown to be a boundless and technologically suitable concept. 
 Following this approach, it becomes possible to include in the round robin tests aspects as 
difficult to evaluate as sampling methodology (spatial and temporal), the analysis of noise charac-
teristics (presence of tones, low frequency noise, impulse…), and the detection of noise sources… 
E-comparisons can be carried out from any part of the world, they can cover almost every as-
pect of testing procedures, and they can be reused with slight changes for periodic interlaboratory 
comparison exercises. 
5. Acknowledgements 
The authors would like to express their gratitude to all the participants in this survey, and es-
pecially to M. Ausejo, K. Jambrošić, J. Kang, G. Moschioni, M. Recuero and J. Romeu. 
REFERENCES  
1. WHO. Guidelines for community noise. ; 1999. 
2. Directive 2002/49/EC of the European Parliament and of the Council of 25 June 2002 Relating to 
the Assessment and Management of Environmental Noise, European Directive U.S.C. (2002). 
3. ISO. ISO 1996-1:2003. Acoustics -- description, measurement and assessment of environmental 
noise -- part 1: Basic quantities and assessment procedures. 2003. 
4. ISO. ISO 1996-2:2007. Acoustics -- description, measurement and assessment of environmental 
noise -- part 2: Determination of environmental noise levels. 2007. 
5. ISO. ISO 1996-3:1987. Acoustics -- description and measurement of environmental noise -- part 
3: Application to noise limits. 1987. 
6. ISO/IEC. ISO/IEC 17025: 2005 general requirements for the competence of testing and calibra-
tion laboratories. 2005. 
7. ISO. ISO/IEC guide 43-1:1997. Proficiency testing by interlaboratory comparisons- part 1: De-
velopment and operation of proficiency testing schemes. 1997. 
8. ISO. ISO 20906:2009. Acoustics -- unattended monitoring of aircraft sound in the vicinity of 
airports. 2009. 
9. ISO. ISO 5725-1:1994. Accuracy (trueness and precision) of measurement methods and results -- 
part 1: General principles and definitions. 1994. 


http://oa.upm.es/12179/1/INVE_MEM_2011_87388.pdf

E-comparisons: A new approach to round robin tests for environmental noise assessment in transport infrastructures

Abstract

Similar works

Full text

Available Versions

Pure OAI Repository

Archivo Digital UPM

Repository TU/e

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Pure OAI Repository