Panel conditioning arises if respondents are influenced by participation in previous surveys, such that their answers differ significantly from the answers of individuals who are interviewed for the first time. Having two panels—a trained one and a completely fresh one—created a unique opportunity for analysing panel conditioning effects. To determine which type of question is sensitive to panel conditioning, 981 trained respondents and 2809 fresh respondents answered nine questions with different question types. The results in this paper show that panel conditioning only arise in knowledge questions. Questions on attitudes, actual behaviour, or facts were not sensitive to panel conditioning. Panel conditioning in knowledge questions was restricted to less-known subjects (more difficult questions), suggesting a relation between panel conditioning and cognition.panel conditioning;re-interviewing;measurement error;panel surveys

Das, J.W.M.

Soest, A.H.O. van

Toepoel, V.

English

Research Papers in Economics

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
No. 2008–04 
 
RELATING QUESTION TYPE TO PANEL CONDITIONING: A 
COMPARISON BETWEEN TRAINED AND FRESH 
RESPONDENTS 
 
By Vera Toepoel, Marcel Das, Arthur van Soest 
 
 
 
January 2008 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ISSN 0924-7815   1 
Relating Question Type to Panel Conditioning: A Comparison between 
Trained and Fresh Respondents 
 
Vera Toepoel
∗, Marcel Das
∗, and Arthur van Soest
∗∗ 
 
Abstract   Panel conditioning arises if respondents are influenced by 
participation in previous surveys, such that their answers differ significantly 
from the answers of individuals who are interviewed for the first time. Having 
two panels—a trained one and a completely fresh one—created a unique 
opportunity for analysing panel conditioning effects. To determine which type 
of question is sensitive to panel conditioning, 981 trained respondents and 
2809 fresh respondents answered nine questions with different question 
types. The results in this paper show that panel conditioning only arise in 
knowledge questions. Questions on attitudes, actual behaviour, or facts were 
not sensitive to panel conditioning. Panel conditioning in knowledge questions 
was restricted to less-known subjects (more difficult questions), suggesting a 
relation between panel conditioning and cognition. 
 
Keywords: panel conditioning, re-interviewing, measurement error, panel 
surveys 
 
JEL codes: C42, C81, C93
                                                 
∗ CentERdata, Tilburg University, postal address: CentERdata, Tilburg University, P.O. Box  
90153, 5000 LE Tilburg, The Netherlands. Corresponding author: Vera Toepoel; e-mail: 
V.Toepoel@uvt.nl 
 
 
∗∗ Tilburg University, Faculty of Economics and Business Administration, Department of 
Econometrics and Operations Research.   2 
1. Introduction 
Trained respondents may answer questions differently than those with 
little or no experience in a panel. This can result in different responses with 
regard to content (e.g. because of increasing knowledge on topics) as well as 
the procedure (question-answering process). Panel members may learn from 
taking surveys. They may feel obliged to prepare for future surveys (increase 
their knowledge), or develop attitudes towards certain topics. In addition, they 
may become familiar with the question-answering process, learn how to 
interpret questions, and make fewer errors than new respondents. Or the 
opposite: experienced respondents may also answer strategically to avoid 
follow-up questions and reduce the burden of their task, speed through the 
survey, and therefore even make more errors. This paper addresses the issue 
of the first form of learning from taking surveys: knowledge on questions in a 
survey. It is investigated which type of question sensitive to panel conditioning 
is. 
 
2. Background 
One of the basic decisions a survey researcher faces is whether or not 
to use trained respondents (using a panel) or fresh respondents (e.g. a cross 
sample). Shariot (1991) discusses advantages and disadvantages of panels. 
There are two important methodological issues associated with the use of 
panel surveys: panel attrition and panel conditioning. Panel conditioning 
arises if re-interviewing causes differences in knowledge, behaviour or 
attitude.    3 
Panel conditioning has been studied in many social sciences, with 
mixed findings. While Williams (1970), Williams and Mallows (1970), Meurs 
et al. (1989), and Waterton and Lievesley (1989) found some evidence 
that respondents are influenced by re-interviewing,  Dennis (2001) and Clinton 
(2001) found little evidence for panel conditioning in the ‘Knowledge Networks' 
panel (an online panel that is representative of the entire US population). Van 
der Zouwen and Van Tilburg (2001) argue that conditioning effects sometimes 
appear and sometimes do not, without a clear indication of the conditions 
under which these effects occur. Trivellato (1999) argues that panel 
participation mainly affects the way in which behaviour is reported (response 
process), while it does not have pervasive effects on behaviour itself. Das et 
al. (2007) and Coombs (1973) argue that panel conditioning only arises for 
knowledge questions, but not in other types of questions. Sturgis et al. (2007) 
try to define a main theory behind panel conditioning: the cognitive stimulus 
hypothesis. Questions asked about certain topics may induce respondents to 
reflect more closely on them after the interview has ended, and possibly to 
talk about them with friends and relatives or to acquire additional information. 
Golob (1990) argues that no panel conditioning effects exist in questions that 
require simple reporting tasks, e.g. that panel conditioning relates to the 
cognitive difficulty in answering questions. Van der Zouwen and Van Tilburg 
(2001), on the other hand, found that panel conditioning did not take place via 
cognitive processes within the respondent’s mind but via the interviewer.  
 
3. Design and implementation   4 
To study the relation between panel conditioning and question type, we used 
two online household panels administrated by CentERdata (see 
www.centerdata.nl for more details about the panels). The first panel, the 
CentERpanel, consists for more than fifteen years. Panel members fill out 
questionnaires every week. Panel duration of respondents varies between 
fifteen years and a few months. The second panel is called the LISS-panel. 
Our questions were included in the first questionnaire presented to 
respondents in this panel. We fielded the questionnaire in June 2007. In the 
CentERpanel 1356 panel members were selected to fill out the questionnaire; 
981 respondents responded (72.3%). In the LISS-panel, 4530 respondents 
were selected; 2809 respondents filled out the questionnaire (62.0%). Due to 
non-response, both panels were not entirely the same with regard to some 
key personal characteristics. Therefore, we used weights based upon sex, 
age, and education to compare the results of both panels. We used nine 
questions on two different subjects: food infection and old-age pensions. 
These subjects were presented to the trained panel multiple times, so we 
thought these would be the most sensitive to bias due to panel conditioning.  
 
4. Results 
Table 1 shows significant differences between trained and fresh respondents 
for the knowledge question about campylobacter
1; 25.2% of the trained 
panels know what campylobacter is compared to 17.0% of the fresh panel. 
Salmonella and cross infection (also knowledge questions) do not show 
differences between the two panels. These concepts are relatively well-
                                                 
1 Campylobacter is a bacterium found in the intestines of many types of animals and is the most 
common bacterial cause of diarrheal illness.   5 
known, while campylobacter is a type of infection that less people know about. 
We found differences in the question about “Stichting Pensioenkijker”, an 
association to promote pension awareness that was mentioned in previous 
interviews the trained panel responded to, as well. Almost twice as much 
trained respondents compared to fresh respondents heard, saw, or read 
something about this association (39.7% in the trained panel compared to 
21.8% in the fresh panel). Other types of questions (attitude, fact, behaviour, 
etc.) were not sensitive to repeated interviewing.  Knowledge questions on 
less-known subjects seem to be sensitive to panel conditioning, indicating that 
panel conditioning relates to the cognitive difficulty in answering questions. 
 
Table 1. A comparison between trained and fresh respondents for different 
Yes/No question types. 
  Type of 
question 
%  
Yes 
Trained 
panel 
% 
Yes 
Fresh 
panel 
Do you know what Campylobacter is?  Knowledge  25.2  17.0* 
Do you know what Salmonella is?  Knowledge  98.4  98.5 
Do you know what Cross infection is?  Knowledge  80.9  79.1 
Did you think about your age of retirement the 
last year? 
Behaviour  60.5  62.3 
Did you ever hear, see, or read something 
about “Stichting Pensioenkijker”? 
Knowledge  39.7  21.8* 
Do you think pensions will be higher about ten 
years from now? 
Attitude  24.1  27.3 
Do you think people will be more satisfied with 
their pensions about ten years from now? 
Attitude  10.2  9.1 
Do you think many people will retire partially in 
the future? 
Attitude  64.0  64.2 
Are you retired?  Fact  21.7  21.8 
*Difference between trained and fresh panel is significant (p<.01). 
 
5. Discussion and Conclusions 
It is important to understand issues related to panel conditioning and their 
potential impact on the quality of research. Panel research gives big   6 
advantages, but the fact that the panel is the foundation on which research 
projects are built, and trained respondents may respond differently than fresh 
respondents, causes concerns with regard to survey quality. This paper 
shows that knowledge questions on less-known subjects are very much 
affected by panel conditioning. When asking these kind of questions, a 
researcher has to be particular careful about the kind of sample used. We 
found that other types of questions are not sensitive to repeated interviewing. 
Our results hint at a relation between panel conditioning and cognitive 
demand. Knowledge questions on difficult subjects are more sensitive to 
panel conditioning. Future research can make the effect between panel 
conditioning and cognition more clear. 
   
6. References 
 
Clinton, Joshua D. 2001. “Panel Bias from Attrition and Conditioning: A Case 
Study of the Knowledge Networks Panel.” Working paper retrieved December 
21 2007 on 
http://www.knowledgenetworks.com/insights/docs/Panel%20Effects.pdf 
 
Coombs, Lologene C. 1973. “Problems of Contamination in Panel Surveys: A 
Brief Report on an Independent Sample; Taiwan 1970.” Studies in Family 
Planning, 4, 257-261. 
 
Das, Marcel, Vera Toepoel, and Arthur van Soest. 2007. “Can I Use a Panel? 
Panel Conditioning and Attrition Bias in Panel Surveys.” CentER Discussion 
Paper 2007-56, CentER, Tilburg University. 
 
Dennis, Michael. 2001. “Are Internet Panels Creating Professional 
Respondents?” Marketing Research, 13, 34-39. 
 
Golob, Thomas F. 1990. “The Dynamics of Household Travel Time 
Expenditures and Car Ownership Decisions.” Transportation Research, 24A, 
443-463. 
 
Meurs, Henk, Leo van Wissen, and Jacqueline Visser. 1989. “Measurement 
Biases in Panel Data.” Transportation, 16, 175-194. 
   7 
Shariot, Trevor. 1991. “Attrition and Rotation in Panel Surveys.” The 
Statistician, 40, 325-331. 
 
Sturgis, Patrick, Nick Allum, and Ian Brunton-Smith. 2007. “Attitudes Over 
Time: The Psychology of Panel Conditioning.” In Methodology in 
Longitudinal Surveys, edited by P. Lynn: Wiley. 
 
Trivellato, Ugo. 1999. “Issues in the Design and Analysis of Panel Studies: A 
Cursory Review.” Quality & Quantity, 33, 339-352. 
 
Waterton, Jennifer and Denise Lievesley. 1989. “Evidence of Conditioning 
Effects in the British Social Attitudes Panel.” In: Panel Surveys, edited by D. 
Kasprzyk, G. Duncan, G. Kalton, and M. Singh. Wiley series in probability and 
mathematical statistics: New York. 
 
Williams, William H. 1970. “The Systematic Bias Effects of Incomplete 
Responses in Rotation Samples.” Public Opinion Quarterly, 33, 593-602. 
 
Williams, William H. and Colin L. Mallows. 1970. “Systematic Biases in Panel 
Surveys Due to Differential Nonresponse.” Journal of the American Statistical 
Association, 65, 1338-1349. 
 
Van der Zouwen, Johannes and Theo van Tilburg. 2001. “Reactivity in Panel 
Studies and its Consequences for Testing Causal Hypotheses.” Sociological 
Methods & Research, 30, 35-56. 
 

Attitudes Over Time: The Psychology of Panel Conditioning.” In Methodology in Longitudinal Surveys, edited by P.

Attrition and Rotation in

Can I Use a Panel? Panel Conditioning and Attrition Bias

Evidence of Conditioning Effects in the British Social Attitudes Panel.” In: Panel Surveys, edited by

Issues in the Design and Analysis of Panel Studies:

Panel Bias from Attrition and Conditioning: A Case Study of the Knowledge Networks Panel.” Working paper retrieved

Systematic Biases in Panel Surveys Due to Differential Nonresponse.”

The Dynamics of Household Travel Time Expenditures and Car Ownership Decisions.” Transportation Research,

The Systematic Bias Effects of Incomplete Responses

Relating Question Type to Panel Conditioning: A Comparison between Trained and Fresh Respondents

Relating Question Type to Panel Conditioning: A Comparison between Trained and Fresh Respondents

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics