Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial

Background: Minimising participant non-response in postal surveys helps to maximise the generalisability of the inferences made from the data collected. The aim of this study was to examine the effect of questionnaire length, personalisation and reminder type on postal survey response rate and quality and to compare the cost-effectiveness of the alternative survey strategies. Methods: In a pilot study for a population study of travel behaviour, physical activity and the environment, 1000 participants sampled from the UK edited electoral register were randomly allocated using a 2 × 2 factorial design to receive one of four survey packs: a personally addressed long (24 page) questionnaire pack, a personally addressed short (15 page) questionnaire pack, a non-personally addressed long questionnaire pack or a non-personally addressed short questionnaire pack. Those who did not return a questionnaire were stratified by initial randomisation group and further randomised to receive either a full reminder pack or a reminder postcard. The effects of the survey design factors on response were examined using multivariate logistic regression. Results: An overall response rate of 17% was achieved. Participants who received the short version of the questionnaire were more likely to respond (OR = 1.48, 95% CI 1.06 to 2.07). In those participants who received a reminder, personalisation of the survey pack and reminder also increased the odds of response (OR = 1.44, 95% CI 1.01 to 1.95). Item non-response was relatively low, but was significantly higher in the long questionnaire than the short (9.8% vs 5.8%; p = .04). The cost per additional usable questionnaire returned of issuing the reminder packs was £23.1 compared with £11.3 for the reminder postcards. Conclusions: In contrast to some previous studies of shorter questionnaires, this trial found that shortening a relatively lengthy questionnaire significantly increased the response. Researchers should consider the trade off between the value of additional questions and a larger sample. If low response rates are expected, personalisation may be an important strategy to apply. Sending a full reminder pack to non-respondents appears a worthwhile, albeit more costly, strategy

Reliability and validity of the transport and physical activity questionnaire (TPAQ) for assessing physical activity behaviour

Background: No current validated survey instrument allows a comprehensive assessment of both physical activity and travel behaviours for use in interdisciplinary research on walking and cycling. This study reports on the test-retest reliability and validity of physical activity measures in the transport and physical activity questionnaire (TPAQ). Methods: The TPAQ assesses time spent in different domains of physical activity and using different modes of transport for five journey purposes. Test-retest reliability of eight physical activity summary variables was assessed using intra-class correlation coefficients (ICC) and Kappa scores for continuous and categorical variables respectively. In a separate study, the validity of three survey-reported physical activity summary variables was assessed by computing Spearman correlation coefficients using accelerometer-derived reference measures. The Bland-Altman technique was used to determine the absolute validity of survey-reported time spent in moderate-to-vigorous physical activity (MVPA). Results: In the reliability study, ICC for time spent in different domains of physical activity ranged from fair to substantial for walking for transport (ICC = 0.59), cycling for transport (ICC = 0.61), walking for recreation (ICC = 0.48), cycling for recreation (ICC = 0.35), moderate leisure-time physical activity (ICC = 0.47), vigorous leisure-time physical activity (ICC = 0.63), and total physical activity (ICC = 0.56). The proportion of participants estimated to meet physical activity guidelines showed acceptable reliability (k = 0.60). In the validity study, comparison of survey-reported and accelerometer-derived time spent in physical activity showed strong agreement for vigorous physical activity (r = 0.72, p,0.001), fair but non-significant agreement for moderate physical activity (r = 0.24, p = 0.09) and fair agreement for MVPA (r = 0.27, p = 0.05). Bland-Altman analysis showed a mean overestimation of MVPA of 87.6 min/week (p = 0.02) (95% limits of agreement 2447.1 to +622.3 min/week). Conclusion: The TPAQ provides a more comprehensive assessment of physical activity and travel behaviours and may be suitable for wider use. Its physical activity summary measures have comparable reliability and validity to those of similar existing questionnaires

Validity of physical activity measures overall and by domain.

<p>Validity of physical activity measures overall and by domain.</p

Questionnaire items and physical activity summary variables in TPAQ.

<p>†Journey purposes: to and from work; business journeys; to and from a place of study or to and from school; for shopping and personal business; to visit friends and relatives and for other social activities. Wt = walking for transport; Ct = cycling for transport; Wr = Walking for recreation; Cr = cycling for recreation; VPA = vigorous physical activity; MPA = moderate physical activity; TPA = Total physical activity; MPA = moderate physical activity; TPA = Total physical activity; MVPA = moderate-to-vigorous physical activity.</p>r<p> = reliability study;</p>v<p> = validity study;</p>f<p> = frequency of participation assessed in reliability study as separate variable;</p><p>♦summed across all journey purposes. The full survey is available at <a href="http://bmjopen.bmj.com/content/suppl/2012/02/04/bmjopen-2011-000694.DC1/bmjopen-2011-000694-s1.pdf" target="_blank">http://bmjopen.bmj.com/content/suppl/2012/02/04/bmjopen-2011-000694.DC1/bmjopen-2011-000694-s1.pdf</a></p><p>Questionnaire items and physical activity summary variables in TPAQ.</p

Sample characteristics.

a<p> Included in analysis.</p>b<p> Numbers do not sum to totals due to missing responses.</p><p>Sample characteristics.</p

Bland-Altman plot for time spent in moderate-to-vigorous physical activity.

<p>The difference between objectively measured time spent in total MVPA (min/week) and self-reported time spent in MVPA excluding cycling (min/week) plotted against objectively measured time spent in MVPA (min/week). Mean difference: 87.6 min/week (p = 0.02); limits of agreement: −447.1 min/week, +622.3 min/week.</p

Intra-class correlations for frequency of participation in recreational physical activity.

<p>T1 = Survey time point 1;</p><p>T2 = Survey time point 2.</p><p>Intra-class correlations for frequency of participation in recreational physical activity.</p

Intra-class correlations for time spent in transport-related and recreational physical activity overall and by domain.

<p>T1 = Survey time point 1;</p><p>T2 = Survey time point 2;</p>†<p>Total physical activity includes vigorous and moderate leisure-time activity and walking and cycling for transport and recreation.</p><p>Intra-class correlations for time spent in transport-related and recreational physical activity overall and by domain.</p

Repeat cross-sectional associations between exposure to a motorway and change in SF-8 physical and mental component score.

<p>Repeat cross-sectional associations between exposure to a motorway and change in SF-8 physical and mental component score.</p

Cross-sectional associations between exposure to a motorway and Warwick-Edinburgh Mental Well-being Scale (short version) score at T2.

<p>Cross-sectional associations between exposure to a motorway and Warwick-Edinburgh Mental Well-being Scale (short version) score at T2.</p