Recently, I sent postcards to many people. The same week, Dr. X sent me a postcard with nearly identical wording. What happened? Both Dr. X and I had been influenced by Dillman’s Mail and Telephone Surveys. Despite its flaws, it is a worthwhile book. I keep this book near me when working on surveys, and I suspect it will have a strong influence on survey research. This review gives a description of the content of Dillman’s book. It goes on to describe the flaws not only to alert the reader but also in the hope that Dillman will write a new edition to overcome these problems. He could convert a gold book into an excellent one.Don Dillman, surveys, telephone, review

JS Armstrong

English

Research Papers in Economics

  1
REVIEWS OF BOOKS ON FOR E C A S T I N G          
Principles of Forecasting Web Site 
 
Don A. Dillman, Mail and Telephone Surveys, New York: John Wiley & Sons, 1978. 
 
Reviewed by: J. Scott Armstrong, University of Pennsylvania; published in the Journal of Business, 54 
(4), 1981, 622-625. 
 
Recently, I sent postcards to many people. The same week, Dr. X sent me a postcard with nearly identical 
wording. What happened? Both Dr. X and I had been influenced by Dillman’s Mail and Telephone Surveys. Despite its 
flaws, it is a worthwhile book. I keep this book near me when working on surveys, and I suspect it will have a strong 
influence on survey research. 
This review gives a description of the content of Dillman’s book. It goes on to describe the flaws not only to 
alert the reader but also in the hope that Dillman will write a new edition to overcome these problems. He could convert a 
gold book into an excellent one. 
“This book describes the step-by-step details of how to conduct successful mail and telephone surveys.” So goes 
the first sentence in the book. That is precisely what the book does and why it is valuable. He even tells what type of paper 
to use m a mail questionnaire and how to fold it. 
The content covers much of what one would need in order to prepare and administer a mail or telephone survey. 
As a result, it is especially valuable to people who are relatively new to the field. Experienced survey researchers will also 
find it valuable; in my own case. I concluded that a number of Dillman’s techniques were better than my own. 
Chapter 1 consists mostly of puffery about “Total Design Method” (TDM). Dillman implies that TDM is 
revolutionary in nature and that it helps to improve the effectiveness of mail and telephone surveys “in ways thought 
impossible only a few years ago” (p. 2). Actually. TDM merely says that many factors affect the success of mail and 
telephone surveys and that these factors interact with one another. Because of this interaction, he says that “exhaustive 
reviews of the research, of which there are several, seemed destined to be unhelpful” (p. 7). But Dillman does not provide 
empirical support to show the importance of interactions. Apparently not much evidence exists on this point. Furthermore, 
he omits reference to Wiseman (1973), a small empirical study which concluded that factor interaction was unimportant. I 
did not find the TDM framework to be helpful. Furthermore. I do find exhaustive reviews to be helpful. I recommend that 
you start the book on page 20. This last part of chapter 1 provides evidence on response rates achieved by surveys using 
TDM. 
Chapter 2 examines the relative advantages of mail, telephone, and face-to-face surveys. Experienced survey 
researchers can skip this chapter. Dillman has little to offer in the way of new evidence or new ideas. 
Experienced researchers may want to skip chapter 3, which describes how to write questions. I found nothing 
new or surprising in this chapter.  
Everyone join in for chapter 4! It contains practical advice on the construction of mail questionnaires: designing 
booklet format and printing procedures, putting the questions in proper order, choosing the first question, formulating the 
pages, designing the front and back covers, and pre-testing. 
Chapter 4 is also useful. It discusses how to implement mail surveys: writing and printing the cover letter, 
preparing the envelope, deciding on postage, identifying the questionnaire. preparing the return envelope, assembling the 
mail-out package, selecting the mail-out date, scheduling follow-up mailings, and responding to problems as they arise. 
This chapter complements that useful book by Erdos (1970). 
Chapters 6 and 7 do a similar and useful step-by-step description for telephone surveys: Some of the key issues 
discussed in chapter 6 are the development of response categories, order of questions, the first few questions, construction   2
of the page for the interviewer. use of screening questions, placement of interviewer and coder instructions, pre-testing. 
and printing of the questionnaire. Chapter 7 discusses sampling from telephone directories, random digit dialing (RDD), 
writing the introduction, the advance letter, selecting respondents within the household, centralized interviewing facilities, 
recruitment and training of interviewers, answering respondent questions, and scheduling the interviewing sessions. 
The book ends with chapter 7, in my opinion. Dillman included an eighth but it contains little information. 
Sample subtitle: “Beyond the TDM.” 
Even the best chapters have serious problems. The reader should understand that the recommended procedures 
often are not sympathetic to the respondent. Dillman implies (p. 24) that one should estimate on the low side when telling 
the respondent how long a telephone survey will last. Then he mentions problems that a researcher might have with RDD 
but does not mention that RDD is an intrusion on the privacy of people who have unlisted telephone numbers. Also, he 
does not discuss payments to people in the sample: In addition to improving the likelihood of a response, payments 
provide a benefit to the respondent (Armstrong 1975). 
Although the Gunning fog index for this book was 17, which means that the readability is moderately difficult 
(Gunning 1959), 1 found it to be relatively easy to read. The writing is at its best, in my opinion, when Dillman is drawing 
on his experience. 
The design of the book leaves much to be desired. With footnotes at the end of each chapter and references at the 
end of the book, the reader keeps busy turning pages. The poor layout and presentation of the tables give the impression 
that the publisher is trying to save money. That is unfortunate for a book that originally cost $19.95 and currently costs 
$24.95. 
The index is incomplete. For example, the book talks about nonresponse bias, but this is difficult to trace via the 
index. This incompleteness of the index is a serious problem in using the book as a reference. 
A rich and growing body of research exists for mail and telephone surveys. Dillman lists about 270 of these in 
his reference bibliography. One useful feature of this bibliography is that it indicates which studies relate to mail and 
which to telephone interviews. The bibliography covers research up through 1976, and the growth is illustrated in 
Table 1. 
Table 1. Growth in Mail and Telephone Research Studies 
Time Period  Mail  Telephone 
1930s 
1940s 
1950s 
1960s 
1970s (through 1976) 
  .8 
3.7 
5.4 
5.9 
7.4 
  .0 
  .2 
1.2 
2.8 
3.9 
 
Although Dillman provided a large set of references, it was not a complete listing. Table 2 lists some 
review studies that were available prior to the publication of Mail and Telephone Surveys and indicates the 
percentage of articles overlooked by Dillman. For example, Linsky (1975) had 58 references to his article; of these 
58, Dillman mentioned all but 14% in his reference bibliography. In my opinion, most of these omitted articles are 
relevant and useful.   3
Table 2. Completeness of Dillman’s References 
Source  Topic (All Mail Surveys  Articles Cited 
in Source 
% Overlooked 
by Dillman 
Armstrong 1975 
Kanuk & Berenson 1975 
Linsky 1975 
Pressley 1975 
Monetary incentives 
Stimulating responses 
Stimulating responses 
Stimulating responses 
  14 
  82 
  58 
107 
36 
16 
14 
19 
 
On the positive side. Dillman’s references are accurately presented. I checked 23 references against their 
original sources and found only one error (the reference to Bender should be 1957, not 1975). This impressed me as 
an excellent record: In contrast, more than one-third of the references in Linsky (1975) contain errors. 
 
The most serious problem with the book is that the detailed recommendations generally are not supported 
by empirical evidence. Partly this is because previous research is not available on some key points. But Dillman 
could have made better use of existing research. For example, he has little to say about nonresponse bias in mail 
surveys (pp. 52-54). Armstrong and Overton (1977) had found 41 empirical studies on this subject; of these, 
Dillman overlooked 20% in his references. But worst of all, he frequently failed to utilize his own references in 
supporting his recommendations. For the example of nonresponse bias, Dillman discussed only two of his 
references. Dillman’s failure to provide empirical evidence on key issues makes one concerned about the validity of 
his recommendations. 
 
Despite its problems. Dillman’s book should be part of the survey researcher’s working library. He has 
shared much of his valuable experience and he brings together some of the evidence. I hope that he will write an 
expanded edition and complete the job he started 
 
 
References 
 
Armstrong, J. Scott (1975), “Monetary incentives in mail surveys,” Public Opinion Quarterly, 39, 222-226. 
 
Armstrong, J. Scott and Terry S. Overton (1977), “Estimating nonresponse bias in mail surveys,” Journal of 
Marketing Research, 14, 396-402. 
 
Erdos, Paul L. (1972), Professional Mail Surveys. New York: McGraw-Hill. 
 
Gunning, Robert (1959), The Technique of Clear Writing. New York: McGraw-Hill. 
 
Kanuk, Leslie and Conrad Berenson (1975), “Mail surveys and response rates: A literature review,” Journal of 
Marketing Research, 12, 440-453. 
 
Linsky, Arnold S. (1975), “Stimulating responses to mailed questionnaires: A review,” Public Opinion Quarterly, 
39, 82-101. 
 
Pressley, Milton M. (1976), Mail Survey Response: A Critically Annotated Bibliography. Greensboro, NC: Faber. 
 
Wiseman, Frederick (1973), “Factor interaction effects in mail survey response rates,” Journal of Marketing 
Research, 10, 330-333. 

Factor interaction effects in mail survey response rates,”

Mail Survey Response: A Critically Annotated Bibliography.

Mail surveys and response rates: A literature review,”

Monetary incentives in mail surveys,”

Professional Mail Surveys.

Stimulating responses to mailed questionnaires:

The Technique of Clear Writing.

Review of Don A. Dillman's Mail and Telephone Surveys, New York: John Wiley & Sons, 1978.

http://128.118.178.162/eps/get/papers/0502/0502043.pdf

Review of Don A. Dillman's Mail and Telephone Surveys, New York: John Wiley & Sons, 1978.

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics