The paper discusses the validity of computer interviewing compared to the paper and pencil one. The results support the hypothesis that computer interviewing is at least as appropriate data gathering technique as classical survey interviewing. Especially when soliciting some highly personal information, as in the case of a research on sexual behavior, microcomputers - being the "cooler media" - might even yield more accuracy.U radu se uspoređuje validnost anketiranja uz pomoć kompjutora u odnosu spram klasičnog anketiranja. Rezultati govore u prilog hipotezi da je
kompjutorsko anketiranje barem podjednako prihvatljiva metoda prikupljanja podataka kao uobičajena metoda papira i olovke. Postoji mogućnost, posebice kada se radi o osjetljivim podacima, kao što je to slučaj u istraživanju seksualnog ponašanja, da kompjutorsko anketiranje - zbog korištenja "hladnijeg medija" - daje točnije rezultate

Bryce Johnson

Victoria Sturtevant

Hrčak - Portal of scientific journals of Croatia

A Comparison of Computer Interviewing With Traditional Paper and Pencil Format: Soliciting Sensitive InformationThe paper discusses the validity of computer interviewing compared to the paper and pencil one. The results support the hypothesis that computer interviewing is at least as appro­priate data gathering technique as classical survey inte/viewing. Especially when soliciting some highly personal information, as in the case of a research on sexual behavior, microcom­puters - being the "cooler media" - might even yield more accuracy.Key words: METHODOLOGY, COMPUTER INTERVIEWING, SEXUAL BEHA­VIORINTRODUCTIONSociologists are continually attempting to develop new techniques to overcome the problems inherent in the implementation and analysis of social surveys. How do we persuade respondents to answer personal questions? How do we ensure that these answers are truthful and reliable? How do we avoid error and reduce the tedium in coding and analysis of the answers?Survey interviewing can benefit from the revolution brought about generally by personal computers and specifically by software developed for marketing research. Some of the earliest computer applications involved mainframe CATI (computer assisted telephone interviewing) systems. More recently, "user-friendly" personal computers have been used effectively in marketing research for self-administration of interviews; these interviews are often speciali­zed and employ complex options such as pair-wise tradeoffs and multiple skip questions. Outside of market research, however, no research has been conducted, to our knowledge, on using computers in self-administered interviews, i.e., those conducted without an inter­viewer, either on the telephone or in person. We see computer technology as soon creating a breakthrough into this area of survey research, which this study is designed to further.BACKGROUNDThere are a number of microcomputer-based interviewing systems now available. The software used in this research (CI2 system developed by Sawtooth Software) gives the re­searcher complete control over what the respondent sees (a weakness of paper and pencil interviewing) and guides the respondent in providing an answer (e.g., with numbers, short typed responses, complex scale and item choices). The system is flexible; virtually any ques­tions can be asked, and certain probing techniques can be used. This system provides ac­curate execution of skip and flow patterns, so that respondents do not view what is inap­propriate or irrelevant. Finally, answers are recorded, creating numeric and ASCII files for formatting and later analysis by PC or mainframe computer statistical packages.Research in marketing analysis comparing interviewing by computer to more conven­tionally administered interviews reports higher levels of respondent interest, shorterperceived time for completing interviews, higher accuracy of response and analysis, and lower cost per interview. It has been suggested, but not yet demonstrated, that the use of the computerBRYCE JOHNSON VICTORIA STURTEVANTSouthern Oregon State College E-mail: brycej@soscl.sosc.osshe.eduUDKj 303.622:681.31 681.31:303.622 Stručni radPrimljeno: 11. rujna 1994.89Johnson, B., Sturtevant, V.: A comparison of computer interviewing . Rev. za soc., Zgb., Vol XXV (1994), No 1-2: 89-95technique might also affect respondent’s willingness to reveal confidential or sensitive infor­mation. The computer is a "cooler," more enveloping medium; respondents sitting in front of a terminal tend to become absorbed in the task to a much greater degree than if they were using paper and pencils as tools.This study, in asking students about their experiences with sex education as related to later sexual attitudes and behavior, addresses the question of the possible benefits of com­puter interviewing. It is hypothesized that subjects will respond more readily to questions which seek intimate self disclosure in computer interviews than to more traditional paper and pencil questionnaires.APPLICATIONSPersonal computers have been employed as interviewing stations in such places as shop­ping malls, banks, professional conventions, trade shows, and service business lobby locations. Taste tests, consumer profile tests, and client satisfaction surveys have been successfully ad­ministered by personal computers.Several questions reqarding the use of the technique have been raised. Some fear that people unfamiliar with computers would be intimidated by them (most particularly older people), feeling alienated and unwilling to sit down in front of new technology. If finally convinced to approach the computer, the respondent might be distracted into quick and superficial answers by his anxiety. Some people might fear that computers are somehow able to secretly identify them and link them to their answers at a later time.Actual evidence in marketing research contradicts these expectations. Those exami­ning the reactions of the average consumer to this technique find respondents react favorably to the computerized interview. Few report difficulty in interacting with the computer; in fact, most find the experience actively enjoyable. Experienced computer users view the self-administered interview as interesting and simple to complete, finishing the interview more rapidly than others; non-experienced users find the interviews even more interesting (O’Brien). The same study shows that, though some respondents were nervous about the computer at first, 94% found it easy to concentrate once the interview had started. Most were willing to repeat the experience.In studies where the length of the actual interview was timed, computer administered interviews were slightly, but not significantly, longer in duration than conventional paper and pencil interviews. Respondents self-administering computer interviews have been reported as underestimating the time actually spent taking the survey (O’Brien, 1985). It has been sug­gested that one reason computer-administered interviews may take longer is that respondents enter a greater number of responses (O’Brien, 1985). On multiple response questions, the computer group tended to punch in more answers and generate grids of follow-up questions. This computer group also showed a greater variance in response, suggesting that answers are more carefully considered. Another finding, that of a more positive response to interviewer- administered questionnaires on Likert scales questionnaire ratings of service, suggests that respondents are less likely to be frank and candid to an interviewer than to a computer.The results of these studies, therefore, indicate that respondents do not respond nega­tively to computer administered questionnaires; in fact, they enjoy completing them and underestimate the time spent in front of the screen. Computer respondents report no diffi­culty in self-administration of or concentration on the questionnaire and analysis of their actual responses suggests they are less worried about the response of the computer to the substance of their answers; hence they are more likely to give honest opinions.Published research shows that the most sensitive or personal questions in surveys, those about personal income, have had a significantly lower refusal rate in computer-administered questionnaires, but can we expect other more confidential questions to reveal the same advantage?90Johnson, B., Sturtevant, V.: A comparison of computer interviewing . Rev. za soc., Zgb., Vol XXV (1994), No 1-2: 89-95THE STUDYThe students in one of the authors’ research methods course designed and coordinated the administration of the questionnaire used for this study. This served an obvious pedago­gical function, but also provided legitimation on campus for the study and potential for future analysis of the substantive responses. The questionnaire included open-ended, scaled, text and option questions.Questions were designed to test the research methods course’s hypothesis that sex education affected later sexual attitudes and behavior; numerous control variables such as parental occupational background and attitudes about sex roles, more broadly, were included. Questions of a personal or potentially embarrassing nature were developed, such as frequency of intercourse, number of pregnancies and abortions, incidence of sexual diseases, and preference of sexual partners. Even more personal questions potentially applicable to the hypothesis of this particular study (such as frequency of masturbation) could have been asked, but could not be justified given the class’s hypothesis. The questionnaire was pretested for clarity, readability and duration.A paper and pencil questionnaire was designed to parallel the machine administered form; nearly identical questions were asked in the same order. (Complicated skips are more cumbersome on paper and pencil and respondents were required to advance through pages if questions were offensive or not appropriate). Those skips, while involving some page turning in the paper and pencil version, are invisible on the computer. At various points, often before potentially sensitive questions, respondents were given the option of either terminating the questionnaire or skipping on to questions less personal in nature.Classes were selected in order to present a cross section of the student body (in terms of age, major, and sex), and those offering possible desensitization to the topic (e.g., human sexuality) were omitted. All students were required by their instructors to report to the interviewing stations, yet they were free to leave any time thereafter.Students were preassigned a number which then randomly designated whether they would self-administer a computer or a paper and pencil questionnaire. Table 1 shows that the random assignment of students provided similar composition of groups in terms of age, sex and major (the computer administrated group is slightly, but not significantly, older and more female, and fewer have declared majors). Paper and pencil questionnaires were admi­nistered in a small classroom with large tables, much like an exam situation, and computer questionnaires were administered in a room set up with seven computer "stations" arranged in a semi-private configuration. Each respondent faced a screen which contained simple directions on the execution of the program. Numerals were most often used to select an answer, and the enter/return key was used for continuing on to the next question, although some typing of longer answers, such as names of occupations and religions, was required. A notation on the screen reminded the respondent that a key could be used to back up to the previous question. Although a student "assistant" was available nearby to help with any problems in self-administration, she was seldom called upon.Table 1: Paper and pencil and computer groups compared according to age, sex and declared majorComputer Paper/PencilMean age 23.1 22.7Sex: Female 60% 56%Male 40% 44%Major Declared 76% 84%(N) (127) (129)91Johnson, B., Sturtevant, V.: A comparison of computer interviewing . Rev. za «k., Zgb., Vol XXV (1994), No 1-2: 89-95Two hundred and fifty-six interviews were completed. The data from the 127 interviews completed on the machines were immediately available for analysis. The data from the 129 paper and pencil interviews were coded by the research students by entering the paper and pencil answers into the computer questionnaire program. Presumably no coding error exists in the machine interviewed data, but we would expect a certain amount of error to be introduced in the conversion stage of the paper and pencil data.RESULTSSome informal observations were made which support previous research on user re­sponse to computer administered questionnaires. As students left the lab we heard remarks such as "I like sociology," "that was fun," "computers aren’t so bad" and "I want to know the results." Post-test questionnaires were not administered, but informally volunteered com­ments suggested that students, even those unfamiliar with computers, found self-administra­tion by computer interesting, easy and fun. Duration of computer administered question­naires was somewhat shorter than that of paper and pencil questions; again, this phenomenon was not officially measured as it was not central to the hypothesis of this study.Analysis of the data relevant to the hypothesis shows some interesting differences between paper and pencil and computer administered questionnaires. Only two students declined to take the questionnaire, one from each group, and few students chose the option of skipping the section of questions concerning sexual behavior. However, of those opting not to answer the more personal, possibly embarrassing questions, a greater proportion were in the paper and pencil group (5%, compared to 2% in the computer administered group).Table 2: Respondents willing to answer personal questions about sexual behaviorComputer Paper/PencilYes 98% 95%No 2% 5%(N) (127) (129)When asked the sex of their partner(s) in the last six months, 17% of the computer group chose either homosexual or bisexual partners while only 9% did so in the paper and pencil group (Table 3).Table 3: Sex of partner(s)Computer Paper/PencilOpposite 82% 91%Same 15% 8%Both 2% 0%(N) (85) (90)Questions about abortions were answered only by women who agreed to respond to this section and who had been pregnant. Of this group, 64% in the machine group reported experiencing abortions compared to only 40% in the paper and pencil group.92Johnson, B., Sturtevant, V.: A comparison of computer interviewing . Rev. za soc., Zgb., Vol XXV (1994), No 1-2: 89-95Table 4: Females ever had abortionComputer Paper/PencilYes 65% 40%No 35% 60%(N) (29) (25)These differences all tend to support the hypothesis that people are more willing to disclose personal information in self-administered computer interviews.An interesting and unexpected, hut explainable, finding is that paper and pencil ques­tionnaires had a higher percentage of individuals claiming greater frequency of sexual en­counters and a greater number of sexual partners: 11% of paper and pencil respondents reported having more than five partners in the past six months, in contrast to 2% of com­puter respondents (Table 5).Table 5: Number of sexual partners in last six monthsComputer Paper/PencilOne 71% 62%> One 27% 27%> Five 2% 11%(N) (85) (90)Similarly, 53% of paper and pencil respondents reported having sex more than once aweek, compared to 38% of computer respondents.Table 6: Frequency of sexual activityComputer Paper/PencilOnce/Month 12% 9%> Once/Month 19% 15%Once/Week 32% 24%> Once/Week 38% 53%(N) (85) (89)The same proportions in each group used similar types of birth control and similar numbers became pregnant. There were no refusals in answering whether or not they had had abortions.DISCUSSIONThis section might better be titled, "Out of the closet and into the locker room: new visions of college sexuality." While this study found that overall answers were similar in the two groups, some results have interesting implications and suggest further research. Few students resisted discussing their sexual activity and preferences; neither group had high rates of refusal on personal questions. In fact, one might speculate that one group, males answering paper and pencil questionnaires, may have found a forum for showing off, perhaps even bragging, about their sexual activity. In contrast, the still more closeted sexual topics93Johnson, B, Sturtevant, V.: A comparison of computer interviewing . Rev. za soc., Zgb., Vol XXV (1994), No 1-2: 89-95such as homosexual partners and abortions (subjects less likely to be bragged about in the locker room) may be underreported using the paper and pencil method.The difference in results between these two groups may reflect actual differences in their behavior, or the paper and pencil group may be exaggerating in their reports of fre­quency and number of sexual encounters and partners (and correspondingly in their respon­sibility for birth control), or the computer group may be underplaying their activity and responsibility. Our guess is that the ambience of the paper and pencil questionnaire more nearly approximated the locker- room, in that respondents not only chatted with one another but felt more flexibility in answering questions. In contrast, we feel that people interacting with the computer, especially those with little experience with the technology, didn’t feel they could allow the same margin of error. One doesn’t "mess around" with a computer; the machine is foreboding to some, or at least commands some authority with its intricate programming, bold graphics and speedy and accurate responses. For the more "closeted" type of topic, the computer may foster a more private and engaging session than is possible with the paper and pencil format.Our low rate of refusals, in both groups, suggests that we should have asked more personal questions. We were surprised that none of the females refused to answer questions about their abortions, although these women constitued only a small, extensively qualified group (they had to be willing to answer the questions, sexually active, and formerly pregnant; only 54 of the sample of 148 women met these prerequisites.)In summary, these findings suggest that methods of interviewing by paper and pencil and computer are parallel, that no compromises are made by employing the computer, that people have no difficulty with self-administration by machine. In fact, there is some sugges­tion that people are more likely to answer honestly to the computer, that they are less willing to exaggerate attributes perceived as being positive and less likely to hide actions that might be negatively sanctioned. Interviewing with computers has unquestionable advantages in terms of speed, accuracy, and cost (if hardware is available), and it combines the positive attributes of a personal interview with a trained interviewer (in terms of probes and managed skips) with paper and pencil (getting people to answer questions which might be too personal for response to a person). This research also suggests that computers may be more useful in getting people to answer honestly questions which are personal.REFERENCESJohnson, Richard M. (1980) "Measurement of Consumer Values Using Computer Interactive Techni­ques." In David B. Montgomery and Dick R. Wittink (eds.) Market Measurement and Analysis. Cambridge, MA: Marketing Science Institutes.McBride, James N. and Richard M. Johnson (1980) "Respondent Reaction to Computer-Interactive Interviewing Techniques." Paper Presented at ESOMAR Conference, Monte Carlo.McBrien, Bernadette D. (1984) "The Role of the Personal Computer in Data Collection, Data Analysis, and Data Presentation: A Case Study." Paper Presented at ESOMAR Conference, Nice, France.------(1985) "The Big Advantages of Little Computers." Journal of Data Collection 25 (Fall): 10-13.McDaniel, Carl and Roger Gates (1982) "Personal vs. Computer Interviewing: Comparisons of Data Quality." Journal of Data Collection 22: 15-20.94Johnson, B., Sturtevant, V.: A comparison of computer interviewing . Rev. za soc., Zgb., Vol XXV (1994), No 1-2: 89-95USPOREDBA KOMPJUTORSKOG I KLASIČNOG ANKETIRANJA: PRIBAVLJANJE OSJETLJIVIH PODATAKABRYCE JOHNSON, VICTORIA STURTEVANTSouthern Oregon State CollegeU radu se uspoređuje validnost anketiranja uz pomoć kompjutora u odnosu spram klasičnog anketiranja. Rezultati govore u prilog hipotezi da je kompjutorsko anketiranje barem podjednako prihvatljiva metoda prikupljanja podataka kao uobičajena metoda papira i olovke. Postoji mogućnost, posebice kada se radi o osjetljivim podacima, kao što je to slučaj u istraživanju seksu­alnog ponašanja, da kompjutorsko anketiranje - zbog korištenja ",hladnijeg medija" - daje točnije rezultate.95

Usporedba kompjutorskog i klasičnog anketiranja: pribavljanje osjetljivih podataka

Abstract

Similar works

Full text

Available Versions

Hrčak - Portal of scientific journals of Croatia