| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
An Experimental Comparison of Web and Telephone Surveys
SCOTT FRICKER is a psychologist at the Bureau of Labor Statistics and a graduate student at the Joint Program in Survey Methodology, University of Maryland. ROGER TOURANGEAU is a research professor at the Institute for Social Research, University of Michigan, and director of the Joint Program in Survey Methodology, University of Maryland. MIRTA GALESIC and TING YAN are graduate students at the Joint Program in Survey Methodology, University of Maryland. The work reported here was conducted as part of the Joint Program in Survey Methodology Practicum. We are grateful to Sarah Dipko, who helped direct the Practicum, to the students in that class, and to the National Science Foundation for its support of the study. We are especially grateful to Robert Bell and Jeri Mulrow at the National Science Foundation for their help in designing the study and to Frauke Kreuter and Carolina Casas-Cordero for their comments on an earlier draft of the paper. Finally, we thank Chintan Turakhia and Dean Williams at Schulman, Ronca, and Bucuvalas, Inc., for their direction of the study at SRBI. The authors contributed equally to the research and are listed in alphabetical order.
Address correspondence to Roger Tourangeau; e-mail: RTourang{at}survey.umd.edu.
We carried out an experiment that compared telephone and Web versions of a questionnaire that assessed attitudes toward science and knowledge of basic scientific facts. Members of a random digit dial (RDD) sample were initially contacted by telephone and answered a few screening questions, including one that asked whether they had Internet access. Those with Internet access were randomly assigned to complete either a Web version of the questionnaire or a computer-assisted telephone interview. There were four main findings. First, although we offered cases assigned to the Web survey a larger incentive, fewer of them completed the online questionnaire; almost all those who were assigned to the telephone condition completed the interview. The two samples of Web users nonetheless had similar demographic characteristics. Second, the Web survey produced less item nonresponse than the telephone survey. The Web questionnaire prompted respondents when they left an item blank, whereas the telephone interviewers accepted "no opinion" answers without probing them. Third, Web respondents gave less differentiated answers to batteries of attitude items than their telephone counterparts. The Web questionnaire presented these items in a grid that may have made their similarity more salient. Finally, Web respondents took longer to complete the knowledge items, particularly those requiring open-ended answers, than the telephone respondents, and Web respondents answered a higher percentage of them correctly. These differences between Web and telephone surveys probably reflect both inherent differences between the two modes and incidental features of our implementation of the survey. The mode differences also vary by item type and by respondent age.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  E. W. Wolfe, P. D. Converse, O. Airen, and N. Bodenhorn Unit and Item Nonresponses and Ancillary Information in Web- and Paper-Based Questionnaires Administered to School Counselors Measurement and Evaluation in Counseling and Development, July 1, 2009; 42(2): 92 - 103. [Abstract] [PDF]
|
|
|
|
 |
|
 J. L. Holland and L. M. Christian The Influence of Topic Interest and Interactive Probing on Responses to Open-Ended Questions in Web Surveys Social Science Computer Review, May 1, 2009; 27(2): 196 - 212. [Abstract] [PDF]
|
|
|
|
 |
|
 D. Heerwegh and G. Loosveldt Face-to-Face versus Web Surveying in a High-Internet-Coverage Population: Differences in Response Quality Public Opin Q, December 1, 2008; 72(5): 836 - 846. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. A. Klofstad, S. Boulianne, and D. Basson Matching the Message to the Medium: Results from an Experiment on Internet Survey Email Contacts Social Science Computer Review, November 1, 2008; 26(4): 498 - 509. [Abstract] [PDF]
|
|
|
|
 |
|
 J. Whitlock, J. Eckenrode, and D. Silverman Self-injurious behaviors in a college population. Pediatrics, June 1, 2006; 117(6): 1939 - 1948. [Abstract] [Full Text] [PDF]
|
|