Public Opinion Quarterly Advance Access first published online on December 4, 2008
This version published online on December 23, 2008
Public Opinion Quarterly, doi:10.1093/poq/nfn050
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
Completion Time and Response Order Effects in Web Surveys
Address correspondence to Neil Malhotra; e-mail: neilm{at}stanford.edu.
The use of the World Wide Web to conduct surveys has grown rapidly over the past decade, raising concerns regarding data quality, questionnaire design, and sample representativeness. This research note focuses on an issue that has not yet been studied: Are respondents who complete self-administered Web surveys more quickly—perhaps taking advantage of participation benefits while minimizing effort—also more prone to response order effects, a manifestation of "satisficing"? I surveyed a random sample of the US adult population over the Web and manipulated the order in which respondents saw the response options. I then assessed whether primacy effects were moderated by the overall length of time respondents took to complete the questionnaires. I found that low-education respondents who filled out the questionnaire most quickly were most prone to primacy effects when completing items with unipolar rating scales. These results have important implications for various aspects of Web survey methodology including panel management, human–computer interaction, and response order randomization.
NEIL MALHOTRAis with the Department of Political Science, Stanford University, Encina Hall West, Room 100, Stanford, CA 94305-6044, USA. I gratefully acknowledge Jon Krosnick, Paul Sniderman, Jed Stiglitz, Daniel Schneider, John Bullock, Matt Levendusky, Elizabeth Popp, Alex Kuo, and anonymous reviewers for valuable suggestions. I thank Diana Mutz of TESS and Poom Nukulkij of Knowledge Networks for their time and assistance. The data collection was funded by Timeshare Experiments in the Social Sciences (TESS).