Public Opinion Quarterly Advance Access originally published online on December 12, 2008
Public Opinion Quarterly 2008 72(5):892-913; doi:10.1093/poq/nfn059
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
This article appears in the following Public Opinion Quarterly issue: Special Issue: Web Survey Methods [View the issue table of contents]
Eye-Tracking Data
New Insights on Response Order Effects and Other Cognitive Shortcuts in Survey Responding
Address correspondence to Mirta Galesic; e-mail: galesic{at}mpib-berlin.mpg.de.
Survey researchers since Cannell have worried that respondents may take various shortcuts to reduce the effort needed to complete a survey. The evidence for such shortcuts is often indirect. For instance, preferences for earlier versus later response options have been interpreted as evidence that respondents do not read beyond the first few options. This is really only a hypothesis, however, that is not supported by direct evidence regarding the allocation of respondent attention. In the current study, we used a new method to more directly observe what respondents do and do not look at by recording their eye movements while they answered questions in a Web survey. The eye-tracking data indicate that respondents do in fact spend more time looking at the first few options in a list of response options than those at the end of the list; this helps explain their tendency to select the options presented first regardless of their content. In addition, the eye-tracking data reveal that respondents are reluctant to invest effort in reading definitions of survey concepts that are only a mouse click away or paying attention to initially hidden response options. It is clear from the eye-tracking data that some respondents are more prone to these and other cognitive shortcuts than others, providing relatively direct evidence for what had been suspected based on more conventional measures.
MIRTA GALESIC is with the Joint Program in Survey Methodology, University of Maryland, College Park, MD 20742, USA and also with the Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany. ROGER TOURANGEAU, MICK P. COUPER AND FREDERICK G. CONRAD are with the Institute for Social Research, University of Michigan, P.O. Box 1248, Ann Arbor, MI 48109, USA and also with the Joint Program in Survey Methodology, University of Maryland, College Park, MD 20742, USA. We are grateful to Scott Fricker, Duane Gilbert, Ting Yan, and Cong Ye for their help in conducting this study and to Rik Pieters, Marc Sebrechts, and an anonymous reviewer for their advice regarding the analysis of the eye-tracking data. This research was supported by a grant from the National Institute for Child Health and Human Development (R01 HD041386-01A1) to Roger Tourangeau, Mick Couper, Fred Conrad, and Reg Baker. The National Institute for Child Health and Human Development is not responsible for the conclusions presented here.