Public Opinion Quarterly Advance Access originally published online on August 14, 2007
Public Opinion Quarterly 2007 71(3):325-348; doi:10.1093/poq/nfm024
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Response Order Effects in Dichotomous Categorical Questions Presented Orally
The Impact of Question and Respondent Attributes
Address correspondence to Allyson L. Holbrook; (e-mail: allyson{at}uic.edu) or Jon A. Krosnick (e-mail: krosnick{at}stanford.edu).
Using data from 548 experiments in telephone surveys conducted by the Gallup Organization, we explored how attributes of questions and respondents moderate response order effects in dichotomous categorical questions. These effects were predominantly recency effects and occurred most in questions that were more difficult to comprehend (especially among respondents with the least education), with response choices that were more difficult to comprehend (because they were complete sentences instead of words or phrases and because they were not mutually exclusive), and that were asked after many prior questions. Recency effects were also more common in questions that explicitly or implicitly encouraged respondents to wait until they had heard all the answer choices before formulating a judgment than in questions that induced respondents to begin formulating a judgment before all the answer choices had been read (especially among the least educated respondents). A study of interviewer behavior revealed patterns of pausing between and within sentences that help to explain why some types of questions are especially prone to recency effects and others are not.
ALLYSON L. HOLBROOK is with the University of Illinois, Chicago, USA. JON A. KROSNICK is with Stanford University, CA, USA. DAVID MOORE is with the University of New Hampshire, and ROGER TOURANGEAU is with the Institute for Social Research, University of Michigan, Ann Arbor, MI, USA and also with Joint Program in Survey Methodology, University of Maryland, College Park, MD, USA.
The survey results reported here were obtained from searches of the iPOLL Databank and other resources provided by the Roper Center for Public Opinion Research, University of Connecticut.