Public Opinion Quarterly Advance Access published online on February 8, 2008
Public Opinion Quarterly, doi:10.1093/poq/nfn005
| ||||||||||||||||||||||||||||||||||||||||||||||||||||
Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?
Address correspondence to Jolene D. Smyth; e-mail: jsmyth2{at}unl.edu
Recent experimental research has shown that respondents to forced-choice questions endorse significantly more options than respondents to check-all questions. This research has challenged the common assumption that these two question formats can be used interchangeably but has been limited to comparisons within a single survey mode. In this paper we use data from a 2004 random sample survey of university students to compare the forced-choice and check-all question formats across web self-administered and telephone interviewer-administered surveys as they are commonly used in survey practice. We find that the within-mode question format effects revealed by previous research and reaffirmed in the current study appear to persist across modes as well; the telephone forced-choice format produces higher endorsement than the web check-all format. These results provide further support for the argument that the check-all and forced-choice question formats do not produce comparable results and are not interchangeable formats. Additional comparisons show that the forced-choice format performs similarly across telephone and web modes.
JOLENE D. SMYTH is with the Survey Research and Methodology Program and the Department of Sociology at the University of Nebraska-Lincoln, PO Box 880241, Lincoln, NE 68588, USA.
LEAH MELANI CHRISTIAN is with Pew Research Center for the People & the Press, 1615 L Street NW, Suite 700, Washington, DC 20036-5610, USA.
DON A. DILLMAN is with Washington State University, Wilson Hall 133, PO Box 644014, Pullman, WA 99164, USA. This is a revision of a paper presented at the Second International Conference on Telephone Survey Methodology, Miami, FL, USA, January 11–15, 2006. Analysis of these data was supported by funds provided to the Washington State University Social and Economic Sciences Research Center (SESRC) under Cooperative Agreement 43-3AEU-1-80055 with the USDA-National Agricultural Statistics Service, supported by the National Science Foundation, Division of Science Resource Statistics. Data collection was financed by the SESRC and additional funds provided to the SESRC by the Gallup Organization.