Public Opinion Quarterly Advance Access originally published online on May 28, 2009
Public Opinion Quarterly 2009 73(2):325-337; doi:10.1093/poq/nfp029
| ||||||||||||||||||||||||||||||||||||||||||||||||||
Open-Ended Questions in Web Surveys
Can Increasing the Size of Answer Boxes and Providing Extra Verbal Instructions Improve Response Quality?
Address correspondence to Jolene D. Smyth; e-mail: jsmyth2{at}unl.edu.
Previous research has revealed techniques to improve response quality in open-ended questions in both paper and interviewer-administered survey modes. The purpose of this paper is to test the effectiveness of similar techniques in web surveys. Using data from a series of three random sample web surveys of Washington State University undergraduates, we examine the effects of visual and verbal answer-box manipulations (i.e., altering the size of the answer box and including an explanation that answers could exceed the size of the box) and the inclusion of clarifying and motivating introductions in the question stem. We gauge response quality by the amount and type of information contained in responses as well as response time and item nonresponse. The results indicate that increasing the size of the answer box has little effect on early responders to the survey but substantially improved response quality among late responders. Including any sort of explanation or introduction that made response quality and length salient also improved response quality for both early and late responders. In addition to discussing these techniques, we also address the potential of the web survey mode to revitalize the use of open-ended questions in self-administered surveys.
JOLENE D. SMYTH is with the Survey Research and Methodology Program and the Department of Sociology, University of Nebraska-Lincoln, 724 Oldfather Hall, Lincoln, NE 68588, USA. DON A. DILLMAN is with the Washington State University, Wilson Hall 133, PO Box 644014, Pullman, WA 99164, USA. LEAH MELANI CHRISTIAN is with the Pew Research Center for the People & the Press, 1615 L Street NW, Suite 700, Washington, DC, 20036-5610, USA. MALLORY MCBRIDE is with the University of Montana, 913 Pullman St. #2, Missoula, MT 59802, USA. This paper is a revised version of a paper presented at the 2006 World Association for Public Opinion Research Meeting held during May 16–21 in Montreal, Quebec, Canada. Research reported here was conducted by the Washington State University Social and Economic Sciences Research Center (SESRC) with support from the USDA-National Agricultural Statistics Service and the National Science Foundation Division of Science Resources Statistics under cooperative agreement no. 43-3AEU-1-80055 (Don A. Dillman, Principal Investigator). Additional support for data collection was provided by the SESRC and the Gallup Organization. The authors wish to thank the reviewers for their helpful comments and suggestions on a previous draft of this paper.