Public Opinion Quarterly Advance Access originally published online on November 25, 2008
Public Opinion Quarterly 2008 72(5):985-1007; doi:10.1093/poq/nfn060
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
This article appears in the following Public Opinion Quarterly issue: Special Issue: Web Survey Methods [View the issue table of contents]
Effects of Design in Web Surveys
Comparing Trained and Fresh Respondents
Address correspondence to Vera Toepoel; e-mail: v.toepoel{at}uvt.nl.
In this paper, we investigate whether there are differences in the effect of instrument design between trained and fresh respondents. In three experiments, we varied the number of items on a screen, the choice of response categories, and the layout of a five-point rating scale. In general, effects of design carry over between trained and fresh respondents. We found little evidence that survey experience influences the question-answering process. Trained respondents seem to be more sensitive to satisficing. The shorter completion time, higher interitem correlations for multiple-item-per-screen formats, and the fact that they select the first response options more often indicate that trained respondents tend to take shortcuts in the response process and study the questions less carefully.
VERA TOEPOEL AND MARCEL DAS are with the CentERdata, Tilburg University, PO Box 90153, 5000 LE Tilburg, The Netherlands. ARTHUR VAN SOEST is with the Department of Econometrics & OR, Tilburg University and also with the Netspar; RAND; OSA; IZA; DIW, PO Box 90153, 5000 LE Tilburg, The Netherlands. We are grateful to Mick Couper, Joachim Winter, two anonymous referees, the (other) participants of the MESS workshop in Zeist, August 2008, and the (other) participants of the Panel Survey Methods Workshop in Essex, July 2008, for useful comments.