Public Opinion Quarterly Advance Access originally published online on January 29, 2009
Public Opinion Quarterly 2008 72(5):1008-1032; doi:10.1093/poq/nfn065
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
This article appears in the following Public Opinion Quarterly issue: Special Issue: Web Survey Methods [View the issue table of contents]
Computing Response Metrics for Online Panels
Address correspondence to Mario Callegaro; e-mail: mcallegaro{at}knowledgenetworks.com; mail at Knowledge Networks, Inc., 1350 Willow Road, Ste. 102, Menlo Park, CA 94025.
As more researchers use online panels for studies, the need for standardized rates to evaluate these studies becomes paramount. There are currently many different ways and conflicting terminology used to compute various metrics for online panels. This paper discusses the sparse literature on how to compute response, refusal, and other rates and proposes a set of formulas and a standardized terminology that can be used to calculate and interpret these metrics for online panel studies. A description of and distinction between probability-based and volunteer opt-in panels is made since not all metrics apply to both types. A review of the existing discussion and recommendations, mostly from international organizations, is presented for background and context. In order to propose response and other metrics, the different stages involved in building an online panel are delineated. Metrics associated with these stages contribute to cumulative response rate formulas that can be used to evaluate studies using online probability-based panels. (Only completion rates can be calculated with opt-in panels.) We conclude with a discussion of the meaning of the different metrics proposed and what we think should be reported for which type of panel.
MARIO CALLEGARO AND CHARLES DISOGRA are with Knowledge Networks, Inc. Previous versions of the paper were presented at the annual Pacific chapter of the American Association for Public Opinion Research conference (AAPOR) in San Francisco in December 2007, at the Political Psychology Research Group at Stanford University in March 2008, and at the 63rd AAPOR Annual conference in May 2008. The authors appreciated the suggestions of these audiences. The authors thank Mike Dennis, Erica Demme, Fran Featherston, Erlina Hendarwan, Ana Villar, and Tom Wells for their useful comments. Willem Saris gave us information about the now discontinued Dutch Telepanel and Marcel Das provided with details of the CentERdata CentERpanel and LISS. We are also very grateful to the two anonymous reviewers and the valuable suggestions made by the POQ editors.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Ansolabehere and D. M. Konisky Public Attitudes Toward Construction of New Power Plants Public Opin Q, September 1, 2009; 73(3): 566 - 577. [Abstract] [Full Text] [PDF]
|
|