Public Opinion Quarterly Advance Access originally published online on April 1, 2009
Public Opinion Quarterly 2009 73(1):56-73; doi:10.1093/poq/nfp009
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||
Use of Expert Ratings as Sampling Strata for a More Cost-Effective Probability Sample of a Rare Population
Address correspondence to Marc N. Elliott; e-mail: elliott{at}rand.org.
We consider situations in which externally observable characteristics allow experts to quickly categorize individual households as likely or unlikely to contain a member of a rare target population. This classification can form the basis of disproportionate stratified sampling such that households classified as "unlikely" are sampled at a lower rate than those classified as "likely," thereby reducing screening costs. Design weights account for this approach and allow unbiased estimates for the target population. We demonstrate that with sensitivity and specificity of expert classification at least 70 percent, and ideally at least 80 percent, our approach can economically increase effective sample size for a rare population. We develop heuristics for implementing this approach and demonstrate that sensitivity drives design effects and screening costs whereas specificity only drives the latter. We demonstrate that the potential gains from this approach increase as the target population becomes rarer. We further show that for most applications, unlikely strata should be sampled at 1/6 to 1/2 the rate of likely strata. This approach was applied to a survey of Cambodian immigrants in which the 82 percent of households rated "unlikely" were sampled at 1/4 the rate as "likely" households, reducing screening from 9.4 to 4.0 approaches per complete. Sensitivity and specificity were 86 percent and 91 percent, respectively. Weighted estimation had a design effect of 1.26, so screening costs per effective sample size were reduced by 47 percent. We also note that in this instance, expert classification appeared to be uncorrelated with survey outcomes of interest among eligibles.
MARC N. ELLIOTT, JUDITH PERLMAN, GRANT N. MARSHALL AND KATRIN HAMBARSOOMIANS are with the RAND, 1776 Main Street, Santa Monica, CA 90401, USA. DANIEL MCCAFFREY is with the RAND, 4570 Fifth Avenue, Suite 600, Pittsburgh, PA 15213, USA. This study was supported by grants R01MH059555 from the National Institute of Mental Health and R01AA013818 from the National Institute on Alcohol Abuse and Alcoholism, both awarded to Grant Marshall (Principal Investigator). Marc Elliott was supported in part by the Centers for Disease Control and Prevention (CDC U48/DP000056, Mark Schuster—Principal Investigator). The contents of the publication are solely the responsibility of the authors and do not necessarily reflect the official views of the sponsors. The authors would like to thank Kate Sommers-Dawes for assistance with the preparation of the manuscript.