Public Opinion Quarterly - recent issues

Editor's Note

Miller, P. V. — 2008-03-14

A Comparison of Address-Based Sampling (ABS) Versus Random-Digit Dialing (RDD) for General Population Surveys

Link, M. W., Battaglia, M. P., Frankel, M. R., Osborn, L., Mokdad, A. H. — 2008-03-14

Valid and reliable public health data are becoming more difficult to obtain through random-digit dial (RDD) telephone surveys. As a result, researchers are evaluating different survey designs (i.e., sampling frame and survey mode combinations) as complements or alternatives to RDD. Traditionally, mail surveys of the general public have been limited due to a lack of a complete sampling frame of households. Recent advances in electronic record keeping, however, have allowed researchers to develop a sample from a frame of addresses (e.g., the U.S. Postal Service Delivery Sequence File, which appears to provide coverage which rivals or possibly exceeds that obtained through RDD sampling methods). To test the use of this frame for surveying adults aged 18 years and older across a wide geographic area, a pilot study was conducted as part of the 2005 Behavioral Risk Factor Surveillance System (BRFSS). The pilot compared use of a traditional, RDD telephone survey methodology to an approach using a mail version of the questionnaire completed by a random sample of households drawn from an address-based frame. The findings indicate that the mail survey approach can achieve higher response rates in low-response-rate states (<40%) than RDD (particularly when two mailings are sent). Additionally, the address frame with mail survey design provides access to cell phone only households and offers cost savings over the telephone approach. The resulting sample, however, significantly overrepresents non-Hispanic whites and people with higher levels of education.

A "Brute Force" Estimation of the Residency Rate for Undetermined Telephone Numbers in an RDD Survey

Kennedy, C., Keeter, S., Dimock, M. — 2008-03-14

In random digit dial (RDD) telephone surveys, some proportion of the sampled telephone numbers cannot definitively be classified as eligible or ineligible. Each call attempt to these numbers generally results in either a busy signal or a ring with no answer. According to the profession's standard definitions, the proportion of these unresolved numbers that are, in fact, eligible is known as "e" and should be accounted for in response rate calculations. We used call records and a directory listed indicator from a survey with an extended calling period (and other features – letters, incentives, etc.) to resolve as many unknown cases as possible and thus derive an empirical estimate (.47) for "e." We found additional support for this estimate by matching the unresolved telephone numbers from a separate survey to residency information from a commercial data vendor. The estimate of "e" applies to national RDD surveys featuring approximately a six-call design.

Effects and Costs of Tracing Strategies on Nonresponse Bias in a Survey of Workers with Low-Back Injury

2008-03-14

Declining rates of participation are an increasing challenge for studies that involve telephone surveys. This study examined the costs of a telephone survey methodology that used increasingly intensive tracing methods to track a pool of claimants who had sustained occupational back injuries. It also compared the respondent sample to people who refused the survey and/or were not located or contacted. 3,181 claimants were drawn from a database maintained by the Missouri Division of Workers’ Compensation (DWC) and 1,475 completed a telephone interview. The DWC database provided data reflecting monetary and disability outcomes for all potential participants; telephone interviews provided additional self-reported data. More intensive tracing strategies improved the representativeness of the sample, yielding more women and minority participants. Relative to less intensive techniques, advanced tracing efforts located people whose DWC records were more similar to claimants not located. While the hands-on tracing efforts reduced the apparent bias of the respondent sample, costs increased as tracing strategies intensified – over $98,000 was spent to trace 1,027 claimants who were never located or contacted. The results suggest some guidelines that may facilitate decision-making for researchers and funders who must balance the trade-off between costs and nonresponse bias when planning surveys.

The Social Roots of the Partisan Gender Gap

Burden, B. C. — 2008-03-14

I suggest that the gender gap in party identification is dependent on question wording and asymmetric stereotypes about men's and women's partisan preferences. A survey experiment reanalyzes the gender gap by comparing the standard partisan battery to an alternative version that emphasizes feelings rather than thoughts. Bringing question wording into closer alignment with theory causes the gender gap to shrink. This happens in part because the "feel" questions find women to be less Democratic than did the "think" questions. Moreover, reduction of the gender gap occurs mostly among highly sophisticated women and not those usually susceptible to question wording effects. Contrary to popular wisdom, men and women appear to be more, not less, alike politically when feelings are primed.

Social Desirability Effects and Support for a Female American President

Streb, M. J., Burrell, B., Frederick, B., Genovese, M. A. — 2008-03-14

Public opinion polls show consistently that a substantial portion of the American public would vote for a qualified female presidential candidate. Because of the controversial nature of such questions, however, the responses may suffer from social desirability effects. In other words, respondents may be purposely giving false answers as not to violate societal norms. Using an unobtrusive measure called the "list experiment," we find that public opinion polls are indeed exaggerating support for a female president. Roughly 26 percent of the public is "angry or upset" about the prospect of a female president. Moreover, this level of dissatisfaction is constant across several demographic groups.

An Experiment on the Measurement of Political Knowledge in Surveys

Sturgis, P., Allum, N., Smith, P. — 2008-03-14

In a series of articles, Mondak and colleagues argue that the conventional way of measuring political knowledge in surveys is flawed. Personality related "propensity to guess" underestimates the level of political knowledge in the population and distorts estimates of between group differences, when a DK alternative is offered. This has led Mondak to recommend the use of closed-ended items on which DKs are not explicitly offered, following best practice in the field of educational testing. In this article, we present the results of an experimental study which calls into question the wisdom of this approach. Our results show little evidence of partial knowledge concealed within DK responses; when people who initially select a DK alternative are subsequently asked to provide a "best guess," they fare statistically no better than chance. We conclude that opinion researchers should be cautious about adopting Mondak's recommendations for the design of political knowledge items in surveys.

Does "Yes or No" on the Telephone Mean the Same as "Check-All-That-Apply" on the Web?

Smyth, J. D., Christian, L. M., Dillman, D. A. — 2008-03-14

Recent experimental research has shown that respondents to forced-choice questions endorse significantly more options than respondents to check-all questions. This research has challenged the common assumption that these two question formats can be used interchangeably but has been limited to comparisons within a single survey mode. In this paper we use data from a 2004 random sample survey of university students to compare the forced-choice and check-all question formats across web self-administered and telephone interviewer-administered surveys as they are commonly used in survey practice. We find that the within-mode question format effects revealed by previous research and reaffirmed in the current study appear to persist across modes as well; the telephone forced-choice format produces higher endorsement than the web check-all format. These results provide further support for the argument that the check-all and forced-choice question formats do not produce comparable results and are not interchangeable formats. Additional comparisons show that the forced-choice format performs similarly across telephone and web modes.

How Too Little Can Give You a Little Too Much: Determining the Number of Household Phone Lines in RDD Surveys

Merkle, D., Langer, G. — 2008-03-14

Accurately assessing the number of household phone lines in random-digit dialed (RDD) surveys for use in weighting has become more complicated over time. This study evaluates phone line weighting by asking an in-depth battery of seven questions designed to measure the number of available land lines in the home more precisely. The results show that this weighting correction is not so simple, and that inadequately parsing the type and use of household phone numbers produces an inaccurate weight. Failing explicitly to remove irrelevant phone numbers, especially cell phones, produces a larger-than-appropriate weighting adjustment. Potentially more important on a practical level is that, when done correctly, and even when "overdone," phone line weighting had little impact across a diverse set of opinion and demographic variables.

The Polls--Review: Reply to Berinsky and Druckman: Success Still Matters

Gelpi, C., Reifler, J. — 2008-03-14

The Polls--Trends: Labor Unions in the United States

Panagopoulos, C., Francia, P. L. — 2008-03-14

In this study, we analyze data on public opinion and attitudes toward labor unions from the iPOLL Databank at the Roper Center for Public Opinion Research (University of Connecticut), the American National Election Study, and the Current Population Survey. Despite recent developments that suggest labor unions are in decline, we find organized labor has maintained reasonably strong public support. Although the data indicate that Americans remain skeptical about how much confidence they can place in unions and their leaders, the results make clear that the public continues to recognize the need for unions to protect the rights of workers. These results hold potentially important implications for the future of organized labor in the United States.

Sarah E. Igo. The Averaged American: Surveys, Citizens, and the Making of a Mass Public

Kohut, A. — 2008-03-14

Manuscript Referees, 2006-2007

2008-03-14

Mitofsky Waksberg: Learning From The Past

Brick, J. M., Tucker, C. — 2007-12-13

Warren Mitofsky and Joseph Waksberg were two of the most influential researchers in telephone surveys and survey research methods. Their development of the Mitofsky–Waksberg method for random digit dialing sampling revolutionized telephone sampling and surveys. Jointly and separately, Waksberg and Mitofsky were instrumental in the development of many innovative methods and applications. This article reviews some of their contributions and discusses how their approaches to these problems may be a model for addressing issues that are being encountered currently in the United States in cell phone research.

Cellular-Only Substitution in the United States as Lifestyle Adoption: Implications for Telephone Survey Coverage

Ehlen, J., Ehlen, P. — 2007-12-13

Historically, the coverage bias from excluding the United States cell-only population from survey samples has been minimal due to the relatively small size of this group. However, the unrelenting growth of this segment has sparked growing concern that telephone surveys of the general public in the United States will become increasingly subject to coverage bias. While there is evidence that demographic weighting can be used to eliminate this bias, the availability of the weights lag behind the rapidly changing cell-only population. To explain the extent of the problem, we propose a reliable model to forecast cell-only population size and demographics. This model posits that a stable behavioral process, the rate of habit retention, can be estimated from prior wireless lifestyle adoption in the United States and may also describe adoption of the cell-only lifestyle. Using measures of incentive and habituation, we test this assumption by predicting changes in the cell-only population size and changes in age demographics. The accuracy of predictions confirms the two adoption behaviors are similar. We then develop forecasts of age demographics through 2009, and show how cell-only lifestyle adoption leads to potential coverage bias that is better addressed through this type of modeling rather than weighting from historical data.

Coverage Bias in Traditional Telephone Surveys of Low-Income and Young Adults

Blumberg, S. J., Luke, J. V. — 2007-12-13

The proportion of adults with only wireless telephones is growing rapidly. Using 2006 data from the National Center for Health Statistics’ National Health Interview Survey, this article is among the first to reveal that noncoverage of this population can result in nonnegligible bias for traditional random-digit-dial landline telephone surveys that do not call wireless telephone numbers. In 2006 in the United States, 17 percent of low-income adults with household income below 200 percent of the federal poverty thresholds, 25 percent of young adults aged 18–29 years, and 32 percent of low-income young adults lived in households with only wireless telephones. Within each of these three subgroups, we compared wireless-only adults and adults with landline telephones on demographic characteristics and 13 key indicators of health status, health behaviors, health care service use, and health care access. Even after statistical adjustments that account for demographic differences between adults living in households with and without landlines, telephone surveys of landlines will underestimate the prevalence of health behaviors, such as binge drinking, smoking, and HIV testing. Obesity may be overestimated and physical activity may be underestimated for low-income young adults. No significant bias is predicted for other measures of health status and health insurance coverage. Sample weighting procedures that incorporate adjustments for multiple demographic characteristics are necessary to help attenuate coverage bias in traditional telephone surveys, but may not be sufficient for behavioral risk factor surveys of low-income and young adults.

Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys

Kennedy, C. — 2007-12-13

The high costs and largely unknown error properties of cellular telephone interviews make screening for cell-only adults a potentially attractive option in dual frame RDD surveys. Screening out adults with landline telephones from the cellular sample does not affect the coverage properties of a dual frame survey, but it may affect other sources of error, especially nonresponse. In this study, data from a 2006 dual frame RDD survey conducted for the Pew Research Center, the Associated Press, and AOL are used to evaluate the effects of implementing a cell-only screener on both the bias and variance of weighted survey estimates. The effect of screening appears to be minimal so long as an adjustment for telephone service is included in the weighting method. Results of an attempt to correct for residual nonresponse due to inaccessibility are also discussed.

What's Missing from National Landline RDD Surveys?: The Impact of the Growing Cell-Only Population

Keeter, S., Kennedy, C., Clark, A., Tompson, T., Mokrzycki, M. — 2007-12-13

The number of cell phone only households has continued to grow – 12.8 percent of all households by the end of 2006, according to the National Health Interview Survey (NHIS). To assess the extent and nature of the potential bias in landline telephone samples created by the absence of cell-only households, the Pew Research Center conducted four independent dual frame studies in 2006, three of which included at least 200 interviews with cell-only respondents. Despite the fact that there are significant differences between cell-only and landline respondents on many important variables, across the four surveys with different substantive content we find that including a cell-only sample with a landline RDD sample produces general population estimates that are nearly identical to those from the landline sample alone. Yet, while the noncoverage problem is currently not damaging estimates for the entire population, we find evidence that it does create biased estimates on certain variables for young adults, 25 percent of whom are cell-only according to the most recent government estimate.

Sampling Telephone Numbers and Adults, Interview Length, and Weighting in The California Health Interview Survey Cell Phone Pilot Study

Brick, J. M., Edwards, W. S., Lee, S. — 2007-12-13

This article describes several features included in a California Health Interview Survey cell phone pilot study that differ from earlier cell phone surveys conducted in the United States. One difference is that the study used a screening design and only adults living in cell-only households were interviewed. Most of the previous studies used dual frame designs. Another difference was the development and implementation of a within-household adult sampling procedure to cover adults when cell phones were shared in the household. The study was also intended to determine if conducting a cell phone interview of the same scope and length as a regular telephone interview was feasible. Most previous surveys were focused on cell phone topics and were shorter than the comparable landline surveys. We then explore some interesting problems that arise in weighting a cell phone survey using a screening design. We conclude with a discussion of planning a new cell phone survey in 2007 that incorporates findings from this study.

Reaching the U.S. Cell Phone Generation: Comparison of Cell Phone Survey Results with an Ongoing Landline Telephone Survey

Link, M. W., Battaglia, M. P., Frankel, M. R., Osborn, L., Mokdad, A. H. — 2007-12-13

Noncoverage rates in U.S. landline-based telephone samples due to cell phone only households (i.e., households with no landline but accessible by cell phone) and the corresponding potential for bias in estimates from surveys that sample only from landline frames are growing issues. Building on some of the few published studies that focus on this problem, a study was conducted in three states (Georgia, New Mexico, and Pennsylvania) as part of the Behavioral Risk Factor Surveillance System (BRFSS), the world's largest ongoing public health telephone survey, to evaluate the effectiveness of conducting the BRFSS interview with a sample drawn from dedicated cell phone telephone exchanges and mixed-use (landline and cell phone) exchanges. Approximately 600 interviews were conducted in each of two groups: cell phone only adults (n = 572) and adults with both a landline and a cell phone (n = 592). Making comparisons with data from the ongoing, landline-based BRFSS survey, we report on response rates, demographic characteristics of respondents, key survey estimates of health conditions and risk behaviors, and survey costs. The methods used in this study have wide application for other U.S. telephone surveys.

The State of Surveying Cell Phone Numbers in the United States: 2007 and Beyond

Lavrakas, P. J., Shuttles, C. D., Steeh, C., Fienberg, H. — 2007-12-13

By the late 1970s, household telephone coverage grew to exceed 90 percent in the United States, and by the mid-1980s telephone surveying of the general public had become commonplace. Nevertheless, 20 years later, the ability of researchers to reach representative samples of the U.S. public via landline (wired) telephone surveys and gather reliable data is being seriously challenged for many reasons, especially those related to cell phones and the growth of the "cell phone only" population. However, at present there exists no widely accepted set of Cell Phone Surveying "best practices" for U.S. survey researchers to follow. Despite what some appear to believe, surveying persons reached on cell phone numbers in the United States currently is a very complex undertaking if one wants to do it "right," i.e. to do it legally, ethically, and in ways that optimally allocate one's finite resources to gather the highest quality data, and to analyze and interpret those data accurately. This final "wrap-up" article in the special issue provides a review of the empirical articles in the issue with a focus on their practical implications for the decisions that researchers need to make regarding sampling, coverage, nonresponse, measurement, and weighting in surveys that include interviews with persons reached on cell phones. The article also highlights the practical implications of a number of legal, ethical, and other issues that relate to surveys in the United States that include cell phone numbers. Surveying the U.S. cell phone population is possible, if at a higher cost than surveying their landline counterparts, and if with less precision than currently can be done surveying the landline population. The next five years should see a considerable growth in the methodological and statistical know-how that the survey community uses to plan, implement, and interpret cell phone surveys. There is a great deal that still must be learned.

Predispositions and Public Support for the President During the War on Terrorism

Ladd, J. M. — 2007-11-23

The terrorist attacks of September 11, 2001 and their aftermath offer a rare opportunity to examine how presidential approval responds to a sudden and severe national security crisis. I utilize the 2000–2002 National Election Studies panel to track change in public attitudes toward George W. Bush over the first two years of his presidency. An advantage of using panel data is that it allows me to go beyond aggregate change in presidential approval to examine how change is related to defense policy predispositions and prior political awareness. I find important differences. Over these two years, those high in political awareness experience priming of their defense predispositions but very little rally effect. In contrast, those low in political awareness experience a rally in support for Bush but very little priming. These results reaffirm that those with different levels of political awareness respond to dramatic messages in distinct ways.

From Agenda Setting to Refusal Setting: Survey Nonresponse as a Function of Media Coverage Across the 2004 Election Cycle

Stroud, N. J., Kenski, K. — 2007-11-23

Past research suggests that there is a relationship between survey response and topic salience, namely that individuals responding to a survey are likely to find the survey topic more salient than nonrespondents do. For election surveys, nonresponse resulting from a lack of salience can influence findings because respondents may be more interested in politics than nonrespondents. The agenda-setting model suggests that media coverage should heighten salience. Thus, as media coverage of political campaigns increases over the course of an election, refusals to a political survey should decline. Using data from the National Annenberg Election Survey (NAES), which was conducted nearly continuously in 2004, this study investigates the issue of nonresponse in a random digit dial telephone survey across the election cycle by examining daily changes in the refusal rates using time-series analysis. Content analyses of the frequencies of presidential campaign stories mentioned in the New York Times and three network news broadcasts were matched against a time series from the NAES to demonstrate that increases in media coverage of the election were negatively related to the survey refusal rate.

Black Class Exceptionalism: Insights from Direct Democracy on the Race Versus Class Debate

Hajnal, Z. L. — 2007-11-23

What implications does the growing economic divide between poor and middle class blacks have for the political arena? Traditional accounts suggest that increased economic diversity should lead to increased political division as the middle class becomes more conservative. Others maintain that race will continue to trump class because of ongoing racial inequality and widespread racial discrimination. I argue for a third alternative. I suggest that for blacks and possibly for other racial minorities increasing class status reinforces race. Class gains may increase the salience of race because economic success often means working in a predominantly white world and experiencing discrimination more regularly. I test these theories using the vote in direct democracy. I find that middle class blacks are more rather than less likely to support a liberal or black agenda. Class works differently for African Americans than for whites.

The True Electorate: A Cross-Validation of Voter Registration Files and Election Survey Demographics

Mcdonald, M. P. — 2007-11-23

I cross-validate the 2004 general election electorate's demographic profile available from voter registration files, the media consortium's National Election Poll or exit poll, and the Current Population Survey within selected states. I find voter files and Current Population Survey to be in general agreement, but the exit poll reports an electorate that is younger and composed of more minorities. All three confirm a pronounced pro-woman turnout gap.

Methodological Comparisons Between CATI Event History Calendar and Standardized Conventional Questionnaire Instruments

Belli, R. F., Smith, L. M., Andreski, P. M., Agrawal, S. — 2007-11-23

Six hundred and twenty-six participants from the Panel Study of Income Dynamics (PSID) were interviewed via telephone retrospectively about life course events by either computer-assisted (CATI) event history calendar (EHC) or standardized CATI conventional questionnaire (CQ) methods, randomly assigned. Experimental retrospective reports, for a reference period up to 30 years, were validated against reports provided annually from the PSID core interviews. Data quality outcome measures included variables associated with marriage, cohabitation, employment, unemployment, residential changes, and cigarette smoking. The EHC provided higher quality retrospective reports for cohabitation, employment, unemployment, and smoking histories; the CQ provided better data quality for marriage history, although what variable was being measured, instead of which method was being used, had the biggest impact on differences in data quality. Both EHC and CQ interviews lasted on average around one hour, with the EHC interviews being on average 10 percent longer. Interviewers preferred the EHC interviews. In both EHC and CQ conditions, respondents generally enjoyed the interviews, and did not find questions difficult. The costs and benefits of both EHC and CQ methods in the collection of life course retrospective reports are discussed.

Visual Context Effects in Web Surveys

Couper, M. P., Conrad, F. G., Tourangeau, R. — 2007-11-23

There are many examples of context effects in survey measurement. Responses to survey questions can be shaped by the order of questions, the format of response options, the broader survey environment, and so on. For Web surveys, the inclusion of visual images is a trivial design issue, but may have consequences for the responses obtained because they change the visual context. We report a series of experiments examining how responses may be affected by the use of images in Web surveys. Specifically, we examine the effect that pictures of a healthy woman exercising versus a sick woman in a hospital bed have on self-rated health. We replicated the experiments in three different surveys, varying such factors as the size and placement of the image and the location of the question within the questionnaire. In general, we find that when exposed to a picture of a fit woman, respondents consistently rate their own health lower than when exposed to a picture of a sick woman.

Mixed-Mode Contacts In Web Surveys: Paper is Not Necessarily Better

Porter, S. R., Whitcomb, M. E. — 2007-11-23

This paper investigates the impact of paper and email contacts on web survey response rates. We use six combinations of paper and email prenotifications and reminders to test the impact of mixed-mode contacts. In addition, we use two survey samples that differ in their relationship with the sponsoring institution to test if the impact of contact mode is conditional on relationship between respondents and the survey researchers. Contrary to previous research, we find little differences in response rates across experimental groups.

Bringing Registration into Models of Vote Overreporting

Fullerton, A. S., Dixon, J. C., Borch, C. — 2007-11-23

Voting is a socially desirable act and a basic form of political participation in the United States. This social desirability sometimes leads respondents in surveys, such as the National Election Study (NES), to claim to have voted when they did not. The methodology of previous studies assumes that people only overreport voting and that the sample of potential overreporters (i.e., nonvalidated voters) is not systematically different from the sample of potential voters. In this research note, we explore several different ways of examining the determinants of overreporting at two different stages (registering and voting) and with a consideration for selection bias. Comparing the traditional probit model used in previous research with sequential and heckit probit models, we find that the determinants of overreporting registering and voting differ substantially. In addition, there is a significant selection effect at the registration stage of overreporting. We conclude with a discussion of contemporary implications for pre-election polling and the postelection analysis of survey data.

Examining Citizens' Beliefs that Government Should Run Like Business

Gangl, A. — 2007-11-23

According to both scholars and cultural critics, a majority of Americans increasingly prefer market mechanisms to political processes or else simply equate democratic government with free markets. Some cultural critics go so far as to identify an emerging consensus among Americans that government is most fair and just when it runs like a business. This manuscript draws on analyses of a split-ballot framing experiment of 400 adults. The results suggest that beliefs that government would be more effective if it were run like a business are somewhat malleable. When people are presented with some of the reasons that democratic government works slowly and deliberatively, they are more likely to evaluate political processes positively compared to when they are exposed to information that suggests that democratic processes should work more efficiently like a business.

The Polls Trends: Public Reactions to Global Health Threats and Infectious Diseases

Ho, S. S., Brossard, D., Scheufele, D. A. — 2007-11-23

Over the past two decades, newly emerging infectious diseases have developed into major global health concerns, sparking intense media coverage, and triggering fears of a global outbreak among public health experts and authorities. This article focuses on trends in American attitudes toward these newly emerged infectious diseases by analyzing poll data over the past 6 years about issues relating to avian flu, severe acute respiratory syndrome, West Nile virus, and anthrax. The polls show that Americans’ attention to news coverage seemed to be event driven, peaking when there were new human or animal cases, and decreasing rapidly when the diseases seemed to have been contained. Americans’ perceptions of threats were usually the highest in the early stages of major outbreaks. The public became more complacent when the outbreaks seemed to be under control. Both behavioral changes and general knowledge remained largely constant, suggesting a limited impact of the various informational and awareness campaigns by governmental agencies in the wake of these pandemics.

Darren W. Davis. Negative Liberty: Public Opinion and the Terrorist Attacks on America

Nisbet, E. C. — 2007-11-23

Richard Gunther, Jose Ramon Montero, and Hans-Jurgen Puhle. Democracy, Intermediation, and Voting on Four Continents.

Howard, P. N. — 2007-11-23

Andrew Kohut and Bruce Stokes. America Against the World: How We are Different and Why We are Disliked.

Page, B. I. — 2007-11-23