Nonresponse in the American Time Use Survey
Who Is Missing from the Data and How Much Does It Matter?
KATHARINE G. ABRAHAM is professor of survey methodology, AARON MAITLAND is a survey methodology graduate student, and SUZANNE M. BIANCHI is professor of sociology, all at the University of Maryland, College Park.
Address correspondence to Katharine G. Abraham; e-mail: kabraham{at}survey.umd.edu.
 |
Abstract |
|---|
 TOP Abstract IntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
This article examines nonresponse in a large government survey, the American Time Use Survey (ATUS), which interviews persons in households previously interviewed in the Current Population Survey. The response rate for the ATUS has been below 60 percent for the first two years of its existence, raising questions about whether the results can be generalized to the target population. The article begins with an analysis of the types of nonresponse encountered in the ATUS. Noncontact accounts for roughly 60 percent of ATUS nonresponse, with refusals accounting for roughly 40 percent. We find little support for the hypothesis that busy people are less likely to respond to the ATUS but find considerable support for the hypothesis that people who are weakly integrated into their communities are less likely to respond, mostly because they are less likely to be contacted. When we compare aggregate estimates of time use calculated using the ATUS base weights without any adjustment for nonresponse, estimates calculated using the ATUS final weights with a nonresponse adjustment, and estimates calculated using weights that incorporate our own nonresponse adjustment based on a propensity model, we find some modest differences, but the three sets of estimates are broadly similar. The article ends with suggestions for further research and analysis.
|
Introduction |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
The well-documented decline in household survey response rates in recent decades (see, for example, Atrostic et al. 2001
; Curtin, Presser and Singer 2005; de Leeuw and de Heer 2002) has led to growing concern among survey researchers about the quality of household survey data. While government survey response rates have tended to be higher than those for private surveys, even the U.S. federal statistical agencies have experienced increasing difficulty in obtaining household survey interviews. The new American Time Use Survey (ATUS) is a case in point. The ATUS is designed to produce comprehensive information on how Americans use their time, information that, among other potential uses, should deepen our understanding of family life, enrich the analysis of social policy alternatives, and support more comprehensive measurement of national output. Despite the survey’s official imprimatur, the ATUS response rate has been below 60 percent, and questions naturally have arisen about whether and how the responses obtained can be generalized to the target population. One feature of the ATUS that facilitates analysis of the causes and consequences of nonresponse to the survey is the existence of unusually rich information about the ATUS nonrespondents. The ATUS sample is drawn from the outgoing rotation groups of the Current Population Survey (CPS). Given their prior participation in the CPS, the process of obtaining responses from those selected for the ATUS sample parallels in certain respects the process of obtaining responses to the follow-on waves of a panel survey. Although this design feature makes it difficult to generalize the findings to strictly cross-sectional surveys, it allows us to test competing hypotheses about the reasons for ATUS nonresponse.
A starting point for thinking about survey nonresponse is to consider the different ways in which it may occur. Groves and Couper (1998) develop a model of nonresponse to household interview surveys that distinguishes between noncontact, refusal, and other reasons for survey nonresponse. Lepkowski and Couper (2002) extend this model to longitudinal panel surveys in which specific individuals, rather than specific housing units, are the unit of observation. In their model, nonresponse in the second and subsequent waves of a panel survey may be the consequence of failure to locate a previously interviewed sample unit, failure to contact the sample unit once located, or refusal by a sample unit that has been contacted. As emphasized by Groves and Couper (1998), different types of nonresponse are likely to have different causes and different consequences.
Survey nonresponse commonly is taken as an indicator of the quality of survey data. In fact, however, nonresponse is a source of bias in survey estimates only to the extent that those who respond are different from those who do not with respect to the characteristic of interest (Groves 2006). Several recent studies have suggested that there is no consistent relationship between survey response rates and bias in survey estimates (see Curtin, Presser, and Singer 2000; Keeter et al. 2000; Merkle and Edelman 2002; and for a synthesis and review, Groves 2006). This raises the possibility that, in many cases, the money spent on intensive efforts to locate, contact, and solicit cooperation from sample members might be better spent on other survey activities. To reach this conclusion for any particular survey, however, requires good evidence on how nonresponse might be affecting the survey estimates.
Two alternative hypotheses about household survey response seem especially relevant to time diary studies such as the ATUS. One is that people who are busy with other activities are less likely to respond. Being busy could lead both to lower contact rates, since busy people may be less frequently at home, and to higher refusal rates, since busy people may be less willing to take the time to respond to a request for survey participation. If true, this would be a particular problem for a time diary study such as the ATUS, since it is precisely the use of time that such studies are designed to measure, and the underrepresentation of busy people could seriously distort the estimates produced (see Abraham and Mackie 2005; Hochschild 1989). For example, if people who work long hours are less likely to respond, time diary estimates might understate average hours of work.
An alternative hypothesis is that a person’s response propensity reflects strength of social integration or, put differently, strength of attachment to the broader community. People with weaker community ties may be difficult to locate because they move away, do not have valid phone numbers, and so on, as well as possibly being more difficult to contact because they are less likely to be at home. A person with weak social ties also may be less receptive to completing a survey interview. If people with weak community ties spend their time differently than other people, differences in response propensities associated with the strength of these ties could bias aggregate time use estimates (Robinson and Godbey 1997 discuss a similar idea). For example, those with weak community attachment may be less likely to engage in volunteer activity, leading to an overstatement of volunteer hours in estimates based on reports from respondents (Abraham, Helms, and Presser 2006).
Some evidence in the literature suggests that busy people may be more difficult to contact for survey interviews. Groves and Couper (1998), for example, report that households in which one would expect at least one adult to be out of the labor force are easier to contact than other households. They also find that those who are difficult to contact spend more hours away from home, but that the same is not true of those who refuse as compared with those who agree to complete an interview once contacted.
The results of several previous studies hint that underrepresentation of busy people could be a problem for time diary studies specifically. Drago et al. (1999) conducted a pilot time diary study of 58 teachers employed at either a "high stress" school or a different "low stress" school. Teachers at the "high stress" school were much less likely to volunteer to participate in the study. Paakkonen (1999) analyzed data from the nationwide Finnish time diary study conducted in 1987–88. Among 10,574 people contacted for the study, 8,540 participated in an initial short interview. Of these, 7,758 completed a time diary. Those who participated in the initial interview but refused to keep the time diary were no more likely to report feeling "rushed" than those who agreed, but they did report working somewhat longer hours.
The results of other studies, however, suggest that busy people may be overrepresented, not underrepresented, in time diary reports. Robinson (1999) examined differences in the distribution of activities reported in the first wave of the 1975 University of Michigan time use survey for those who did and did not participate in the second wave of the same survey. Those who did not respond to the second wave had reported in the first wave that they spent less time working and doing housework, but more time sleeping and watching television. Similar results were obtained using data from a later time use study conducted in 1985 (Robinson and Godbey 1997). Knulst and van den Broek (1999) examined rates of response to the several official time diary studies conducted in the Netherlands since 1975 for groups defined on the basis of their age, gender, urbanization of place of residence, position in the family, and position in the labor market. The Dutch time use studies required completion of a 7-day diary, and nonresponse rose from about one-quarter of the survey sample in 1975 to about three-quarters in 1995. Response rates generally were higher, rather than lower, for those groups in which respondents reported longer hours of paid work and larger total time commitments.
Prior research on household survey nonresponse that is relevant to the "social integration" hypothesis has proxied the strength of an individual’s or a household’s community attachment in different ways. A consistent finding in the literature is that household survey response rates are lower for those who live in urbanized areas (Groves, 2006). Groves and Couper (1998) report that single-person households, households without children, and occupants of multiunit structures, all of which they characterize as more socially isolated, tend to have lower cooperation rates. In keeping with some earlier research, however, they find no evidence of lower cooperation among those who have moved within the past five years. In a study of panel nonresponse in the Survey of Income and Program Participation (SIPP), Rizzo, Kalton, and Brick (1996) find lower response rates for people living in rental housing or in a household headed by someone other than a family member. Similarly, Zabel (1998) reports that, both in the SIPP and in the Panel Study of Income Dynamics, renters are more likely than owners to drop out of the survey from one wave to the next. These analyses do not differentiate, however, between noncontacts and refusals. Lepkowski and Couper (2002) study nonresponse in the second wave of two longitudinal household surveys. Among other results, they find that people who rent rather than own their home are more difficult to locate and, in one of the two surveys they study, are also more likely to refuse a survey request. We are not aware of research that has looked specifically at the effects of social integration on response rates in time diary studies.
The plan of the remainder of the article is as follows. After describing the collection of the ATUS data and our categorization of survey nonresponse, we examine descriptive statistics concerning the disposition of cases included in the ATUS survey sample. Next we explore the bivariate association between sample members’ characteristics and the likelihood of responding to the ATUS, looking at indicators of "busyness" and "social integration." We then fit a response propensity model to the data and use the results from this model to adjust survey weights to account for differences in the probability of response associated with a range of individual and household characteristics. The article concludes with a discussion of the implications of the ATUS nonresponse analysis and suggests avenues for further research.
|
Data and Methods |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
The American Time Use Survey is conducted by the U.S. Census Bureau with funding from the U.S. Bureau of Labor Statistics (BLS). All ATUS data are collected using computer-assisted telephone interviewing (CATI). The survey was first conducted in 2003. The analysis we report is based on ATUS data collected during 2004.1
SAMPLE DESIGN
The target population for the ATUS is the U.S. civilian noninstitutionalized population age 15 or older. Individuals chosen for participation in the ATUS are selected randomly from households completing the eighth wave of participation in the CPS, the monthly household survey that is the source of official U.S. labor force statistics. The CPS sample overrepresents small states; a first stage of selection for the ATUS sample eliminates this overrepresentation. Households then are stratified by the race/ethnicity of the householder, the presence and age of children in the household, and the number of adults in adult-only households. The rates at which households are sampled for the ATUS differ across these strata. In the third stage of sample selection, one randomly selected person age 15 or older in each sampled household is designated for participation in the ATUS. Each sample member is assigned a designated day for which time use information will be collected, and telephone interviews are conducted on the day following the designated day. The ATUS diary days are distributed across the days of the week, with 10 percent allocated to each of the weekdays Monday through Friday, 25 percent to Saturdays and 25 percent to Sundays, and distributed evenly across the weeks of the year.
ATUS interviews generally are conducted between two and four months after the household’s last CPS interview. If the selected person cannot be contacted on his/her assigned interview date, he/she may be called on the same day the following week. Sample members for whom no telephone number is available (approximately 5 percent of the total) are sent a letter offering an incentive of $40 to participate in the study and asking that they call the telephone center on a specified day to complete the interview. People who have moved away are considered ineligible for participation and dropped from the sample. Efforts to contact an eligible sample member may be continued for up to eight weeks.
People in households that were selected for the CPS but did not complete a wave-eight interview have no chance of being selected for the ATUS. Over the recent past, response rates for the eighth-month-in-sample basic CPS questionnaire have averaged about 94 percent. CPS weights that incorporate an adjustment for CPS nonresponse are used in selecting the ATUS sample and constructing the ATUS estimation weights. Still, to the extent that nonresponding CPS households differ from responding households with similar demographic characteristics, there is the potential for bias in the ATUS estimates. Unfortunately, we have no means of assessing any bias from this source and do not consider it further.
VARIABLES
In analyzing the ATUS we make use of case disposition information provided on the accompanying survey methodology file. We are especially interested in distinguishing among completed interviews (C), noncontacts (NC), refusals (R), and other noninterviews (O). In the ATUS the interviewer may be unable to contact a designated respondent because that person has moved away or is absent from the household for other reasons; because the interviewer does not have a valid telephone number for the household; or because the designated respondent is never available to speak to the interviewer. The ATUS survey methodology file categorizes designated respondents who have moved away as ineligible (NE), assigns designated respondents who are absent for other reasons such as illness to the "other noninterview" category (O), and considers designated respondents for whom the survey interviewer does not have a valid phone number to be of unknown eligibility (UE). Only those cases for which the validity of a respondent’s phone number is established but the interviewer does not succeed in speaking with the respondent are categorized as noncontacts (NC). We categorize all of these cases as noncontacts.2 For some purposes, we look separately at noncontact due to the designated respondent having moved away or being absent from the household for other reasons (category NC-1), to bad contact information (NC-2), or to difficulty in finding the designated respondent at home (NC-3). We use the official case disposition codes to identify refusals.3 Most of those we categorize as "other noninterviews" (O) are cases involving language barriers. Appendix A displays the detailed case disposition codes recorded on the ATUS survey methodology file, together with the official and our alternate grouping of those codes into broader case disposition categories.
Beyond the information collected as part of the ATUS interview, additional information about the ATUS sample members and their households is available from the CPS interviews in which they participated. The ATUS-CPS data file provided by the BLS contains most of the information collected as part of the last basic CPS interview for each ATUS household, together with identifiers that allow the ATUS-CPS records to be linked to the ATUS interview records. Important for our purposes, the ATUS-CPS file contains records not only for ATUS respondents and the other members of their households but also for people picked as ATUS respondents who did not complete the survey and the members of their households. A few pieces of information relevant to the analysis of survey nonresponse—specifically, whether the household rented or owned its housing unit and whether the household was located in a central city—are not included on the ATUS-CPS data file but were obtained from the relevant CPS basic interview files.
To test our hypotheses about ATUS nonresponse, we must identify individual and household characteristics that can proxy for "busyness" and "social integration." All else the same, we expect people who work longer hours or have children in the home to be busier. Among those who are married, given their own hours of work, people whose spouses work longer hours also may be busier. These observable characteristics admittedly are crude indicators of how busy someone is—and how busy people feel may matter more than how busy they actually are—but if "busyness" is important, we would expect to see some association between these proxies and the survey response, contact, and cooperation rates.
A second set of individual and household characteristics proxy for the strength of respondents’ integration into their communities. Many seem likely to be associated with residential stability, which directly affects the probability of noncontact due to not locating a sample member, but some may also affect the motivation an individual feels to cooperate in a survey. Married people living with their spouse may be better integrated into their communities than people who are not married. People who are married but separated from their spouse may be more difficult to locate and also less willing to spend time talking to a survey interviewer.4 Hours of work may be an indicator of "busyness," but being out of the labor force may also indicate weak social integration. Similarly, the presence of children may affect not only "busyness" but also "social integration." People in households that include children, especially children age 6–17, may be less likely to move and more strongly connected to their communities through their children’s schools. Homeowners can be expected to have stronger ties to their communities than renters, and the same may be true of people who live in nonmetropolitan areas. Finally, we have created a variable that captures whether people are living in households that include adults who are not related to the householder, reasoning that such households may tend to be more transient. For completeness, we also have created a variable that captures the presence in the household of other adults who are relatives of the householder.
In addition to the characteristics that relate to either the "busyness" or the "social integration" hypothesis, we consider the ATUS sample member’s sex, age, race/ethnicity, household income, education, region, and telephone status as potential influences on survey response. The construction of the individual and household characteristic variables used in our analysis is outlined in appendix B.
ANALYSIS PLAN
Our analysis begins with simple tabulations of response outcomes for people with different characteristics. We make use of AAPOR response rate RR2 (American Association for Public Opinion Research 2006):
|
| (1) |
where C represents completed and sufficient partial interviews, R refusals, NC noncontacts, O other noninterviews, and UE cases of unknown eligibility (though in fact there are no UE cases in our preferred categorization scheme). We also consider contact rate CON1:
|
| (2) |
and cooperation rate COOP2:
|
| (3) |
Note that the response rate equals the product of the contact rate and the cooperation rate. The noncontact rate is the complement of the contact rate. For some purposes, we are interested in the prevalence of different types of noncontact, as well as in the refusal rate and the other noninterview rate.5 All of these survey outcome rates are tabulated by hours worked, by the presence of children in the household, by housing tenure (own versus rent), and so on. ATUS base weights are used for the calculations.
After examining the simple tabulations, we estimate multivariate logistic regressions of the factors that determine response outcome—response, contact, cooperation, and so on. The logistic regression for each modeled outcome is estimated independently using weighted data. Standard errors for the estimates from the regressions are estimated using a replicate variance method proposed by Fay (1989) that accounts for the increased variance associated with the clustering of the ATUS sample relative to the variance that would have been expected for a simple random sample of the same size.6
All of the explanatory variables included in the logistic regression models are dichotomous. A dummy variable coefficient that is significant and positive (negative) implies that having the characteristic in question raises (lowers) the probability of the outcome being modeled. The size of these effects on the probability of the modeled outcome, however, depends on the baseline against which the effect is calculated. To assist in interpreting the logistic regression results, we have calculated the implied change in the probability of the outcome of interest associated with having versus not having each specified characteristic, evaluated at the average probability of observing the outcome for the sample as a whole. Rather than reporting the coefficient estimates, we report these marginal probability effects. The statistical significance of the estimated marginal effects can be determined based on the magnitude and standard errors of the corresponding logistic regression coefficients.7
A further question we consider is whether reweighting the data to account for differences in response propensities makes a material difference to estimated patterns of time use. This might be the case if differences in response propensities associated with observable characteristics also have a systematic association with how people use their time. We use the estimated response propensities based on the weighted logistic regression coefficients to calculate nonresponse adjustment factors equal to the inverse of the estimated response propensity for each survey respondent.8
Because different days of the week are represented in different proportions in the survey data and this was not accounted for in the survey base weights, we also make an adjustment to ensure that each day of the week (Sunday through Saturday) received one-seventh of the total of the final survey weights. Our final weight for each respondent thus is equal to:
|
| (4) |
where Wfinal is the final weight, Wbase is the ATUS base weight, Wnonresponse is the propensity-score-based weight adjustment factor that accounts for differences across observations in their response propensities, and Wday is the day-of-week adjustment factor. We compare time use estimates prepared using these weights with estimates unweighted for nonresponse propensity and with estimates produced using the official ATUS estimation weights.9
|
Results |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
Our analysis of the ATUS data first examines the distribution of response outcomes, and then considers the personal characteristics associated with different outcomes and how these associations affect the survey estimates.
ATUS RESPONSE OUTCOMES
Sample dispositions for the 2004 ATUS are shown in table 1. The first column in the top panel of the table shows the number of sample members assigned to each major sample disposition category based on the codes from the survey methodology file provided by BLS; the second column shows the unweighted percentage distribution of these cases for the portion of the sample considered to be eligible respondents; and the third column shows the weighted percentage distribution. The reported figures imply an unweighted (weighted) response rate for the ATUS of 54.6 (56.1) percent.10
|
The bottom panel of table 1 is similar to the top panel, except that, consistent with our understanding of the AAPOR guidelines, we assign more cases to the noncontact category. The data make clear the importance of problems with contacting respondents as a source of nonresponse in the 2004 ATUS. Using our classification scheme, noncontact accounts for roughly 60 percent of all survey nonresponse, with refusals accounting for between 35 and 40 percent and other reasons for the small number remaining. In addition, because the alternative disposition category structure places fewer cases in the not eligible category, the estimated response rate is a bit lower than that obtained using the official ATUS disposition codes.
BIVARIATE ASSOCIATIONS
In table 2 we tabulate survey outcome rates for the 2004 ATUS. Overall nonresponse rates offer little support for the hypothesis that busy people are less likely to respond to the American Time Use Survey. People who work full-time (35–44 hours a week) have lower response rates than people who work part-time, but the response rate for people who work more than full-time is comparable to that for people who work part-time, and both are higher than the response rate for people who do not work at all. Among married people, those whose spouses work very long hours have the highest response rates. The presence of children in the household is not strongly related to response propensity.
|
Noncontact due to the designated respondent never being available to talk to the survey interviewer (NC-3) does rise with hours of work. Presence of children does not have a consistent effect on this category of noncontact, however, and none of the "busyness" proxies have an obvious relationship to the refusal rate.
In contrast, there are consistent differences in response rates across groups that conform to the prediction of the "social integration" hypothesis. Response rates are relatively low for people who are out of the labor force, and also for people who are separated or have never been married. Renters’ response rates are nearly 15 percentage points lower than homeowners’ response rates. People identified as living in a central city are approximately 9 percentage points less likely to respond than people living in a nonmetropolitan area. People who live in households that include an adult who is not related to the householder are roughly 13 percentage points less likely to respond than people who live in households where everyone is related to the householder. Differences in the incidence of noncontact, especially noncontact related to the respondent being absent from the household or to bad contact information having been recorded for the respondent (NC-1/2), account for most of these differences.
MULTIVARIATE MODELS
Table 3 summarizes the implied marginal probability effects derived from the multivariate logistic regressions with response outcomes as the dependent variables. To illustrate the interpretation of the estimates reported in the table, the figure shown in the "Widowed" row of the "Nonresponse" column indicates that, evaluated at the mean probability of nonresponse, being widowed rather than never married (the omitted group) raises the nonresponse rate by an estimated 3.68 percentage points. Statistically significant estimated effects are shown in bold. Like the tabulations reported in table 2, the multivariate results offer little support for the "busyness" hypothesis. All else the same, part-time workers are less likely to be nonrespondents than either those who do not work or those who work longer hours, and married people whose spouse works very long hours have lower nonresponse probabilities than others. As was also true in the simple tabulations, however, we find that longer hours of work are associated with a higher probability that the respondent will not be available to talk with the interviewer (NC-3).
|
Something we did not examine in table 2 was the interaction between marital status and presence of children in the household. The presence of children has no significant effect on survey response for married sample members, but the presence of children age 6–17 actually raises the probability of response for unmarried sample members. This finding is at odds with the "busyness" hypothesis but lends support to the "social integration" hypothesis, insofar as having school-age children can be supposed to engage single parents in their communities.
Most of the "social integration" variables discussed in connection with table 2 have statistically significant effects in the multivariate nonresponse model. Response probabilities are significantly lower for renters as compared with homeowners, people who live in metropolitan areas, and people who live in households that include adults not related to the householder. In this model, people who live in households that include other adult relatives of the householder also have lower response rates. Again, most of these differences in response rates reflect differences in the probability of contact.
EFFECT OF REWEIGHTING ON TIME USE ESTIMATES
For everyone who completed the ATUS interview, the estimated logistic regression coefficients can be used to calculate the probability that a person with that set of characteristics would have responded to the survey.11 The differences in response propensities across individuals with different characteristics are sizable. Taking the two most extreme examples, the implied response rate for a young black male with less than a high school education who is separated from his spouse, lives in a rented housing unit in a central city in the South, works 35–44 hours per week, resides in a household that includes young children, adult relatives, and adults who are not related to the householder, has no telephone, and did not provide household income information to the CPS interviewer, is just 5.1 percent. At the other end of the scale, the implied response rate for a married white female homeowner age 56–65 with a graduate education who lives in a nonmetropolitan area in the Midwest, has a telephone, works part-time and has a spouse who works 45 hours or more per week, resides in a household that includes no children or other adults, and has a reported income in the range $40,000–75,000 is 87.8 percent. As described in the previous section, we calculate a final weight for each survey respondent that incorporates the inverse of their response propensity based on the logistic regression results and a day-of-week adjustment.
The official ATUS estimates reported by BLS also are calculated using weights that incorporate nonresponse and day-of-week adjustments. The official weights control the estimated totals from the respondent sample along the dimensions of race, sex, age, presence of children, and education. Compared with the official weights, our weights are based on somewhat less detailed age breaks and more detailed education breaks. In addition, we account for all of the other individual and household characteristics shown in table 3. The official weighting procedures control the day-of-week distribution within each month to the actual representation of days within that month; we adjust the weight totals for the year as a whole so that each day of the week gets one-seventh of the total weight. A final difference is that the official weights control for whether the respondent was offered an incentive to participate, whereas we do not.12
In order to see the effect on the ATUS estimates of adjusting for survey nonresponse, and also to learn whether adjusting for differences in nonresponse related to factors not taken into account in the official weight construction procedures leads to different conclusions about the effects of nonresponse on the survey estimates, table 4 reports three different sets of weighted time use estimates based on the 2004 ATUS. The most notable feature of the three sets of estimates is their similarity. Reweighting the data to account for nonresponse associated with observable characteristics raises the estimates of average time spent sleeping and watching television, and reduces the estimate of average time spent in household chores, but the changes are very small, on the order of 5 minutes per day. Further, the purely demographic adjustments used to produce the official ATUS final weights yield estimates very similar to those based on a more extensive set of personal and household characteristics.
|
|
Discussion and Conclusions |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
As a first step in exploring whether the nonresponse we observe in the ATUS is a source of bias in the survey estimates, we constructed new weights for the survey that account for differences in response propensities associated with a variety of observable characteristics. Reweighting the data in this way has relatively little effect on aggregate estimates of time use. Although there are differences in the patterns of time use associated with individuals’ observable characteristics and the probability of responding to the ATUS differs with respect to these same characteristics, the net effects on the survey estimates of reweighting the data to take this into account are not very large.
These findings do not rule out nonresponse as a source of bias in the ATUS estimates. It is possible that there are differences in the characteristics of respondents and nonrespondents for which we have not been able to account, and that these characteristics are strongly associated with how people spend their time. Abraham, Helms, and Presser (2006) find that people selected for the ATUS sample who reported volunteer activity in the September 2002 CPS Volunteer Supplement were much more likely to respond to the ATUS than people who did not. Estimates of volunteer activity based on the ATUS thus seem very likely to suffer from nonresponse bias, since the survey sample consists disproportionately of people identified through their CPS responses as active volunteers. Even though the nonresponse weighting adjustments described in the present article did not have much effect on the survey estimates we examined, there may be other individual or household characteristics we did not observe that, if taken into account, would make a larger difference, at least for estimates of time devoted to certain activities.
There are several avenues that might be explored in further efforts to better understand the effects of nonresponse, and especially noncontact, on the ATUS estimates. First, the responses of recent movers—people who entered the CPS sample between the fifth and the eighth survey waves—could be compared with those of people who have not moved recently. Second, it may be possible to use information collected through CPS supplements completed by ATUS sample member households, such as the October school enrollment supplements or the biennial displaced worker supplements, to shed additional light on material differences between ATUS respondents and nonrespondents.
Third, the BLS has made available call history data for all of the cases selected for the 2004 ATUS. If we can assume that designated ATUS respondents who were difficult to contact, based on the number of telephone calls required to reach them or other indicators, are more similar to those who did not respond than are designated respondents who were easy to contact, a comparison of the responses received from "difficult" and "easy" respondents could be informative about the direction of noncontact bias in the survey.
The BLS also may wish to consider the collection of additional information to shed light on the potential existence and magnitude of nonresponse, and especially noncontact, bias in the ATUS. A nonresponse follow-up survey designed to locate and interview those not contacted for the ATUS to learn more about them could be useful. Another strategy might be to add a small number of supplemental questions to the interviews conducted with outgoing CPS rotation group households over several months, again with the goal of obtaining better information about those selected for the ATUS who end up not responding.
|
Supplementary Data |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
Supplementary data are available online at http://pubopq.oxfordjournals.org/.
|
Appendix A |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
|
|
Appendix B |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
|
|
Acknowledgements |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
The first draft of this paper was prepared for the American Time Use Survey Early Results Conference, Bethesda, MD, December 8–9, 2005. Dorinda Allard and Kristina Shelley of the Bureau of Labor Statistics patiently answered all of our many questions about the data. Mary Edith Bozylinsky provided invaluable assistance in preparing the data for analysis. The authors are grateful for comments from Michael Brick, Frauke Kreuter, Stanley Presser, Richard Valliant, participants in the ATUS Early Results Conference, and two anonymous reviewers.
|
Footnotes |
|---|
1. In the course of developing our hypotheses regarding the causes and consequences of nonresponse in the ATUS, we carried out a variety of preliminary analyses using the 2003 data, and we did not want to test these hypotheses using the same set of observations. None of our conclusions would have been altered had we used 2003 data.
2. Some very small share of those we reassign from the NE and UE categories to the NC category may have joined the Armed Forces or been institutionalized subsequent to their final CPS interview, making them ineligible for the survey. These disqualifying events are so rare, however, that categorizing the groups in question as noncontacts seems most appropriate.
3. Some "soft" refusals—cases in which a respondent simply avoids ever speaking to the survey interviewer—may be included among those we categorize as noncontacts, but there is no way to identify these "soft" refusals based on the available information.
4. The married but separated category includes a small number of people who report that they are married but are neither the householder nor the spouse of the householder. We were unable to determine whether these individuals’ spouses resided in the same household and assigned them to the separated category.
5. See American Association for Public Opinion Research (2006) for further discussion of various survey outcome rate measures and the relationships among them.
6. The SAS-callable procedure RLOGIST in SUDAAN, a statistical software package for the analysis of survey data collected using complex sample designs, was used to calculate the standard errors of the logistic regression parameters. The necessary replicate weights were provided by BLS. Further details are available from the authors on request.
7. For coefficient estimates from the multivariate models, together with their standard errors, please see supplementary data online.
8. As discussed by Little and Vartivarian (2003), an argument can be made for using unweighted rather than weighted logistic regression models as the basis for nonresponse weight adjustment, but in our case the two models produce very similar coefficient estimates and applying the two sets of weight adjustments yields virtually identical time use estimates. For consistency of presentation, we have retained the weighted coefficients.
9. The approach we take in this section of the article is similar to that employed by Rizzo, Kalton, and Brick (1996) in their study of panel nonresponse in the Survey of Income and Program Participation.
10. BLS reports an unweighted ATUS response rate 57.3 percent for 2004. There are two main reasons why the response rate reported by BLS differs from that which we have calculated using the official case disposition codes. First, our rate is based on the set of cases for which a final disposition was obtained during the calendar year in question; the BLS response rate is based on the set of cases initiated during the calendar year and thus covers a slightly different time period. Second, the BLS response rate was calculated prior to editing of the survey data. In 2004 the data collected for several hundred cases were evaluated during editing to be of poor quality, and these cases were recoded from completed interviews to refusals. Working with the edited data thus produces somewhat lower response rates.
11. More precisely, the coefficient estimates can be used to calculate the probability that a person with given characteristics would be a nonrespondent, and one minus that probability then equals the person’s response propensity.
12. U.S. Bureau of Labor Statistics and U.S. Census Bureau (2005) provides an overview of how the ATUS weights are constructed, and Tupek (2004a, 2004b) gives additional details.
|
References |
|---|
|
TOPAbstractIntroductionData and MethodsResultsDiscussion and ConclusionsSupplementary DataAppendix AAppendix BAcknowledgementsReferences |
|---|
Abraham Katharine G., Helms Sara, Presser Stanley. (2006) ldquo;Effects of Survey Nonresponse on Inferences about Volunteer Work. Unpublished manuscript, University of Maryland.
Abraham Katharine G. and Mackie Christopher. (2005) Beyond the Market: Designing Nonmarket Accounts for the United States(National Academies Press, Washington, DC).
American Association for Public Opinion Research (AAPOR). (2006) Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 4th ed (AAPOR, Lenexa, KS).
Atrostic B. K., Bates Nancy, Burt Geraldine, Silberstein Adriana. (2001) Nonresponse in U.S. Government Household Surveys: Consistent Measures, Recent Trends, and New Insights. Journal of Official Statistics 17:2209–26.
Curtin Richard, Presser Stanley, Singer Eleanor. (2000) Effects of Response Rate Changes on the Index of Consumer Sentiment. Public Opinion Quarterly 64:413–28.[Abstract]
Curtin Richard. (2005) Changes in Telephone Survey Nonresponse over the Past Quarter Century. Public Opinion Quarterly 69:87–98.
de Leeuw Edith and de Heer Wim. (2002) Trends in Household Survey Nonresponse: A Longitudinal and International Comparison. In Groves Robert M., Dillman Don A., Eltinge John L., Little Roderick J. A. (Eds.). Survey Nonresponse(Wiley, New York) pp. 41–54.
Drago Robert, et al. (1999) Time for Surveys: Do Busy People Complete Time Diaries? Society and Leisure 21:2555–62.
Fay Robert E. (1989) Theoretical Application of Weighting for Variance Calculation. Proceedings of the Section on Survey Research Methods of the American Statistical Association(American Statistical Association, Alexandria, VA) pp. 212–17.
Groves Robert M. (2006) Nonresponse Rates and Nonresponse Error in Household Surveys. Public Opinion Quarterly 70:646–75.
Groves Robert M. and Couper Mick P. (1998) Nonresponse in Household Surveys(John Wiley and Sons, New York).
Hochschild Arlie. (1989) The Second Shift: Working Parents and the Revolution at Home(Viking, New York).
Keeter Scott, Miller Carolyn, Kohut Andrew, Groves Robert, Presser Stanley. (2000) Consequences of Reducing Nonresponse in a Large National Telephone Survey. Public Opinion Quarterly 64:125–48.[Abstract]
Knulst Wim and van den Broek Andries. (1999) Do Time-Use Surveys Succeed in Measuring ‘Busyness’: Some Observations of the Dutch Case. Society and Leisure 21:2563–72.
Lepkowski James and Couper Mick. (2002) Nonresponse in the Second Wave of Longitudinal Household Surveys. In Groves Robert M., Dillman Don A., Eltinge John L., Little Roderick J. A. (Eds.). Survey Nonresponse(Wiley, New York) pp. 259–72.
Little Roderick and Vartivarian Sonya. (2003) On Weighting the Rates in Nonresponse Weights. Statistics in Medicine 22:1589–99.[CrossRef][ISI][Medline]
Merkle Daniel and Edelman Murray. (2002) Nonresponse in Exit Polls: A Comprehensive Analysis. In Groves Robert M., Dillman Don A., Eltinge John L., Little Roderick J. A. (Eds.). Survey Nonresponse(Wiley, New York) pp. 243–57.
Paakkonen Hannu. (1999) Are Busy People Under- or Over-Represented in National Time Budget Surveys. Society and Leisure 21:2573–82.
Rizzo Lou, Kalton Graham, Michael Brick J. (1996) A Comparison of Some Weighting Adjustment Methods for Panel Nonresponse. Survey Methodology 22:143–53.
Robinson John P. (1999) Activity Patterns of Time-Diary Dropouts. Society and Leisure 21:2551–54.
Robinson John P. and Godbey Geoffrey. (1997) Time for Life: The Surprising Ways That Americans Spend Their Time(Pennsylvania State University Press, University Park).
Tupek Alan R. ( October 5, 2004a) Revised Weighting Specifications for the American Time Use Survey. Document no. ATUS-06-R, Memorandum to Chester E. Bowie, Demographic Surveys Division, U.S. Census Bureau.
Tupek Alan R. ( September 30, 2004b) Weighting Specifications for the American Time Use Survey for 2004. Document no. ATUS-11, Draft memorandum to Chester E. Bowie, Demographic Surveys Division, U.S. Census Bureau.
U.S. Bureau of Labor Statistics and U.S. Census Bureau. (2005) American Time Use Survey User’s Guide: 2003–2004. Available online athttp://www.bls.gov/tus/atususersguide.pdf (accessed September 21, 2005).
Zabel Jeffrey E. (1998) An Analysis of Attrition in the Panel Study of Income Dynamics and the Survey of Income and Program Participation with an Application to a Model of Labor Market Behavior. Journal of Human Resources 33:2479–506.[CrossRef][ISI]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||