IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/22333.html
   My bibliography  Save this paper

Difficulty to Reach Respondents and Nonresponse Bias: Evidence from Large Government Surveys

Author

Listed:
  • Ori Heffetz
  • Daniel B. Reeves

Abstract

How high is unemployment? How low is labor force participation? Is obesity more prevalent among men? How large are household expenditures? We study the sources of the relevant official statistics—the Current Population Survey (CPS), the Behavioral Risk Factor Surveillance System (BRFSS), and the Consumer Expenditure Survey (CEX)—and find that the answers depend on whether we look at easy- or at difficult-to-reach respondents, measured by the number of call and visit attempts made by interviewers. A challenge to the (conditionally-)random-nonresponse assumption, these findings empirically substantiate the theoretical warning against making population-wide estimates from surveys with low response rates.

Suggested Citation

  • Ori Heffetz & Daniel B. Reeves, 2016. "Difficulty to Reach Respondents and Nonresponse Bias: Evidence from Large Government Surveys," NBER Working Papers 22333, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:22333
    Note: AG LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w22333.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alan B. Krueger & Alexandre Mas & Xiaotong Niu, 2017. "The Evolution of Rotation Group Bias: Will the Real Unemployment Rate Please Stand Up?," The Review of Economics and Statistics, MIT Press, vol. 99(2), pages 258-264, May.
    2. Adrian Chadi, 2019. "Dissatisfied with life or with being interviewed? Happiness and the motivation to participate in a survey," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 53(3), pages 519-553, October.
    3. Luc Behaghel & Bruno Crépon & Marc Gurgand & Thomas Le Barbanchon, 2015. "Please Call Again: Correcting Nonresponse Bias in Treatment Effect Models," The Review of Economics and Statistics, MIT Press, vol. 97(5), pages 1070-1080, December.
    4. Adrian Chadi & Clemens Hetschko, 2018. "The magic of the new: How job changes affect job satisfaction," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 27(1), pages 23-39, March.
    5. Paul P. Biemer & Patrick Chen & Kevin Wang, 2013. "Using level-of-effort paradata in non-response adjustments with application to field surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 147-168, January.
    6. Richard Williams, 2011. "Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects," CHI11 Stata Conference 9, Stata Users Group.
    7. Ori Heffetz & Matthew Rabin, 2013. "Conclusions Regarding Cross-Group Differences in Happiness Depend on Difficulty of Reaching Respondents," American Economic Review, American Economic Association, vol. 103(7), pages 3001-3021, December.
    8. Oswald, Andrew J. & Wu, Stephen, 2010. "Objective Confirmation of Subjective Measures of Human Well-being: Evidence from the USA," IZA Discussion Papers 4695, Institute of Labor Economics (IZA).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Albert Cheng & Gema Zamarro & Bart Orriens, 2020. "Personality as a Predictor of Unit Nonresponse in an Internet Panel," Sociological Methods & Research, , vol. 49(3), pages 672-698, August.
    2. Warnke, Arne Jonas, 2017. "An investigation of record linkage refusal and its implications for empirical research," ZEW Discussion Papers 17-031, ZEW - Leibniz Centre for European Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jan-Emmanuel De Neve & George Ward & Femke De Keulenaer & Bert Van Landeghem & Georgios Kavetsos & Michael I. Norton, 2018. "The Asymmetric Experience of Positive and Negative Economic Growth: Global Evidence Using Subjective Well-Being Data," The Review of Economics and Statistics, MIT Press, vol. 100(2), pages 362-375, May.
    2. de Pinto Marco & Goerke Laszlo, 2019. "Efficiency Wages in Cournot-Oligopoly," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 19(4), pages 1-13, October.
    3. Borgschulte, Mark & Cho, Heepyung & Lubotsky, Darren, 2022. "Partisanship and survey refusal," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 332-357.
    4. Bernhardt, Robert & Wunnava, Phanindra V., 2020. "The CPS Citizenship Question and Survey Refusals: Causal and Semi-Causal Evidence Featuring a Two-Stage Regression Discontinuity Design," IZA Discussion Papers 13350, Institute of Labor Economics (IZA).
    5. Lovett, Nicholas & Xue, Yuhan, 2020. "Family first or the kindness of strangers? Foster care placements and adult outcomes," Labour Economics, Elsevier, vol. 65(C).
    6. Makarski, Krzysztof & Tyrowicz, Joanna, 2019. "On welfare effects of increasing retirement age," Journal of Policy Modeling, Elsevier, vol. 41(4), pages 718-746.
    7. Wu, Wenjie, 2015. "Rail access and subjective well-being: Evidence from quality of life surveys," Journal of Comparative Economics, Elsevier, vol. 43(2), pages 456-470.
    8. Sonja C. Kassenboehmer & Stefanie Schurer, 2018. "Survey item-response behavior as an imperfect proxy for unobserved ability: Theory and application," Working Papers 2018-035, Human Capital and Economic Opportunity Working Group.
    9. de New, Sonja C. & Schurer, Stefanie & Leung, Felix, 2015. "Testing the Validity of Item Non-Response as a Proxy for Cognitive and Non-Cognitive Skills," IZA Discussion Papers 8874, Institute of Labor Economics (IZA).
    10. Nikolova, Elena & Sanfey, Peter, 2016. "How much should we trust life satisfaction data? Evidence from the Life in Transition Survey," Journal of Comparative Economics, Elsevier, vol. 44(3), pages 720-731.
    11. Yamada, Katsunori & Sato, Masayuki, 2013. "Another avenue for anatomy of income comparisons: Evidence from hypothetical choice experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 89(C), pages 35-57.
    12. Teresa Molina Millán & Karen Macours, 2017. "Attrition in randomized control trials: Using tracking information to correct bias," FEUNL Working Paper Series novaf:wp1702, Universidade Nova de Lisboa, Faculdade de Economia.
    13. Wuyts Celine & Loosveldt Geert, 2020. "Measurement of Interviewer Workload within the Survey and an Exploration of Workload Effects on Interviewers’ Field Efforts and Performance," Journal of Official Statistics, Sciendo, vol. 36(3), pages 561-588, September.
    14. Clark, Andrew E. & Senik, Claudia & Yamada, Katsunori, 2017. "When experienced and decision utility concur: The case of income comparisons," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 70(C), pages 1-9.
    15. Van Landeghem, Bert & Vandeplas, Anneleen, 2018. "The relationship between status and happiness: Evidence from the caste system in rural India," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 77(C), pages 62-71.
    16. Rémi Yin & Anthony Lepinteur & Andrew E Clark & Conchita d'Ambrosio, 2021. "Life Satisfaction and the Human Development Index Across the World," Working Papers halshs-03174513, HAL.
    17. Schwandt, Hannes, 2013. "Unmet Aspirations as an Explanation for the Age U-shape in Human Wellbeing," IZA Discussion Papers 7604, Institute of Labor Economics (IZA).
    18. Budría, Santiago & Ferrer-i-Carbonell, Ada, 2012. "Income Comparisons and Non-Cognitive Skills," IZA Discussion Papers 6419, Institute of Labor Economics (IZA).
    19. Dorsett, Richard & Oswald, Andrew J., 2014. "Human Well-being and In-Work Benefits: A Randomized Controlled Trial," IZA Discussion Papers 7943, Institute of Labor Economics (IZA).
    20. Eugenio Proto & Andrew J. Oswald, 2017. "National Happiness and Genetic Distance: A Cautious Exploration," Economic Journal, Royal Economic Society, vol. 127(604), pages 2127-2152, September.

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health
    • J60 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:22333. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.