IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v46y2017i3p456-489.html
   My bibliography  Save this article

Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys

Author

Listed:
  • Thomas Klausch
  • Barry Schouten
  • Joop J. Hox

Abstract

This study evaluated three types of bias—total, measurement, and selection bias (SB)—in three sequential mixed-mode designs of the Dutch Crime Victimization Survey: telephone, mail, and web, where nonrespondents were followed up face-to-face (F2F). In the absence of true scores, all biases were estimated as mode effects against two different types of benchmarks. In the single-mode benchmark (SMB), effects were evaluated against a F2F reference survey. In an alternative analysis, a “hybrid-mode benchmark†(HMB) was used, where effects were evaluated against a mix of the measurements of a web survey and the SB of a F2F survey. A special reinterview design made available additional auxiliary data exploited in estimation for a range of survey variables. Depending on the SMB and HMB perspectives, a telephone, mail, or web design with a F2F follow-up (SMB) or a design involving only mail and/or web but not a F2F follow-up (HMB) is recommended based on the empirical findings.

Suggested Citation

  • Thomas Klausch & Barry Schouten & Joop J. Hox, 2017. "Evaluating Bias of Sequential Mixed-mode Designs Against Benchmark Surveys," Sociological Methods & Research, , vol. 46(3), pages 456-489, August.
    • Handle: RePEc:sae:somere:v:46:y:2017:i:3:p:456-489
    DOI: 10.1177/0049124115585362
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124115585362
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124115585362?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    2. Donald B. Rubin, 2003. "Nested multiple imputation of NMES via partially incompatible MCMC," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 57(1), pages 3-18, February.
    3. Vannieuwenhuyze Jorre T.A. & Loosveldt Geert & Molenberghs Geert, 2014. "Evaluating Mode Effects in Mixed-Mode Survey Data Using Covariate Adjustment Models," Journal of Official Statistics, Sciendo, vol. 30(1), pages 1-21, March.
    4. Thomas Klausch & Joop Hox & Barry Schouten, 2015. "Selection error in single- and mixed mode surveys of the Dutch general population," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 945-961, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. van de Walle, Dominique & Mu, Ren, 2007. "Fungibility and the flypaper effect of project aid: Micro-evidence for Vietnam," Journal of Development Economics, Elsevier, vol. 84(2), pages 667-685, November.
    2. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    3. Turner, Alex J. & Fichera, Eleonora & Sutton, Matt, 2021. "The effects of in-utero exposure to influenza on mental health and mortality risk throughout the life-course," Economics & Human Biology, Elsevier, vol. 43(C).
    4. Roxana Elena Manea, 2021. "School Feeding Programmes, Education and Food Security in Rural Malawi," CIES Research Paper series 63-2020, Centre for International Environmental Studies, The Graduate Institute.
    5. repec:ags:jrapmc:122316 is not listed on IDEAS
    6. Gunther Bensch & Jörg Peters, 2013. "Alleviating Deforestation Pressures? Impacts of Improved Stove Dissemination on Charcoal Consumption in Urban Senegal," Land Economics, University of Wisconsin Press, vol. 89(4), pages 676-698.
    7. Michael Lechner & Ruth Miquel & Conny Wunsch, 2011. "Long‐Run Effects Of Public Sector Sponsored Training In West Germany," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 742-784, August.
    8. Ralf Becker & Maggy Fostier, 2015. "Evaluating non-compulsory educational interventions - the case of peer assisted study groups," Economics Discussion Paper Series 1509, Economics, The University of Manchester.
    9. Katie Meara & Francesco Pastore & Allan Webster, 2020. "The gender pay gap in the USA: a matching study," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(1), pages 271-305, January.
    10. Lou, Jiehong & Shen, Xingchi & Niemeier, Deb, 2020. "Are stay-at-home orders more difficult to follow for low-income groups?," Journal of Transport Geography, Elsevier, vol. 89(C).
    11. Gernandt, Johannes & Maier, Michael & Pfeiffer, Friedhelm & Rat-Wirtzler, Julie, 2006. "Distributional effects of the high school degree in Germany," ZEW Discussion Papers 06-088, ZEW - Leibniz Centre for European Economic Research.
    12. Naguib, Costanza, 2019. "Estimating the Heterogeneous Impact of the Free Movement of Persons on Relative Wage Mobility," Economics Working Paper Series 1903, University of St. Gallen, School of Economics and Political Science.
    13. Lechner, Michael & Sari, Nazmi, 2015. "Labor market effects of sports and exercise: Evidence from Canadian panel data," Labour Economics, Elsevier, vol. 35(C), pages 1-15.
    14. Alan Manning & Barbara Petrongolo, 2005. "The Part-Time Pay Penalty," CEP Discussion Papers dp0679, Centre for Economic Performance, LSE.
    15. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    16. Bodory, Hugo & Huber, Martin, 2018. "The causalweight package for causal inference in R," FSES Working Papers 493, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    17. Toro, Weily & Tigre, Robson & Sampaio, Breno, 2015. "Daylight Saving Time and incidence of myocardial infarction: Evidence from a regression discontinuity design," Economics Letters, Elsevier, vol. 136(C), pages 1-4.
    18. Caloffi, Annalisa & Freo, Marzia & Ghinoi, Stefano & Mariani, Marco & Rossi, Federica, 2022. "Assessing the effects of a deliberate policy mix: The case of technology and innovation advisory services and innovation vouchers," Research Policy, Elsevier, vol. 51(6).
    19. Sascha O. Becker & Marco Caliendo, 2007. "Sensitivity analysis for average treatment effects," Stata Journal, StataCorp LP, vol. 7(1), pages 71-83, February.
    20. Augurzky, Boris & Kluve, Jochen, 2004. "Assessing the performance of matching algorithms when selection into treatment is strong," RWI Discussion Papers 21, RWI - Leibniz-Institut für Wirtschaftsforschung.
    21. Emilia Del Bono & Andrea Weber & Rudolf Winter-Ebmer, 2012. "Clash Of Career And Family: Fertility Decisions After Job Displacement," Journal of the European Economic Association, European Economic Association, vol. 10(4), pages 659-683, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:46:y:2017:i:3:p:456-489. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.