IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v44y2015i3p391-426.html
   My bibliography  Save this article

Measurement Error Calibration in Mixed-mode Sample Surveys

Author

Listed:
  • Bart Buelens
  • Jan A. van den Brakel

Abstract

Mixed-mode surveys are known to be susceptible to mode-dependent selection and measurement effects, collectively referred to as mode effects. The use of different data collection modes within the same survey may reduce selectivity of the overall response but is characterized by measurement errors differing across modes. Inference in sample surveys generally proceeds by correcting for selectivity—for example, by applying calibration estimators—and ignoring measurement error. When a survey is conducted repeatedly, such inferences are valid only if the measurement error remains constant between surveys. In sequential mixed-mode surveys, it is likely that the mode composition of the overall response differs between subsequent editions of the survey, leading to variations in the total measurement error and invalidating classical inferences. An approach to inference in these circumstances, which is based on calibrating the mode composition of the respondents toward fixed levels, is proposed. Assumptions and risks are discussed and explored in a simulation and applied to the Dutch crime victimization survey.

Suggested Citation

  • Bart Buelens & Jan A. van den Brakel, 2015. "Measurement Error Calibration in Mixed-mode Sample Surveys," Sociological Methods & Research, , vol. 44(3), pages 391-426, August.
    • Handle: RePEc:sae:somere:v:44:y:2015:i:3:p:391-426
    DOI: 10.1177/0049124114532444
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124114532444
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0049124114532444?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2006. "Design of Web Questionnaires : The Effect of Layout in Rating Scales," Other publications TiSEM 9401f6c5-0275-400e-ac97-c, Tilburg University, School of Economics and Management.
    2. Vera Toepoel & Corrie Vis & Marcel Das & Arthur van Soest, 2009. "Design of Web Questionnaires," Sociological Methods & Research, , vol. 37(3), pages 371-392, February.
    3. Annette Jäckle & Caroline Roberts & Peter Lynn, 2010. "Assessing the Effect of Data Collection Mode on Measurement," International Statistical Review, International Statistical Institute, vol. 78(1), pages 3-20, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anna DeCastellarnau, 2018. "A classification of response scale characteristics that affect data quality: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1523-1559, July.
    2. Dana Garbarski & Nora Cate Schaeffer & Jennifer Dykema, 2019. "The Effects of Features of Survey Measurement on Self-Rated Health: Response Option Order and Scale Orientation," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 14(2), pages 545-560, April.
    3. Brauner, Jacob, 2020. "Are Smileys Valid Answers? Survey Data Quality with Innovative Item Formats," SocArXiv dk9bc, Center for Open Science.
    4. Zawojska, Ewa & Czajkowski, Mikotaj, 2017. "Are preferences stated in web vs. personal interviews different? A comparison of willingness to pay results for a large multi-country study of the Baltic Sea eutrophication reduction," Annual Meeting, 2017, June 18-21, Montreal, Canada 258604, Canadian Agricultural Economics Society.
    5. Debra Wright & Matt Sloan & Kirsten Barrett, 2012. "Is There a Trade-off Between Quality and Cost? Telephone Versus Face-to-Face Interviewing of Persons with Disabilities," Mathematica Policy Research Reports cb6067df035641e99a913d534, Mathematica Policy Research.
    6. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    7. Jorre T. A. Vannieuwenhuyze & Geert Loosveldt, 2013. "Evaluating Relative Mode Effects in Mixed-Mode Surveys:," Sociological Methods & Research, , vol. 42(1), pages 82-104, February.
    8. Mackeben, Jan, 2020. "Mode Effects in the Fourth Wave of the Linked Personnel Panel (LPP) Employee Survey," FDZ Methodenreport 202005_en, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    9. Grewenig, Elisabeth & Lergetporer, Philipp & Simon, Lisa & Werner, Katharina & Woessmann, Ludger, 2018. "Can Online Surveys Represent the Entire Population?," IZA Discussion Papers 11799, Institute of Labor Economics (IZA).
    10. Chatpong Tangmanee & Phattharaphong Niruttinanon, 2019. "Web Survey’s Completion Rates: Effects of Forced Responses, Question Display Styles, and Subjects’ Attitude," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 8(1), pages 20-29, January.
    11. Jäckle, Annette & Roberts, Caroline, 2012. "Causes of mode effects: separating out interviewer and stimulus effects in comparisons of face-to-face and telephone surveys," ISER Working Paper Series 2012-27, Institute for Social and Economic Research.
    12. Carina Cornesse & Annelies G. Blom, 2023. "Response Quality in Nonprobability and Probability-based Online Panels," Sociological Methods & Research, , vol. 52(2), pages 879-908, May.
    13. Wenz, Alexander, 2017. "Completing web surveys on mobile devices: does screen size affect data quality?," ISER Working Paper Series 2017-05, Institute for Social and Economic Research.
    14. Fabo, B., 2017. "Towards an understanding of job matching using web data," Other publications TiSEM b8b877f2-ae6a-495f-b6cc-9, Tilburg University, School of Economics and Management.
    15. Weijters, Bert & Millet, Kobe & Cabooter, Elke, 2021. "Extremity in horizontal and vertical Likert scale format responses. Some evidence on how visual distance between response categories influences extreme responding," International Journal of Research in Marketing, Elsevier, vol. 38(1), pages 85-103.
    16. de Leeuw, E.D. & Hox, J.J.C.M. & Scherpenzeel, A.C., 2011. "Mode effect or question wording? Measurement error in mixed mode surveys," Other publications TiSEM 4218c762-6d80-4dfc-97ee-8, Tilburg University, School of Economics and Management.
    17. Bernhard Schimpl-Neimanns, 2013. "Methodische Herausforderungen bei der Erfassung von Bildung und Ausbildung im Mikrozensus," RatSWD Working Papers 221, German Data Forum (RatSWD).
    18. Giorgio Piccitto & Aart C. Liefbroer & Tom Emery, 2022. "Does the Survey Mode Affect the Association Between Subjective Well-being and its Determinants? An Experimental Comparison Between Face-to-Face and Web Mode," Journal of Happiness Studies, Springer, vol. 23(7), pages 3441-3461, October.
    19. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Other publications TiSEM 21fbcc01-3c5d-4942-b41c-7, Tilburg University, School of Economics and Management.
    20. Pirmin Fessler & Maximilian Kasy & Peter Lindner, 2018. "Survey mode effects on measured income inequality," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 16(4), pages 487-505, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:44:y:2015:i:3:p:391-426. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.