IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0231500.html
   My bibliography  Save this article

Propensity score adjustment using machine learning classification algorithms to control selection bias in online surveys

Author

Listed:
  • Ramón Ferri-García
  • María del Mar Rueda

Abstract

Modern survey methods may be subject to non-observable bias, from various sources. Among online surveys, for example, selection bias is prevalent, due to the sampling mechanism commonly used, whereby participants self-select from a subgroup whose characteristics differ from those of the target population. Several techniques have been proposed to tackle this issue. One such is Propensity Score Adjustment (PSA), which is widely used and has been analysed in various studies. The usual method of estimating the propensity score is logistic regression, which requires a reference probability sample in addition to the online nonprobability sample. The predicted propensities can be used for reweighting using various estimators. However, in the online survey context, there are alternatives that might outperform logistic regression regarding propensity estimation. The aim of the present study is to determine the efficiency of some of these alternatives, involving Machine Learning (ML) classification algorithms. PSA is applied in two simulation scenarios, representing situations commonly found in online surveys, using logistic regression and ML models for propensity estimation. The results obtained show that ML algorithms remove selection bias more effectively than logistic regression when used for PSA, but that their efficacy depends largely on the selection mechanism employed and the dimensionality of the data.

Suggested Citation

  • Ramón Ferri-García & María del Mar Rueda, 2020. "Propensity score adjustment using machine learning classification algorithms to control selection bias in online surveys," PLOS ONE, Public Library of Science, vol. 15(4), pages 1-19, April.
  • Handle: RePEc:plo:pone00:0231500
    DOI: 10.1371/journal.pone.0231500
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0231500
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0231500&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0231500?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jelke Bethlehem, 2010. "Selection Bias in Web Surveys," International Statistical Review, International Statistical Institute, vol. 78(2), pages 161-188, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ramón Ferri-García & María del Mar Rueda, 2022. "Variable selection in Propensity Score Adjustment to mitigate selection bias in online surveys," Statistical Papers, Springer, vol. 63(6), pages 1829-1881, December.
    2. Lehmann, Nico & Sloot, Daniel & Schüle, Christopher & Ardone, Armin & Fichtner, Wolf, 2023. "The motivational drivers behind consumer preferences for regional electricity – Results of a choice experiment in Southern Germany," Energy Economics, Elsevier, vol. 120(C).
    3. Giulia Casu & Marco Giovanni Mariani & Rita Chiesa & Dina Guglielmi & Paola Gremigni, 2021. "The Role of Organizational Citizenship Behavior and Gender between Job Satisfaction and Task Performance," IJERPH, MDPI, vol. 18(18), pages 1-15, September.
    4. Bertram, Christine & Rehdanz, Katrin, 2015. "The role of urban green space for human well-being," Ecological Economics, Elsevier, vol. 120(C), pages 139-152.
    5. Galperin, Hernan & Arcidiacono, Malena, 2021. "Employment and the gender digital divide in Latin America: A decomposition analysis," Telecommunications Policy, Elsevier, vol. 45(7).
    6. Luis Castro-Martín & María del Mar Rueda & Ramón Ferri-García, 2020. "Estimating General Parameters from Non-Probability Surveys Using Propensity Score Adjustment," Mathematics, MDPI, vol. 8(11), pages 1-14, November.
    7. Grilli, Gianluca & Curtis, John, 2021. "An evaluation of public initiatives to change behaviours that affect water quality," Papers WP696, Economic and Social Research Institute (ESRI).
    8. Brian Fabo & Sharon Sarah Belli, 2017. "(Un)beliveable wages? An analysis of minimum wage policies in Europe from a living wage perspective," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 6(1), pages 1-11, December.
    9. Kawamura, Tetsuya & Mori, Tomoharu & Motonishi, Taizo & Ogawa, Kazuhito, 2021. "Is Financial Literacy Dangerous? Financial Literacy, Behavioral Factors, and Financial Choices of Households," Journal of the Japanese and International Economies, Elsevier, vol. 60(C).
    10. Paige Coyne & Zach Staffell & Sarah J. Woodruff, 2021. "Recreational Screen Time Use among a Small Sample of Canadians during the First Six Months of the COVID-19 Pandemic," IJERPH, MDPI, vol. 18(23), pages 1-9, December.
    11. Yoo, Nari & Jang, Sou Hyun, 2024. "Does social empathy moderate fear-induced minority blaming during the COVID-19 pandemic?," Social Science & Medicine, Elsevier, vol. 346(C).
    12. Berton, Fabio & Migheli Matteo, 2015. "Estimating the marginal rate of substitution between wage and employment protection," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201529, University of Turin.
    13. Felderer Barbara & Kirchner Antje & Kreuter Frauke, 2019. "The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias," Journal of Official Statistics, Sciendo, vol. 35(1), pages 93-115, March.
    14. Lang, Megan & Ligon, Ethan, 2022. "SMS Surveys of Selected Expenditures," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt7p7336h5, Department of Agricultural & Resource Economics, UC Berkeley.
    15. Curtis, John & Breen, Benjamin & O'Reilly, Paul, 2016. "Recreational Angling Tournaments: Participants’ Expenditures," Papers WP546, Economic and Social Research Institute (ESRI).
    16. Lehmann, Nico & Sloot, Daniel & Ardone, Armin & Fichtner, Wolf, 2021. "The limited potential of regional electricity marketing – Results from two discrete choice experiments in Germany," Energy Economics, Elsevier, vol. 100(C).
    17. Booth, Hollie & Mourato, Susana & Milner-Gulland, E.J., 2022. "Investigating acceptance of marine tourism levies, to cover the opportunity costs of conservation for coastal communities," Ecological Economics, Elsevier, vol. 201(C).
    18. Shen, Xuejing & Li, Shaoping & Liu, Chengfang & Luo, Renfu & Chen, Yuting, 2021. "Online Learning during the COVID-19 Pandemic Among Primary and High School Students in Rural China," 2021 Conference, August 17-31, 2021, Virtual 315351, International Association of Agricultural Economists.
    19. Tony Ross-Hellauer & Thomas Klebel & Petr Knoth & Nancy Pontika, 2024. "Value dissonance in research(er) assessment: individual and perceived institutional priorities in review, promotion, and tenure," Science and Public Policy, Oxford University Press, vol. 51(3), pages 337-351.
    20. Emmert, Martin & Hessemer, Stefanie & Meszmer, Nina & Sander, Uwe, 2014. "Do German hospital report cards have the potential to improve the quality of care?," Health Policy, Elsevier, vol. 118(3), pages 386-395.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0231500. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.