IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/qrwg4.html
   My bibliography  Save this paper

Social Media, Web, and Panel Surveys: Using Non- Probability Samples in Social and Policy Research

Author

Listed:
  • Lehdonvirta, Vili
  • Oksanen, Atte

    (University of Tampere)

  • Räsänen, Pekka
  • Blank, Grant

Abstract

The use of online surveys has grown rapidly in social science and policy research, surpassing more established methods. We argue that a better understanding is needed, especially of the strengths and weaknesses of non-probability online surveys that can be conducted relatively quickly and cheaply. We describe two common approaches to non-probability online surveys – river and panel sampling – and theorize their inherent selection biases: topical self-selection and economic self-selection. We conduct an empirical comparison of two river samples (Facebook and web-based) and one panel sample (from a major survey research company) with benchmark data grounded in a comprehensive population registry. We examine (1) how closely the online samples correspond with the benchmark, and (2) their usefulness in studying a non-demographic subpopulation. The river samples diverge from the benchmark on demographic variables and yield much higher means on non-demographic variables, even after weighting; we attribute this to topical self-selection. The panel is closer to the benchmark. When examining the characteristics of a non-demographic subpopulation, we detect no differences between the river and panel samples. We conclude that non-probability online surveys don’t replace probability surveys, but augment the researcher’s toolkit with new digital practices, such as exploratory studies of small and emerging non-demographic subpopulations.

Suggested Citation

  • Lehdonvirta, Vili & Oksanen, Atte & Räsänen, Pekka & Blank, Grant, 2020. "Social Media, Web, and Panel Surveys: Using Non- Probability Samples in Social and Policy Research," OSF Preprints qrwg4, Center for Open Science.
  • Handle: RePEc:osf:osfxxx:qrwg4
    DOI: 10.31219/osf.io/qrwg4
    as

    Download full text from publisher

    File URL: https://osf.io/download/5e8aff5b430166009ca0c239/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/qrwg4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Malhotra, Neil & Krosnick, Jon A., 2007. "The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples," Political Analysis, Cambridge University Press, vol. 15(3), pages 286-323, July.
    2. Jelke Bethlehem, 2010. "Selection Bias in Web Surveys," International Statistical Review, International Statistical Institute, vol. 78(2), pages 161-188, August.
    3. repec:nas:journl:v:115:y:2018:p:12441-12446 is not listed on IDEAS
    4. Lehdonvirta, Vili, 2018. "Flexibility in the Gig Economy: Managing Time on Three Online Piecework Platforms," SocArXiv k3hy4, Center for Open Science.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stefano Visintin & Kea Tijdens & Stephanie Steinmetz & Pablo de Pedraza, 2015. "Task implementation heterogeneity and wage dispersion," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-24, December.
    2. Ramón Ferri-García & María del Mar Rueda, 2022. "Variable selection in Propensity Score Adjustment to mitigate selection bias in online surveys," Statistical Papers, Springer, vol. 63(6), pages 1829-1881, December.
    3. Lehmann, Nico & Sloot, Daniel & Schüle, Christopher & Ardone, Armin & Fichtner, Wolf, 2023. "The motivational drivers behind consumer preferences for regional electricity – Results of a choice experiment in Southern Germany," Energy Economics, Elsevier, vol. 120(C).
    4. Gilles Paché, 2020. "Inside Delivery Platforms: The Covid-19 Pandemic And After," Post-Print hal-03041080, HAL.
    5. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    6. Lasse J. Jessen & Sebastian Koehne & Patrick Nüß & Jens Ruhose, 2024. "Socioeconomic Inequality in Life Expectancy: Perception and Policy Demand," CESifo Working Paper Series 10940, CESifo.
    7. Galperin, Hernan & Arcidiacono, Malena, 2021. "Employment and the gender digital divide in Latin America: A decomposition analysis," Telecommunications Policy, Elsevier, vol. 45(7).
    8. Aaron C. Sparks & Heather Hodges & Sarah Oliver & Eric R. A. N. Smith, 2020. "Confidence in Local, National, and International Scientists on Climate Change," Sustainability, MDPI, vol. 13(1), pages 1-13, December.
    9. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    10. Grilli, Gianluca & Curtis, John, 2021. "An evaluation of public initiatives to change behaviours that affect water quality," Papers WP696, Economic and Social Research Institute (ESRI).
    11. Niki Panteli & Andriana Rapti & Dora Scholarios, 2020. "‘If He Just Knew Who We Were’: Microworkers’ Emerging Bonds of Attachment in a Fragmented Employment Relationship," Work, Employment & Society, British Sociological Association, vol. 34(3), pages 476-494, June.
    12. Yáñez-Valdés, Claudia & Guerrero, Maribel & Barros-Celume, Sebastián & Ibáñez, María J., 2023. "Winds of change due to global lockdowns: Refreshing digital social entrepreneurship research paradigm," Technological Forecasting and Social Change, Elsevier, vol. 190(C).
    13. Brian Fabo & Sharon Sarah Belli, 2017. "(Un)beliveable wages? An analysis of minimum wage policies in Europe from a living wage perspective," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 6(1), pages 1-11, December.
    14. Arfive Gandhi & Yudho Giri Sucahyo, 2021. "Architecting an Advanced Maturity Model for Business Processes in the Gig Economy: A Platform-Based Project Standardization," Economies, MDPI, vol. 9(4), pages 1-21, November.
    15. Yoo, Nari & Jang, Sou Hyun, 2024. "Does social empathy moderate fear-induced minority blaming during the COVID-19 pandemic?," Social Science & Medicine, Elsevier, vol. 346(C).
    16. Berton, Fabio & Migheli Matteo, 2015. "Estimating the marginal rate of substitution between wage and employment protection," Department of Economics and Statistics Cognetti de Martiis. Working Papers 201529, University of Turin.
    17. Felderer Barbara & Kirchner Antje & Kreuter Frauke, 2019. "The Effect of Survey Mode on Data Quality: Disentangling Nonresponse and Measurement Error Bias," Journal of Official Statistics, Sciendo, vol. 35(1), pages 93-115, March.
    18. Marie Nilsen & Trond Kongsvik & Stian Antonsen, 2022. "Taming Proteus: Challenges for Risk Regulation of Powerful Digital Labor Platforms," IJERPH, MDPI, vol. 19(10), pages 1-23, May.
    19. Magdalena Smyk & Joanna Tyrowicz & Lucas van der Velde, 2021. "A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator," Sociological Methods & Research, , vol. 50(1), pages 429-464, February.
    20. Booth, Hollie & Mourato, Susana & Milner-Gulland, E.J., 2022. "Investigating acceptance of marine tourism levies, to cover the opportunity costs of conservation for coastal communities," Ecological Economics, Elsevier, vol. 201(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:qrwg4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.