IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v58y2024i4d10.1007_s11135-024-01835-8.html
   My bibliography  Save this article

The return of non-probability sample: the electoral polls at the time of internet and social media

Author

Listed:
  • Giovanni Franco

    (Sapienza University of Rome)

Abstract

For the past 80 years survey researchers have used a probability sampling framework. Probability samples have a well-defined set of quality criteria that have been organized around the concept of Total Survey Error (TSE). Non-probability samples do not fit within this framework very well and some possible alternatives to TSE are explored. In recent years, electoral polls have undergone changes as a result of the dispersion of public opinion due mostly, but not only, to the development of the web and social media. From a methodological point of view, the main changes concerned sampling and data collection techniques. The aim of the article is to provide a critical contribution to the methodological debate on electoral polls with particular attention to the samples used which appear to be more similar to non-probability samples than to the traditional probability samples used for many decades in electoral polls. We will explore several new approaches that attempt to make inference possible even when a survey sample does not match the classic probability sample. We will also discuss a set of post hoc adjustments that have been suggested as ways to reduce the bias in estimates from non-probability samples; these adjustments use auxiliary data in an effort to deal with selection and other biases. Propensity score adjustment is the most well know of these techniques. The empirical section of the article analyzes a database of 1793 electoral polls conducted in Italy from January 2017 to July 2023.

Suggested Citation

  • Giovanni Franco, 2024. "The return of non-probability sample: the electoral polls at the time of internet and social media," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(4), pages 3811-3830, August.
    • Handle: RePEc:spr:qualqt:v:58:y:2024:i:4:d:10.1007_s11135-024-01835-8
    DOI: 10.1007/s11135-024-01835-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-024-01835-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-024-01835-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Melanie Revilla, 2015. "Comparison of the quality estimates in a mixed-mode and a unimode design: an experiment from the European Social Survey," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 1219-1238, May.
    2. Britta Busse & Marek Fuchs, 2012. "The components of landline telephone survey coverage bias. The relative importance of no-phone and mobile-only populations," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(4), pages 1209-1225, June.
    3. J. Michael Brick & Douglas Williams, 2013. "Explaining Rising Nonresponse Rates in Cross-Sectional Surveys," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 36-59, January.
    4. Couper, Mick P. & Kapteyn, Arie & Schonlau, Matthias & Winter, Joachim, 2007. "Noncoverage and nonresponse in an Internet survey," Munich Reprints in Economics 20093, University of Munich, Department of Economics.
    5. Matthias Schonlau & Arthur van Soest & Arie Kapteyn & Mick Couper, 2009. "Selection Bias in Web Surveys and the Use of Propensity Scores," Sociological Methods & Research, , vol. 37(3), pages 291-318, February.
    6. Jelke Bethlehem, 2010. "Selection Bias in Web Surveys," International Statistical Review, International Statistical Institute, vol. 78(2), pages 161-188, August.
    7. Giovanni Di Franco & Michele Santurro, 2021. "Machine learning, artificial neural networks and social research," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(3), pages 1007-1025, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Crossley, Thomas F. & Fisher, Paul & Low, Hamish, 2021. "The heterogeneous and regressive consequences of COVID-19: Evidence from high quality panel data," Journal of Public Economics, Elsevier, vol. 193(C).
    2. Grewenig, Elisabeth & Lergetporer, Philipp & Simon, Lisa & Werner, Katharina & Woessmann, Ludger, 2018. "Can Online Surveys Represent the Entire Population?," IZA Discussion Papers 11799, Institute of Labor Economics (IZA).
    3. Arthur van Soest & Arie Kapteyn, 2009. "Mode and Context Effects in Measuring Household Assets," Working Papers 200949, Geary Institute, University College Dublin.
    4. Mohorko Anja & Leeuw Edith de & Hox Joop, 2013. "Internet Coverage and Coverage Bias in Europe: Developments Across Countries and Over Time," Journal of Official Statistics, Sciendo, vol. 29(4), pages 609-622, December.
    5. Stefano Visintin & Kea Tijdens & Stephanie Steinmetz & Pablo de Pedraza, 2015. "Task implementation heterogeneity and wage dispersion," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-24, December.
    6. Joanne W. Hsu & Brooke H. McFall, 2015. "Mode effects in mixed-mode economic surveys: Insights from a randomized experiment," Finance and Economics Discussion Series 2015-8, Board of Governors of the Federal Reserve System (U.S.).
    7. Lipps Oliver & Pekari Nicolas, 2016. "Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone in an Election Survey," Journal of Official Statistics, Sciendo, vol. 32(1), pages 165-186, March.
    8. Camilla Salvatore, 2023. "Inference with non-probability samples and survey data integration: a science mapping study," METRON, Springer;Sapienza Università di Roma, vol. 81(1), pages 83-107, April.
    9. Grewenig, Elisabeth & Lergetporer, Philipp & Simon, Lisa & Werner, Katharina & Woessmann, Ludger, 2023. "Can internet surveys represent the entire population? A practitioners’ analysis," European Journal of Political Economy, Elsevier, vol. 78(C).
    10. Lang, Megan & Ligon, Ethan, 2022. "SMS Surveys of Selected Expenditures," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt7p7336h5, Department of Agricultural & Resource Economics, UC Berkeley.
    11. repec:aia:aiaswp:wp76 is not listed on IDEAS
    12. Arthur van Soest & Arie Kapteyn, 2009. "Mode and Context Effects in Measuring Household Assets," Working Papers 200949, Geary Institute, University College Dublin.
    13. Ramón Ferri-García & María del Mar Rueda, 2022. "Variable selection in Propensity Score Adjustment to mitigate selection bias in online surveys," Statistical Papers, Springer, vol. 63(6), pages 1829-1881, December.
    14. Lehmann, Nico & Sloot, Daniel & Schüle, Christopher & Ardone, Armin & Fichtner, Wolf, 2023. "The motivational drivers behind consumer preferences for regional electricity – Results of a choice experiment in Southern Germany," Energy Economics, Elsevier, vol. 120(C).
    15. Guzi, Martin & de Pedraza, Pablo, 2013. "A Web Survey Analysis of the Subjective Well-being of Spanish Workers," IZA Discussion Papers 7618, Institute of Labor Economics (IZA).
    16. Galperin, Hernan & Arcidiacono, Malena, 2021. "Employment and the gender digital divide in Latin America: A decomposition analysis," Telecommunications Policy, Elsevier, vol. 45(7).
    17. Grilli, Gianluca & Curtis, John, 2021. "An evaluation of public initiatives to change behaviours that affect water quality," Papers WP696, Economic and Social Research Institute (ESRI).
    18. Borgschulte, Mark & Cho, Heepyung & Lubotsky, Darren, 2022. "Partisanship and survey refusal," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 332-357.
    19. Katharine G. Abraham, 2022. "Big Data and Official Statistics," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 68(4), pages 835-861, December.
    20. Stéphane Legleye & Géraldine Charrance & Nicolas Razafindratsima & Nathalie Bajos & Aline Bohet & Caroline Moreau, 2018. "The Use of a Nonprobability Internet Panel to Monitor Sexual and Reproductive Health in the General Population," Sociological Methods & Research, , vol. 47(2), pages 314-348, March.
    21. Brian Fabo & Sharon Sarah Belli, 2017. "(Un)beliveable wages? An analysis of minimum wage policies in Europe from a living wage perspective," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 6(1), pages 1-11, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:58:y:2024:i:4:d:10.1007_s11135-024-01835-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.