IDEAS home Printed from https://ideas.repec.org/a/vrs/offsta/v32y2016i1p165-186n8.html
   My bibliography  Save this article

Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone in an Election Survey

Author

Listed:
  • Lipps Oliver

    (FORS – Swiss Centre of Expertise in the Social Sciences, c/o University of Lausanne Quartier Mouline Lausanne 1015, Lausanne, Switzerland.)

  • Pekari Nicolas

    (FORS – Swiss Centre of Expertise in the Social Sciences, c/o University of Lausanne Quartier Mouline Lausanne 1015, Lausanne, Switzerland.)

Abstract

The objective of this article is to understand how the change of mode from telephone to web affects data quality in terms of sample representation and substantive variable bias. To this end, an experiment, consisting of a web survey with and without a prepaid incentive, was conducted alongside the telephone Swiss election survey. All three designs used identical questionnaires and probability samples drawn from a national register of individuals.First, our findings show that differences in completion rates mostly reflect different levels of coverage in the two modes. Second, incentives in the web survey strongly increase completion rates of all person groups, with the exception of people without Internet access or limited computer literacy. Third, we find voting behavior to be much closer to official figures in the web with the incentive version compared to the two other designs. However, this is partly due to the different sociodemographic compositions of the samples. Other substantive results suggest that the incentive version includes harder-to-reach respondents. Unit costs are much lower in the two web designs compared to the telephone, including when a relatively high incentive is used. We conclude that in countries with high Internet penetration rates such as Switzerland, web surveys are already likely to be highly competitive.

Suggested Citation

  • Lipps Oliver & Pekari Nicolas, 2016. "Sample Representation and Substantive Outcomes Using Web With and Without Incentives Compared to Telephone in an Election Survey," Journal of Official Statistics, Sciendo, vol. 32(1), pages 165-186, March.
    • Handle: RePEc:vrs:offsta:v:32:y:2016:i:1:p:165-186:n:8
    DOI: 10.1515/jos-2016-0008
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jos-2016-0008
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jos-2016-0008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ansolabehere, Stephen & Schaffner, Brian F., 2014. "Does Survey Mode Still Matter? Findings from a 2010 Multi-Mode Comparison," Political Analysis, Cambridge University Press, vol. 22(3), pages 285-303, July.
    2. Mohorko Anja & Leeuw Edith de & Hox Joop, 2013. "Internet Coverage and Coverage Bias in Europe: Developments Across Countries and Over Time," Journal of Official Statistics, Sciendo, vol. 29(4), pages 609-622, December.
    3. Britta Busse & Marek Fuchs, 2012. "The components of landline telephone survey coverage bias. The relative importance of no-phone and mobile-only populations," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(4), pages 1209-1225, June.
    4. Matthias Schonlau & Arthur van Soest & Arie Kapteyn & Mick Couper, 2009. "Selection Bias in Web Surveys and the Use of Propensity Scores," Sociological Methods & Research, , vol. 37(3), pages 291-318, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giovanni Franco, 2024. "The return of non-probability sample: the electoral polls at the time of internet and social media," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(4), pages 3811-3830, August.
    2. Ana Gomes & José G. Dias, 2023. "Is there a Common Digital Market in the European Union? Implications for the European Digitalization Strategy," Journal of Common Market Studies, Wiley Blackwell, vol. 61(3), pages 797-814, May.
    3. Crossley, Thomas F. & Fisher, Paul & Low, Hamish, 2021. "The heterogeneous and regressive consequences of COVID-19: Evidence from high quality panel data," Journal of Public Economics, Elsevier, vol. 193(C).
    4. Guzi, Martin & de Pedraza, Pablo, 2013. "A Web Survey Analysis of the Subjective Well-being of Spanish Workers," IZA Discussion Papers 7618, Institute of Labor Economics (IZA).
    5. Helbling, Marc & Jungkunz, Sebastian, 2020. "Social divides in the age of globalization," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 43(6), pages 1187-1210.
    6. Sakshaug Joseph W. & Wiśniowski Arkadiusz & Ruiz Diego Andres Perez & Blom Annelies G., 2019. "Supplementing Small Probability Samples with Nonprobability Samples: A Bayesian Approach," Journal of Official Statistics, Sciendo, vol. 35(3), pages 653-681, September.
    7. Maciej Berȩsewicz & Dagmara Nikulin, 2021. "Estimation of the size of informal employment based on administrative records with non‐ignorable selection mechanism," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(3), pages 667-690, June.
    8. Piatak Jaclyn, 2023. "Do Sociocultural Factors Drive Civic Engagement? An Examination of Political Interest and Religious Attendance," Nonprofit Policy Forum, De Gruyter, vol. 14(2), pages 185-204, April.
    9. Stéphane Legleye & Géraldine Charrance & Nicolas Razafindratsima & Nathalie Bajos & Aline Bohet & Caroline Moreau, 2018. "The Use of a Nonprobability Internet Panel to Monitor Sexual and Reproductive Health in the General Population," Sociological Methods & Research, , vol. 47(2), pages 314-348, March.
    10. David M. Konisky & Llewelyn Hughes & Charles H. Kaylor, 2016. "Extreme weather events and climate change concern," Climatic Change, Springer, vol. 134(4), pages 533-547, February.
    11. Michael A. Cacciatore & Glen J. Nowak & Nathaniel J. Evans, 2018. "It's Complicated: The 2014–2015 U.S. Measles Outbreak and Parents’ Vaccination Beliefs, Confidence, and Intentions," Risk Analysis, John Wiley & Sons, vol. 38(10), pages 2178-2192, October.
    12. McFadden, Brandon R. & Malone, Trey, 2018. "How will mandatory labeling of genetically modified food nudge consumer decision-making?," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 77(C), pages 186-194.
    13. Heng Chen & Geoffrey Dunbar & Q. Rallye Shen, 2020. "The Mode is the Message: Using Predata as Exclusion Restrictions to Evaluate Survey Design," Advances in Econometrics, in: Essays in Honor of Cheng Hsiao, volume 41, pages 341-357, Emerald Group Publishing Limited.
    14. Hildebrand Sean, 2015. "Coerced Confusion? Local Emergency Policy Implementation After September 11," Journal of Homeland Security and Emergency Management, De Gruyter, vol. 12(2), pages 273-298, June.
    15. Craig F. Berning & Brian E. Roe, 2017. "Assessing the National Bioengineered Food Disclosure Standard of 2016: Can Americans Access Electronic Disclosure Information?," Sustainability, MDPI, vol. 9(6), pages 1-9, May.
    16. Anderson, Brilé & Bernauer, Thomas, 2016. "How much carbon offsetting and where? Implications of efficiency, effectiveness, and ethicality considerations for public opinion formation," Energy Policy, Elsevier, vol. 94(C), pages 387-395.
    17. Aaron Martin & Timothy B Gravelle & Erik Baekkeskov & Jenny Lewis & Yoshi Kashima, 2019. "Enlisting the support of trusted sources to tackle policy problems: The case of antimicrobial resistance," PLOS ONE, Public Library of Science, vol. 14(3), pages 1-9, March.
    18. Grewenig, Elisabeth & Lergetporer, Philipp & Simon, Lisa & Werner, Katharina & Woessmann, Ludger, 2018. "Can Online Surveys Represent the Entire Population?," IZA Discussion Papers 11799, Institute of Labor Economics (IZA).
    19. Timmons, Shane & Barjaková, Martina & Robertson, Deirdre & Belton, Cameron & Lunn, Pete, 2020. "Public understanding and perceptions of the COVID-19 Test-and-Trace system," Research Series, Economic and Social Research Institute (ESRI), number SUSTAT96.
    20. Knox, Melissa A. & Oddo, Vanessa M. & Walkinshaw, Lina Pinero & Jones-Smith, Jessica, 2020. "Is the public sweet on sugary beverages? Social desirability bias and sweetened beverage taxes," Economics & Human Biology, Elsevier, vol. 38(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vrs:offsta:v:32:y:2016:i:1:p:165-186:n:8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.sciendo.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.