IDEAS home Printed from https://ideas.repec.org/a/eee/eejocm/v14y2015icp17-31.html
   My bibliography  Save this article

Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys?

Author

Listed:
  • Liebe, Ulf
  • Glenk, Klaus
  • Oehlmann, Malte
  • Meyerhoff, Jürgen

Abstract

Web surveys are becoming increasingly popular in survey research including stated preference surveys. Compared with face-to-face, telephone and mail surveys, web surveys may contain a different and new source of measurement error and bias: the type of device that respondents use to answer the survey questions. This is the first study that tests whether the use of mobile devices, tablets or smartphones, affects survey characteristics and stated preferences in a web-based choice experiment. The web survey on expanding renewable energy production in Germany was carried out with 3182 respondents, of which 12% used a mobile device. Propensity score matching is used to account for selection bias in the use of mobile devices for survey completion. We find that mobile device users spent more time than desktop/laptop users to answer the survey. Yet, desktop/laptop users and mobile device users do not differ in acquiescence tendency as an indicator of extreme response patterns. For mobile device users only, we find a negative correlation between screen size and interview length and a positive correlation between screen size and acquiescence tendency. In the choice experiment data, we do not find significant differences in the tendency to choose the status quo option and scale between both subsamples. However, some of the estimates of implicit prices differ, albeit not in a unidirectional fashion. Model results for mobile device users indicate a U-shaped relationship between error variance and screen size. Together, the results suggest that using mobile devices is not detrimental to survey quality.

Suggested Citation

  • Liebe, Ulf & Glenk, Klaus & Oehlmann, Malte & Meyerhoff, Jürgen, 2015. "Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys?," Journal of choice modelling, Elsevier, vol. 14(C), pages 17-31.
    • Handle: RePEc:eee:eejocm:v:14:y:2015:i:c:p:17-31
    DOI: 10.1016/j.jocm.2015.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1755534515000068
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jocm.2015.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jill Windle & John Rolfe, 2011. "Comparing Responses from Internet and Paper-Based Collection Methods in more Complex Stated Preference Environmental Valuation Surveys," Economic Analysis and Policy, Elsevier, vol. 41(1), pages 83-97, March.
    2. Matthew Blackwell & Stefano Iacus & Gary King & Giuseppe Porro, 2009. "cem: Coarsened exact matching in Stata," Stata Journal, StataCorp LP, vol. 9(4), pages 524-546, December.
    3. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521766555.
    4. Riccardo Scarpa & John M. Rose, 2008. "Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why ," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 52(3), pages 253-282, September.
    5. Gregory L. Poe & Kelly L. Giraud & John B. Loomis, 2005. "Computational Methods for Measuring the Difference of Empirical Distributions," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(2), pages 353-365.
    6. Riccardo Scarpa & Mara Thiene & Kenneth Train, 2008. "Utility in Willingness to Pay Space: A Tool to Address Confounding Random Scale Effects in Destination Choice to the Alps," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 90(4), pages 994-1010.
    7. Swait, Joffre & Adamowicz, Wiktor, 2001. "Choice Environment, Market Complexity, and Consumer Behavior: A Theoretical and Empirical Approach for Incorporating Decision Complexity into Models of Consumer Choice," Organizational Behavior and Human Decision Processes, Elsevier, vol. 86(2), pages 141-167, November.
    8. DeShazo, J. R. & Fermo, German, 2002. "Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency," Journal of Environmental Economics and Management, Elsevier, vol. 44(1), pages 123-143, July.
    9. Iacus, Stefano & King, Gary & Porro, Giuseppe, 2009. "cem: Software for Coarsened Exact Matching," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 30(i09).
    10. Hess, Stephane & Stathopoulos, Amanda, 2013. "Linking response quality to survey engagement: A combined random scale and latent variable approach," Journal of choice modelling, Elsevier, vol. 7(C), pages 1-12.
    11. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    12. Stephane Hess & Amanda Stathopoulos & Andrew Daly, 2012. "Allowing for heterogeneous decision rules in discrete choice models: an approach and four case studies," Transportation, Springer, vol. 39(3), pages 565-591, May.
    13. Scarpa, R. & Thiene, M. & Train, K., 2008. "Appendix to Utility in WTP space: a tool to address confounding random scale effects in destination choice to the Alps," American Journal of Agricultural Economics APPENDICES, Agricultural and Applied Economics Association, vol. 90(4), pages 1-9, January.
    14. Marta-Pedroso, Cristina & Freitas, Helena & Domingos, Tiago, 2007. "Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews," Ecological Economics, Elsevier, vol. 62(3-4), pages 388-398, May.
    15. Nielsen, Jytte Seested, 2011. "Use of the Internet for willingness-to-pay surveys: A comparison of face-to-face and web-based interviews," Resource and Energy Economics, Elsevier, vol. 33(1), pages 119-129, January.
    16. John Rose & Iain Black, 2006. "Means matter, but variance matter too: Decomposing response latency influences on variance heterogeneity in stated preference experiments," Marketing Letters, Springer, vol. 17(4), pages 295-310, December.
    17. Søren Olsen, 2009. "Choosing Between Internet and Mail Survey Modes for Choice Experiment Surveys Considering Non-Market Goods," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 44(4), pages 591-610, December.
    18. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Reithmayer, Corrinna & Danne, Michael & Mußhoff, Oliver, 2019. "Look at that! - The effect pictures have on consumer preferences for in ovo gender determination as an alternative to culling male chicks," DARE Discussion Papers 1907, Georg-August University of Göttingen, Department of Agricultural Economics and Rural Development (DARE).
    2. Karlõševa, Aljona & Nõmmann, Sulev & Nõmmann, Tea & Urbel-Piirsalu, Evelin & Budziński, Wiktor & Czajkowski, Mikołaj & Hanley, Nick, 2016. "Marine trade-offs: Comparing the benefits of off-shore wind farms and marine protected areas," Energy Economics, Elsevier, vol. 55(C), pages 127-134.
    3. Dugstad, Anders & Grimsrud, Kristine & Kipperberg, Gorm & Lindhjem, Henrik & Navrud, Ståle, 2020. "Acceptance of wind power development and exposure – Not-in-anybody's-backyard," Energy Policy, Elsevier, vol. 147(C).
    4. Anders Dugstad & Kristine Grimsrud & Gorm Kipperberg & Henrik Lindhjem & Ståle Navrud, 2020. "Scope elasticity and economic significance in discrete choice experiments," Discussion Papers 942, Statistics Norway, Research Department.
    5. Vass, Caroline M. & Boeri, Marco & Poulos, Christine & Turner, Alex J., 2022. "Matching and weighting in stated preferences for health care," Journal of choice modelling, Elsevier, vol. 44(C).
    6. Anders Dugstad & Kristine Grimsrud & Gorm Kipperberg & Henrik Lindhjem & Ståle Navrud, 2020. "Acceptance of national wind power development and exposure. A case-control choice experiment approach," Discussion Papers 933, Statistics Norway, Research Department.
    7. Larisa Boboc & Daniel Moise, 2016. "Consumer Perspective upon the Marketing of Online Surveys," International Conference on Marketing and Business Development Journal, The Bucharest University of Economic Studies, vol. 2(1), pages 150-156, July.
    8. Anders Dugstad & Kristine M. Grimsrud & Gorm Kipperberg & Henrik Lindhjem & Ståle Navrud, 2021. "Scope Elasticity of Willingness to pay in Discrete Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 80(1), pages 21-57, September.
    9. Uttam Khanal & Clevo Wilson & Shunsuke Managi & Boon Lee & Viet-Ngu Hoang & Robert Gifford, 2018. "Psychological influence on survey incentives: valuing climate change adaptation benefits in agriculture," Environmental Economics and Policy Studies, Springer;Society for Environmental Economics and Policy Studies - SEEPS, vol. 20(2), pages 305-324, April.
    10. Askill H. Halse & Stefan Flügel & Marco Kouwenhoven & Gerard Jong & Hanne B. Sundfør & Nina Hulleberg & Guri N. Jordbakke & Henrik Lindhjem, 2023. "A minute of your time: The impact of survey recruitment method and interview location on the value of travel time," Transportation, Springer, vol. 50(5), pages 1553-1584, October.
    11. Skeie, Magnus Aa. & Lindhjem, Henrik & Skjeflo, Sofie & Navrud, Ståle, 2019. "Smartphone and tablet effects in contingent valuation web surveys – No reason to worry?," Ecological Economics, Elsevier, vol. 165(C), pages 1-1.
    12. Tomas Badura & Silvia Ferrini & Michael Burton & Amy Binner & Ian J. Bateman, 2020. "Using Individualised Choice Maps to Capture the Spatial Dimensions of Value Within Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 75(2), pages 297-322, February.
    13. Glenk, Klaus & Meyerhoff, Jürgen & Colombo, Sergio & Faccioli, Michela, 2024. "Enhancing the face validity of choice experiments: A simple diagnostic check," Ecological Economics, Elsevier, vol. 221(C).
    14. Erlend Dancke Sandorf & Kristine Grimsrud & Henrik Lindhjem, 2022. "Ponderous, Proficient or Professional? Survey Experience and Smartphone Effects in Stated Preference Research," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 81(4), pages 807-832, April.
    15. William F. Vásquez & Jennifer M. Trudeau & Jessica Alicea‐Planas, 2021. "Immediate and informative feedback during a pandemic: Using stated preference analysis to predict vaccine uptake rates," Health Economics, John Wiley & Sons, Ltd., vol. 30(12), pages 3123-3137, December.
    16. Liebe, Ulf & Glenk, Klaus & von Meyer-Höfer, Marie & Spiller, Achim, 2019. "A web survey application of real choice experiments," Journal of choice modelling, Elsevier, vol. 33(C).
    17. Kaitlynn Sandstrom‐Mistry & Frank Lupi & Hyunjung Kim & Joseph A. Herriges, 2023. "Comparing water quality valuation across probability and non‐probability samples," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 45(2), pages 744-761, June.
    18. Vondolia, Godwin K. & Navrud, Ståle, 2019. "Are non-monetary payment modes more uncertain for stated preference elicitation in developing countries?," Journal of choice modelling, Elsevier, vol. 30(C), pages 73-87.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhifeng Gao & Lisa A. House & Jing Xie, 2016. "Online Survey Data Quality and Its Implication for Willingness-to-Pay: A Cross-Country Comparison," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 64(2), pages 199-221, June.
    2. Penn, Jerrod & Hu, Wuyang, 2016. "Making the Most of Cheap Talk in an Online Survey," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236171, Agricultural and Applied Economics Association.
    3. Jed J. Cohen & Johannes Reichl, 2022. "Comparing Internet and phone survey mode effects across countries and research contexts," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 66(1), pages 44-71, January.
    4. Gillespie Rob & Kragt Marit E., 2012. "Accounting for Nonmarket Impacts in a Benefit-Cost Analysis of Underground Coal Mining in New South Wales, Australia," Journal of Benefit-Cost Analysis, De Gruyter, vol. 3(2), pages 1-29, May.
    5. Matthews, Yvonne & Scarpa, Riccardo & Marsh, Dan, 2017. "Using virtual environments to improve the realism of choice experiments: A case study about coastal erosion management," Journal of Environmental Economics and Management, Elsevier, vol. 81(C), pages 193-208.
    6. Sandorf, Erlend Dancke & Aanesen, Margrethe & Navrud, Ståle, 2016. "Valuing unfamiliar and complex environmental goods: A comparison of valuation workshops and internet panel surveys with videos," Ecological Economics, Elsevier, vol. 129(C), pages 50-61.
    7. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    8. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    9. Tobias Börger, 2016. "Are Fast Responses More Random? Testing the Effect of Response Time on Scale in an Online Choice Experiment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 65(2), pages 389-413, October.
    10. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    11. West, Grant H. & Snell, Heather & Kovacs, Kent & Nayga, Rodolfo M., 2020. "Estimation of the preferences for the intertemporal services from groundwater," 2020 Annual Meeting, July 26-28, Kansas City, Missouri 304220, Agricultural and Applied Economics Association.
    12. Liebe, Ulf & Glenk, Klaus & von Meyer-Höfer, Marie & Spiller, Achim, 2019. "A web survey application of real choice experiments," Journal of choice modelling, Elsevier, vol. 33(C).
    13. Morten Mørkbak & Tove Christensen & Dorte Gyrd-Hansen, 2010. "Choke Price Bias in Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 45(4), pages 537-551, April.
    14. Ding, Ye & Nayga Jr, Rodolfo M. & Zeng, Yinchu & Yang, Wei & Arielle Snell, Heather, 2022. "Consumers’ valuation of a live video feed in restaurant kitchens for online food delivery service," Food Policy, Elsevier, vol. 112(C).
    15. Guimarães, Maria Helena & Nunes, Luís Catela & Madureira, Lívia & Santos, José Lima & Boski, Tomasz & Dentinho, Tomaz, 2015. "Measuring birdwatchers preferences: A case for using online networks and mixed-mode surveys," Tourism Management, Elsevier, vol. 46(C), pages 102-113.
    16. Punel, Aymeric & Stathopoulos, Amanda, 2017. "Modeling the acceptability of crowdsourced goods deliveries: Role of context and experience effects," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 105(C), pages 18-38.
    17. Reithmayer, Corrina & Danne, Michael & Mußhoff, Oliver, 2019. "Look at that! – The effect pictures have on consumer preferences for in ovo gender determination as an alternative to culling male chicks," Department of Agricultural and Rural Development (DARE) Discussion Papers 298419, Georg-August-Universitaet Goettingen, Department of Agricultural Economics and Rural Development (DARE).
    18. Czajkowski, Mikołaj & Zawojska, Ewa & Meade, Norman & da Motta, Ronaldo Seroa & Welsh, Mike & Ortiz, Ramon Arigoni, 2024. "On the inference about a willingness-to-pay distribution using contingent valuation data," Ecological Economics, Elsevier, vol. 222(C).
    19. Börger, Tobias & Hattam, Caroline & Burdon, Daryl & Atkins, Jonathan P. & Austen, Melanie C., 2014. "Valuing conservation benefits of an offshore marine protected area," Ecological Economics, Elsevier, vol. 108(C), pages 229-241.
    20. Mohammed Hussen Alemu & Søren Bøye Olsen, 2020. "An analysis of the impacts of tasting experience and peer effects on consumers’ willingness to pay for novel foods," Agribusiness, John Wiley & Sons, Ltd., vol. 36(4), pages 653-674, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:eejocm:v:14:y:2015:i:c:p:17-31. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/journal-of-choice-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.