IDEAS home Printed from https://ideas.repec.org/a/kap/transp/v47y2020i1d10.1007_s11116-018-9962-8.html
   My bibliography  Save this article

A new twist on the gig economy: conducting surveys on Amazon Mechanical Turk

Author

Listed:
  • Laurie A. Garrow

    (Georgia Institute of Technology)

  • Ziran Chen

    (Georgia Institute of Technology
    Amazon.com Inc.)

  • Mohammad Ilbeigi

    (Bowling State University)

  • Virginie Lurkin

    (TU/e, Eindhoven University of Technology)

Abstract

There is growing interest in using online outsourcing platforms that are part of the “gig economy” to conduct surveys for academic research. This interest has been driven in part by the belief that compared to traditional survey data collection methods, internet-based marketplaces such as Amazon Mechanical Turk (MTurk) enable one to collect survey data cheaper and faster from a larger, more diverse participant pool. However, many have questioned whether models based on survey data from these online marketplaces are similar to models based on survey data from more traditional platforms. To investigate this research question, we used MTurk and Qualtrics (a traditional market research firm) to survey air travelers. Our results showed that MTurk and Qualtrics respondents had distinct socio-demographic characteristics, but we found no statistical evidence for different air trip characteristics. In our data, proportionately more MTurk respondents were in the younger, single, male, and lower-income categories than for Qualtrics respondents. We found that airline itinerary choice models estimated from the MTurk and Qualtrics survey data were similar, with the key difference related to price sensitivities. Although our results provide evidence that MTurk can be used for travel demand modeling applications, we offer words of caution for others planning to conduct surveys in online marketplaces, particularly for those seeking to recruit more than 1000 participants or for those targeting specific geographic areas.

Suggested Citation

  • Laurie A. Garrow & Ziran Chen & Mohammad Ilbeigi & Virginie Lurkin, 2020. "A new twist on the gig economy: conducting surveys on Amazon Mechanical Turk," Transportation, Springer, vol. 47(1), pages 23-42, February.
    • Handle: RePEc:kap:transp:v:47:y:2020:i:1:d:10.1007_s11116-018-9962-8
    DOI: 10.1007/s11116-018-9962-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11116-018-9962-8
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11116-018-9962-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Landers, Richard N. & Behrend, Tara S., 2015. "An Inconvenient Truth: Arbitrary Distinctions Between Organizational, Mechanical Turk, and Other Convenience Samples," Industrial and Organizational Psychology, Cambridge University Press, vol. 8(2), pages 142-164, June.
    2. Highhouse, Scott & Zhang, Don, 2015. "The New Fruit Fly for Applied Psychological Research," Industrial and Organizational Psychology, Cambridge University Press, vol. 8(2), pages 179-183, June.
    3. Lurkin, Virginie & Garrow, Laurie A. & Higgins, Matthew J. & Newman, Jeffrey P. & Schyns, Michael, 2017. "Accounting for price endogeneity in airline itinerary choice models: An application to Continental U.S. markets," Transportation Research Part A: Policy and Practice, Elsevier, vol. 100(C), pages 228-246.
    4. Schmidt, Gordon B., 2015. "Fifty Days an MTurk Worker: The Social and Motivational Context for Amazon Mechanical Turk Workers," Industrial and Organizational Psychology, Cambridge University Press, vol. 8(2), pages 165-171, June.
    5. Helveston, John Paul & Liu, Yimin & Feit, Elea McDonnell & Fuchs, Erica & Klampfl, Erica & Michalek, Jeremy J., 2015. "Will subsidies drive electric vehicle adoption? Measuring consumer preferences in the U.S. and China," Transportation Research Part A: Policy and Practice, Elsevier, vol. 73(C), pages 96-112.
    6. Krupa, Joseph S. & Rizzo, Donna M. & Eppstein, Margaret J. & Brad Lanute, D. & Gaalema, Diann E. & Lakkaraju, Kiran & Warrender, Christina E., 2014. "Analysis of a consumer survey on plug-in hybrid electric vehicles," Transportation Research Part A: Policy and Practice, Elsevier, vol. 64(C), pages 14-31.
    7. Louviere,Jordan J. & Hensher,David A. & Swait,Joffre D. With contributions by-Name:Adamowicz,Wiktor, 2000. "Stated Choice Methods," Cambridge Books, Cambridge University Press, number 9780521788304.
    8. Smith, Nicholas A. & Sabat, Isaac E. & Martinez, Larry R. & Weaver, Kayla & Xu, Shi, 2015. "A Convenient Solution: Using MTurk To Sample From Hard-To-Reach Populations," Industrial and Organizational Psychology, Cambridge University Press, vol. 8(2), pages 220-228, June.
    9. Berinsky, Adam J. & Huber, Gregory A. & Lenz, Gabriel S., 2012. "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk," Political Analysis, Cambridge University Press, vol. 20(3), pages 351-368, July.
    10. Cantor, David E. & Blackhurst, Jennifer V. & Cortes, Juan David, 2014. "The clock is ticking: The role of uncertainty, regulatory focus, and level of risk on supply chain disruption decision making behavior," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 72(C), pages 159-172.
    11. Mariana L. Nicolae & Mark E. Ferguson & Laurie A. Garrow, 2016. "Measuring the Benefit of Offering Auxiliary Services: Do Bag-Checkers Differ in Their Sensitivities to Airline Itinerary Attributes?," Production and Operations Management, Production and Operations Management Society, vol. 25(10), pages 1689-1708, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Reema Bera & Bhargab Maitra, 2021. "Analyzing Prospective Owners’ Choice Decision towards Plug-in Hybrid Electric Vehicles in Urban India: A Stated Preference Discrete Choice Experiment," Sustainability, MDPI, vol. 13(14), pages 1-24, July.
    2. Yongyou Nie & Enci Wang & Qinxin Guo & Junyi Shen, 2018. "Examining Shanghai Consumer Preferences for Electric Vehicles and Their Attributes," Sustainability, MDPI, vol. 10(6), pages 1-16, June.
    3. Abotalebi, Elnaz & Scott, Darren M. & Ferguson, Mark R., 2019. "Why is electric vehicle uptake low in Atlantic Canada? A comparison to leading adoption provinces," Journal of Transport Geography, Elsevier, vol. 74(C), pages 289-298.
    4. Chiambaretto, Paul, 2021. "Air passengers’ willingness to pay for ancillary services on long-haul flights," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 147(C).
    5. Deena A. Isom & Hunter M. Boehme & Toniqua C. Mikell & Stephen Chicoine & Marion Renner, 2021. "Status Threat, Social Concerns, and Conservative Media: A Look at White America and the Alt-Right," Societies, MDPI, vol. 11(3), pages 1-20, July.
    6. Gökçe Esenduran & James A. Hill & In Joon Noh, 2020. "Understanding the Choice of Online Resale Channel for Used Electronics," Production and Operations Management, Production and Operations Management Society, vol. 29(5), pages 1188-1211, May.
    7. Jianlong Wu & Zhongji Yang & Xiaobo Hu & Hongqi Wang & Jing Huang, 2018. "Exploring Driving Forces of Sustainable Development of China’s New Energy Vehicle Industry: An Analysis from the Perspective of an Innovation Ecosystem," Sustainability, MDPI, vol. 10(12), pages 1-24, December.
    8. Guo, Yuntao & Souders, Dustin & Labi, Samuel & Peeta, Srinivas & Benedyk, Irina & Li, Yujie, 2021. "Paving the way for autonomous Vehicles: Understanding autonomous vehicle adoption and vehicle fuel choice under user heterogeneity," Transportation Research Part A: Policy and Practice, Elsevier, vol. 154(C), pages 364-398.
    9. Aravena, C. & Denny, E., 2021. "The impact of learning and short-term experience on preferences for electric vehicles," Renewable and Sustainable Energy Reviews, Elsevier, vol. 152(C).
    10. Shanmugavel, Nagarajan & Balakrishnan, Janarthanan, 2023. "Influence of pro-environmental behaviour towards behavioural intention of electric vehicles," Technological Forecasting and Social Change, Elsevier, vol. 187(C).
    11. Huang, Youlin & Qian, Lixian & Tyfield, David & Soopramanien, Didier, 2021. "On the heterogeneity in consumer preferences for electric vehicles across generations and cities in China," Technological Forecasting and Social Change, Elsevier, vol. 167(C).
    12. Xiao, Xu & Chen, Zi-Rui & Nie, Pu-Yan, 2020. "Analysis of two subsidies for EVs: Based on an expanded theoretical discrete-choice model," Energy, Elsevier, vol. 208(C).
    13. Will Bryant & Stephanie M. Merritt, 2021. "Unethical Pro-organizational Behavior and Positive Leader–Employee Relationships," Journal of Business Ethics, Springer, vol. 168(4), pages 777-793, February.
    14. Mandys, F., 2021. "Electric vehicles and consumer choices," Renewable and Sustainable Energy Reviews, Elsevier, vol. 142(C).
    15. Elnaz Abotalebi & Mark R. Ferguson & Moataz Mohamed & Darren M. Scott, 2020. "Design of a survey to assess prospects for consumer electric mobility in Canada: a retrospective appraisal," Transportation, Springer, vol. 47(3), pages 1223-1250, June.
    16. Tanaka, Makoto & Ida, Takanori & Murakami, Kayo & Friedman, Lee, 2014. "Consumers’ willingness to pay for alternative fuel vehicles: A comparative discrete choice analysis between the US and Japan," Transportation Research Part A: Policy and Practice, Elsevier, vol. 70(C), pages 194-209.
    17. Xue, Yixi & Zhang, Yi & Wang, Zhuoli & Tian, Shuo & Xiong, Qian & Li, Lydia Qianqian, 2023. "Effects of incentive policies on the purchase intention of electric vehicles in China: Psychosocial value and family ownership," Energy Policy, Elsevier, vol. 181(C).
    18. Boden, Joe & Maier, Erik & Wilken, Robert, 2020. "The effect of credit card versus mobile payment on convenience and consumers’ willingness to pay," Journal of Retailing and Consumer Services, Elsevier, vol. 52(C).
    19. Langbroek, Joram H.M. & Franklin, Joel P. & Susilo, Yusak O., 2016. "The effect of policy incentives on electric vehicle adoption," Energy Policy, Elsevier, vol. 94(C), pages 94-103.
    20. Chenlei Xue & Huaguo Zhou & Qunqi Wu & Xueying Wu & Xingbo Xu, 2021. "Impact of Incentive Policies and Other Socio-Economic Factors on Electric Vehicle Market Share: A Panel Data Analysis from the 20 Countries," Sustainability, MDPI, vol. 13(5), pages 1-12, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:transp:v:47:y:2020:i:1:d:10.1007_s11116-018-9962-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.