IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0245460.html
   My bibliography  Save this article

Pay for performance, satisfaction and retention in longitudinal crowdsourced research

Author

Listed:
  • Elena M Auer
  • Tara S Behrend
  • Andrew B Collmus
  • Richard N Landers
  • Ahleah F Miles

Abstract

In the social and cognitive sciences, crowdsourcing provides up to half of all research participants. Despite this popularity, researchers typically do not conceptualize participants accurately, as gig-economy worker-participants. Applying theories of employee motivation and the psychological contract between employees and employers, we hypothesized that pay and pay raises would drive worker-participant satisfaction, performance, and retention in a longitudinal study. In an experiment hiring 359 Amazon Mechanical Turk Workers, we found that initial pay, relative increase of pay over time, and overall pay did not have substantial influence on subsequent performance. However, pay significantly predicted participants' perceived choice, justice perceptions, and attrition. Given this, we conclude that worker-participants are particularly vulnerable to exploitation, having relatively low power to negotiate pay. Results of this study suggest that researchers wishing to crowdsource research participants using MTurk might not face practical dangers such as decreased performance as a result of lower pay, but they must recognize an ethical obligation to treat Workers fairly.

Suggested Citation

  • Elena M Auer & Tara S Behrend & Andrew B Collmus & Richard N Landers & Ahleah F Miles, 2021. "Pay for performance, satisfaction and retention in longitudinal crowdsourced research," PLOS ONE, Public Library of Science, vol. 16(1), pages 1-17, January.
    • Handle: RePEc:plo:pone00:0245460
    DOI: 10.1371/journal.pone.0245460
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0245460
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0245460&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0245460?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Johnston, Hannah & Land-Kazlauskas, Chris., 2018. "Organizing on-demand representation, voice, and collective bargaining in the gig economy," ILO Working Papers 994981993502676, International Labour Organization.
    2. Neil Stewart & Jesse Chandler & Gabriele Paolacci, "undated". "Crowdsourcing Samples in Cognitive Science," Mathematica Policy Research Reports c63e922cf7554604a919e1f18, Mathematica Policy Research.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andersson, Henrik & Scholtz, Henrik & Zheng, Jiakun, 2023. "Measuring regret theory in the health and financial domain," TSE Working Papers 23-1449, Toulouse School of Economics (TSE).
    2. Lars Hornuf & Daniel Vrankar, 2022. "Hourly Wages in Crowdworking: A Meta-Analysis," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 64(5), pages 553-573, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alex Veen & Tom Barratt & Caleb Goods, 2020. "Platform-Capital’s ‘App-etite’ for Control: A Labour Process Analysis of Food-Delivery Work in Australia," Work, Employment & Society, British Sociological Association, vol. 34(3), pages 388-406, June.
    2. Behl, Abhishek & Jayawardena, Nirma & Ishizaka, Alessio & Gupta, Manish & Shankar, Amit, 2022. "Gamification and gigification: A multidimensional theoretical approach," Journal of Business Research, Elsevier, vol. 139(C), pages 1378-1393.
    3. Niki Panteli & Andriana Rapti & Dora Scholarios, 2020. "‘If He Just Knew Who We Were’: Microworkers’ Emerging Bonds of Attachment in a Fragmented Employment Relationship," Work, Employment & Society, British Sociological Association, vol. 34(3), pages 476-494, June.
    4. Gernot Grabher & Erwin van Tuijl, 2020. "Uber-production: From global networks to digital platforms," Environment and Planning A, , vol. 52(5), pages 1005-1016, August.
    5. Ciotti, Fabrizio & Hornuf, Lars & Stenzhorn, Eliza, 2021. "Lock-In Effects in Online Labor Markets," LIDAM Discussion Papers CORE 2021014, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    6. repec:cup:judgdm:v:14:y:2019:i:2:p:170-178 is not listed on IDEAS
    7. Evgeny Nikulchev & Dmitry Ilin & Pavel Kolyasnikov & Shamil Magomedov & Anna Alexeenko & Alexander N. Kosenkov & Andrey Sokolov & Artem Malykh & Victoria Ismatullina & Sergey Malykh, 2021. "Isolated Sandbox Environment Architecture for Running Cognitive Psychological Experiments in Web Platforms," Future Internet, MDPI, vol. 13(10), pages 1-17, September.
    8. Azzellini, Dario & Greer, Ian & Umney, Charles, 2019. "Limits of the platform economy: Digitalization and marketization in live music," Working Paper Forschungsförderung 154, Hans-Böckler-Stiftung, Düsseldorf.
    9. -, 2021. "Employment Situation in Latin America and the Caribbean: Decent work for platform workers in Latin America," Coediciones, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL), number 46956 edited by Eclac.
    10. Marie Nilsen & Trond Kongsvik & Stian Antonsen, 2022. "Taming Proteus: Challenges for Risk Regulation of Powerful Digital Labor Platforms," IJERPH, MDPI, vol. 19(10), pages 1-23, May.
    11. Matthieu Raoelison & Wim De Neys, 2019. "Do we de-bias ourselves?: The impact of repeated presentation on the bat-and-ball problem," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 14(2), pages 170-178, March.
    12. Shuili Du & Mayowa T. Babalola & Premilla D’Cruz & Edina Dóci & Lucia Garcia-Lorenzo & Louise Hassan & Gazi Islam & Alexander Newman & Ernesto Noronha & Suzanne Gils, 2024. "The Ethical, Societal, and Global Implications of Crowdsourcing Research," Journal of Business Ethics, Springer, vol. 193(1), pages 1-16, August.
    13. Carlo Vercellone & Francesco Brancaccio & Alfonso Giuliani & Federico Puletti & Giulia Rocchi & Pierluigi Vattimo, 2018. "Data-driven disruptive commons-based models," Working Papers halshs-01952141, HAL.
    14. Carlo Vercellone & Francesco Brancaccio & Alfonso Giuliani & Federico Puletti & Giulia Rocchi & Pierluigi Vattimo, 2018. "Data-driven disruptive commons-based models," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01952141, HAL.
    15. Berde, Éva, 2018. "Uber és taxi egymás mellett - új piaci modellek hagyományos árdiszkriminációval [Uber and taxi firms side by side. The Ublyft" business model with traditional price discrimination]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(6), pages 650-666.
    16. Jiachun Lu & Lutz Kaufmann & Craig R. Carter, 2021. "How Informal Exchanges Impact Formal Sourcing Collaboration (and What Supply Managers Can Do about It)," Journal of Supply Chain Management, Institute for Supply Management, vol. 57(4), pages 26-62, October.
    17. Schweinsberg, Martin & Feldman, Michael & Staub, Nicola & van den Akker, Olmo R. & van Aert, Robbie C.M. & van Assen, Marcel A.L.M. & Liu, Yang & Althoff, Tim & Heer, Jeffrey & Kale, Alex & Mohamed, Z, 2021. "Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 165(C), pages 228-249.
    18. Serpil ÇİĞDEM, 2019. "Endüstri 4.0 ve Dijital Emek Platformlarının İnsana Yakışır İş Bağlamında Değerlendirilmesi," Journal of Social Policy Conferences, Istanbul University, Faculty of Economics, vol. 0(77), pages 157-199, December.
    19. Keller, Berndt, 2020. "Interest representation and industrial relations in the age of digitalization ‒ an outline [Interessenvertretung und Arbeitsbeziehungen im Zeitalter der Digitalisierung - ein Überblick]," Industrielle Beziehungen. Zeitschrift für Arbeit, Organisation und Management, Verlag Barbara Budrich, vol. 27(3), pages 255-285.
    20. Schneider, Nathan, 2022. "Governable Spaces: A Feminist Architecture for Platform Policy," MediArXiv 9d6et, Center for Open Science.
    21. Julián Vicens & Josep Perelló & Jordi Duch, 2018. "Citizen Social Lab: A digital platform for human behavior experimentation within a citizen science framework," PLOS ONE, Public Library of Science, vol. 13(12), pages 1-20, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0245460. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.