IDEAS home Printed from https://ideas.repec.org/a/spr/jbecon/v94y2024i9d10.1007_s11573-024-01196-6.html
   My bibliography  Save this article

How has the COVID-19 pandemic shaped behavior in crowdsourcing? The role of online labor market training

Author

Listed:
  • Evangelos Mourelatos

    (University of Oulu)

  • Jaakko Simonen

    (University of Oulu)

  • Simo Hosio

    (University of Oulu)

  • Daniil Likhobaba

    (Crowdsourcing Research Group, Toloka)

  • Dmitry Ustalov

    (Crowdsourcing Research Group, Toloka)

Abstract

The outbreak of the COVID-19 pandemic and the associated lockdown measures have been a shock to market systems worldwide, affecting both the supply and demand of labor. Intensified by this pandemic-driven recession, online labor markets are in many ways at the core of the economic and policy debates about their technological innovation, which could be used as a way of economic reform and recovery. In this work, we focus on crowdsourcing, which is a specific type of online labor. We apply a unique dataset of labor data to investigate the effects of online training, a policy that was provided to requesters by the platform during the COVID-19 period. Our findings suggest that workers indirectly finance on-the-job online training by accepting lower wages during the pandemic. By utilizing a difference-in-difference research design, we also provide causal evidence that online training results in lower job completion time and the probability of being discontinued. Our findings show that both employers and employees in our online labor context reacted to the pandemic by participating in online labor procedures with different risk strategies and labor approaches. Our findings provide key insights for several groups of crowdsourcing stakeholders, including policy-makers, platform owners, hiring managers, and workers. Managerial and practical implications in relation to how online labor markets react to external shocks are discussed.

Suggested Citation

  • Evangelos Mourelatos & Jaakko Simonen & Simo Hosio & Daniil Likhobaba & Dmitry Ustalov, 2024. "How has the COVID-19 pandemic shaped behavior in crowdsourcing? The role of online labor market training," Journal of Business Economics, Springer, vol. 94(9), pages 1201-1244, November.
    • Handle: RePEc:spr:jbecon:v:94:y:2024:i:9:d:10.1007_s11573-024-01196-6
    DOI: 10.1007/s11573-024-01196-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11573-024-01196-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11573-024-01196-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Moshe A. Barach & John J. Horton, 2021. "How Do Employers Use Compensation History? Evidence from a Field Experiment," Journal of Labor Economics, University of Chicago Press, vol. 39(1), pages 193-218.
    2. Hayo, Bernd & Seifert, Wolfgang, 2003. "Subjective economic well-being in Eastern Europe," Journal of Economic Psychology, Elsevier, vol. 24(3), pages 329-348, June.
    3. Mourelatos, Evangelos, 2023. "Does Mood affect Sexual and Gender Discrimination in Hiring Choices? Evidence from Online Experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    4. David Fairris & Roberto Pedace, 2004. "The Impact of Minimum Wages on Job Training: An Empirical Exploration with Establishment Data," Southern Economic Journal, John Wiley & Sons, vol. 70(3), pages 566-583, January.
    5. Christopher T. Stanton & Catherine Thomas, 2016. "Landing the First Job: The Value of Intermediaries in Online Hiring," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 83(2), pages 810-854.
    6. Ajay Agrawal & John Horton & Nicola Lacetera & Elizabeth Lyons, 2015. "Digitization and the Contract Labor Market: A Research Agenda," NBER Chapters, in: Economic Analysis of the Digital Economy, pages 219-250, National Bureau of Economic Research, Inc.
    7. John J. Horton, 2019. "Buyer Uncertainty About Seller Capacity: Causes, Consequences, and a Partial Solution," Management Science, INFORMS, vol. 65(8), pages 3518-3540, August.
    8. Adams-Prassl, Abi & Boneva, Teodora & Golin, Marta & Rauh, Christopher, 2020. "Inequality in the impact of the coronavirus shock: Evidence from real time surveys," Journal of Public Economics, Elsevier, vol. 189(C).
    9. Hensvik, Lena & Le Barbanchon, Thomas & Rathelot, Roland, 2021. "Job search during the COVID-19 crisis," Journal of Public Economics, Elsevier, vol. 194(C).
    10. Kikuchi, Shinnosuke & Kitao, Sagiri & Mikoshiba, Minamo, 2021. "Who suffers from the COVID-19 shocks? Labor market heterogeneity and welfare consequences in Japan," Journal of the Japanese and International Economies, Elsevier, vol. 59(C).
    11. Altig, Dave & Baker, Scott & Barrero, Jose Maria & Bloom, Nicholas & Bunn, Philip & Chen, Scarlet & Davis, Steven J. & Leather, Julia & Meyer, Brent & Mihaylov, Emil & Mizen, Paul & Parker, Nicholas &, 2020. "Economic uncertainty before and during the COVID-19 pandemic," Journal of Public Economics, Elsevier, vol. 191(C).
    12. Adams-Prassl, A. & Boneva, T. & Golin, M & Rauh, C., 2020. "Inequality in the Impact of the Coronavirus Shock: New Survey Evidence for the UK," Cambridge Working Papers in Economics 2023, Faculty of Economics, University of Cambridge.
    13. David Johnson & John Barry Ryan, 2020. "Amazon Mechanical Turk workers can provide consistent and economically meaningful data," Southern Economic Journal, John Wiley & Sons, vol. 87(1), pages 369-385, July.
    14. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    15. Alberto Bayo-Moriones & Pedro Ortín-à ngel, 2006. "Internal Promotion versus External Recruitment in Industrial Plants in Spain," ILR Review, Cornell University, ILR School, vol. 59(3), pages 451-470, April.
    16. Michele Cantarella & Chiara Strozzi, 2021. "Workers in the crowd: the labor market impact of the online platform economy [An evaluation of instrumental variable strategies for estimating the effects of catholic schooling]," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 30(6), pages 1429-1458.
    17. David Fairris & Roberto Pedace, 2004. "The Impact of Minimum Wages on Job Training: An Empirical Exploration with Establishment Data," Southern Economic Journal, John Wiley & Sons, vol. 70(3), pages 566-583, January.
    18. Liliana Cori & Fabrizio Bianchi & Ennio Cadum & Carmen Anthonj, 2020. "Risk Perception and COVID-19," IJERPH, MDPI, vol. 17(9), pages 1-6, April.
    19. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    20. Guido Matias Cortes & Eliza Forsythe, 2023. "Heterogeneous Labor Market Impacts of the COVID-19 Pandemic," ILR Review, Cornell University, ILR School, vol. 76(1), pages 30-55, January.
    21. David H. Autor, 2001. "Wiring the Labor Market," Journal of Economic Perspectives, American Economic Association, vol. 15(1), pages 25-40, Winter.
    22. Forsythe, Eliza & Kahn, Lisa B. & Lange, Fabian & Wiczer, David, 2020. "Labor demand in the time of COVID-19: Evidence from vacancy postings and UI claims," Journal of Public Economics, Elsevier, vol. 189(C).
    23. Mourelatos, Evaggelos & Giannakopoulos, Nicholas & Tzagarakis, Manolis, 2020. "Productivity outcomes in online labor markets and within-task complexity and difficultly," GLO Discussion Paper Series 739, Global Labor Organization (GLO).
    24. Daniel Fernández & Daphna Harel & Panos Ipeirotis & Tara McAllister, 2019. "Statistical considerations for crowdsourced perceptual ratings of human speech productions," Journal of Applied Statistics, Taylor & Francis Journals, vol. 46(8), pages 1364-1384, June.
    25. Lawrence F. Katz & Alan B. Krueger, 2019. "The Rise and Nature of Alternative Work Arrangements in the United States, 1995–2015," ILR Review, Cornell University, ILR School, vol. 72(2), pages 382-416, March.
    26. Arindrajit Dube & Jeff Jacobs & Suresh Naidu & Siddharth Suri, 2020. "Monopsony in Online Labor Markets," American Economic Review: Insights, American Economic Association, vol. 2(1), pages 33-46, March.
    27. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    28. Marios Kokkodis & Panagiotis G. Ipeirotis, 2016. "Reputation Transferability in Online Labor Markets," Management Science, INFORMS, vol. 62(6), pages 1687-1706, June.
    29. Patricia Heuser & Peter Letmathe & Matthias Schinner, 2022. "Workforce planning in production with flexible or budgeted employee training and volatile demand," Journal of Business Economics, Springer, vol. 92(7), pages 1093-1124, September.
    30. John J. Horton, 2017. "The Effects of Algorithmic Labor Market Recommendations: Evidence from a Field Experiment," Journal of Labor Economics, University of Chicago Press, vol. 35(2), pages 345-385.
    31. Jason Chan & Jing Wang, 2018. "Hiring Preferences in Online Labor Markets: Evidence of a Female Hiring Bias," Management Science, INFORMS, vol. 64(7), pages 2973-2994, July.
    32. Chapkovski, Philipp, 2023. "Conducting interactive experiments on Toloka," Journal of Behavioral and Experimental Finance, Elsevier, vol. 37(C).
    33. Paul Calanter & Daniela Zisu, 2022. "EU Policies to Combat the Energy Crisis," Global Economic Observer, "Nicolae Titulescu" University of Bucharest, Faculty of Economic Sciences;Institute for World Economy of the Romanian Academy, vol. 10(1), pages 26-33, June.
    34. Stephany, Fabian & Dunn, Michael & Sawyer, Steven & Lehdonvirta, Vili, 2020. "Distancing Bonus or Downscaling Loss? The Changing Livelihood of US Online Workers in Times of COVID-19," SocArXiv vmg34, Center for Open Science.
    35. Saleh Al-Omoush, Khaled & Orero-Blat, Maria & Ribeiro-Soriano, Domingo, 2021. "The role of sense of community in harnessing the wisdom of crowds and creating collaborative knowledge during the COVID-19 pandemic," Journal of Business Research, Elsevier, vol. 132(C), pages 765-774.
    36. Stephanie Duchek, 2020. "Organizational resilience: a capability-based conceptualization," Business Research, Springer;German Academic Association for Business Research, vol. 13(1), pages 215-246, April.
    37. Amanda Pallais, 2014. "Inefficient Hiring in Entry-Level Labor Markets," American Economic Review, American Economic Association, vol. 104(11), pages 3565-3599, November.
    38. Gurca, Andrei & Bagherzadeh, Mehdi & Velayati, Rezvan, 2023. "Aligning the crowdsourcing type with the problem attributes to improve solution search efficacy," Technovation, Elsevier, vol. 119(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mueller-Langer, Frank & Gómez-Herrera, Estrella, 2022. "Mobility restrictions and the substitution between on-site and remote work: Empirical evidence from a European online labour market," Information Economics and Policy, Elsevier, vol. 58(C).
    2. Kazakova, E. & Sandomirskaia, M. & Suvorov, A. & Khazhgerieva, A. & Shavshin, R., 2023. "Platforms, online labor markets, and crowdsourcing. Part 1. Traditional online labor market," Journal of the New Economic Association, New Economic Association, vol. 60(3), pages 120-148.
    3. Alejandra Bellatin & Gabriela Galassi, 2022. "What COVID-19 May Leave Behind: Technology-Related Job Postings in Canada," Staff Working Papers 22-17, Bank of Canada.
    4. Christopher T. Stanton & Catherine Thomas, 2020. "The Gig Economy Beyond Local Services and Transportation," CESifo Forum, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 21(03), pages 21-26, September.
    5. Wu, Jianxin & Zhan, Xiaoling & Xu, Hui & Ma, Chunbo, 2023. "The economic impacts of COVID-19 and city lockdown: Early evidence from China," Structural Change and Economic Dynamics, Elsevier, vol. 65(C), pages 151-165.
    6. Liang, Xiao & Rozelle, Scott & Yi, Hongmei, 2022. "The impact of COVID-19 on employment and income of vocational graduates in China: Evidence from surveys in January and July 2020," China Economic Review, Elsevier, vol. 75(C).
    7. Lea Immel & Florian Neumeier & Andreas Peichl, 2022. "The Unequal Consequences of the Covid‐19 Pandemic: Evidence from a Large Representative German Population Survey," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 68(2), pages 471-496, June.
    8. Seifried, Mareike, 2021. "Transitions from offline to online labor markets: The relationship between freelancers' prior offline and online work experience," ZEW Discussion Papers 21-101, ZEW - Leibniz Centre for European Economic Research.
    9. Cristiano Codagnone & Fabienne Abadie & Federico Biagi, 2016. "The Future of Work in the ‘Sharing Economy’. Market Efficiency and Equitable Opportunities or Unfair Precarisation?," JRC Research Reports JRC101280, Joint Research Centre.
    10. Bas Scheer & Wiljan van den Berge & Maarten Goos & Alan Manning & Anna Salomons, 2022. "Alternative Work Arrangements and Worker Outcomes: Evidence from Payrolling," CPB Discussion Paper 435, CPB Netherlands Bureau for Economic Policy Analysis.
    11. Blanas, Sotiris & Oikonomou, Rigas, 2023. "COVID-induced economic uncertainty, tasks and occupational demand," Labour Economics, Elsevier, vol. 81(C).
    12. Hensvik, Lena & Le Barbanchon, Thomas & Rathelot, Roland, 2021. "Job search during the COVID-19 crisis," Journal of Public Economics, Elsevier, vol. 194(C).
    13. Stefan Pichler & Katherine Wen & Nicolas R. Ziebarth, 2021. "Positive Health Externalities of Mandating Paid Sick Leave," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(3), pages 715-743, June.
    14. Alan Benson & Aaron Sojourner & Akhmed Umyarov, 2020. "Can Reputation Discipline the Gig Economy? Experimental Evidence from an Online Labor Market," Management Science, INFORMS, vol. 66(5), pages 1802-1825, May.
    15. Mourelatos, Evangelos, 2023. "Does Mood affect Sexual and Gender Discrimination in Hiring Choices? Evidence from Online Experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    16. Taiyo Fukai & Hidehiko Ichimura & Keisuke Kawata, 2021. "Describing the impacts of COVID-19 on the labor market in Japan until June 2020," The Japanese Economic Review, Springer, vol. 72(3), pages 439-470, July.
    17. Laurel Wheeler & Robert Garlick & Eric Johnson & Patrick Shaw & Marissa Gargano, 2022. "LinkedIn(to) Job Opportunities: Experimental Evidence from Job Readiness Training," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 101-125, April.
    18. Mourelatos, Evangelos, 2021. "Personality and Ethics on Online Labor Markets: How mood influences ethical perceptions," EconStor Preprints 244735, ZBW - Leibniz Information Centre for Economics.
    19. Jason Chan & Jing Wang, 2018. "Hiring Preferences in Online Labor Markets: Evidence of a Female Hiring Bias," Management Science, INFORMS, vol. 64(7), pages 2973-2994, July.
    20. Pajarinen, Mika & Rouvinen, Petri & Claussen, Jörg & Hakanen, Jari & Kovalainen, Anne & Kretschmer, Tobias & Poutanen, Seppo & Seifried, Mareike & Seppänen, Laura, 2018. "Upworkers in Finland: Survey Results," ETLA Reports 85, The Research Institute of the Finnish Economy.

    More about this item

    Keywords

    Online labor markets; Crowdsourcing; COVID-19 pandemic; Task completion; Online training;
    All these keywords.

    JEL classification:

    • D40 - Microeconomics - - Market Structure, Pricing, and Design - - - General
    • J20 - Labor and Demographic Economics - - Demand and Supply of Labor - - - General
    • J40 - Labor and Demographic Economics - - Particular Labor Markets - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jbecon:v:94:y:2024:i:9:d:10.1007_s11573-024-01196-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.