IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v42y2013i1p287-294.html
   My bibliography  Save this article

Does incentive provision increase the quality of peer review? An experimental study

Author

Listed:
  • Squazzoni, Flaminio
  • Bravo, Giangiacomo
  • Takács, Károly

Abstract

Although peer review is crucial for innovation and experimental discoveries in science, it is poorly understood in scientific terms. Discovering its true dynamics and exploring adjustments which improve the commitment of everyone involved could benefit scientific development for all disciplines and consequently increase innovation in the economy and the society. We have reported the results of an innovative experiment developed to model peer review. We demonstrate that offering material rewards to referees tends to decrease the quality and efficiency of the reviewing process. Our findings help to discuss the viability of different options of incentive provision, supporting the idea that journal editors and responsible of research funding agencies should be extremely careful in offering material incentives on reviewing, since these might undermine moral motives which guide referees’ behavior.

Suggested Citation

  • Squazzoni, Flaminio & Bravo, Giangiacomo & Takács, Károly, 2013. "Does incentive provision increase the quality of peer review? An experimental study," Research Policy, Elsevier, vol. 42(1), pages 287-294.
    • Handle: RePEc:eee:respol:v:42:y:2013:i:1:p:287-294
    DOI: 10.1016/j.respol.2012.04.014
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733312001230
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2012.04.014?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ernst Fehr & Klaus M. Schmidt, 1999. "A Theory of Fairness, Competition, and Cooperation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(3), pages 817-868.
    2. Nicola Lacetera, 2009. "Different Missions and Commitment Power in R&D Organizations: Theory and Evidence on Industry-University Alliances," Organization Science, INFORMS, vol. 20(3), pages 565-582, June.
    3. Daniel S. Hamermesh, 1994. "Facts and Myths about Refereeing," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 153-163, Winter.
    4. Juin-jen Chang & Ching-chong Lai, 2001. "Is It Worthwhile to Pay Referees?," Southern Economic Journal, John Wiley & Sons, vol. 68(2), pages 457-463, October.
    5. Herbert Gintis, 2000. "Strong Reciprocity and Human Sociality," UMASS Amherst Economics Working Papers 2000-02, University of Massachusetts Amherst, Department of Economics.
    6. Kenneth Arrow, 1962. "Economic Welfare and the Allocation of Resources for Invention," NBER Chapters, in: The Rate and Direction of Inventive Activity: Economic and Social Factors, pages 609-626, National Bureau of Economic Research, Inc.
    7. Johnson, Noel D. & Mislin, Alexandra A., 2011. "Trust games: A meta-analysis," Journal of Economic Psychology, Elsevier, vol. 32(5), pages 865-889.
    8. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    9. Hendrik P. van Dalen & K?ne Henkens, 2005. "Signals in science - On the importance of signaling in gaining attention in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 209-233, August.
    10. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    11. Roach, Michael & Sauermann, Henry, 2010. "A taste for science? PhD scientists' academic orientation and self-selection into research careers in industry," Research Policy, Elsevier, vol. 39(3), pages 422-434, April.
    12. Haeussler, Carolin, 2011. "Information-sharing in academia and the industry: A comparative study," Research Policy, Elsevier, vol. 40(1), pages 105-122, February.
    13. Richard R. Nelson, 1959. "The Simple Economics of Basic Scientific Research," Journal of Political Economy, University of Chicago Press, vol. 67(3), pages 297-297.
    14. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    15. Flaminio Squazzoni & Károly Takács, 2011. "Social Simulation That 'Peers into Peer Review'," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 14(4), pages 1-3.
    16. David N. Laband, 1990. "Is There Value-Added from the Review Process in Economics?: Preliminary Evidence from Authors," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(2), pages 341-352.
    17. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    18. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    19. Almås, Ingvild & Cappelen, Alexander W. & Sørensen, Erik Ø. & Tungodden, Bertil, 2015. "Fairness and the Development of Inequality Acceptance," Discussion Paper Series in Economics 18/2015, Norwegian School of Economics, Department of Economics.
    20. Scott Stern, 2004. "Do Scientists Pay to Be Scientists?," Management Science, INFORMS, vol. 50(6), pages 835-853, June.
    21. Partha, Dasgupta & David, Paul A., 1994. "Toward a new economics of science," Research Policy, Elsevier, vol. 23(5), pages 487-521, September.
    22. Bruno S. Frey & Reto Jegen, 2001. "Motivation Crowding Theory," Journal of Economic Surveys, Wiley Blackwell, vol. 15(5), pages 589-611, December.
    23. Greiner, Ben, 2004. "An Online Recruitment System for Economic Experiments," MPRA Paper 13513, University Library of Munich, Germany.
    24. Andreas Ortmann & John Fitzgerald & Carl Boeing, 2000. "Trust, Reciprocity, and Social History: A Re-examination," Experimental Economics, Springer;Economic Science Association, vol. 3(1), pages 81-100, June.
    25. Engers, Maxim & Gans, Joshua S, 1998. "Why Referees Are Not Paid (Enough)," American Economic Review, American Economic Association, vol. 88(5), pages 1341-1349, December.
    26. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    27. Pollak, Robert A & Wachter, Michael L, 1975. "The Relevance of the Household Production Function and Its Implications for the Allocation of Time," Journal of Political Economy, University of Chicago Press, vol. 83(2), pages 255-277, April.
    28. Paul A. David, 2004. "Can 'Open Science' be Protected from the Evolving Scheme of IPR Protections?," Journal of Institutional and Theoretical Economics (JITE), Mohr Siebeck, Tübingen, vol. 160(1), pages 9-34, March.
    29. Riccardo Boero & Giangiacomo Bravo & Marco Castellani & Francesco Laganà & Flaminio Squazzoni, 2009. "Pillars of Trust: An Experimental Study on Reputation and Its Effects," Sociological Research Online, , vol. 14(5), pages 49-67, November.
    30. Berg Joyce & Dickhaut John & McCabe Kevin, 1995. "Trust, Reciprocity, and Social History," Games and Economic Behavior, Elsevier, vol. 10(1), pages 122-142, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vladimir Batagelj & Anuška Ferligoj & Flaminio Squazzoni, 2017. "The emergence of a field: a network analysis of research on peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 503-532, October.
    2. Simone Righi & Károly Takács, 2017. "The miracle of peer review and development in science: an agent-based model," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 587-607, October.
    3. Raj Chetty & Emmanuel Saez & Laszlo Sandor, 2014. "What Policies Increase Prosocial Behavior? An Experiment with Referees at the Journal of Public Economics," Journal of Economic Perspectives, American Economic Association, vol. 28(3), pages 169-188, Summer.
    4. Richard R Snell, 2015. "Menage a Quoi? Optimal Number of Peer Reviewers," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-14, April.
    5. Takahashi, Hiromasa & Shen, Junyi & Ogawa, Kazuhito, 2016. "An experimental examination of compensation schemes and level of effort in differentiated tasks," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 61(C), pages 12-19.
    6. Vivian M Nguyen & Neal R Haddaway & Lee F G Gutowsky & Alexander D M Wilson & Austin J Gallagher & Michael R Donaldson & Neil Hammerschlag & Steven J Cooke, 2015. "How Long Is Too Long in Contemporary Peer Review? Perspectives from Authors Publishing in Conservation Biology Journals," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-20, August.
    7. Mario Paolucci & Francisco Grimaldo, 2014. "Mechanism change in a simulation of peer review: from junk support to elitism," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(3), pages 663-688, June.
    8. Charles Ayoubi & Boris Thurm, 2023. "Knowledge diffusion and morality: Why do we freely share valuable information with Strangers?," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 32(1), pages 75-99, January.
    9. Marco Seeber & Alberto Bacchelli, 2017. "Does single blind peer review hinder newcomers?," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 567-585, October.
    10. Olgica Nedić & Aleksandar Dekanski, 2016. "Priority criteria in peer review of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 15-26, April.
    11. Simone Righi & Karoly Takacs, 2016. "The Miracle of Peer Review and Development inScience: An Agent-Based Model," Department of Economics 0090, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    12. Christian Matt & Christian Hoerndlein & Thomas Hess, 2017. "Let the crowd be my peers? How researchers assess the prospects of social peer review," Electronic Markets, Springer;IIM University of St. Gallen, vol. 27(2), pages 111-124, May.
    13. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    14. García, J.A. & Montero-Parodi, J.J. & Rodriguez-Sánchez, Rosa & Fdez-Valdivia, J., 2023. "How to motivate a reviewer with a present bias to work harder," Journal of Informetrics, Elsevier, vol. 17(4).
    15. Federico Bianchi & Francisco Grimaldo & Giangiacomo Bravo & Flaminio Squazzoni, 2018. "The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 1401-1420, September.
    16. Houqiang Yu & Yian Liang & Yinghua Xie, 2024. "Can peer review accolade awards motivate reviewers? A large-scale quasi-natural experiment," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-13, December.
    17. Sergio Copiello, 2018. "On the money value of peer review," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 613-620, April.
    18. Bianchi, Federico & Grimaldo, Francisco & Squazzoni, Flaminio, 2019. "The F3-index. Valuing reviewers for scholarly journals," Journal of Informetrics, Elsevier, vol. 13(1), pages 78-86.
    19. Siddarth Srinivasan & Jamie Morgenstern, 2021. "Auctions and Peer Prediction for Academic Peer Review," Papers 2109.00923, arXiv.org, revised May 2023.
    20. Monica Aniela Zaharie & Marco Seeber, 2018. "Are non-monetary rewards effective in attracting peer reviewers? A natural experiment," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1587-1609, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Simeth, Markus & Raffo, Julio D., 2013. "What makes companies pursue an Open Science strategy?," Research Policy, Elsevier, vol. 42(9), pages 1531-1543.
    2. Gans, Joshua S. & Murray, Fiona E. & Stern, Scott, 2017. "Contracting over the disclosure of scientific knowledge: Intellectual property and academic publication," Research Policy, Elsevier, vol. 46(4), pages 820-835.
    3. Ofer H. Azar, 2006. "The Academic Review Process: How Can We Make it More Efficient?," The American Economist, Sage Publications, vol. 50(1), pages 37-50, March.
    4. Azar, Ofer H., 2008. "Evolution of social norms with heterogeneous preferences: A general model and an application to the academic review process," Journal of Economic Behavior & Organization, Elsevier, vol. 65(3-4), pages 420-435, March.
    5. Ganglmair, Bernhard & Holcomb, Alex & Myung, Noah, 2020. "Expectations of reciprocity when competitors share information: Experimental evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 170(C), pages 244-267.
    6. Güth, Werner & Mugera, Harriet & Musau, Andrew & Ploner, Matteo, 2014. "Deterministic versus probabilistic consequences of trust and trustworthiness: An experimental investigation," Journal of Economic Psychology, Elsevier, vol. 42(C), pages 28-40.
    7. Simeth, Markus & Lhuillery, Stephane, 2015. "How do firms develop capabilities for scientific disclosure?," Research Policy, Elsevier, vol. 44(7), pages 1283-1295.
    8. repec:wip:wpaper:6 is not listed on IDEAS
    9. Stephan, Paula E., 2010. "The Economics of Science," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 217-273, Elsevier.
    10. Henry Sauermann & Wesley M. Cohen, 2010. "What Makes Them Tick? Employee Motives and Firm Innovation," Management Science, INFORMS, vol. 56(12), pages 2134-2153, December.
    11. Rotolo, Daniele & Camerani, Roberto & Grassano, Nicola & Martin, Ben R., 2022. "Why do firms publish? A systematic literature review and a conceptual framework," Research Policy, Elsevier, vol. 51(10).
    12. Charles Ayoubi & Boris Thurm, 2023. "Knowledge diffusion and morality: Why do we freely share valuable information with Strangers?," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 32(1), pages 75-99, January.
    13. Daniel Woods & Maroš Servátka, 2019. "Nice to you, nicer to me: Does self-serving generosity diminish the reciprocal response?," Experimental Economics, Springer;Economic Science Association, vol. 22(2), pages 506-529, June.
    14. Bartoš, Vojtěch & Levely, Ian, 2021. "Sanctioning and trustworthiness across ethnic groups: Experimental evidence from Afghanistan," Journal of Public Economics, Elsevier, vol. 194(C).
    15. Henry Sauermann & Michael Roach, 2011. "Not All Scientists pay to be Scientists:," DRUID Working Papers 11-03, DRUID, Copenhagen Business School, Department of Industrial Economics and Strategy/Aalborg University, Department of Business Studies.
    16. Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014. "Open access to data: An ideal professed but not practised," Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
    17. Erik O. Kimbrough & Alexander Vostroknutov, 2016. "Norms Make Preferences Social," Journal of the European Economic Association, European Economic Association, vol. 14(3), pages 608-638, June.
    18. Bjorn Bartling & Ernst Fehr & Klaus M. Schmidt, 2012. "Screening, Competition, and Job Design: Economic Origins of Good Jobs," American Economic Review, American Economic Association, vol. 102(2), pages 834-864, April.
    19. Nikos Nikiforakis & Helen Mitchell, 2014. "Mixing the carrots with the sticks: third party punishment and reward," Experimental Economics, Springer;Economic Science Association, vol. 17(1), pages 1-23, March.
    20. Margit Osterloh & Bruno S. Frey, 2010. "Academic rankings and research governance," IEW - Working Papers 482, Institute for Empirical Research in Economics - University of Zurich.
    21. Giovanni Di Bartolomeo & Stefano Papa, 2019. "The Effects of Physical Activity on Social Interactions: The Case of Trust and Trustworthiness," Journal of Sports Economics, , vol. 20(1), pages 50-71, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:42:y:2013:i:1:p:287-294. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.