IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0189220.html
   My bibliography  Save this article

Reinforcement learning account of network reciprocity

Author

Listed:
  • Takahiro Ezaki
  • Naoki Masuda

Abstract

Evolutionary game theory predicts that cooperation in social dilemma games is promoted when agents are connected as a network. However, when networks are fixed over time, humans do not necessarily show enhanced mutual cooperation. Here we show that reinforcement learning (specifically, the so-called Bush-Mosteller model) approximately explains the experimentally observed network reciprocity and the lack thereof in a parameter region spanned by the benefit-to-cost ratio and the node’s degree. Thus, we significantly extend previously obtained numerical results.

Suggested Citation

  • Takahiro Ezaki & Naoki Masuda, 2017. "Reinforcement learning account of network reciprocity," PLOS ONE, Public Library of Science, vol. 12(12), pages 1-8, December.
    • Handle: RePEc:plo:pone00:0189220
    DOI: 10.1371/journal.pone.0189220
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0189220
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0189220&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0189220?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. repec:cup:cbooks:9780511771576 is not listed on IDEAS
    2. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    3. Jonathan Bendor & Dilip Mookherjee & Debraj Ray, 2001. "Aspiration-Based Reinforcement Learning In Repeated Interaction Games: An Overview," International Game Theory Review (IGTR), World Scientific Publishing Co. Pte. Ltd., vol. 3(02n03), pages 159-174.
    4. Jillian J Jordan & David G Rand & Samuel Arbesman & James H Fowler & Nicholas A Christakis, 2013. "Contagion of Cooperation in Static and Fluid Social Networks," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-10, June.
    5. Hisashi Ohtsuki & Christoph Hauert & Erez Lieberman & Martin A. Nowak, 2006. "A simple rule for the evolution of cooperation on graphs and social networks," Nature, Nature, vol. 441(7092), pages 502-505, May.
    6. Siddharth Suri & Duncan J Watts, 2011. "Cooperation and Contagion in Web-Based, Networked Public Goods Experiments," PLOS ONE, Public Library of Science, vol. 6(3), pages 1-18, March.
    7. Cassar, Alessandra, 2007. "Coordination and cooperation in local, random and small world networks: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 58(2), pages 209-230, February.
    8. Jelena Grujić & Torsten Röhl & Dirk Semmann & Manfred Milinski & Arne Traulsen, 2012. "Consistent Strategy Updating in Spatial and Non-Spatial Behavioral Experiments Does Not Promote Cooperation in Social Networks," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-8, November.
    9. Jelena Grujić & Constanza Fosco & Lourdes Araujo & José A Cuesta & Angel Sánchez, 2010. "Social Experiments in the Mesoscale: Humans Playing a Spatial Prisoner's Dilemma," PLOS ONE, Public Library of Science, vol. 5(11), pages 1-9, November.
    10. Easley,David & Kleinberg,Jon, 2010. "Networks, Crowds, and Markets," Cambridge Books, Cambridge University Press, number 9780521195331, October.
    11. Giulio Cimini & Angel Sanchez, 2015. "How Evolutionary Dynamics Affects Network Reciprocity in Prisoner’s Dilemma," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 18(2), pages 1-22.
    12. Hirokazu Shirado & Feng Fu & James H. Fowler & Nicholas A. Christakis, 2013. "Quality versus quantity of social ties in experimental cooperative networks," Nature Communications, Nature, vol. 4(1), pages 1-8, December.
    13. Bendor Jonathan & Mookherjee Dilip & Ray Debraj, 2001. "Reinforcement Learning in Repeated Interaction Games," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 1(1), pages 1-44, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Correia, A.D. & Leestmaker, L.L. & Stoof, H.T.C. & Broere, J.J., 2022. "Asymmetric games on networks: Towards an Ising-model representation," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 593(C).
    2. Han, Xu & Zhao, Xiaowei & Xia, Haoxiang, 2022. "Hybrid learning promotes cooperation in the spatial prisoner’s dilemma game," Chaos, Solitons & Fractals, Elsevier, vol. 164(C).
    3. Zhang, Liming & Huang, Changwei & Li, Haihong & Dai, Qionglin & Yang, Junzhong, 2021. "Cooperation guided by imitation, aspiration and conformity-driven dynamics in evolutionary games," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 561(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yali Dong & Cong Li & Yi Tao & Boyu Zhang, 2015. "Evolution of Conformity in Social Dilemmas," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-12, September.
    2. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    3. Alberto Antonioni & Maria Paula Cacault & Rafael Lalive & Marco Tomassini, 2013. "Coordination on Networks: Does Topology Matter?," PLOS ONE, Public Library of Science, vol. 8(2), pages 1-11, February.
    4. Syngjoo Choi & Edoardo Gallo & Shachar Kariv, 2015. "Networks in the laboratory," Cambridge Working Papers in Economics 1551, Faculty of Economics, University of Cambridge.
    5. Li, Yan & Ye, Hang, 2015. "Effect of migration based on strategy and cost on the evolution of cooperation," Chaos, Solitons & Fractals, Elsevier, vol. 76(C), pages 156-165.
    6. Jelena Grujić & Torsten Röhl & Dirk Semmann & Manfred Milinski & Arne Traulsen, 2012. "Consistent Strategy Updating in Spatial and Non-Spatial Behavioral Experiments Does Not Promote Cooperation in Social Networks," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-8, November.
    7. Milena Tsvetkova & Claudia Wagner & Andrew Mao, 2018. "The emergence of inequality in social groups: Network structure and institutions affect the distribution of earnings in cooperation games," PLOS ONE, Public Library of Science, vol. 13(7), pages 1-16, July.
    8. Fabio Della Rossa & Fabio Dercole & Anna Di Meglio, 2020. "Direct Reciprocity and Model-Predictive Strategy Update Explain the Network Reciprocity Observed in Socioeconomic Networks," Games, MDPI, vol. 11(1), pages 1-28, March.
    9. Tsvetkova, Milena & Wagner, Claudia & Mao, Andrew, 2018. "The emergence of inequality in social groups: network structure and institutions affect the distribution of earnings in cooperation games," LSE Research Online Documents on Economics 89716, London School of Economics and Political Science, LSE Library.
    10. Dennie van Dolder & Vincent Buskens, 2014. "Individual Choices in Dynamic Networks: An Experiment on Social Preferences," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-16, April.
    11. Hadzibeganovic, Tarik & Stauffer, Dietrich & Han, Xiao-Pu, 2018. "Interplay between cooperation-enhancing mechanisms in evolutionary games with tag-mediated interactions," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 496(C), pages 676-690.
    12. Jillian J Jordan & David G Rand & Samuel Arbesman & James H Fowler & Nicholas A Christakis, 2013. "Contagion of Cooperation in Static and Fluid Social Networks," PLOS ONE, Public Library of Science, vol. 8(6), pages 1-10, June.
    13. Tim Johnson & Oleg Smirnov, 2020. "Temporal assortment of cooperators in the spatial prisoner's dilemma," Papers 2011.14440, arXiv.org.
    14. Pei-Pei Liu & Vasiliy Safin & Barry Yang & Christian C Luhmann, 2015. "Direct and Indirect Influence of Altruistic Behavior in a Social Network," PLOS ONE, Public Library of Science, vol. 10(10), pages 1-18, October.
    15. Jia, Danyang & Li, Tong & Zhao, Yang & Zhang, Xiaoqin & Wang, Zhen, 2022. "Empty nodes affect conditional cooperation under reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 413(C).
    16. Konno, Tomohiko, 2013. "An imperfect competition on scale-free networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(21), pages 5453-5460.
    17. Siegfried Berninghaus & Werner Güth & M. Vittoria Levati & Jianying Qiu, 2006. "Satisficing in sales competition: experimental evidence," Papers on Strategic Interaction 2006-32, Max Planck Institute of Economics, Strategic Interaction Group.
    18. Floriana Gargiulo & José J Ramasco, 2012. "Influence of Opinion Dynamics on the Evolution of Games," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-7, November.
    19. Marcin Dziubinski & Jaideep Roy, 2007. "Endogenous Selection of Aspiring and Rational rules in Coordination Games," CEDI Discussion Paper Series 07-14, Centre for Economic Development and Institutions(CEDI), Brunel University.
    20. Rense Corten & Stephanie Rosenkranz & Vincent Buskens & Karen S Cook, 2016. "Reputation Effects in Social Networks Do Not Promote Cooperation: An Experimental Test of the Raub & Weesie Model," PLOS ONE, Public Library of Science, vol. 11(7), pages 1-17, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0189220. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.