IDEAS home Printed from https://ideas.repec.org/a/eee/ecomod/v439y2021ics0304380020304142.html
   My bibliography  Save this article

A multi-armed bandit algorithm speeds up the evolution of cooperation

Author

Listed:
  • Cazzolla Gatti, Roberto

Abstract

Most evolutionary biologists consider selfishness an intrinsic feature of our genes and as the best choice in social situations. During the last years, prolific research has been conducted on the mechanisms that can allow cooperation to emerge “in a world of defectors” to become an evolutionarily stable strategy. A big debate started with the proposal by W.D. Hamilton of “kin selection” in terms of cost sustained by the cooperators and benefits received by related conspecifics. After this, four other main rules for the evolution of cooperation have been suggested. However, one of the main problems of these five rules is the assumption that the payoffs obtained by either cooperating or defeating are quite well known by the parties before they interact and do not change during the time or after repeated encounters. This is not always the case in real life. Following each rule blindly, there is a risk for individuals to get stuck in an unfavorable situation. Axelrod (1984) highlighted that the main problem is how to obtain benefits from cooperation without passing through several trials and errors, which are slow and painful. With a better understanding of this process, individuals can use their foresight to speed up the evolution of cooperation. Here I show that a multi-armed bandit (MAB) model, a classic problem in decision sciences, is naturally employed by individuals to opt for the best choice most of the time, accelerating the evolution of the altruistic behavior and solving the abovementioned problems. A common MAB model that applies extremely well to the evolution of cooperation is the epsilon-greedy (ε-greedy) algorithm. This algorithm, after an initial period of exploration (which can be considered as biological history), greedily exploits the best option ε% of the time and explores other options the remaining percentage of times (1-ε%). Through the epsilon-greedy decision-making algorithm, cooperation evolves as a multilevel process nested in the hierarchical levels that exist among the five rules for the evolution of cooperation. This reinforcement learning, a subtype of artificial intelligence, with trials and errors, provides a powerful tool to better understand and even probabilistically quantify the chances cooperation has to evolve in a specific situation.

Suggested Citation

  • Cazzolla Gatti, Roberto, 2021. "A multi-armed bandit algorithm speeds up the evolution of cooperation," Ecological Modelling, Elsevier, vol. 439(C).
    • Handle: RePEc:eee:ecomod:v:439:y:2021:i:c:s0304380020304142
    DOI: 10.1016/j.ecolmodel.2020.109348
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304380020304142
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ecolmodel.2020.109348?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Noa Truskanov & Yasmin Emery & Redouan Bshary, 2020. "Juvenile cleaner fish can socially learn the consequences of cheating," Nature Communications, Nature, vol. 11(1), pages 1-9, December.
    2. Ashleigh S. Griffin & Stuart A. West & Angus Buckling, 2004. "Cooperation and competition in pathogenic bacteria," Nature, Nature, vol. 430(7003), pages 1024-1027, August.
    3. David L. Hull, 1999. "Evolutionists red in tooth and claw," Nature, Nature, vol. 398(6726), pages 385-385, April.
    4. Hanning Chen & Yunlong Zhu & Kunyuan Hu, 2009. "Cooperative Bacterial Foraging Optimization," Discrete Dynamics in Nature and Society, Hindawi, vol. 2009, pages 1-17, November.
    5. Lester B. Lave, 1962. "An Empirical Approach to the Prisoners' Dilemma Game," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 76(3), pages 424-436.
    6. Martin A. Nowak & Corina E. Tarnita & Edward O. Wilson, 2010. "The evolution of eusociality," Nature, Nature, vol. 466(7310), pages 1057-1062, August.
    7. Sean B. Carroll, 2001. "Chance and necessity: the evolution of morphological complexity and diversity," Nature, Nature, vol. 409(6823), pages 1102-1109, February.
    8. Gatti, Roberto Cazzolla & Hordijk, Wim & Kauffman, Stuart, 2017. "Biodiversity is autocatalytic," Ecological Modelling, Elsevier, vol. 346(C), pages 70-76.
    9. Jamieson C. Gorrell & Andrew G. McAdam & David W. Coltman & Murray M. Humphries & Stan Boutin, 2010. "Adopting kin enhances inclusive fitness in asocial red squirrels," Nature Communications, Nature, vol. 1(1), pages 1-4, December.
    10. Martin A. Nowak & Karl Sigmund, 2005. "Evolution of indirect reciprocity," Nature, Nature, vol. 437(7063), pages 1291-1298, October.
    11. Ugo Bastolla & Miguel A. Fortuna & Alberto Pascual-García & Antonio Ferrera & Bartolo Luque & Jordi Bascompte, 2009. "The architecture of mutualistic networks minimizes competition and increases biodiversity," Nature, Nature, vol. 458(7241), pages 1018-1020, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roberto Cazzolla Gatti & Roger Koppl & Brian D. Fath & Stuart Kauffman & Wim Hordijk & Robert E. Ulanowicz, 2020. "On the emergence of ecological and economic niches," Journal of Bioeconomics, Springer, vol. 22(2), pages 99-127, July.
    2. Liu, Yuan & Cao, Lixuan & Wu, Bin, 2022. "General non-linear imitation leads to limit cycles in eco-evolutionary dynamics," Chaos, Solitons & Fractals, Elsevier, vol. 165(P2).
    3. Zhang, Yifan & Shu, Gang & Li, Ya, 2017. "Strategy-updating depending on local environment enhances cooperation in prisoner’s dilemma game," Applied Mathematics and Computation, Elsevier, vol. 301(C), pages 224-232.
    4. Du, Faqi & Fu, Feng, 2013. "Quantifying the impact of noise on macroscopic organization of cooperation in spatial games," Chaos, Solitons & Fractals, Elsevier, vol. 56(C), pages 35-44.
    5. Mengyuan Zhou, 2022. "Does the Source of Inheritance Matter in Bequest Attitudes? Evidence from Japan," Journal of Family and Economic Issues, Springer, vol. 43(4), pages 867-887, December.
    6. Wendelin Schnedler & Nina Lucia Stephan, 2020. "Revisiting a Remedy Against Chains of Unkindness," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 72(3), pages 347-364, July.
    7. Cristina Fiera & Jan Christian Habel & Werner Ulrich, 2018. "Neutral colonisations drive high beta-diversity in cavernicole springtails (Collembola)," PLOS ONE, Public Library of Science, vol. 13(1), pages 1-12, January.
    8. Mengyuan Zhou, 2019. "The Effect of the Source of Inheritance on Bequest Attitudes: Evidence from Japan," Keio-IES Discussion Paper Series 2019-018, Institute for Economics Studies, Keio University.
    9. Alós-Ferrer, Carlos & Garagnani, Michele, 2020. "The cognitive foundations of cooperation," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 71-85.
    10. Cinzia Di Novi & Rowena Jacobs & Matteo Migheli, 2013. "The quality of life of female informal caregivers: from Scandinavia to the Mediterranean Sea," Working Papers 084cherp, Centre for Health Economics, University of York.
    11. Catherine C Eckel & Enrique Fatas & Sara Godoy & Rick K Wilson, 2016. "Group-Level Selection Increases Cooperation in the Public Goods Game," PLOS ONE, Public Library of Science, vol. 11(8), pages 1-13, August.
    12. Colton Brehm & Astrid Layton, 2021. "Nestedness of eco‐industrial networks: Exploring linkage distribution to promote sustainable industrial growth," Journal of Industrial Ecology, Yale University, vol. 25(1), pages 205-218, February.
    13. Charness, Gary & Du, Ninghua & Yang, Chun-Lei, 2011. "Trust and trustworthiness reputations in an investment game," Games and Economic Behavior, Elsevier, vol. 72(2), pages 361-375, June.
    14. Som B Ale & Joel S Brown & Amy T Sullivan, 2013. "Evolution of Cooperation: Combining Kin Selection and Reciprocal Altruism into Matrix Games with Social Dilemmas," PLOS ONE, Public Library of Science, vol. 8(5), pages 1-8, May.
    15. M. Kleshnina & K. Kaveh & K. Chatterjee, 2020. "The role of behavioural plasticity in finite vs infinite populations," Papers 2009.13160, arXiv.org.
    16. Maier-Rigaud, Frank P. & Apesteguia, José, 2003. "The Role of Choice in Social Dilemma Experiments," Bonn Econ Discussion Papers 22/2003, University of Bonn, Bonn Graduate School of Economics (BGSE).
    17. Andrew W. Bausch, 2014. "Evolving intergroup cooperation," Computational and Mathematical Organization Theory, Springer, vol. 20(4), pages 369-393, December.
    18. de Melo, Gioia & Piaggio, Matías, 2015. "The perils of peer punishment: Evidence from a common pool resource framed field experiment," Ecological Economics, Elsevier, vol. 120(C), pages 376-393.
    19. Liang, Rizhou & Zhang, Jiqiang & Zheng, Guozhong & Chen, Li, 2021. "Social hierarchy promotes the cooperation prevalence," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 567(C).
    20. Ping Zhu & Guiyi Wei, 2014. "Stochastic Heterogeneous Interaction Promotes Cooperation in Spatial Prisoner's Dilemma Game," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-10, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecomod:v:439:y:2021:i:c:s0304380020304142. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/ecological-modelling .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.