IDEAS home Printed from https://ideas.repec.org/a/eee/chsofr/v184y2024ics096007792400523x.html
   My bibliography  Save this article

Emergence of anti-coordinated patterns in snowdrift game by reinforcement learning

Author

Listed:
  • Ding, Zhen-Wei
  • Zhang, Ji-Qiang
  • Zheng, Guo-Zhong
  • Cai, Wei-Ran
  • Cai, Chao-Ran
  • Chen, Li
  • Wang, Xu-Ming

Abstract

Patterns emerging through self-organization in nature have sparked considerable interest across various disciplines, owing to their significance in comprehending collective behaviors. The anti-coordination phenomenon is widespread in both social systems and ecosystems. However, within the context of the SnowDrift Game (SDG), considered as a typical anti-coordination game, the occurrence of anti-coordination patterns is surprisingly rare. In this work, we introduce a model called the Two-Agent, Two-Action Reinforcement Learning Evolutionary Game (2 × 2 RLEG), and utilize it in the SDG on different regular lattices. We unveil intriguing phenomena represented by Anti-Coordinated domains (AC-domains) within spatial patterns, which hold a significant sway over the cooperation preference within populations. The frustration contributes substantially to the diversity of AC-domains within parameter space. To comprehend the underlying mechanism, we have developed a perturbation theory to analyze the stability of different AC-domains comprising action patterns and identify continuous phase transitions within parameter space. Our theory effectively partitions the parameter space into several areas, each characterized by unique features of domains that determine the cooperation preferences within the population. Our work offers insights into the emergence of spatial patterns and their impact on cooperation in nature, contributing to the development of theories that analyze their structural complexities.

Suggested Citation

  • Ding, Zhen-Wei & Zhang, Ji-Qiang & Zheng, Guo-Zhong & Cai, Wei-Ran & Cai, Chao-Ran & Chen, Li & Wang, Xu-Ming, 2024. "Emergence of anti-coordinated patterns in snowdrift game by reinforcement learning," Chaos, Solitons & Fractals, Elsevier, vol. 184(C).
  • Handle: RePEc:eee:chsofr:v:184:y:2024:i:c:s096007792400523x
    DOI: 10.1016/j.chaos.2024.114971
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S096007792400523X
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.chaos.2024.114971?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Benjamin Allen & Gabor Lippner & Yu-Ting Chen & Babak Fotouhi & Naghmeh Momeni & Shing-Tung Yau & Martin A. Nowak, 2017. "Evolutionary dynamics on any population structure," Nature, Nature, vol. 544(7649), pages 227-230, April.
    2. Li, Wen-Jing & Chen, Zhi & Jin, Ke-Zhong & Wang, Jun & Yuan, Lin & Gu, Changgui & Jiang, Luo-Luo & Perc, Matjaž, 2022. "Options for mobility and network reciprocity to jointly yield robust cooperation in social dilemmas," Applied Mathematics and Computation, Elsevier, vol. 435(C).
    3. Zhang, Jing & Li, Zhao & Zhang, Jiqiang & Ma, Lin & Zheng, Guozhong & Chen, Li, 2023. "Emergence of oscillatory cooperation in a population with incomplete information," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 617(C).
    4. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    5. Ni, Y.C. & Xu, C. & Hui, P.M. & Johnson, N.F., 2009. "Cooperative behavior in evolutionary snowdrift game with bounded rationality," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 388(23), pages 4856-4862.
    6. Jia, Ning & Ma, Shoufeng, 2013. "Evolution of cooperation in the snowdrift game among mobile players with random-pairing and reinforcement learning," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(22), pages 5700-5710.
    7. Christoph Hauert & Michael Doebeli, 2004. "Spatial structure often inhibits the evolution of cooperation in the snowdrift game," Nature, Nature, vol. 428(6983), pages 643-646, April.
    8. Jeff Gore & Hyun Youk & Alexander van Oudenaarden, 2009. "Snowdrift game dynamics and facultative cheating in yeast," Nature, Nature, vol. 459(7244), pages 253-256, May.
    9. Meron, Ehud, 2012. "Pattern-formation approach to modelling spatially extended ecosystems," Ecological Modelling, Elsevier, vol. 234(C), pages 70-82.
    10. Han, Jia-Xu & Wang, Rui-Wu, 2023. "Complex interactions promote the frequency of cooperation in snowdrift game," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 609(C).
    11. Pietro Guarnieri & Lorenzo Spadoni, 2024. "Norms and anti-coordination: elicitation and priming in an El Farol Bar Game experiment," Discussion Papers 2024/303, Dipartimento di Economia e Management (DEM), University of Pisa, Pisa, Italy.
    12. Tobias Reichenbach & Mauro Mobilia & Erwin Frey, 2007. "Mobility promotes and jeopardizes biodiversity in rock–paper–scissors games," Nature, Nature, vol. 448(7157), pages 1046-1049, August.
    13. Ma, Lin & Zhang, Jiqiang & Zheng, Guozhong & Liang, Rizhou & Chen, Li, 2023. "Emergence of cooperation in a population with bimodal response behaviors," Chaos, Solitons & Fractals, Elsevier, vol. 171(C).
    14. Segismundo S. Izquierdo & Luis R. Izquierdo & Nicholas M. Gotts, 2008. "Reinforcement Learning Dynamics in Social Dilemmas," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 11(2), pages 1-1.
    15. Flores, Lucas S. & Amaral, Marco A. & Vainstein, Mendeli H. & Fernandes, Heitor C.M., 2022. "Cooperation in regular lattices," Chaos, Solitons & Fractals, Elsevier, vol. 164(C).
    16. Izquierdo, Luis R. & Izquierdo, Segismundo S. & Gotts, Nicholas M. & Polhill, J. Gary, 2007. "Transient and asymptotic dynamics of reinforcement learning in games," Games and Economic Behavior, Elsevier, vol. 61(2), pages 259-276, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sarkar, Bijan, 2021. "The cooperation–defection evolution on social networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 584(C).
    2. Qi Su & Lei Zhou & Long Wang, 2019. "Evolutionary multiplayer games on graphs with edge diversity," PLOS Computational Biology, Public Library of Science, vol. 15(4), pages 1-22, April.
    3. Li, Ya & Chen, Shanxiong & Niu, Ben, 2018. "Reward depending on public funds stimulates cooperation in spatial prisoner’s dilemma games," Chaos, Solitons & Fractals, Elsevier, vol. 114(C), pages 38-45.
    4. Jia, Danyang & Li, Tong & Zhao, Yang & Zhang, Xiaoqin & Wang, Zhen, 2022. "Empty nodes affect conditional cooperation under reinforcement learning," Applied Mathematics and Computation, Elsevier, vol. 413(C).
    5. Zhong, Shiquan & Jia, Ning & Ma, Shoufeng, 2014. "Iterated snowdrift game among mobile agents with myopic expected-reward based decision rule: Numerical and analytical research," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 414(C), pages 6-18.
    6. Jorge Peña & Yannick Rochat, 2012. "Bipartite Graphs as Models of Population Structures in Evolutionary Multiplayer Games," PLOS ONE, Public Library of Science, vol. 7(9), pages 1-13, September.
    7. Yang, Zhengzhi & Zheng, Lei & Perc, Matjaž & Li, Yumeng, 2024. "Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game," Applied Mathematics and Computation, Elsevier, vol. 463(C).
    8. Zhang, Lan & Huang, Changwei, 2023. "Preferential selection to promote cooperation on degree–degree correlation networks in spatial snowdrift games," Applied Mathematics and Computation, Elsevier, vol. 454(C).
    9. Fulin Guo, 2023. "Experience-weighted attraction learning in network coordination games," Papers 2310.18835, arXiv.org.
    10. Brian McLoone & Wai-Tong Louis Fan & Adam Pham & Rory Smead & Laurence Loewe, 2018. "Stochasticity, Selection, and the Evolution of Cooperation in a Two-Level Moran Model of the Snowdrift Game," Complexity, Hindawi, vol. 2018, pages 1-14, February.
    11. Wang, Xianjia & Yang, Zhipeng & Liu, Yanli & Chen, Guici, 2023. "A reinforcement learning-based strategy updating model for the cooperative evolution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 618(C).
    12. Fabio Della Rossa & Fabio Dercole & Anna Di Meglio, 2020. "Direct Reciprocity and Model-Predictive Strategy Update Explain the Network Reciprocity Observed in Socioeconomic Networks," Games, MDPI, vol. 11(1), pages 1-28, March.
    13. Bai, Pengzhou & Qiang, Bingzhuang & Zou, Kuan & Huang, Changwei, 2024. "Preferential selection based on adaptive attractiveness induce by reinforcement learning promotes cooperation," Chaos, Solitons & Fractals, Elsevier, vol. 180(C).
    14. Li, Bin-Quan & Wu, Zhi-Xi & Guan, Jian-Yue, 2022. "Critical thresholds of benefit distribution in an extended snowdrift game model," Chaos, Solitons & Fractals, Elsevier, vol. 157(C).
    15. Sakiyama, Tomoko, 2021. "A power law network in an evolutionary hawk–dove game," Chaos, Solitons & Fractals, Elsevier, vol. 146(C).
    16. Luis R. Izquierdo & Segismundo S. Izquierdo & José Manuel Galán & José Ignacio Santos, 2009. "Techniques to Understand Computer Simulations: Markov Chain Analysis," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 12(1), pages 1-6.
    17. Cheng, Jiangjiang & Mei, Wenjun & Su, Wei & Chen, Ge, 2023. "Evolutionary games on networks: Phase transition, quasi-equilibrium, and mathematical principles," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 611(C).
    18. Wang, Weijie & Xu, Zhehang & Hua, Shijia & Cui, Longqing & Zhang, Jianlin & Meng, Fanyuan, 2024. "Threshold-initiated spatial public goods games," Chaos, Solitons & Fractals, Elsevier, vol. 184(C).
    19. Wang, Chaoqian & Lin, Zongzhe & Rothman, Dale S., 2022. "Public goods game on coevolving networks driven by the similarity and difference of payoff," Chaos, Solitons & Fractals, Elsevier, vol. 162(C).
    20. Lv, Shaojie & Li, Jiaying & Zhao, Changheng, 2023. "The evolution of cooperation in voluntary public goods game with shared-punishment," Chaos, Solitons & Fractals, Elsevier, vol. 172(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:chsofr:v:184:y:2024:i:c:s096007792400523x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Thayer, Thomas R. (email available below). General contact details of provider: https://www.journals.elsevier.com/chaos-solitons-and-fractals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.