IDEAS home Printed from https://ideas.repec.org/a/wsi/acsxxx/v17y2014i03n04ns0219525914500131.html
   My bibliography  Save this article

Unstable Dynamics Of Adaptation In Unknown Environment Due To Novelty Seeking

Author

Listed:
  • ARKADY ZGONNIKOV

    (University of Aizu, Tsuruga, Ikki-Machi, Aizu-Wakamatsu, Fukushima 965-8580, Japan)

  • IHOR LUBASHEVSKY

    (University of Aizu, Tsuruga, Ikki-Machi, Aizu-Wakamatsu, Fukushima 965-8580, Japan)

Abstract

Learning and adaptation play great role in emergent socio-economic phenomena. Complex dynamics has been previously found in the systems of multiple learning agents interacting via a simple game. Meanwhile, the single agent adaptation is considered trivially stable. We advocate the idea that adopting a more complex model of the individual behavior may result in a more diverse spectrum of macro-level behaviors. We develop an adaptation model based on the reinforcement learning framework extended by an additional processing channel. We scrutiny the dynamics of the single agent adapting to the unknown environment; the agent is biased by novelty seeking, the intrinsic inclination for exploration. We demonstrate that the behavior of the novelty-seeking agent may be inherently unstable. One of the surprising results is that under certain conditions the increase of the novelty-seeking level may cause the agent to switch from the non-rational to the strictly rational behavior. Our results give evidence to the hypothesis that the intrinsic motives of agents should be paid no less attention than the extrinsic ones in the models of complex socio-economic systems.

Suggested Citation

  • Arkady Zgonnikov & Ihor Lubashevsky, 2014. "Unstable Dynamics Of Adaptation In Unknown Environment Due To Novelty Seeking," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 17(03n04), pages 1-17.
  • Handle: RePEc:wsi:acsxxx:v:17:y:2014:i:03n04:n:s0219525914500131
    DOI: 10.1142/S0219525914500131
    as

    Download full text from publisher

    File URL: http://www.worldscientific.com/doi/abs/10.1142/S0219525914500131
    Download Restriction: Access to full text is restricted to subscribers

    File URL: https://libkey.io/10.1142/S0219525914500131?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fudenberg, Drew & Levine, David, 1998. "Learning in games," European Economic Review, Elsevier, vol. 42(3-5), pages 631-639, May.
    2. Drew Fudenberg & David K. Levine, 1998. "The Theory of Learning in Games," MIT Press Books, The MIT Press, edition 1, volume 1, number 0262061945, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Galbiati, Marco & Soramäki, Kimmo, 2011. "An agent-based model of payment systems," Journal of Economic Dynamics and Control, Elsevier, vol. 35(6), pages 859-875, June.
    2. Ianni, A., 2002. "Reinforcement learning and the power law of practice: some analytical results," Discussion Paper Series In Economics And Econometrics 203, Economics Division, School of Social Sciences, University of Southampton.
    3. ,, 2011. "Manipulative auction design," Theoretical Economics, Econometric Society, vol. 6(2), May.
    4. Benaïm, Michel & Hofbauer, Josef & Hopkins, Ed, 2009. "Learning in games with unstable equilibria," Journal of Economic Theory, Elsevier, vol. 144(4), pages 1694-1709, July.
    5. Dieter Balkenborg & Rosemarie Nagel, 2016. "An Experiment on Forward vs. Backward Induction: How Fairness and Level k Reasoning Matter," German Economic Review, Verein für Socialpolitik, vol. 17(3), pages 378-408, August.
    6. William L. Cooper & Tito Homem-de-Mello & Anton J. Kleywegt, 2015. "Learning and Pricing with Models That Do Not Explicitly Incorporate Competition," Operations Research, INFORMS, vol. 63(1), pages 86-103, February.
    7. Siegfried Berninghaus & Werner Güth & M. Vittoria Levati & Jianying Qiu, 2006. "Satisficing in sales competition: experimental evidence," Papers on Strategic Interaction 2006-32, Max Planck Institute of Economics, Strategic Interaction Group.
    8. Ball, Richard, 2017. "Violations of monotonicity in evolutionary models with sample-based beliefs," Economics Letters, Elsevier, vol. 152(C), pages 100-104.
    9. Tsakas, Elias & Voorneveld, Mark, 2009. "The target projection dynamic," Games and Economic Behavior, Elsevier, vol. 67(2), pages 708-719, November.
    10. Sandholm,W.H., 2003. "Excess payoff dynamics, potential dynamics, and stable games," Working papers 5, Wisconsin Madison - Social Systems.
    11. Yoo, Seung Han, 2014. "Learning a population distribution," Journal of Economic Dynamics and Control, Elsevier, vol. 48(C), pages 188-201.
    12. Anthony Ziegelmeyer & Frédéric Koessler & Kene Boun My & Laurent Denant-Boèmont, 2008. "Road Traffic Congestion and Public Information: An Experimental Investigation," Journal of Transport Economics and Policy, University of Bath, vol. 42(1), pages 43-82, January.
    13. DeJong, D.V. & Blume, A. & Neumann, G., 1998. "Learning in Sender-Receiver Games," Other publications TiSEM 4a8b4f46-f30b-4ad2-bb0c-1, Tilburg University, School of Economics and Management.
    14. Alessandro Lizzeri & Marciano Siniscalchi, 2008. "Parental Guidance and Supervised Learning," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(3), pages 1161-1195.
    15. Ilya R. P. Cuypers & Youtha Cuypers & Xavier Martin, 2017. "When the target may know better: Effects of experience and information asymmetries on value from mergers and acquisitions," Strategic Management Journal, Wiley Blackwell, vol. 38(3), pages 609-625, March.
    16. Mario Gilli, 2002. "Rational Learning in Imperfect Monitoring Games," Working Papers 46, University of Milano-Bicocca, Department of Economics, revised Mar 2002.
    17. Brit Grosskopf & Ido Erev & Eldad Yechiam, 2006. "Foregone with the Wind: Indirect Payoff Information and its Implications for Choice," International Journal of Game Theory, Springer;Game Theory Society, vol. 34(2), pages 285-302, August.
    18. Salle, Isabelle & Yildizoglu, Murat & Zumpe, Martin & Sénégas, Marc-Alexandre, 2017. "Coordination through social learning in a general equilibrium model," Journal of Economic Behavior & Organization, Elsevier, vol. 141(C), pages 64-82.
    19. Gilles Grandjean & Ana Mauleon & Vincent Vannetelbosch, 2017. "Strongly rational sets for normal-form games," Economic Theory Bulletin, Springer;Society for the Advancement of Economic Theory (SAET), vol. 5(1), pages 35-46, April.
    20. John T. Scholz & Cheng‐Lung Wang, 2009. "Learning to Cooperate: Learning Networks and the Problem of Altruism," American Journal of Political Science, John Wiley & Sons, vol. 53(3), pages 572-587, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wsi:acsxxx:v:17:y:2014:i:03n04:n:s0219525914500131. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tai Tone Lim (email available below). General contact details of provider: http://www.worldscinet.com/acs/acs.shtml .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.