IDEAS home Printed from https://ideas.repec.org/p/vnm/wpdman/40.html
   My bibliography  Save this paper

Pack light on the move: Exploitation and exploration in a dynamic environment

Author

Listed:
  • Marco LiCalzi

    (Department of Management, Università Ca' Foscari Venezia)

  • Davide Marchiori

    (Department of Management, Università Ca' Foscari Venezia)

Abstract

This paper revisits a recent study by Posen and Levinthal (2012) on the exploration/exploitation tradeoff for a multi-armed bandit problem, where the reward probabilities undergo random shocks. We show that their analysis suffers two shortcomings: it assumes that learning is based on stale evidence, and it overlooks the steady state. We let the learning rule endogenously discard stale evidence, and we perform the long run analyses. The comparative study demonstrates that some of their conclusions must be qualified.

Suggested Citation

  • Marco LiCalzi & Davide Marchiori, 2013. "Pack light on the move: Exploitation and exploration in a dynamic environment," Working Papers 4, Venice School of Management - Department of Management, Università Ca' Foscari Venezia.
    • Handle: RePEc:vnm:wpdman:40
    as

    Download full text from publisher

    File URL: http://virgo.unive.it/wpideas/storage/2013wp4.pdf
    File Function: First version, 2013
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Duffy, John, 2006. "Agent-Based Models and Human Subject Experiments," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 19, pages 949-1011, Elsevier.
    2. Leigh Tesfatsion & Kenneth L. Judd (ed.), 2006. "Handbook of Computational Economics," Handbook of Computational Economics, Elsevier, edition 1, volume 2, number 2.
    3. James G. March, 1991. "Exploration and Exploitation in Organizational Learning," Organization Science, INFORMS, vol. 2(1), pages 71-87, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Friederike Wall, 2016. "Agent-based modeling in managerial science: an illustrative survey and study," Review of Managerial Science, Springer, vol. 10(1), pages 135-193, January.
    2. Yoo, Seung Han, 2014. "Learning a population distribution," Journal of Economic Dynamics and Control, Elsevier, vol. 48(C), pages 188-201.
    3. Pascal Seppecher & Isabelle Salle & Dany Lang, 2019. "Is the market really a good teacher?," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 299-335, March.
    4. Giuseppe Attanasi & Samuele Centorrino & Ivan Moscati, 2011. "Double Auction Equilibrium and Efficiency in a Classroom Experimental Search Market," LERNA Working Papers 11.03.337, LERNA, University of Toulouse.
    5. Flaminio Squazzoni, 2010. "The impact of agent-based models in the social sciences after 15 years of incursions," History of Economic Ideas, Fabrizio Serra Editore, Pisa - Roma, vol. 18(2), pages 197-234.
    6. Hommes, Cars H., 2006. "Heterogeneous Agent Models in Economics and Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 23, pages 1109-1186, Elsevier.
    7. Hommes, Cars, 2011. "The heterogeneous expectations hypothesis: Some evidence from the lab," Journal of Economic Dynamics and Control, Elsevier, vol. 35(1), pages 1-24, January.
    8. Takahiro Ezaki & Yutaka Horita & Masanori Takezawa & Naoki Masuda, 2016. "Reinforcement Learning Explains Conditional Cooperation and Its Moody Cousin," PLOS Computational Biology, Public Library of Science, vol. 12(7), pages 1-13, July.
    9. Michael Neugart & Matteo G. Richiardi, 2012. "Agent-based models of the labor market," LABORatorio R. Revelli Working Papers Series 125, LABORatorio R. Revelli, Centre for Employment Studies.
    10. Julissa Alexandra Galarza-Villamar & Mariette McCampbell & Cees Leeuwis & Francesco Cecchi, 2021. "Adding Emergence and Spatiality to a Public Bad Game for Studying Dynamics in Socio-Ecological Systems (Part I): The Design of Musa-Game for Integrative Analysis of Collective Action in Banana Disease," Sustainability, MDPI, vol. 13(16), pages 1-21, August.
    11. Paola Tubaro, 2011. "Computational Economics," Chapters, in: John B. Davis & D. Wade Hands (ed.), The Elgar Companion to Recent Economic Methodology, chapter 10, Edward Elgar Publishing.
    12. Vincze, János & Varga, Gergely, 2016. "Megtakarítási típusok - egy adaptív-evolúciós megközelítés [Types of saving - an adaptive-evolutionary approach]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(2), pages 162-187.
    13. Kluger, Brian D. & McBride, Mark E., 2011. "Intraday trading patterns in an intelligent autonomous agent-based stock market," Journal of Economic Behavior & Organization, Elsevier, vol. 79(3), pages 226-245, August.
    14. Albert Banal-Estañol & Augusto Rupérez-Micola, 2010. "Are agent-based simulations robust? The wholesale electricity trading case," Economics Working Papers 1214, Department of Economics and Business, Universitat Pompeu Fabra.
    15. Paola Tubaro, 2009. "Agent-based Computational Economics: a Methodological Appraisal," EconomiX Working Papers 2009-42, University of Paris Nanterre, EconomiX.
    16. Shu-Heng Chan & Shu G. Wang, 2010. "Emergent Complexity in Agent-Based Computational Economics," ASSRU Discussion Papers 1017, ASSRU - Algorithmic Social Science Research Unit.
    17. LeBaron, Blake, 2006. "Agent-based Computational Finance," Handbook of Computational Economics, in: Leigh Tesfatsion & Kenneth L. Judd (ed.), Handbook of Computational Economics, edition 1, volume 2, chapter 24, pages 1187-1233, Elsevier.
    18. Delli Gatti,Domenico & Fagiolo,Giorgio & Gallegati,Mauro & Richiardi,Matteo & Russo,Alberto (ed.), 2018. "Agent-Based Models in Economics," Cambridge Books, Cambridge University Press, number 9781108400046, November.
    19. Cars Hommes & Florian Wagener, 2008. "Complex Evolutionary Systems in Behavioral Finance," Tinbergen Institute Discussion Papers 08-054/1, Tinbergen Institute.
    20. Lengnick, Matthias, 2013. "Agent-based macroeconomics: A baseline model," Journal of Economic Behavior & Organization, Elsevier, vol. 86(C), pages 102-120.

    More about this item

    Keywords

    adaptation; learning; turbulence; multi-armed bandit problem;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vnm:wpdman:40. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Daria Arkhipova (email available below). General contact details of provider: https://edirc.repec.org/data/mdvenit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.