IDEAS home Printed from https://ideas.repec.org/a/spr/joecth/v73y2022i1d10.1007_s00199-020-01328-3.html
   My bibliography  Save this article

Robust experimentation in the continuous time bandit problem

Author

Listed:
  • Farzad Pourbabaee

    (University of California)

Abstract

We study the experimentation dynamics of a decision maker (DM) in a two-armed bandit setup (Bolton and Harris in Econometrica 67(2):349–374, 1999), where the agent holds ambiguous beliefs regarding the distribution of the return process of one arm and is certain about the other one. The DM entertains Multiplier preferences à la Hansen and Sargent (Am. Econ. Rev. 91(2):60–66, 2001), thus we frame the decision making environment as a two-player differential game against nature in continuous time. We characterize the DM’s value function and her optimal experimentation strategy that turns out to follow a cut-off rule with respect to her belief process. The belief threshold for exploring the ambiguous arm is found in closed form and is shown to be increasing with respect to the ambiguity aversion index. We then study the effect of provision of an unambiguous information source about the ambiguous arm. Interestingly, we show that the exploration threshold rises unambiguously as a result of this new information source, thereby leading to more conservatism. This analysis also sheds light on the efficient time to reach for an expert opinion.

Suggested Citation

  • Farzad Pourbabaee, 2022. "Robust experimentation in the continuous time bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 73(1), pages 151-181, February.
    • Handle: RePEc:spr:joecth:v:73:y:2022:i:1:d:10.1007_s00199-020-01328-3
    DOI: 10.1007/s00199-020-01328-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00199-020-01328-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00199-020-01328-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gustavo Manso, 2011. "Motivating Innovation," Journal of Finance, American Finance Association, vol. 66(5), pages 1823-1860, October.
    2. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Misspecification," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 6, pages 155-216, World Scientific Publishing Co. Pte. Ltd..
    3. Christopher Anderson, 2012. "Ambiguity aversion in multi-armed bandit problems," Theory and Decision, Springer, vol. 72(1), pages 15-33, January.
    4. Hansen, Lars Peter & Sargent, Thomas J., 2011. "Robustness and ambiguity in continuous time," Journal of Economic Theory, Elsevier, vol. 146(3), pages 1195-1223, May.
    5. Bonatti, Alessandro & Hörner, Johannes, 2017. "Learning to disagree in a game of experimentation," Journal of Economic Theory, Elsevier, vol. 169(C), pages 234-269.
    6. Larry G. Epstein & Shaolin Ji, 2022. "Optimal Learning Under Robustness and Time-Consistency," Operations Research, INFORMS, vol. 70(3), pages 1317-1329, May.
    7. Heidhues, Paul & Rady, Sven & Strack, Philipp, 2015. "Strategic experimentation with private payoffs," Journal of Economic Theory, Elsevier, vol. 159(PA), pages 531-551.
    8. Li, Jian, 2019. "The K-armed bandit problem with multiple priors," Journal of Mathematical Economics, Elsevier, vol. 80(C), pages 22-38.
    9. Godfrey Keller & Sven Rady & Martin Cripps, 2005. "Strategic Experimentation with Exponential Bandits," Econometrica, Econometric Society, vol. 73(1), pages 39-68, January.
    10. Yaoyao Wu & Jinqiang Yang & Zhentao Zou, 2018. "Ambiguity sharing and the lack of relative performance evaluation," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 66(1), pages 141-157, July.
    11. Larry G. Epstein & Martin Schneider, 2007. "Learning Under Ambiguity," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 74(4), pages 1275-1303.
    12. Robert J. Meyer & Yong Shi, 1995. "Sequential Choice Under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem," Management Science, INFORMS, vol. 41(5), pages 817-834, May.
    13. Jianjun Miao & Alejandro Rivera, 2016. "Robust Contracts in Continuous Time," Econometrica, Econometric Society, vol. 84(4), pages 1405-1440, July.
    14. Gilboa, Itzhak & Schmeidler, David, 1989. "Maxmin expected utility with non-unique prior," Journal of Mathematical Economics, Elsevier, vol. 18(2), pages 141-153, April.
    15. Yulei Luo, 2017. "Robustly Strategic Consumption–Portfolio Rules with Informational Frictions," Management Science, INFORMS, vol. 63(12), pages 4158-4174, December.
    16. Weitzman, Martin L, 1979. "Optimal Search for the Best Alternative," Econometrica, Econometric Society, vol. 47(3), pages 641-654, May.
    17. Frank Riedel, 2009. "Optimal Stopping With Multiple Priors," Econometrica, Econometric Society, vol. 77(3), pages 857-908, May.
    18. Godfrey Keller & Sven Rady, 1999. "Optimal Experimentation in a Changing Environment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(3), pages 475-507.
    19. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    20. Patrick Bolton & Christopher Harris, 1999. "Strategic Experimentation," Econometrica, Econometric Society, vol. 67(2), pages 349-374, March.
    21. Epstein, Larry G. & Schneider, Martin, 2003. "Recursive multiple-priors," Journal of Economic Theory, Elsevier, vol. 113(1), pages 1-31, November.
    22. Massimo Marinacci, 2002. "Learning from ambiguous urns," Statistical Papers, Springer, vol. 43(1), pages 143-151, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Farzad Pourbabaee, 2024. "Reputation, learning and project choice in frictional economies," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 78(4), pages 1075-1115, December.
    2. Matt Van Essen & John Wooders, 2023. "Dual auctions for assigning winners and compensating losers," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 76(4), pages 1069-1114, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Farzad Pourbabaee, 2021. "Robust Experimentation in the Continuous Time Bandit Problem," Papers 2104.00102, arXiv.org.
    2. Li, Jian, 2019. "The K-armed bandit problem with multiple priors," Journal of Mathematical Economics, Elsevier, vol. 80(C), pages 22-38.
    3. Paul Viefers, 2012. "Should I Stay or Should I Go?: A Laboratory Analysis of Investment Opportunities under Ambiguity," Discussion Papers of DIW Berlin 1228, DIW Berlin, German Institute for Economic Research.
    4. Alexander Zimper, 2011. "Do Bayesians Learn Their Way Out of Ambiguity?," Decision Analysis, INFORMS, vol. 8(4), pages 269-285, December.
    5. Daniele Pennesi, 2013. "Asset Prices in an Ambiguous Economy," Carlo Alberto Notebooks 315, Collegio Carlo Alberto.
    6. Bhattacharjee, Swagata, 2022. "Dynamic contracting for innovation under ambiguity," Games and Economic Behavior, Elsevier, vol. 132(C), pages 534-552.
    7. Li, Jing, 2018. "Essays on model uncertainty in financial models," Other publications TiSEM 202cd910-7ef1-4db4-94ae-d, Tilburg University, School of Economics and Management.
    8. Cosmin L. Ilut & Martin Schneider, 2022. "Modeling Uncertainty as Ambiguity: a Review," NBER Working Papers 29915, National Bureau of Economic Research, Inc.
    9. Jang, Bong-Gyu & Lee, Seungkyu & Lim, Byung Hwa, 2016. "Robust consumption and portfolio rules with time-varying model confidence," Finance Research Letters, Elsevier, vol. 18(C), pages 342-352.
    10. Peter G. Hansen, 2021. "New Formulations of Ambiguous Volatility with an Application to Optimal Dynamic Contracting," Papers 2101.12306, arXiv.org.
    11. Swagata Bhattacharjee, 2019. "Dynamic Contracting for Innovation Under Ambiguity," Working Papers 1022, Ashoka University, Department of Economics, revised Aug 2019.
    12. Battigalli, P. & Francetich, A. & Lanzani, G. & Marinacci, M., 2019. "Learning and self-confirming long-run biases," Journal of Economic Theory, Elsevier, vol. 183(C), pages 740-785.
    13. Hansen, Peter G., 2022. "New formulations of ambiguous volatility with an application to optimal dynamic contracting," Journal of Economic Theory, Elsevier, vol. 199(C).
    14. Moreno Othón M., 2014. "Consumption of Durable Goods under Ambiguity," Working Papers 2014-02, Banco de México.
    15. Yacine Aït-Sahalia & Felix Matthys & Emilio Osambela & Ronnie Sircar, 2021. "When Uncertainty and Volatility Are Disconnected: Implications for Asset Pricing and Portfolio Performance," NBER Working Papers 29195, National Bureau of Economic Research, Inc.
    16. Michael Barnett & Greg Buchak & Constantine Yannelis, 2023. "Epidemic responses under uncertainty," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 120(2), pages 2208111120-, January.
    17. Agarwal, Vikas & Arisoy, Y. Eser & Naik, Narayan Y., 2017. "Volatility of aggregate volatility and hedge fund returns," Journal of Financial Economics, Elsevier, vol. 125(3), pages 491-510.
    18. Nengjiu Ju & Jianjun Miao, 2012. "Ambiguity, Learning, and Asset Returns," Econometrica, Econometric Society, vol. 80(2), pages 559-591, March.
    19. Saghafian, Soroush, 2018. "Ambiguous partially observable Markov decision processes: Structural results and applications," Journal of Economic Theory, Elsevier, vol. 178(C), pages 1-35.
    20. Maccheroni, Fabio & Marinacci, Massimo & Rustichini, Aldo, 2006. "Dynamic variational preferences," Journal of Economic Theory, Elsevier, vol. 128(1), pages 4-44, May.

    More about this item

    Keywords

    Model uncertainty; Dynamic experimentation; Variational preferences; Information valuation; Ambiguous diffusion;
    All these keywords.

    JEL classification:

    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C73 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - Stochastic and Dynamic Games; Evolutionary Games
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joecth:v:73:y:2022:i:1:d:10.1007_s00199-020-01328-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.