IDEAS home Printed from https://ideas.repec.org/a/eee/mateco/v80y2019icp22-38.html
   My bibliography  Save this article

The K-armed bandit problem with multiple priors

Author

Listed:
  • Li, Jian

Abstract

This paper studies the impact of ambiguity in the classic K-armed bandit problem. I show that two classic results are robust to ambiguity in the multiple-priors model: (i) In the one-armed bandit, the optimal plan is a switching strategy characterized by a multiple-priors Gittins–Jones index. (ii) The seminal Gittins–Jones theorem is generalized to the multiple-priors K-armed bandit case. Introducing ambiguity has two implications. First, in the K-armed bandit case, the incentive to experiment with an arm decreases in its own perceived ambiguity and increases in other arms’ ambiguity, differing from the comparative statics on risk. This suggests that ambiguity might explain the widely observed underexperimentation in new technologies and consumer products. Second, I characterize an upper bound for the multiple-priors Gittins–Jones index, as the lower envelope of the classic single-prior Gittins–Jones index for every prior lying in the multiple-priors set. I show with a counterexample that this upper bound can be strict, and I identify sufficient conditions under which this upper bound is exact.

Suggested Citation

  • Li, Jian, 2019. "The K-armed bandit problem with multiple priors," Journal of Mathematical Economics, Elsevier, vol. 80(C), pages 22-38.
  • Handle: RePEc:eee:mateco:v:80:y:2019:i:c:p:22-38
    DOI: 10.1016/j.jmateco.2018.10.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304406818301198
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jmateco.2018.10.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gilboa Itzhak & Schmeidler David, 1993. "Updating Ambiguous Beliefs," Journal of Economic Theory, Elsevier, vol. 59(1), pages 33-49, February.
    2. Christopher Anderson, 2012. "Ambiguity aversion in multi-armed bandit problems," Theory and Decision, Springer, vol. 72(1), pages 15-33, January.
    3. Gajdos, T. & Hayashi, T. & Tallon, J.-M. & Vergnaud, J.-C., 2008. "Attitude toward imprecise information," Journal of Economic Theory, Elsevier, vol. 140(1), pages 27-65, May.
    4. Maccheroni, Fabio & Marinacci, Massimo & Rustichini, Aldo, 2006. "Dynamic variational preferences," Journal of Economic Theory, Elsevier, vol. 128(1), pages 4-44, May.
    5. Miao, Jianjun & Wang, Neng, 2011. "Risk, uncertainty, and option exercise," Journal of Economic Dynamics and Control, Elsevier, vol. 35(4), pages 442-461, April.
    6. ,, 2011. "Dynamic choice under ambiguity," Theoretical Economics, Econometric Society, vol. 6(3), September.
    7. Epstein Larry G & Seo Kyoungwon, 2011. "Symmetry or Dynamic Consistency?," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 11(1), pages 1-14, June.
    8. Cerreia-Vioglio, Simone & Maccheroni, Fabio & Marinacci, Massimo & Montrucchio, Luigi, 2013. "Ambiguity and robust statistics," Journal of Economic Theory, Elsevier, vol. 148(3), pages 974-1049.
      • Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Luigi Montrucchio, 2011. "Ambiguity and Robust Statistics," Working Papers 382, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    9. Robert J. Meyer & Yong Shi, 1995. "Sequential Choice Under Ambiguity: Intuitive Solutions to the Armed-Bandit Problem," Management Science, INFORMS, vol. 41(5), pages 817-834, May.
    10. Yoram Halevy, 2007. "Ellsberg Revisited: An Experimental Study," Econometrica, Econometric Society, vol. 75(2), pages 503-536, March.
    11. Bruno Strulovici, 2010. "Learning While Voting: Determinants of Collective Experimentation," Econometrica, Econometric Society, vol. 78(3), pages 933-971, May.
    12. Gilboa, Itzhak & Schmeidler, David, 1989. "Maxmin expected utility with non-unique prior," Journal of Mathematical Economics, Elsevier, vol. 18(2), pages 141-153, April.
    13. Frank Riedel, 2009. "Optimal Stopping With Multiple Priors," Econometrica, Econometric Society, vol. 77(3), pages 857-908, May.
    14. Lars Peter Hansen & Thomas J Sargent, 2014. "Robust Control and Model Uncertainty," World Scientific Book Chapters, in: UNCERTAINTY WITHIN ECONOMIC MODELS, chapter 5, pages 145-154, World Scientific Publishing Co. Pte. Ltd..
    15. Nishimura, Kiyohiko G. & Ozaki, Hiroyuki, 2004. "Search and Knightian uncertainty," Journal of Economic Theory, Elsevier, vol. 119(2), pages 299-333, December.
    16. Ghirardato, Paolo & Marinacci, Massimo, 2002. "Ambiguity Made Precise: A Comparative Foundation," Journal of Economic Theory, Elsevier, vol. 102(2), pages 251-289, February.
    17. , & ,, 2007. "Updating preferences with multiple priors," Theoretical Economics, Econometric Society, vol. 2(3), September.
    18. , G. & ,, 2010. "Symmetry of evidence without evidence of symmetry," Theoretical Economics, Econometric Society, vol. 5(3), September.
    19. Epstein, Larry G. & Schneider, Martin, 2003. "Recursive multiple-priors," Journal of Economic Theory, Elsevier, vol. 113(1), pages 1-31, November.
    20. Cesaltina Pacheco Pires, 2002. "A Rule For Updating Ambiguous Beliefs," Theory and Decision, Springer, vol. 53(2), pages 137-152, September.
    21. Epstein, Larry G & Wang, Tan, 1994. "Intertemporal Asset Pricing Under Knightian Uncertainty," Econometrica, Econometric Society, vol. 62(2), pages 283-322, March.
    22. Massimo Marinacci, 2002. "Learning from ambiguous urns," Statistical Papers, Springer, vol. 43(1), pages 143-151, January.
    23. Kreps, David M & Porteus, Evan L, 1978. "Temporal Resolution of Uncertainty and Dynamic Choice Theory," Econometrica, Econometric Society, vol. 46(1), pages 185-200, January.
    24. Kahn, Barbara E & Sarin, Rakesh K, 1988. "Modeling Ambiguity in Decisions under Uncertainty," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 15(2), pages 265-272, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Farzad Pourbabaee, 2022. "Robust experimentation in the continuous time bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 73(1), pages 151-181, February.
    2. Cosmin L. Ilut & Martin Schneider, 2022. "Modeling Uncertainty as Ambiguity: a Review," NBER Working Papers 29915, National Bureau of Economic Research, Inc.
    3. Battigalli, P. & Francetich, A. & Lanzani, G. & Marinacci, M., 2019. "Learning and self-confirming long-run biases," Journal of Economic Theory, Elsevier, vol. 183(C), pages 740-785.
    4. Farzad Pourbabaee, 2021. "High Dimensional Decision Making, Upper and Lower Bounds," Papers 2105.00545, arXiv.org.
    5. Farzad Pourbabaee, 2021. "Robust Experimentation in the Continuous Time Bandit Problem," Papers 2104.00102, arXiv.org.
    6. Pourbabaee, Farzad, 2021. "High dimensional decision making, upper and lower bounds," Economics Letters, Elsevier, vol. 204(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Jian, 2020. "Preferences for partial information and ambiguity," Theoretical Economics, Econometric Society, vol. 15(3), July.
    2. Alexander Zimper, 2011. "Do Bayesians Learn Their Way Out of Ambiguity?," Decision Analysis, INFORMS, vol. 8(4), pages 269-285, December.
    3. Georgalos, Konstantinos, 2021. "Dynamic decision making under ambiguity: An experimental investigation," Games and Economic Behavior, Elsevier, vol. 127(C), pages 28-46.
    4. Chambers, Robert G. & Melkonyan, Tigran, 2009. "Smoothing preference kinks with information," Mathematical Social Sciences, Elsevier, vol. 58(2), pages 173-189, September.
    5. Massimo Guidolin & Francesca Rinaldi, 2013. "Ambiguity in asset pricing and portfolio choice: a review of the literature," Theory and Decision, Springer, vol. 74(2), pages 183-217, February.
    6. Zimper, Alexander, 2009. "Half empty, half full and why we can agree to disagree forever," Journal of Economic Behavior & Organization, Elsevier, vol. 71(2), pages 283-299, August.
    7. Massimo Marinacci, 2015. "Model Uncertainty," Journal of the European Economic Association, European Economic Association, vol. 13(6), pages 1022-1100, December.
    8. Daniele Pennesi, 2013. "Asset Prices in an Ambiguous Economy," Carlo Alberto Notebooks 315, Collegio Carlo Alberto.
    9. Hill, Brian, 2020. "Dynamic consistency and ambiguity: A reappraisal," Games and Economic Behavior, Elsevier, vol. 120(C), pages 289-310.
    10. José Heleno Faro & Ana Santos, 2023. "Updating variational (Bewley) preferences," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 75(1), pages 207-228, January.
    11. Paul Viefers, 2012. "Should I Stay or Should I Go?: A Laboratory Analysis of Investment Opportunities under Ambiguity," Discussion Papers of DIW Berlin 1228, DIW Berlin, German Institute for Economic Research.
    12. Karni, Edi & Maccheroni, Fabio & Marinacci, Massimo, 2015. "Ambiguity and Nonexpected Utility," Handbook of Game Theory with Economic Applications,, Elsevier.
    13. Farzad Pourbabaee, 2021. "Robust Experimentation in the Continuous Time Bandit Problem," Papers 2104.00102, arXiv.org.
    14. Zimper, Alexander, 2012. "Asset pricing in a Lucas fruit-tree economy with the best and worst in mind," Journal of Economic Dynamics and Control, Elsevier, vol. 36(4), pages 610-628.
    15. Faro, José Heleno & Lefort, Jean-Philippe, 2019. "Dynamic objective and subjective rationality," Theoretical Economics, Econometric Society, vol. 14(1), January.
    16. Hui Chen & Nengjiu Ju & Jianjun Miao, 2014. "Dynamic Asset Allocation with Ambiguous Return Predictability," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 17(4), pages 799-823, October.
    17. Nengjiu Ju & Jianjun Miao, 2012. "Ambiguity, Learning, and Asset Returns," Econometrica, Econometric Society, vol. 80(2), pages 559-591, March.
    18. Farzad Pourbabaee, 2022. "Robust experimentation in the continuous time bandit problem," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 73(1), pages 151-181, February.
    19. Spyros Galanis, 2021. "Dynamic consistency, valuable information and subjective beliefs," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 71(4), pages 1467-1497, June.
    20. Gaurab Aryal & Ronald Stauber, 2014. "Trembles in extensive games with ambiguity averse players," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 57(1), pages 1-40, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:mateco:v:80:y:2019:i:c:p:22-38. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jmateco .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.