IDEAS home Printed from https://ideas.repec.org/a/eee/spapps/v117y2007i9p1357-1371.html
   My bibliography  Save this article

On Gittins' index theorem in continuous time

Author

Listed:
  • Bank, Peter
  • Küchler, Christian

Abstract

We give a new and comparably short proof of Gittins' index theorem for dynamic allocation problems of the multi-armed bandit type in continuous time under minimal assumptions. This proof gives a complete characterization of optimal allocation strategies as those policies which follow the current leader among the Gittins indices while ensuring that a Gittins index is at an all-time low whenever the associated project is not worked on exclusively. The main tool is a representation property of Gittins index processes which allows us to show that these processes can be chosen to be pathwise lower semi-continuous from the right and quasi-lower semi-continuous from the left. Both regularity properties turn out to be crucial for our characterization and the construction of optimal allocation policies.

Suggested Citation

  • Bank, Peter & Küchler, Christian, 2007. "On Gittins' index theorem in continuous time," Stochastic Processes and their Applications, Elsevier, vol. 117(9), pages 1357-1371, September.
    • Handle: RePEc:eee:spapps:v:117:y:2007:i:9:p:1357-1371
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304-4149(07)00016-6
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Samuel N. Cohen & Tanut Treetanthiploet, 2019. "Gittins' theorem under uncertainty," Papers 1907.05689, arXiv.org, revised Jun 2021.
    2. Diana M. Negoescu & Kostas Bimpikis & Margaret L. Brandeau & Dan A. Iancu, 2018. "Dynamic Learning of Patient Response Types: An Application to Treating Chronic Diseases," Management Science, INFORMS, vol. 64(8), pages 3469-3488, August.
    3. Li, Xiao & Li, Yuqiang & Wu, Xianyi, 2023. "Empirical Gittins index strategies with ε-explorations for multi-armed bandit problems," Computational Statistics & Data Analysis, Elsevier, vol. 180(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:spapps:v:117:y:2007:i:9:p:1357-1371. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/505572/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.