IDEAS home Printed from https://ideas.repec.org/a/eee/spapps/v119y2009i6p1792-1822.html
   My bibliography  Save this article

Sequential tracking of a hidden Markov chain using point process observations

Author

Listed:
  • Bayraktar, Erhan
  • Ludkovski, Michael

Abstract

We study finite horizon optimal switching problems for hidden Markov chain models with point process observations. The controller possesses a finite range of strategies and attempts to track the state of the unobserved state variable using Bayesian updates over the discrete observations. Such a model has applications in economic policy making, staffing under variable demand levels and generalized Poisson disorder problems. We show regularity of the value function and explicitly characterize an optimal strategy. We also provide an efficient numerical scheme and illustrate our results with several computational examples.

Suggested Citation

  • Bayraktar, Erhan & Ludkovski, Michael, 2009. "Sequential tracking of a hidden Markov chain using point process observations," Stochastic Processes and their Applications, Elsevier, vol. 119(6), pages 1792-1822, June.
    • Handle: RePEc:eee:spapps:v:119:y:2009:i:6:p:1792-1822
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304-4149(08)00140-3
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Arjas, Elja & Haara, Pentti & Norros, Ikka, 1992. "Filtering the histories of a partially observed marked point process," Stochastic Processes and their Applications, Elsevier, vol. 40(2), pages 225-250, March.
    2. Ceci, Claudia & Gerardi, Anna, 1998. "Partially observed control of a Markov jump process with counting observations: equivalence with the separated problem," Stochastic Processes and their Applications, Elsevier, vol. 78(2), pages 245-260, November.
    3. Erhan Bayraktar & Savas Dayanik, 2006. "Poisson Disorder Problem with Exponential Penalty for Delay," Mathematics of Operations Research, INFORMS, vol. 31(2), pages 217-233, May.
    4. Peter Muller & Bruno Sanso & Maria De Iorio, 2004. "Optimal Bayesian Design by Inhomogeneous Markov Chain Simulation," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 788-798, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bäuerle Nicole & Gilitschenski Igor & Hanebeck Uwe, 2015. "Exact and approximate hidden Markov chain filters based on discrete observations," Statistics & Risk Modeling, De Gruyter, vol. 32(3-4), pages 159-176, December.
    2. Erhan Bayraktar & Michael Ludkovski, 2010. "Inventory management with partially observed nonstationary demand," Annals of Operations Research, Springer, vol. 176(1), pages 7-39, April.
    3. Liang, Zhibin & Bayraktar, Erhan, 2014. "Optimal reinsurance and investment with unobservable claim size and intensity," Insurance: Mathematics and Economics, Elsevier, vol. 55(C), pages 156-166.
    4. Pavel V. Gapeev, 2016. "Bayesian Switching Multiple Disorder Problems," Mathematics of Operations Research, INFORMS, vol. 41(3), pages 1108-1124, August.
    5. Naveed Chehrazi & Peter W. Glynn & Thomas A. Weber, 2019. "Dynamic Credit-Collections Optimization," Management Science, INFORMS, vol. 67(6), pages 2737-2769, June.
    6. Nicole Bauerle & Igor Gilitschenski & Uwe D. Hanebeck, 2014. "Exact and Approximate Hidden Markov Chain Filters Based on Discrete Observations," Papers 1411.0849, arXiv.org, revised Dec 2014.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vanessa Didelez, 2008. "Graphical models for marked point processes based on local independence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(1), pages 245-264, February.
    2. Pavel V. Gapeev & Hessah Al Motairi, 2018. "Perpetual American Defaultable Options in Models with Random Dividends and Partial Information," Risks, MDPI, vol. 6(4), pages 1-15, November.
    3. Erhan Bayraktar & Michael Ludkovski, 2010. "Inventory management with partially observed nonstationary demand," Annals of Operations Research, Springer, vol. 176(1), pages 7-39, April.
    4. Erhan Bayraktar & H. Poor, 2008. "Optimal time to change premiums," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 68(1), pages 125-158, August.
    5. Jin Feng, 2002. "A Stochastic Filtering Approach To Survival Analysis," Statistical Inference for Stochastic Processes, Springer, vol. 5(1), pages 23-53, January.
    6. Brandejsky, Adrien & de Saporta, Benoîte & Dufour, François, 2013. "Optimal stopping for partially observed piecewise-deterministic Markov processes," Stochastic Processes and their Applications, Elsevier, vol. 123(8), pages 3201-3238.
    7. Olivier David & Arnaud Le Rouzic & Christine Dillmann, 2022. "Optimization of sampling designs for pedigrees and association studies," Biometrics, The International Biometric Society, vol. 78(3), pages 1056-1066, September.
    8. Jukka Corander, 2023. "A conversation with Elja Arjas (Helsinki, November 2021 and March 2022)," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 50(1), pages 3-12, March.
    9. Herbert Lee & Matthew Taddy & Genetha Gray, 2010. "Selection of a Representative Sample," Journal of Classification, Springer;The Classification Society, vol. 27(1), pages 41-53, March.
    10. Ekin, Tahir & Naveiro, Roi & Ríos Insua, David & Torres-Barrán, Alberto, 2023. "Augmented probability simulation methods for sequential games," European Journal of Operational Research, Elsevier, vol. 306(1), pages 418-430.
    11. Giorgia Callegaro & Claudia Ceci & Giorgio Ferrari, 2020. "Optimal reduction of public debt under partial observation of the economic growth," Finance and Stochastics, Springer, vol. 24(4), pages 1083-1132, October.
    12. Tahir Ekin & Nicholas G. Polson & Refik Soyer, 2017. "Augmented nested sampling for stochastic programs with recourse and endogenous uncertainty," Naval Research Logistics (NRL), John Wiley & Sons, vol. 64(8), pages 613-627, December.
    13. Suhan Altay & Katia Colaneri & Zehra Eksi, 2017. "Portfolio optimization for a large investor controlling market sentiment under partial information," Papers 1706.03567, arXiv.org.
    14. Tevfik Aktekin & Tahir Ekin, 2016. "Stochastic call center staffing with uncertain arrival, service and abandonment rates: A Bayesian perspective," Naval Research Logistics (NRL), John Wiley & Sons, vol. 63(6), pages 460-478, September.
    15. Ekin, Tahir & Aktekin, Tevfik, 2021. "Decision making under uncertain and dependent system rates in service systems," European Journal of Operational Research, Elsevier, vol. 291(1), pages 335-348.
    16. Ekin, Tahir, 2018. "Integrated maintenance and production planning with endogenous uncertain yield," Reliability Engineering and System Safety, Elsevier, vol. 179(C), pages 52-61.
    17. Daniel R. Cavagnaro & Richard Gonzalez & Jay I. Myung & Mark A. Pitt, 2013. "Optimal Decision Stimuli for Risky Choice Experiments: An Adaptive Approach," Management Science, INFORMS, vol. 59(2), pages 358-375, February.
    18. Pavel V. Gapeev, 2016. "Bayesian Switching Multiple Disorder Problems," Mathematics of Operations Research, INFORMS, vol. 41(3), pages 1108-1124, August.
    19. Buonaguidi, B., 2022. "The disorder problem for diffusion processes with the ϵ-linear and expected total miss criteria," Statistics & Probability Letters, Elsevier, vol. 189(C).
    20. Savas Dayanik & Semih Onur Sezer, 2006. "Compound Poisson Disorder Problem," Mathematics of Operations Research, INFORMS, vol. 31(4), pages 649-672, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:spapps:v:119:y:2009:i:6:p:1792-1822. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/505572/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.