IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v172y2022ics0167947322000597.html
   My bibliography  Save this article

Flexible estimation of the state dwell-time distribution in hidden semi-Markov models

Author

Listed:
  • Pohle, Jennifer
  • Adam, Timo
  • Beumer, Larissa T.

Abstract

Hidden semi-Markov models generalise hidden Markov models by explicitly modelling the time spent in a given state, the so-called dwell time, using some distribution defined on the natural numbers. While the (shifted) Poisson and negative binomial distribution provide natural choices for such distributions, in practice, parametric distributions can lack the flexibility to adequately model the dwell times. To overcome this problem, a penalised maximum likelihood approach is proposed that allows for a flexible and data-driven estimation of the dwell-time distributions without the need to make any distributional assumption. This approach is suitable for direct modelling purposes or as an exploratory tool to investigate the latent state dynamics. The feasibility and potential of the suggested approach is illustrated in a simulation study and by modelling muskox movements in northeast Greenland using GPS tracking data. The proposed method is implemented in the R-package PHSMM which is available on CRAN.

Suggested Citation

  • Pohle, Jennifer & Adam, Timo & Beumer, Larissa T., 2022. "Flexible estimation of the state dwell-time distribution in hidden semi-Markov models," Computational Statistics & Data Analysis, Elsevier, vol. 172(C).
    • Handle: RePEc:eee:csdana:v:172:y:2022:i:c:s0167947322000597
    DOI: 10.1016/j.csda.2022.107479
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947322000597
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2022.107479?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nguyet Nguyen, 2018. "Hidden Markov Model for Stock Trading," IJFS, MDPI, vol. 6(2), pages 1-17, March.
    2. Roland Langrock & Thomas Kneib & Alexander Sohn & Stacy L. DeRuiter, 2015. "Nonparametric inference in hidden Markov models using P-splines," Biometrics, The International Biometric Society, vol. 71(2), pages 520-528, June.
    3. Bulla, Jan & Bulla, Ingo, 2006. "Stylized facts of financial time series and hidden semi-Markov models," Computational Statistics & Data Analysis, Elsevier, vol. 51(4), pages 2192-2209, December.
    4. Bulla, Jan & Bulla, Ingo & Nenadic, Oleg, 2010. "hsmm -- An R package for analyzing hidden semi-Markov models," Computational Statistics & Data Analysis, Elsevier, vol. 54(3), pages 611-619, March.
    5. Langrock, R. & Zucchini, W., 2011. "Hidden Markov models with arbitrary state dwell-time distributions," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 715-724, January.
    6. Timo Adam & Roland Langrock & Christian H. Weiß, 2019. "Penalized estimation of flexible hidden Markov models for time series of counts," METRON, Springer;Sapienza Università di Roma, vol. 77(2), pages 87-104, August.
    7. Roland Langrock & Timo Adam & Vianey Leos‐Barajas & Sina Mews & David L. Miller & Yannis P. Papastamatiou, 2018. "Spline‐based nonparametric inference in general state‐switching models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 179-200, August.
    8. Ruth King & Roland Langrock, 2016. "Semi-Markov Arnason–Schwarz models," Biometrics, The International Biometric Society, vol. 72(2), pages 619-628, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maruotti, Antonello & Petrella, Lea & Sposito, Luca, 2021. "Hidden semi-Markov-switching quantile regression for time series," Computational Statistics & Data Analysis, Elsevier, vol. 159(C).
    2. Morteza Amini & Afarin Bayat & Reza Salehian, 2023. "hhsmm: an R package for hidden hybrid Markov/semi-Markov models," Computational Statistics, Springer, vol. 38(3), pages 1283-1335, September.
    3. Ting Wang & Jiancang Zhuang & Kazushige Obara & Hiroshi Tsuruoka, 2017. "Hidden Markov modelling of sparse time series from non-volcanic tremor observations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(4), pages 691-715, August.
    4. Antonello Maruotti & Antonio Punzo, 2021. "Initialization of Hidden Markov and Semi‐Markov Models: A Critical Evaluation of Several Strategies," International Statistical Review, International Statistical Institute, vol. 89(3), pages 447-480, December.
    5. Roland Langrock & Timo Adam & Vianey Leos‐Barajas & Sina Mews & David L. Miller & Yannis P. Papastamatiou, 2018. "Spline‐based nonparametric inference in general state‐switching models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 179-200, August.
    6. Lennart Oelschlager & Timo Adam, 2020. "Detecting bearish and bullish markets in financial time series using hierarchical hidden Markov models," Papers 2007.14874, arXiv.org.
    7. Wang, Ting & Bebbington, Mark, 2013. "Identifying anomalous signals in GPS data using HMMs: An increased likelihood of earthquakes?," Computational Statistics & Data Analysis, Elsevier, vol. 58(C), pages 27-44.
    8. Shi, Yue & Punzo, Antonio & Otneim, Håkon & Maruotti, Antonello, 2023. "Hidden semi-Markov models for rainfall-related insurance claims," Discussion Papers 2023/17, Norwegian School of Economics, Department of Business and Management Science.
    9. Sofia Ruiz-Suarez & Vianey Leos-Barajas & Juan Manuel Morales, 2022. "Hidden Markov and Semi-Markov Models When and Why are These Models Useful for Classifying States in Time Series Data?," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 27(2), pages 339-363, June.
    10. Adam, Timo & Mayr, Andreas & Kneib, Thomas, 2022. "Gradient boosting in Markov-switching generalized additive models for location, scale, and shape," Econometrics and Statistics, Elsevier, vol. 22(C), pages 3-16.
    11. Milan Kumar Das & Anindya Goswami, 2019. "Testing of binary regime switching models using squeeze duration analysis," International Journal of Financial Engineering (IJFE), World Scientific Publishing Co. Pte. Ltd., vol. 6(01), pages 1-20, March.
    12. Hunt, Julien & Devolder, Pierre, 2011. "Semi Markov regime switching interest rate models and minimal entropy measure," LIDAM Discussion Papers ISBA 2011010, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    13. Vlad Stefan Barbu & Guglielmo D’Amico & Thomas Gkelsinis, 2021. "Sequential Interval Reliability for Discrete-Time Homogeneous Semi-Markov Repairable Systems," Mathematics, MDPI, vol. 9(16), pages 1-18, August.
    14. Jan Bulla, 2010. "Hidden Markov models with t components. Increased persistence and other aspects," Quantitative Finance, Taylor & Francis Journals, vol. 11(3), pages 459-475.
    15. Mevin B. Hooten & Ruth King & Roland Langrock, 2017. "Guest Editor’s Introduction to the Special Issue on “Animal Movement Modeling”," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 22(3), pages 224-231, September.
    16. Kunal Saha & Vinodh Madhavan & Chandrashekhar G. R. & David McMillan, 2020. "Pitfalls in long memory research," Cogent Economics & Finance, Taylor & Francis Journals, vol. 8(1), pages 1733280-173, January.
    17. Andrea Arfè & Stefano Peluso & Pietro Muliere, 2021. "The semi-Markov beta-Stacy process: a Bayesian non-parametric prior for semi-Markov processes," Statistical Inference for Stochastic Processes, Springer, vol. 24(1), pages 1-15, April.
    18. Guglielmo D'Amico & Filippo Petroni, 2020. "A micro-to-macro approach to returns, volumes and waiting times," Papers 2007.06262, arXiv.org.
    19. Jennifer Pohle & Roland Langrock & Floris M. Beest & Niels Martin Schmidt, 2017. "Selecting the Number of States in Hidden Markov Models: Pragmatic Solutions Illustrated Using Animal Movement," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 22(3), pages 270-293, September.
    20. Milan Kumar Das & Anindya Goswami & Sharan Rajani, 2019. "Inference of Binary Regime Models with Jump Discontinuities," Papers 1910.10606, arXiv.org, revised Mar 2022.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:172:y:2022:i:c:s0167947322000597. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.